What is the error margin of goal line technology?

During the Premier League match between Aston Villa and Fulham on April 5, Lewis Holtby had a shot which was cleared off the line and adjudicated on by the goal line technology used by the league. Plenty of opinions were available on Twitter in the aftermath of this incident as it was so tight but none focused on the most important aspect, namely how accurate is the system?

Firstly, take a look at Lewis Holtby’s shot, which can be found at around the 3:30 mark on the video below.


After this happened, Al Jazeera presenter Richard Keys tweeted the following:

The suggestion from Keys is therefore that his own eyes are better at identifying whether a goal had been scored than the Hawkeye technology used by the Premier League. In order to assess this, we need to know two things; the error margin of the  Hawkeye system and the error margin of Keys’ eyes in these situations. Hawkeye seems to claim 5mm but I have been unable to find an independent assessment, only claims from Hawkeye themselves. As for Richard Keys’  eyes, we will have to make do with the human eye and I am hoping that readers might be able to point me in the right direction on this as a quick google search hasn’t provided me with an answer.


Soon after Keys’ tweet came another from Sporting Intelligence’s Nick Harris:


The problem here is that it is presented as if Hawkeye is 100% correct which we know is not the case given the company’s own literature on its accuracy. The image which ‘proves’ it isn’t a goal is a computer generated image from the Hawkeye system but how accurate is that system? If the ball was over the line by less than 5mm, the system could be wrong if the company’s own claim of a 5mm error margin is correct.


The two media opinions on Hawkeye’s decision, and others like them from twitter are surprising in the main because they either put 100% faith in the human eye or 100% faith in the technology. Both though have error margins and in the end, the choice of goal line technology above anything else should be made on one simple test – is the system being used better than all of the other alternatives?


Posted in Uncategorized | Tagged , , , , , , | 14 Comments

Updated: Is this season’s Manchester United really Everton in disguise?

There was an error in the second graphic in the original post. This has now been corrected and the post updated. It leads to a slightly different conclusion. I have also added the lines for the rest of the season.

Earlier this year, Constantinos Chappas, an analytical football blogger based in Cyprus tweeted a chart showing that Manchester United this season were following the same trajectory of results as Everton last year. Cue plenty of retweeting and much hilarity at David Moyes turning Manchester United into Everton. However, like many things looking at football matches in chronological order, could it be misleading?


In order to investigate the Manchester United 2013/2014 against Everton 2012/2013 phenomenon further, I have employed my trusty ISG coefficient which compares results in matches played with those achieved in exactly the same fixtures a year earlier. There is, of course, one problem – what should the fixture earlier this season between Manchester United and Everton be compared to? Given the hypothesis that Manchester United this term are equal to Everton last, it only seems appropriate to use David Moyes as the factor so Manchester United at home to Everton in 2013/2014 is Everton at home to Manchester United last term. Any disagreements with this are welcome in the comments or by tweeting me at my usual address. The two graphics – one comparing the 30 matches to date chronologically and the second comparing direct fixtures are below:

Click on the graphic to go to the full size version

So, although both graphs have shown a similar pattern for much of the season, Moyes’ United have recently overtaken Moyes’ Everton when considering the exact same matches from one season to the next . Everton’s final total last season was 63 points so Manchester United need at least another 13 from their final eight fixtures this term to improve on this points total.



Finally, congratulations to the member of Sky Sports staff who finally got this little nugget of information on television this week as you can see above. Personally I prefer the graphics as they show the full season rather than a single moment in time though so I have the Excel sheets if you need them next time.


Posted in Uncategorized | Tagged , , , | Leave a comment

538 bracket coverage doesn’t differentiate itself from old media

Currently the top story on Nate Silver’s relaunched media venture fivethirtyeight.com suggests that the odds of entering a perfect March Madness bracket are much much better than many people think. The problem is that the headline is misleading at best. Does that statement remind you of anything? Old media perhaps?


For the unitiated, the March Madness bracket is made up of 63 college basketball matches which will be played to decide the best college basketball team in the US. The aim is to predict the winners of each match correctly before a single match has been played.

Whilst the approximately one in 7.5 billion chance quoted in the headline is correct – although it wasn’t initially when the incorrect match-ups were present – it refers to the chance that all 63 favourites will win their matches. This number is therefore just as misleading as the one in 9.2 quintillion chance that can be calculated by assuming – clearly wrongly – that each match is a coin toss between the two teams. Given that Silver and his team have built a statistical model to assess the chances for each match-up, the chance of a perfect bracket can genuinely be calculated from history to provide a much smaller range. What was the chance in each of the last 10 years for example? More useful but still rather dull in my opinion.

The other misleading thing here is that backing each favourite is something which many many many people will do and therefore, in the highly unlikely case – one in 7.5 billion of course – that all 63 favourites won, Warren Buffet’s billion dollars would therefore be split multiple ways. The return of investment would be even worse than the approximately 14 cents on the dollar that it is for a single winner. To put this into perspective, players have around a one in 14 million chance of winning the UK Lottery but if you pick numbers 1, 2, 3, 4, 5 and 6, you will be one of around 10,000 people doing so. Therefore, even with a 14 million pound jackpot, you will win just £1400. This is admittedly explained in the article at fivethirtyeight.com but much much further down.

There is a great story to be told here but fivethirtyeight.com has just chosen not to tell it, namely WHY is it so hard to enter a correct bracket? Put simply, the reason is that we know that there will almost certainly be surprise results but we don’t know how many there will be or which matches will provide those shocks. Using the FA Cup as an analogy for this for my European readers, it is highly unlikely that there will be no surprise results in the 64 matches which take place in the third round of the competition. If it were possible to select the matches in which those surprises would happen correctly and backing the favourites in the others to go through, getting rich quickly would be a cinch.

The known – that there will almost certainly be shocks – and two unknowns – that we don’t know how many or which matches – are what make prediction so incredibly hard. If people were aware of this simple fact, they might think more about what they are doing when gambling. As one of my twitter followers said yesterday, ‘if people understood probability, Las Vegas would be a crossroads in the desert’. Isn’t this a role that 538.com could play when it comes to its sports section?

Media stands at a crossroads as old media business models don’t work anymore and both new and old media are still searching for feasible business models. Nate Silver has been offered a huge opportunity by ESPN to develop something which could be part of the future of media but this current ‘big’ story just seems to me to be old media dressed up as new. I believe in data journalism but I believe in making it accessible without dumbing it down too much. There are plenty of numbers in this article but not enough story. The story is always what matters the most.

Posted in Uncategorized | Tagged , , , , , | 3 Comments

The FA Cup ‘giantkillings’ that never were

FA Cup holders Wigan Athletic beat Crystal Palace in this year’s FA Cup fourth round, a match which seemed to sow doubt in the minds of some reporters in terms of how they should present a result which would normally be seen as an FA Cup giantkilling. Could the FA Cup holders really be described as giantkillers? The fact is that this result typified something in the competition which is becoming more and more common, namely the giantkilling that never was.

The fixture between Wigan and Crystal Palace was already the second in this year’s competition in which a Premier League team was unsurprisingly knocked out by a team from a division below. The first was a round earlier when Nottingham Forest beat a second string West Ham United team as Sam Allardyce rested his players for a League Cup semi-final at Manchester City in which his charges would also be thrashed.

The Guardian’s Paul Wilson was clearly uncertain of how to assess Wigan’s achievement against Crystal Palace: “It is not often that the holders of the FA Cup can be considered giantkillers, and it is even more debatable whether Crystal Palace can be considered lofty enough to fit the Goliath billing…”

Richard Bott in the Daily Mail was even more torn, describing Wigan’s victory as a “minor shock”. The fact is that the giantkilling that never was is becoming a more common feature of the FA Cup as Premier League managers from teams in the Threatened 13 line up second string players in fixtures against opposition from outside the elite.

The table below contains all of the FA Cup matches from the last 10 seasons in which a Premier League club was eliminated by a team from outside the competition but the lower team was the favourite to win with the bookmakers.


It is perhaps surprising that the top of the list, and thus the most likely ‘giantkilling that wasn’t’ of all, was perpetrated by a League Two club against a Premier League team. How could a team two divisions below the Premier League be such a strong favourite against one of the country’s top 20 clubs? Quite simply, Blackpool manager Ian Holloway lined up what was effectively a reserve team and opponents Southampton were one of the best teams in League One. Indeed, the Saints were promoted at the end of that season, putting them in the same division as relegated Blackpool, and were a Premier League team themselves a year later.

Saturday’s victors Wigan Athletic were even the underdogs against a League Two club, Swindon Town, in a supposed giantkilling act two seasons ago. Like Blackpool a year earlier, Wigan fielded a largely young and inexperienced team in this match and lost 2-1 to a high flying Swindon team which would finish the season as champions. Swindon’s victory was described by the BBC as “a remarkable win”. It wasn’t, as the bookmakers’ odds on the match confirm. Swindon Town were the favourites, albeit marginal, to win against their more illustrious opposition that day.

It has long been a feature of the FA Cup that the Premier League teams fighting for the title and European places have fielded under-strength teams against lower opposition but, judging from the above table, it seems that this is now routine for the rest of the competition. The data suggests that this was not widespread until around 2009 but from that year on, the Premier League’s Threatened 13 clubs are being regularly knocked out by lower league opposition in matches which, due to the line-ups being fielded, cannot be regarded as shocks.

Now that the FA Cup draw has been blown wide open by the best four teams remaining being drawn against one another in round five, perhaps Sam Allardyce and Tony Pulis (although he made fewer changes than some) might regret their decisions to field weakened teams in winnable matches. After all, Wigan Athletic won the competition last season, in part due to a relatively smooth set of draws on the way to the final. Then again, Wigan were also relegated to the Championship and, in the end, avoiding the drop is probably more important for everyone associated with these clubs.



Posted in Uncategorized | Tagged , , , , , , , , | Leave a comment

What is a giantkilling?

The 2013-2014 FA Cup fourth round is in full swing with Arsenal’s 4-0 victory against Coventry City and Nottingham Forest’s goalless draw with Preston North End already consigned to the history books. Part of the romance of a cup competition are the giantkilling acts but the nature of these has been utterly transformed by the enormous quality differences across the clubs these days and often the biggest shocks are now taking place in matches involving two clubs from the same division.

To illustrate this phenomenon, I have identified every FA Cup match from the last 10 seasons in which the winner had odds of at least 6 according to Odds Portal which, for those who are more familiar with fractional odds, equates to 5/1. No matches in the 2004/2005, 2006/2007 or 2011/2012 FA Cup competitions were won by teams with odds this high whereas four fixtures in this year’s third round contained shocks of this magnitude.

The most interesting thing in the above table is the sudden appearance, shown in bold, over the last two seasons of huge shocks in matches involving two clubs from the same division. Last season there were three, including the FA Cup final itself, probably the most surprising result in an FA Cup final for 25 years, since Wimbledon beat Liverpool 1-0 in 1988. This season, however, has already matched that figure with Cardiff, Swansea and Crystal Palace’s third round victories at St James’ Park, Old Trafford and the Hawthorns respectively. As a comparison, there was only one surprise of this magnitude in matches involving two teams from the same division in the eight seasons prior to last term; eventual 2007/2008 FA Cup winners Portsmouth’s quarter-final win at Manchester United. Huddersfield Town’s replay win at Leicester City last season was the first time in the 10 season sample that a Championship team had won an FA Cup fixture at odds of 6 or more at a fellow Championship team.

Cardiff’s third round victory at fellow Premier League team Newcastle United a few weeks ago is even the joint biggest surprise in this year’s competition, with non-league Kidderminster’s 3-2 win at League One ‘giants’ Peterborough United.

An FA Cup giantkilling has always been defined as knocking out a team from a division above but at a time when Nottingham Forest’s win against West Ham United was actually to be expected given Forest’s winning odds of 1.75 (between 8/11 and 4/5), is this the best definition? Sheffield United’s 2-1 win away to Aston Villa is an even more extreme example as, despite there being two divisions between the pair, this ‘shock’ was actually smaller than Crystal Palace’s win at Villa’s West Midlands neighbours West Bromwich Albion. At the time, both West Bromwich Albion and Crystal Palace occupied the bottom half of the Premier League table.


Posted in Uncategorized | Tagged , , , , , | 2 Comments

Manchester City kings of the big matches

Matches against the Superior 7 this season do not produce the same neat clusters as were presented in the previous post on matches against the Threatened 13. Obviously this is partially due to less matches being played and it remains to be seen how much of an overlap will remain once the season has been completed. Manchester City are the kings of these big matches this term with 15 points out of a possible 18 but they need to continue to produce these sorts of results to make up for their deficit in comparison to Arsenal in fixtures against the Threatened 13.

City’s big matches to get tougher
With 42 matches between the Superior 7 clubs over the season, it is a rare round of matches which doesn’t contain at least one. On Premier League matchday 23 for example, Tottenham Hotspur will host Manchester City whilst Liverpool and Everton meet in the 222nd Merseyside derby. Five days after visiting White Hart Lane, Manchester City will complete their home fixtures against the Superior 7 by playing Chelsea at the Etihad Stadium. Therefore City’s title credentials will be tested in their next two matches, particularly as they have dropped points against the Threatened 13 in comparison to their title rivals. The main reason that City top the table of results against the Superior 7 is due to having won all five at home. Their sole away fixture to date against one of these teams ended in a 2-1 defeat at Stamford Bridge.

Premier League table of results against Superior 7 clubs

Cann Table S7

Superior 7 clubs shown in green, Threatened 13 clubs shown in red

United’s poor big match results
Manchester City’s rivals Manchester United have taken just five points from matches against fellow Superior 7 opponents despite playing eight of these fixtures so far. Newcastle United, West Bromwich Albion, Aston Villa and Sunderland have all taken more points from the top-7.  United are also no more successful in these matches than Southampton, Stoke City and Cardiff City. However, it is not unusual for teams to perform like this and yet still finish in the Champions League places; Arsenal managed to take just 10 points from the Superior 7 last season and Tottenham Hotspur totalled just 12 points a season earlier.

Moyes in big matches
Staying with Manchester United, their manager David Moyes appears to have a reputation for not being successful in these bigger fixtures of the season but Everton were joint second best of all clubs in matches against the Superior 7 in 2010/2011 and joint fourth best last term. In 2011/2012, 11 clubs were better than Everton although Champions League winners that season Chelsea were not amongst them.

Man City record
The highest number of points achieved against the Superior 7 since this echelon came into existence in the 2010/2011 season is 25 by Manchester City in their title winning year. City will probably need to repeat this performance to add a second Premier League trophy to the one they won two years ago.

Posted in Uncategorized | Tagged , , , , , | Leave a comment

Arsenal on target for title via results against Threatened 13

My last post introduced the concept of The Superior 7 and the Threatened 13 in the Premier League. Now that the competition is moving into its business end, it is instructive to have a look at how these different match-ups look in terms of results.The table which follows confirms these two clusters of teams and also suggests that Arsenal should perhaps be taken more seriously as potential title winners. It also provides a new, and possibly more insightful, way of looking at the Premier League’s relegation battle.

Cann Table
This post focuses on Premier League matches played this season against the Threatened 13. I have visualized these as a Cann Table (thanks to Thomas at @catenaccio for the design) as this shows the gaps between the teams and is therefore clearer than a standard table. If you are unfamiliar with the concept of Cann Tables, they are gaining in popularity as seen in this article. To keep it simple, the table is also shown based on points and not points per match. Matches played against the Threatened 13 range from 12 (Swansea City) to 16 (Arsenal, Liverpool, Manchester City and Everton).

CannTable matches v T13

Gap is clear
The gap between the two clusters is clear in the Cann Table above. The Superior Seven are currently taking more points from these matches than any Threatened 13 club. This is also the case when looking at points per match with Everton the worst of the Superior 7 (2.06 points per match) and Newcastle and Southampton the best of the Threatened 13 (1.86 points per match).

Arsenal on target
Arsenal’s dominance of these matches, which has led to the Gunners dropping just five points out of a possible 48, is also interesting as a continuation of this is probably the key to the Londoners winning this season’s Premier League. With a total of 78 points available in matches against the clearly weaker Threatened 13 clubs, dropping few of those points makes it is less necessary to pick up a lot of good results in the much tougher matches against the Superior 7. Manchester United’s title win last season illustrated this very nicely and will be a future subject for this series of posts. If Arsenal can take 65 or more points from these 26 matches, they will probably need less than 1.5 points per match from their ‘big’ games against the Superior 7 to win the title.

Moving onto the relegation battle, my self defined goal for avoiding the drop is to take 1.5 points per match from the 24 matches against fellow Threatened 13 clubs. A team doing this will get 36 points without having to get anything from their 14 matches against the Superior 7 which should take the pressure off in those matches as few points can be expected from them.

Breaking the threshold
There are currently just five of the Threatened 13 who are hitting this threshold; Southampton (1.86), Newcastle United (1.86), Swansea City (1.67), Norwich City (1.57) and Hull City (1.54). A potential concern for Hull City supporters is that their club has won just one of their last seven of these matches and defeat at Selhurst Park in their next fixture will push them below the required threshold for the first time this season.

In trouble
Four clubs are averaging just one point per match or fewer in fixtures against the Threatened 13 and this quartet could be in real trouble if this does not improve as it means that points will have to be picked up against the big Premier League clubs who inhabit the Superior 7 cluster. Sunderland are the worst with 0.8 points per match against their Threatened 13 bedfellows whilst Cardiff City, West Bromwich Albion and West Ham United all average a point per match in these fixtures.

Two echelons
One of the criticisms of the article which introduced this concept is that it includes Southampton and Newcastle United amongst the Threatened 13. To perhaps make it a bit clearer, the classification is based on the long term and not the short term. This split in the Premier League is clear as a long term trend and Newcastle United’s recent decline from a higher position has confirmed this. There may be an argument in the future for classifying Newcastle as an Excellent Eight team perhaps but not until they consistently perform at the level they have shown this season. As for Southampton, this is their first season of promise and would need to be repeated to be regarded as apart from the relegation fighters. Their injury problems, which have led to poorer results is also clearly the point here. If their squad was stronger, they wouldn’t be affected so badly. Finally, the suggestion that this pair are mid-table teams is also not strong in my opinion as both fought against the drop last term and the concept of a team aiming to finish mid-table makes little sense. Either clubs are battling to win the championship, get into Europe, or they have survival as a first priority.

Finally, keep an eye on those Arsenal matches against the Threatened 13. Can they hit the magic 65+ points figure? If so, the Gunners just might be difficult to stop.

Posted in Uncategorized | Tagged , , , , , , , , , , , , , , , | 1 Comment