26 Predictions: English Premier League forecasting laid bare

With the help of Constantinos Chappas (follow him on Twitter at @cchappas, I have collected 26 pre-season Premier League predictions together, 13 which are at least partially model based, and 13 from the media. The models select Manchester City as title favourites but the journalists favour Chelsea. Who will come out on top?

Having spent part of the summer reading Nassim Nicholas Taleb’s Antifragile, I have become interested in the idea of “skin in the game”. Basically, this concept means that an individual puts his own funds at risk in the pursuit of a goal. He has “skin in the game”. Taleb is basically suggesting that predictors should be putting their money where their mouths are, although he is referring to the predictors of the financial markets. I agree, this applies very well to sports predictors. If someone is running a tipping service for example, I want to see their betting slips. So, how does this relate to this exercise?

Everyone predicts stuff at the beginning of every football season but it is rare that people look back on these predictions to see which came out best? I have only seen James Grayson (@jameswgrayson) do this in order to compare his own models to various benchmarks. With a growing interest in football modelling, it seemed to Constantinos and I that it would be useful to assess these models along with predictions from media experts.  Plus everyone loves to see a prediction so by collecting them all in one place, we are providing a service to our Twitter followers and blog readers.

Anyway, without further ado, here are the predictions. Please click on them to see them full size and thus more readable. They are ordered by the average of each group as measured by the mean. Median has also been included for comparison.

First the models:

Then the experts

Model based predictions come with points so that makes them easy to assess using mean square error or James’s technique of using standard deviations. The media predictions don’t come with points though so to make them ready for assessment, we have calculated the average points per position over the last 18 seasons (20 club era) of the Premier League and applied these to the media predictions. It is a little unfair but this method of assessment is better than just counting how many positions were called exactly right as it provides a more complete picture. All suggestions for alternative assessment methods – using points or positions – are very welcome and can be sent to me via the comments or twitter.

We have tried to use predictions from individuals and individual models where possible but unfortunately The Guardian aggregated its predictions which makes theirs more a “Wisdom of the Guardian” forecast. It would have been better to source the individual forecasts in order to identify who amongst the Guardian football writers is good at forecasting but in this case it wasn’t possible. The Pinnacle Sports prediction represents the betting side and is from one hour before kick-off of last Saturday’s first match. All of these forecasts were published on blogs, websites or on Twitter.

Of course, it is still possible to have luck with a single season prediction so we plan to make this a series which returns to as many of the same people and models as possible next year. This will eventually cancel single season luck out and identify the truly good predictors.

Just looking at the averages for the objective and subjective predictions – “wisdom of the crowd” from the two camps – there are some interesting similarities and differences. As mentioned at the top, Manchester City will be 2013/2014 Premier League champions according to the statistical models but Chelsea will be according to our group of experts. Both groups are in agreement that the same four teams who filled the top four places last season will also do so this term.

I had thought that there would be some major differences between the two “Wisdom” groups when it came to mid-table because it is so competitive and random variation will undoubtedly play a part in the distribution of the final rankings amongst the clubs. This makes the agreement on Swansea City to finish eighth even more remarkable. The two groups are never more than two places away from each other in predicting the finishing position of each team.

Finally relegation. During the Premier League era there has been just one occasion on which all three promoted teams have been relegated but this is what the average of both groups predict. Perhaps remarkably, Hull City AND Crystal Palace are predicted to go down in 25 of the 26 individual predictions. Even that would be above the Premier League average.

The only time all three promoted clubs did go down was 1997/1998. During the summer, a friend provided one of the most memorable – and accurate – predictions that I have ever come across. He suggested that the three relegated clubs would be Barnsley, Bolton and “whoever signs Paul Warhurst”. Warhust subsequently joined Crystal Palace who duly joined the other two in being relegated.