Predicting the Premier League

At the beginning of last season, I collected 26 pre-season predictions for the Premier League with one half coming from a variety of statistical models and one half from the experts in the media. As expected, the statistical models came out on top as a group and the best individual prediction also came from that group, with blogger James Grayson‘s  model based on TSR (Total Shot Ratio) and PDO (a luck proxy). The champion of the subjective predictions was NBC’s Joe Prince-Wright who beat all but two of the statistical models and finished far ahead of the rest of the experts. For this season, this process has been expanded to ask a number of new questions about predicting the Premier League and the number of forecasts collected has more than doubled to 56.

Let’s begin by looking at the results of last season which were kindly put together by James Grayson. Below is the graphic showing last season’s results in terms of the standard deviations of the difference between the predictions and the actual league table. James explains why this measure was used here:

“Standard deviation was used to evaluate the predictions as it gives an excellent indication as to the uncertainty and spread in a given set of predictions. It can also be applied well to a 20 team league. For a quick worked example, a set of predictions with a standard deviation of 10 points would have predicted the points totals of roughly 13 teams correctly to within 10 points, and the points totals of roughly 19 teams correctly to within 20 points. The smaller the standard deviation, the better the set of predictions was.”

 

 

This season
For the 2014/2015 season, eight of the statistical modellers are back as well as a handful of the media experts. However, a group of fans has been added and the total sample of predictors has been hugely increased. This season’s competition includes 27 predictions from statistical models, 18 from media experts and nine from fans, making a total of 56 different forecasts altogether. The fan group will be expanded next season, if this exercise continues, in order to include fans from each Premier League club. Both of the individual group winners from last season are back which means that their consistency of prediction, as well as that of some others, can be looked at. Were they lucky or are they really better than the rest? In addition, a number of predictions have been made after the transfer window closed? Will those predictions be better than those made pre-season? Hopefully James Grayson will again tweet the state of play after each match round as he did last season and I will revisit the statistical models in more detail here on Scoreboard Journalism throughout the season. The full Excel sheet is also available at the end of this post for anyone to download and play with. I welcome any other assessments of these predictions if people want to publish them but please refer back to this blog. For now though, here is the 2014/2015 Premier League prediction rank order based on an average of all entrants with some comment on any major differences between groups.

1. Chelsea

The Premier League title race is a two-way battle between Chelsea and Manchester City according to these predictions. Averaging the ranks of everyone out, it is Chelsea who come out on top but there is a difference between the statistical modellers and the rest. In general, stat models prefer Manchester City whereas media and fans were more likely to predict Chelsea for the title.

2. Manchester City

Second place for last season’s champions according to the total sample of predictions and even less variance in where they are forecast to finish than Chelsea. Only one statistical model and four media experts think that Manchester City will finish outside the top two this season.

3. Arsenal

The Gunners are favoured more than Liverpool by both media experts and fans but the statistical models as a group predict Liverpool to finish in third position.

4. Liverpool

Statistical models look much more favourably on last season’s Premier League runners-up than the other groups and expect the Reds to finish third. Media experts go out on a limb in their forecast that Liverpool will fail to qualify for the Champions League.

5. Manchester United

Only four of the 56 predictions have Manchester United finishing in seventh place or lower this season. One forecast, a post transfer window statistical model, based on squad age and transfer value, makes United 2014/2015 Premier League champions. Media experts as a whole are the only group to put Manchester United in the top four.

6. Tottenham Hotspur

The smallest variance outside Chelsea and Manchester City when it comes to final rank amongst our predictions, Tottenham are predicted to finish below the top-7 by only four entries and above fourth by only one. Again, there is a difference between the groups as the media experts rate Spurs seventh whereas the other two groups put them sixth.

7. Everton

Again, relative homogeneity in the assessment of Everton across all of the forecasters with only two placing the Toffeemen above the top five and seven below the top seven. The media experts are more impressed with Everton than either the stat modellers or fans with a forecast of sixth position as average rank.

8. Newcastle United

The statistical modellers and fans disagree with the media experts here with both placing Newcastle United a place lower in ninth. However, each of those groups chooses a different team to finish eighth.

9. Southampton

After all the upheaval in the summer at St Mary’s it is not surprising to find that the Saints have one of the highest variances in predictions across the 56 forecasts. However, it is not the highest. Southampton occupy almost every position outside the top three with entries predicting them to finish as high as fourth and as low as 19^th. Statistical models place them eighth on average but the other two groups are not as optimistic – the media experts’ forecast is a 10^th place finish and fans have put them as low as 13^th.

10. Stoke City

Stoke City are the fans’ favourite to finish eighth whereas the media experts put them ninth and statistical models 11^th.

11. Swansea City

Solid mid-table finish for the Swans with some small disagreement between the three groups. Statistical models are the most optimistic, placing them in the top half in 10^th with media experts forecasting 11^th and fans 13^th.

12. Sunderland

It is the fans who push Sunderland up this high with an average forecast of 10^th position. Both the statistical modellers and media experts put them lower than this average in 13^th.

13. West Ham United

Another team with a big variance in forecasts of final position from eighth position to bottom, West Ham United are favoured most by the fans whose average prediction for them is 11^th. Media experts put them 14^th and statistical modellers 15^th.

14. Hull City

Statistical modellers are the only one of the three groups to foresee a relegation battle for Hull City this season with an average finishing position of 17^th. This is a big difference from the media experts who place Steve Bruce’s team 12^th on average. Fan forecasts average 15^th position.

15. Aston Villa

Here, statistical modellers are slightly out of synch with the other two groups, putting Villa 14^th whereas fans and media experts seem more problems for the Villans placing them two places lower in 16^th.

16. West Bromwich Albion

The Baggies are the team that is most difficult to predict given the 56 forecasts submitted and this is reflected in the average rank for each of the three groups. Both the media experts and fans foresee a relegation struggle which ultimately fails to keep West Brom up, placing them 19^th and joint 18^th respectively. In the case of the fans, West Brom are rated at a similar level as promoted Burnley. In contrast, the statistical modellers predict West Brom to finish as high as 12^th. Individual predictions put The Baggies anywhere between 9^th and bottom of this season’s Premier League.

17. Crystal Palace

Media experts like the chances of Crystal Palace to survive again but the statistical modellers and fans both put them in the bottom three this time around.

18. Queens Park Rangers

Fans must like Harry Redknapp as they are far more optimistic about QPR’s chances forecasting a 12^th place finish. Statistical modellers put them 19^th and media experts 17^th.

19. Leicester City

Leicester City are the most likely of the promoted trio to stay up say the statistical modellers whose average forecast for them is 15^th. Fans don’t rate them at all though and place them bottom on average. Burnley and West Brom are the only teams worse than Leicester according to the media.

20. Burnley

All three groups predict that Burnley will be relegated and two put them in last place.

 

Questions
So, we should be able to investigate a number of questions this season including:

• Is there a consistency in prediction for those who also entered last season or are their performances partially due to luck?
• Will the statistical models again turn out to be the best?
• Which types of statistical model work the best?
• How do forecasts made after the transfer window had shut perform? Are they better or are they too influenced by results of the three matches played?

 

There are probably many many more things that we can investigate. In the meantime, please download the full dataset here (EPL20142015FINAL) and work with it yourself. When points were not submitted, I used the average per position from the 20-team Premier League seasons. If you’d like to pass anything on from your work with the dataset, please get in touch with me via Twitter at @simongleave.

Finally, I’d like to thank everyone for taking part, including those who didn’t necessarily know that they were.