How Did the WGF Rankings Do At Predicting the World Cup?
Here at WGF, we’re about accountability. We have rankings, predictions, odds, and percentages all over this site. But if those numbers are no good, we need to improve. The only way to do that is measure the performance of the rankings against some sort of benchmark. And what better benchmark than other rankings.
Back in March, we participated in The Roon Ba’s World League of Rankings to predict that day’s 64 matches. By the way, Mark has a fantastic site at The Roon Ba, which you should definitely check out if you’re interested in any type of international team sports. He has them all. Anyway, although we did well, it was evident that Strength of Schedule did not carry enough weight. We adjusted the weighting slightly, and have definitely seen improved results.
The WGF Rankings Explained
Prior to the start of the World Cup, we posted our Win Probabilities for Every Match. This was a direct output of our rankings and the formula we use to convert those rankings to percentages. As a result of those percentages, we multiplied win probability by 3, and draw probability by 1, to get each team’s expected number of points from a match. We then added a team’s 3 matches to get an expected point total.
Group A
| Team | Predicted | Actual | Variance |
| Brazil | 7.982 | 7 | 0.982 |
| Croatia | 3.887 | 3 | 0.887 |
| Mexico | 2.973 | 7 | 4.027 |
| Cameroon | 1.947 | 0 | 1.947 |
Group B
| Team | Predicted | Actual | Variance |
| Spain | 5.937 | 3 | 2.937 |
| Chile | 5.887 | 6 | 0.113 |
| Netherlands | 4.328 | 9 | 4.672 |
| Australia | 0.636 | 0 | 0.636 |
Group C
| Team | Predicted | Actual | Variance |
| Colombia | 6.25 | 9 | 2.75 |
| Greece | 3.579 | 4 | 0.421 |
| Ivory Coast | 3.434 | 3 | 0.434 |
| Japan | 3.105 | 1 | 2.105 |
Group D
| Team | Predicted | Actual | Variance |
| England | 5.531 | 1 | 4.531 |
| Uruguay | 5.342 | 6 | 0.658 |
| Italy | 3.528 | 3 | 0.528 |
| Costa Rica | 2.026 | 7 | 4.974 |
Group E
| Team | Predicted | Actual | Variance |
| France | 6.233 | 7 | 0.767 |
| Switzerland | 4.057 | 6 | 1.943 |
| Ecuador | 3.945 | 4 | 0.055 |
| Honduras | 2.185 | 0 | 2.185 |
Group F
| Team | Predicted | Actual | Variance |
| Argentina | 6.655 | 9 | 2.345 |
| Bosnia-Herzegovina | 5.43 | 3 | 2.43 |
| Nigeria | 3.828 | 4 | 0.172 |
| Iran | 0.857 | 1 | 0.143 |
Group G
| Team | Predicted | Actual | Variance |
| Germany | 6.168 | 7 | 0.832 |
| Portugal | 4.578 | 4 | 0.578 |
| USA | 3.572 | 4 | 0.428 |
| Ghana | 2.119 | 1 | 1.119 |
Group H
| Team | Predicted | Actual | Variance |
| Belgium | 6.206 | 9 | 2.794 |
| Russia | 5.871 | 2 | 3.871 |
| Algeria | 2.794 | 4 | 1.206 |
| Korea Republic | 1.751 | 1 | 0.751 |
By Confederation
| Confederation | Predicted | Actual | Variance |
| AFC | 1.587 | 0.750 | -0.837 |
| CAF | 2.824 | 2.400 | -0.424 |
| CONCACAF | 2.689 | 4.500 | 1.811 |
| CONMEBOL | 6.010 | 6.833 | 0.823 |
| UEFA | 5.026 | 4.692 | -0.333 |
Great job WGF. You only got 16 of 32 teams within 1 point of their actual number and were off by over 4 points on 4 separate teams. That’s no good.
…Actually, it’s not too bad. When you take the average of the variances (all absolute differences), we end up off by an average of 1.694. Alex Olshansky over at Tempo-Free Soccer took a before and after look at how some other rankings did, including Elo, SPI, and Oddsportal. WGF was not included in the project. It turns out that the best of the bunch (Elo and SPI) were off by more than 1.9 points per team. That’s a pretty big difference from our 1.694. How did that happen?
Remember when we said at the beginning that we really ramped up our Strength of Schedule metric after the March results? Turns out that had a big impact. SOS influence hurt teams in CAF and AFC, while boosting teams in CONMEBOL.
The lowest any of the systems had AFC averaging is Oddsportal’s 2.66. We had AFC averaging 1.587. They truly averaged 0.75 points. We also had CONMEBOL averaging 6.01 points. SPI had the next highest at 5.81, and the number turned out to be 6.833. Only Elo had an estimate for CAF lower than ours, and their true number turned out to be even lower than that. Similar to everyone else, we underestimated CONCACAF, but we were still far better than the betting public.
Specific Team Variances
Without any bias whatsoever, here are the teams where we had a variance of over 1 point from the composite projections:
| Team | WGF | Composite | Actual |
| Chile | 5.89 | 4.19 | 6 |
| Australia | 0.64 | 1.77 | 0 |
| Italy | 3.53 | 4.76 | 3 |
| Bosnia-Herzegovina | 5.43 | 4.26 | 3 |
| Iran | 0.86 | 2.41 | 1 |
| Korea Republic | 1.75 | 3.47 | 1 |
5 out of the 6 teams where our rankings varied from the composite, our projection was more accurate. Only Bosnia did us dirty. Overall this is a pretty big win for us, and it’s pretty good to do so well against many respected systems.
We’ll keep predicting and keep working to improve our rankings, and always appreciate any feedback. Thanks for reading.
