World Cup 2018: Does form matter for teams competing in Russia?
9 June 2018
England fans know the drill all too well - the national team heads into a major international football tournament having qualified with a near-perfect record. Hopes are high, but then... well, you know what happens - lacklustre performances or penalty shoot-out heartbreak, followed by an early flight home. So what really determines the success or failure of a team going into a major international football tournament like the World Cup? Is it a side's quality (class) or its recent performances (form)? Reality Check has teamed up with the BBC's statistics department to try to answer one of the biggest debates in football - how much does form matter? To do this, we built a computer program that predicts football results by analysing ratings data. The program crunches data from Elo ratings, a points-based ranking system for international football teams, which takes into account the importance of a game and the strength of the opposition. Unlike Fifa's equivalent, Elo ratings also factor in the size of a win and whether a team is at home. The computer analysis compares the respective class and form of two teams to calculate how many goals each side is likely to score when they meet. It would then predict which team would win, or if the match would be a draw. You might also like: In the program, class was defined as a team's Elo rating before the match. This rating is cumulative and has been built up over decades of international play. A nation's form was the points change in a team's Elo rating over the previous two years - this is a common interval between international football tournaments. Because of the infrequency of international matches, a large timeframe was needed to build up a picture of form. The program also contains data from thousands of international tournament and qualifying matches since 1960, including the number of goals scored by each team, as well as their class and form (as defined by Elo ratings). It used this data to replay hundreds of real-life World Cup finals matches between 1962 and 2014. These predictions were then compared to the actual results. When the program based its predictions on class, it correctly predicted 76% of wins and losses. In comparison, the accuracy dropped to 52% when its predictions were based on form. Although the program didn't get it right 100% of the time for either type of prediction, these results still suggest that a team's class is a much better predictor of its World Cup performance than form. Whilst crunching the numbers, it also emerged that the program had another use: running "what if" scenarios. Many fans will have wondered what might have happened if a penalty shootout had gone the other way, and how that would have affected the wider tournament outcome. So we selected a few stand-out matches from early knockout rounds of recent World Cups, and asked the program to predict what could have happened afterwards... Argentina 2-2 England aet (Argentina win 4-3 on penalties) (last 16, 1998) If they had made it past Argentina, England would have faced the Netherlands in the quarter-finals, but the program predicted they would have lost to Dennis Bergkamp and co. Spain 1-1 Republic of Ireland aet (Spain win 3-2 on penalties) (last 16, 2002) If Mick McCarthy's Irish team had emerged victorious against Spain, the program predicted they would have beaten tournament co-hosts South Korea in the quarter-finals, before losing to Germany in the semis. England 0-0 Portugal aet (Portugal win 3-1 on penalties) (quarter-finals, 2006) If England's golden generation had seen off Portugal, the program predicted they could have made it all the way to the final - having dispatched France in the semis - to face Italy, before falling agonisingly short. Uruguay 1-1 Ghana aet (Uruguay win 4-2 on penalties) (quarter-finals, 2010) A win over a Luis Suarez-inspired Uruguay would have led Ghana to a semi-final defeat by the Netherlands, the program predicted. Netherlands 0-0 Costa Rica aet (Netherlands win 4-3 on penalties) (quarter-finals, 2014) Despite topping a tough group in the opening round, the program predicted that Costa Rica's fairytale journey would have been ended by Argentina in the semi-finals.
The program crunches data from Elo ratings, a points-based ranking system for international football teams, which takes into account the importance of a game and the strength of the opposition. Unlike Fifa's equivalent, Elo ratings also factor in the size of a win and whether a team is at home. The computer analysis compares the respective class and form of two teams to calculate how many goals each side is likely to score when they meet. It would then predict which team would win, or if the match would be a draw. You might also like: In the program, class was defined as a team's Elo rating before the match. This rating is cumulative and has been built up over decades of international play. A nation's form was the points change in a team's Elo rating over the previous two years - this is a common interval between international football tournaments. Because of the infrequency of international matches, a large timeframe was needed to build up a picture of form. The program also contains data from thousands of international tournament and qualifying matches since 1960, including the number of goals scored by each team, as well as their class and form (as defined by Elo ratings). It used this data to replay hundreds of real-life World Cup finals matches between 1962 and 2014. These predictions were then compared to the actual results.
255_0
How does the program work?
When the program based its predictions on class, it correctly predicted 76% of wins and losses. In comparison, the accuracy dropped to 52% when its predictions were based on form. Although the program didn't get it right 100% of the time for either type of prediction, these results still suggest that a team's class is a much better predictor of its World Cup performance than form.
255_1
What was the final score?