## 4 Experimental Setup  
The data set used to evaluate the rating system and to analyze the values for the estimated parameters, including player ratings, contains 52,083 matches in total, dating from August 2008 to June 2017. This includes 16,339 matches from the four divisions of the English league system and 566 matches from the English League Cup. The two top divisions of Spain, France, Germany, and Italy, covering 24,206 matches in total, are also included. Moreover, top divisions from the Netherlands and Portugal are included as well, with 4574 matches. A number of matches (5142) from the UEFA Champions League and Europa League are present, as well as 1256 matches from the Euro 2012, Euro 2016, World Cup 2010, and World Cup 2014 competitions. The latter also encompasses qualification matches for the Euros and the European qualification for both World Cups. When tuning the hyperparameters of the offensive-defensive plus-minus ratings, a subset of the aforementioned data set is used, consisting of 38,126 matches. This subset is formed by discarding matches from Portugal and the Netherlands, as well as matches from the two lowest levels of the English league system and the English League Cup. The two data sets contain, respectively, 30,673 and 26,619 unique players. Estimating the rating model on the full data set is done by solving an unconstrained quadratic program with 61,466 variables and 711,453 squared expressions in the objective function (1)-(8). To evaluate the new ratings, two measures are used. The first aims to express the validity of the ratings, that is, how well they represent the true abilities of the players. To this end, the ratings are used in a prediction model that outputs probabilities for the outcomes of matches. The process follows a sliding window approach, where, first, all matches up to a given point in time are used to calculate player ratings. For the next match, a covariate \(x_{PM}\) is calculated based on the average total rating of the players in the starting line-up of the home team, minus the average total rating of the players in \begin{table}
\begin{tabular}{l l r r r r} \hline \hline  & & \multicolumn{2}{c}{**H.LHS**} & \multicolumn{2}{c}{_A.LHS_} \\ \hline  & **League** & **O.COMP** & **D.COMP** & **O.COMP** & **D.COMP** \\ \hline Spain & Primera Division & 8.617 & \(-\)7.700 & 7.944 & \(-\)8.167 \\ Spain & Segunda Division & 0.550 & \(-\)3.300 & 3.056 & \(-\)0.833 \\ England & Premier League & 0.917 & & & \(-\)1.167 \\ Germany & Bundesliga & 0.550 & & & \(-\)0.500 \\ Netherlands & Eredivisie & 0.367 & & & \(-\)0.333 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Coefficients for \(\beta^{O,COMP}\) and \(\beta^{D,COMP}\) in a selected segment, scaled by \(11/|P^{O}|\) and \(11/|P^{D}|\), but not by segment weights \(w\) or segment duration \(d/90\). the starting line-up of the away team. The value for the covariate together with the observed match outcome (home win, draw, away win) forms an observation. When a sufficient number of observations have been collected, an ordered logit regression model [30] is built with the aim of predicting outcomes of future matches when only the covariate value is known. This prediction is evaluated by calculating its quadratic loss [31], also known as thr Brier score, and the average quadratic loss over a large number of matches is used to indicate validity: better ratings should make it easier to predict future match outcomes, leading to a lower quadratic loss. The quadratic loss is a proper score and, like alternatives such as the rank probability score [32], is a commonly used measure to evaluate match outcome predictions in soccer [33]. Let the outcomes of a match be numbered 1, 2, and 3; let \(p_{1}\), \(p_{2}\), and \(p_{3}\) be the probabilities for the three outcomes; use \(d_{j}\ =\ 1\) to indicate that match outcome \(j\in\{1,2,3\}\) was observed, with \(d_{j}=0\) otherwise.