# Stochastic Extensions of the Elo Rating System
## 4 Computational Study
### Basketball Results
#### 4.2.1 Static Elo  
We implemented the algorithm described above for \(m=20\) and \(F_{X}\), corresponding to a normal variable \(X\sim N(0,200)\), in a training set of seasons 2000-2001 to 2009-2010, and in four different ways to test the effectiveness of the Elo model and the significance of its parameters:  
* Without Elo, simply assuming \(E[p_{k,a_{k}}]=\overline{p_{home}}:=|M|^{-1}\sum_{k=1}^{|M|}p_{k,a_{k}}:F_{X} (L)\). * Fixing \(K=0\) and minimizing \(MSE\) as a function of \(L\) (no change in strength). * Fixing \(L=0\) and minimizing \(MSE\) as a function of \(K\) (no home advantage). * Minimizing \(MSE\) as a function of \(K\) and \(L\) (standard Elo system). To avoid overfitting in \(K\) and \(L\), we also implemented the algorithm for the optimal values \(K^{*}\) and \(L^{*}\) in a validation set consisting of seasons 2010-2011 to 2023-2024. The mean squared errors in the training and testing sets are summarized in Table 1:  
We can see straightaway that the Elo explains a significant part of the variance (around \(11\%\)), and both parameters are useful, since setting them to zero increased the mean squared error. The reduction was similar in the training and testing sets, so cross-validation shows the robustness of the algorithm in Section 4.1. Notice also that the \(K\)-factor and home field advantage are similar to those of chess, where \(|L^{*}|\approx 50\)[1] (Chapter 8.93). The sign of \(L^{*}\) only reflects that in basketball (and American sports in general) the visiting team is listed first. We note that setting \(K=0\) (which amounts to leaving the rating of a team unchanged until they are relegated) increased the variance, but it still had predictive power, as evidenced by the ROC curves shown in Figure 2.  
Finally, setting \(L=0\) barely changed the curve, because the map from \(F_{X}(z)\) to \(F_{X}(z+L)\) is a monotone increasing bijection, and the order of the expected result under the two models are the same for equal \(K^{*}\) (and indeed the \(K\) factors were very similar). After fitting our model and obtaining the sample, we could expect that for a subsample of matches with rating difference \(r_{A}-r_{B}\approx r\), the average result would be close to \(F_{X}(r+L)\), but this was not the case, as evidenced by Figure 3.  
We observe that the normal distribution function \(F_{X}\) overestimates the effect of the rating difference in the expected result of a match. We can obtain a better predictor if we multiply \(r_{A}-r_{B}\) by a \"dampening\" constant \(D\leq 1\), as in Figure 3 of [17]. When we minimized the \(MSE\) on \(K,L\), and \(D\) (only using \(D\) for the calculation of the mean squared error, since otherwise we would just be scaling \(X\)), we obtained optimal parameters \(K^{*}\approx 19.61\), \(L^{*}\approx-58.9\) and \(D^{*}\approx 0.786\), reducing the \(MSE\) to 0.21064 in the test set. \begin{table}
\begin{tabular}{c c c c} \hline \(K^{*}\) & \(L^{*}\) & \(MSE\) & \(MSE_{\text{test}}\) \\ \hline No Elo & \(-53.78\) & 0.23876 & 0.24394 \\ \hline
0 & \(-60.89\) & 0.26498 & 0.29590 \\ \hline
14.71 & 0 & 0.22333 & 0.22308 \\ \hline
16.19 & \(-60.74\) & 0.21179 & 0.21744 \\ \hline \end{tabular}
\end{table} 
Table 1: Mean squared error of different Elo models vs. sample variance. Figure 2: ROC curve for each of the Elo system implementations in the training set (a) and test set (b). This seemed a small improvement in \(MSE\), but we could test the significance of the reduction in variance by looking at the variables \(X:=(p_{A}-F_{X}(L+r_{A}-r_{B}))^{2}\) and \(Y:=(p_{A}-F_{X}(L+Dr_{A}-Dr_{B}))^{2}\) for a random game. In order to check if \(Z:=X-Y\) had a positive mean from our sample of games, we sampled the difference \(z_{k}=(p_{k}^{a}-F_{X}(L+r_{a_{k}}-r_{b}))^{2}-(p_{k}^{a}-F_{X}(L+Dr_{a_{k}}-Dr _{b_{k}}))^{2}\) and argued that by the central limit theorem, the sample mean of \(Z\) approximately followed a normal distribution with \(n\) times the sample variance of \(Z\).