## 2 Materials and Methods
### Computing SHAP Values in Repeated Nested Cross-Validation  
Let us consider a dataset composed of \(N\) samples with \(M\) features and a \(K\)-fold nCV procedure (see Figure 1). This strategy involves nesting two \(K\)-fold CV loops, where the inner loop is used to optimize, for example, model hyperparameters. The outer loop gives an unbiased estimate of the performance of the best model [25]. The procedure starts by splitting the dataset into \(K\) folds (outer CV): one fold is kept as a test set of the outer CV, while the other \(K-1\) folds (the training set of the outer CV) are, in turn, split into \(K\) inner folds, that is, \(K-1\) for training and the \(K\)-th for validation, to provide an unbiased evaluation of the model fit on the inner training set while tuning the model's hyperparameters. Once the best combination of hyperparameters that maximized the performance metrics in the validation set has been found, the model with that combination of hyperparameters is re-trained on the outer training set and tested on the test set kept out from the outer CV. The nested CV is repeated \(R\) times with different random seeds to make different data splitting of the \(K\) folds. This procedure can be used both for regression and classification tasks. The pseudo-code for the computation of representative SHAP values in the training and test sets of the outer CV is illustrated in Algorithm 1. For each repetition \(r\) of the outer CV loop, we compute SHAP values \(\phi_{nk}^{ir}\) of every sample \(n\) and feature \(i\) for the \(k\) round (split iteration) of the outer CV separately for the training and test using the SHAP Python module \(Explainer\)[38]. Then, for each sample \(n\), we compute a representative SHAP value for the training \((\phi_{train})_{n}^{ir}\) and test \((\phi_{test})_{n}^{ir}\) sets. For the training set, we compute \((\phi_{train})_{n}^{ir}\) as the average of the SHAP values overall for the \(K-1\) folds of the outer CV as follows:  
\[(\phi_{train})_{n}^{ir}=\frac{1}{K-1}\sum_{k=1}^{K-1}\Big{(}\phi_{nk}^{ir} \Big{)}. \tag{3}\]  
Since, in the \(r\) repeated CV, a sample \(n\) belongs to one fold used as a test only, namely fold \(k^{*}\), the representative SHAP value of the test set of that sample \(n\) is simply  
\[(\phi_{test})_{n}^{ir}=\phi_{nk^{*}}^{ir}. \tag{4}\]  
```
\(X\leftarrow\textit{Dataset}\quad\#\textit{Data table with N samples and M features}\left(\textit{N rows}\times\textit{M columns}\right)\) \(y\leftarrow\textit{Target}\) model\(\leftarrow\textit{Classifier}\) \(\textit{Explainer}\gets SHAP.Explainer()\quad\#\textit{Shap function which computes SHAP values}\) train_folds_shape_values \(\leftarrow\) 0 # Initialized matrix \((\textit{N rows}\times\textit{M columns})\) test_folds_shape_values \(\leftarrow\) 0 # Initialized matrix \((\textit{N rows}\times\textit{M columns})\) for\(r\) in \(1,\ldots,\textit{R}\)do \(fold\_splits\gets split(K)\) \(\textit{innerCV}(fold\_splits,model)\) \(\textit{outerCV}(innerCV,K).fit(X,y)\) for\(k\) in \(fold\_splits\)do \(X\_train,y\_train\gets fold\_splits.train(k)\) \(X\_test,y\_test\gets fold\_splits.test(k)\) \(\textit{best\_k\_model}\leftarrow\textit{outer\_CV}[k].best\_model\) \(\textit{best\_k\_model}.fit(X\_train,y\_train)\) \(\textit{train\_shap\_values}\leftarrow\textit{Explainer}(best\_k\_model(X\_train))\) \(\textit{test\_shap\_values}\leftarrow\textit{Explainer}(best\_k\_model(X\_test))\) \(\textit{train\_folds\_shap\_values}\leftarrow\textit{train\_folds\_shap\_values}+\frac{ train\_shap\_values}{\textit{test\_folds\_shap\_values}\leftarrow\textit{test\_folds\_shap\_values}+\textit{test\_shap\_values}\) endfor endfor average_train_folds_shape_values \(=\textit{train\_folds\_shap\_values}/\textit{R}\quad\#\,\phi_{train}\) average_test_folds_shape_values \(=\textit{test\_folds\_shap\_values}/\textit{R}\quad\#\,\phi_{test}\)
```  
**Algorithm 1**\(N\): number of samples; \(M\): number of features; \(K\): number of folds; \(R\): number of repetitions.