## 2 Materials and Methods
The input to the meta-model is the output of base regression models from the first layer and some selected input features. This architecture is directly generalized to a multi-layered architecture, in which there are multiple layers of individual regression models between the input training data and the output meta-model. Figure 1: Stacked regression. The first layer comprises base regression models. The second layer consists of the meta-model. The input to the meta-model is the output of base regression models from the first layer (denoted by the blue arrows) and some selected input features (denoted by the red arrow). Algorithms 1 and 2 formally show the stacked regression training algorithm with the simultaneous estimation of hyperparameters of individual base and meta-models. The output of the procedure is the average performance of stacked regression \(H_{R}\) over K _Testing sets_, (Step 2 in Algorithm 1), as well as the optimal hyperparameters \(\theta_{i}^{*}\) (Step 1.2, Algorithm 1). Algorithm 2 shows separately the algorithm for training stacked regression \(H_{R}\) with additional L-fold cross-validation [15; 16]. The entire Algorithm 2 actually corresponds to Step 1.1.a.3 of the algorithm from Algorithm 1. The goal of additional cross-validation is to utilize the entire training dataset efficiently and to get a more reliable estimate of the meta-model performance. The L-fold cross-validation process involves the following steps (see Algorithm 1 for a more detailed description):  
_Step 1_: Divide the original _Training validation set_\(V\) into L equal-size subsets. _Step 2_: For each fold \(l\) (where \(l\) ranges from 1 to L):  
Use L-1 folds for training the base regressions and make predictions on the remaining fold \(l\) (so-called _out-of-fold predictions_). Store these predictions as meta-features for fold \(l\). These are the inputs for the meta-model in this specific fold. _Step 3_: Once the predictions (meta-features) for all L folds have been obtained, combine them to create a new dataset, the _meta-training dataset_. _Step 4_: Train the meta-model on the _meta-training dataset_, where the target variable is still the actual target values from the original training dataset. A _meta-training dataset_ consisting of the _out-of-fold predictions_ formed in Step 2 prevents the leakage of target information into it, thus reducing the possibility of overfitting. ```
0: Training data \(D=\{(X_{i},y_{i}),i=1,2,\ldots,N\}\), T--number of base models
0: A stacked regression \(H_{R}\) Randomly partition \(D\) into \(K\) equal size subsets \(D=\{D_{1},D_{2},\ldots,D_{K}\}\)
1:for\(k=1,2



Algorithm 1 are omitted, while _Training validation_ set \(V\) in Algorithm 2 is replaced by the entire _Training set_ defined in Algorithm 1. The stacked regression model with two layers can be directly extended to an arbitrary number of layers, in which each previous layer generates new features for the next layer of the models. In order to obtain an out-of-fold training set for the final meta-model, multi-layer models require an additional nested cross-validation loop for each additional layer. It is clear that this type of complexity of architecture and training can be practically feasible only in the case of sufficient computing resources and large training sets. Since our training data has only 79 instances, we decided on the simplest two-layer architecture from Figure 1, and the choice of default hyperparameters of all used models, both in the first and in the second layer of the architecture.