## 3 Materials and Methods
### Optimization of a Set of Independent Variables by Feature Selection Techniques
#### 3.5.1 RReliefF Algorithm  
The RReliefF algorithm belongs to the Filter technique for optimizing a set of variables. Relief ([PERSON] and [PERSON], 1992 [27; 28]) and its extension ReliefF ([PERSON], 1994 [29]) are \"context-aware\" algorithms that assess the quality of model variables for solving classification problems where there is strong interdependence among predictors [30]. Unlike the previous two, the Regression ReliefF (RReliefF) algorithm is not limited to category dependent variables only. It is used for regression tasks in which it \"penalizes\" predictors that give different prediction values for adjacent observations with the same values of the dependent variable. In this case, the observation represents one row in the input data matrix, i.e., one input vector. On the other hand, this algorithm \"rewards\" predictors that give different prediction values for neighboring observations with different output values [31]. RReliefF uses intermediate weights to calculate the final predictor weight coefficients; if the two nearest neighbors are considered, the following notation is used:  
* \(W_{j}\) is the weighting coefficient of the predictor \(F_{j}\);
* \(W_{dy}\) is the weighting coefficient for different values of the dependent variable \(y\);
* \(W_{dj}\) is the weighting coefficient for different predictor values \(F_{j}\);
* \(W_{dy\wedge\text{dj}}\) is the weighting coefficient for different values of \(y\) and different values of the predictor \(F_{j}\)[31].  
The weighting coefficients, \(W_{dy}\), \(W_{dj}\), \(W_{dy\wedge j,dj}\) and \(W_{j}\), are equal to zero at the beginning of the algorithm. The algorithm iteratively selects a random observation \(x_{r}\) and a k-nearest observation for \(x_{r}\). For each nearest neighbor \(x_{q}\), intermediate weights are updated as follows [31]:  
\[W_{dy}^{i}=W_{dy}^{i-1}+\Delta_{g}(x_{r},x_{q})\cdot d_{rq} \tag{2}\]  
\[W_{dj}^{i}=W_{dj}^{i-1}+\Delta_{j}(x_{r},x_{q})\cdot d_{rq} \tag{3}\]  
\[W_  
defined by the expression \(\Delta_{j}(x_{r},x_{q})\)[31]. When \(x_{r}\) represents the value of the \(j\)-th predictor for the observation \(x_{r}\), and \(x_{qj}\) is the value of the \(j\)-th predictor for the observation \(x_{q}\), then  
\[\Delta_{j}(x_{r},x_{q})=\frac{\left|x_{rj}-x_{qj}\right|}{\max(F_{j})-\min(F_{j })} \tag{6}\]  
After updating all intermediate weights, RReliefF calculates the weighting coefficients of the predictor \(W_{j}\) according to Equation [31]:  
\[W_{j}=\frac{W_{dy\wedge dj}}{W_{dy}}-\frac{W_{dj}-W_{dy\wedge dj}}{m-W_{dy}} \tag{7}\]  
In order to select the optimal set of predictors in the model in addition to the values of weighting coefficients, it is necessary to define the Relevance Threshold (\(RT\)) as the limit of the significance of independent variables [32]. According to the criterion set in this way, all predictors with \(W_{j}\geq RT\) participate in the creation of the model. Generally, that threshold has a value in the interval between 0 and 1, and more precisely, its value is calculated according to the following expression based on [PERSON]'s inequality [32]:  
\[0<RT<\frac{1}{\sqrt{\alpha\cdot t}} \tag{8}\]  
where \(\alpha\) is the probability of accepting an insignificant feature as significant (type I errors or first type error) and \(t\) is the number of training observations for updating \(W_{j}\), out of a total of \(n\) observations. Within the stated limits, the selection of \(RT\) is arbitrary, where there is a probability that not all variables with \(W_{j}\) above the defined threshold will necessarily be significant because some unimportant variables are expected to have a positive weighting coefficient by chance [32].