## 4 Modeling
### Machine-Learning Techniques and Best Metrics
#### 4.2.2 Regression  
The metric used for regression problems is the Root Relative Squared Error (RRSE). Let \(y_{i}\) be each of the values to be predicted in the test set, let \(p_{i}\) be the predictions that a regression method outputs for those values, and let \(\overline{y}\) be the mean value of \(y_{i}\) in the test set. RRSE is then defined as follows:  
\[\sqrt{\frac{\sum(y_{i}-p_{i})^{2})}{\sum(y_{i}-\overline{y})^{2}}} \tag{7}\]  
Therefore, the values of this metric are always greater than, or equal to, zero. In the ideal case of a regressor predicting without error, the RRSE would be zero. In a naive regressor that would always predict the mean \(\overline{y}\), both the numerator and the denominator of Equation (7) would be equal, then the resulting RRSE value will be 1 (or 100% if expressed as a percentage). The reason for having chosen RRSE as a metric is precisely because it provides a figure that takes this naive regressor as a baseline. A regressor could have an RRSE greater than one, although it would obviously be of no practical interest. The regression methods used in the experiment were:  
1. Function-based methods such as Linear Regression, Support Vector Machines (SVM), and Neural Networks: 1. Linear Regression creates a function that is a linear combination of the input variables, which minimizes the quadratic error of the training set. The Akaike criterion, as the WEKA implementation default, is used to select the attributes of the model. 2. The simplest version of SVM for regression is the linear SVM [47]. As in the case of linear regression, we also have a model that is a linear combination of the input variables (i.e., a hyperplane), but this time the optimization process ignores the instances that are less than \(\epsilon\) away from that hyperplane (i.e., they are within the \"margin\"). The distance, \(\epsilon\), is a parameter to specify that is denoted by C in most implementations. The implementation used in the experiment was LibLinear [40], for which the C parameter was optimized by cross validation, to minimize RRSE, using WEKA's Multisearch package. The SVM for regression, like its version for classification, can use a kernel, and generate the hyperplane in a different space, leading to a more complex geometry than in the original space, which is more appropriate if the behavior of the variable to be predicted is not linear. Therefore, in the experiment we have also used an SVM regressor with a Radial Basis Function kernel (SVM-RBF). In the case of this kernel, besides optimizing C, it is necessary to optimize the bandwidth parameter. As in other methods, Multisearch was also used, using internal cross validation and aiming at minimizing the RRSE. The SVM-RBF implementation used in the experiment is the regression version of LibSVM [39]. 3. The neural network method used is the same as that described for classification (i.e., a Multilayer Perceptron [37]). As in the classification, only an intermediate layer was used. The number of neurons in this layer is also given by the heuristics (number of independent variables + 1)/2. Likewise, the learning rate and momentum parameters were optimized with Multisearch, again through internal cross validation, to minimize the RRSE. kNN [48] works similarly to kNN for classification. The difference is that, once the k instances closest to the test instance are found, the class mean for those k instances is predicted. As for classification, the value of k was also optimized in the experiments for each cross-validation partition with the best value obtained for k ranging between 1 and 10. 3. Regression trees. Two types of decision trees for regression were used: 1. [label=()] 2. M5P [49] which is a type of tree in which a linear regression is created with the instances in each leaf, so the prediction that is returned is the one of the linear regression corresponding to the leaf in which the test instance falls. 3. REP-Tree [33] is WEKA's \"native\" decision tree for both classification and regression.