## 3 Linear and Non-Linear Predictive Analytics Comparison Framework: L&NL-PAC
### Model Comparison Methodology  
A double cross-validation scheme was employed in this work to compare the different methods. The root mean squared error, RMSE, was the metric used to assess and compare the predictive performance. Furthermore, multiple runs of Monte Carlo double cross-validation were conducted to characterize the variability of the predicted RMSE for each regression method, resulting in a more robust analysis of their relative performances. In each run of Monte Carlo double cross-validation, the input spectral dataset was split into training and external validation sets, using a stratified scheme based on the percentiles. To perform this step, the reference response measurements (analytical determinations) were grouped into five intervals according to the percentiles (20 th, 40 th, 60 th, and 80 th), and 80% of the samples in each group of percentile intervals were used for model training, while the remaining 20% were reserved for the external validation set. During model training, another stratified k-fold cross-validation based on the response percentiles was used to select suitable hyperparameters (Table 2) for each regression method. This was done using seven-fold cross-validation wherein the data were partitioned into seven folds, six used for calibration and one for validation, with the procedure being repeated seven times using a different validation fold each time. Then, each final model was built using the seven folds and the respective best hyperparameter(s) and was applied to predict the external validation set (or independent test set), based on which the corresponding RMSEs were obtained and stored. This process was repeated 30 times and the RMSEs for each run of Monte Carlo double cross-validation were saved for analysis. It is important to note that, in each run, all regression methods made use of the same training dataset for model building, and their performance was assessed in the same validation set (i.e., their RMSEs were correlated). The distribution of RMSEs over all runs of the Monte Carlo double cross-validation characterized the performance of each regression method, and methods with lower RMSE values were preferred. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Method** & **Hyperparameter** & **Range Values** \\ \hline \multirow{3}{*}{ \begin{tabular}{c} FSR \\ RR \\ LASSO \\ EN \\ SVR Linear \\ PCR \\ PCR \\ PLS \\ iPLS \\ RF \\ BoRT \\ \end{tabular}  } & \(p_{\text{\emph{center}}}\) & 0.05 \\  & 0.01 \\  & 0.002; 0.02; 2; 20 \\  & 0.001; 0.01; 0.1; 10 \\  & 0.001; 0.01; 1 \\  & 0.002; 0.02; 2; 20 \\  & 1:min(20, n, \(p\))] \\  & \(\varepsilon\) & 0.005; 0.01; 0.05; 0.1 \\  & \(a_{\text{\emph{PCR}}}\) & [:minThe distribution of the RMSE from the double cross-validation constitutes an informative source to compare the performance of different regression methods. However, visually comparing the RMSE distributions can be cumbersome due to the high number of methods included in this study (16 in total). Thus, besides RMSE, an additional key performance indicator (KPI) was devised to facilitate the ranking of the methods. As an additional advantage, this KPI was based on a rigorous statistical approach of hypothesis testing, allowing for detecting whether differences in performance were statistically significant or not. The KPI included in L&NL-PAC is similar to the one utilized in the PAC framework, and more details can be obtained in the original paper [35]. To compute this KPI, every pair of regression methods was considered, and their average RMSE was compared using a statistical hypothesis test, namely a paired \(t\)-test. This allows for the assessment of whether the average RMSE is statistically different across each pair of methods or not. If the differences were found not to be statistically significant, we considered this to be a \"tie\", and both methods under comparison received 1 point. When a statistically significant difference was observed, a \"win\" was attributed to the method with the lowest RMSE and it received 2 points.