# Towards a Vectorial Approach to Predict Beef Farm Performance
## 3 Results
### Comparisons of ST-GP and VE-GP with Other ML Methods  
Now, we discuss the results of the experimental comparison between the ST-GP, VE-GP, and the other ML methods presented previously. As already explained, in addition to ST-GP, KNN, NN, and GLMNET also exploited the information on 2017 with a target in 2018, whereas LSTM was involved in the VE-GP to process vectorial variables for 2014-2017 and the 2018 target. The results reported in Section 3.1 for the ST-GP compared to the VE-GP are also supported by the corresponding boxplots in Figure 2. The median and mean RMSE values are reported in Table 9.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Prediction Model** & **Fitness on Test** & **N. of Variables** & **\% of Variables** \\ \hline
**ST-GP** & & & \\ model 1 & 0.1335 & 9 & 50\% \\ model 2 & 0.1207 & 6 & 33\% \\ model 3 & 0.1143 & 11 & 61\% \\ model 4 & 0.1383 & 8 & 44\% \\ model 5 & 0.1392 & 7 & 39\% \\ model 6 & 0.1439 & 7 & 39\% \\ model 7 & 0.1395 & 8 & 44\% \\ model 8 & 0.1370 & 6 & 33\% \\ model 9 & 0.1285 & 15 & 83\% \\ model 10 & 0.1184 & 7 & 39\% \\ \hline
**VE-GP** & & & \\ model 1 & 0.1117 & 5 & 26\% \\ model 2 & 0.1016 & 3 & 16\% \\ model 3 & 0.1044 & 9 & 47\% \\ model 4 & 0.1085 & 8 & 42\% \\ model 5 & 0.1134 & 3 & 16\% \\ model 6 & 0.0998 & 8 & 42\% \\ model 7 & 0.1018 & 4 & 21\% \\ model 8 & 0.1149 & 4 & 21\% \\ model 9 & 0.0999 & 8 & 42\% \\ model 10 & 0.1121 & 3 & 16\% \\ \hline \hline \end{tabular}
\end{table}
Table 8: Fitness on the test set, number of involved variables, and corresponding percentage for each model evolved by ST-GP (upper table) and VE-GP (lower table) in each of the 10 runs.  
Figure 2: RMSEs on both the learning and test sets for the different algorithms. Learning results are plotted in yellow (**left**) and test results are plotted in blue (**right**) for each technique.  
The Kruskal-Wallis nonparametric test, performed for all the considered methods with a significance level of \(\alpha=0.05\), was applied to investigate the RMSE achieved among the learning sets and the test sets separately. The resulting \(p\)-values (\(t\ll 0.001\)) showed extremely significant differences in median performance between the methods, considering both stages. The pairwise Wilcoxon tests provided with Bonferroni correction \(\alpha=0.05/15=0.0033\) was hence performed among all compared techniques. Among the learning set, STGP was significantly different from all other methods, resulting in a poor performance. Likewise, KNN was significantly different with respect to both NN and LSTM, as well as the comparison between VEGFP and LSTM. Concerning the RMSE achieved on the test sets, STGP performed poorly with respect to other methods, showing greater, significantly different values for the RMSE on average. On the contrary, GLMNET's performance was significantly better than KNN's and NN's, as well as LSTM's compared to KNN and VEGFP, respectively. Consequently, the following pairs of methods did not show significantly different performance: VEGFP-KNN, VEGFP-NN, VEGFP-GLMNET, VEGFP-LSTM, as well as the pair LSTM and GLMNET.  
As a further study, we also compared learning and test fitness distributions obtained by the single methods in order to determine the occurrence of overfitting. The Wilcoxon signed rank test showed that the two distributions for KNN and the two obtained with NN were different, since the corresponding p-values were extremely significant (Wilcoxon: \(p\ll 0.001\)), as well as the median RMSE (Kruskal-Wallis test: \(p\ll 0.001\)). Concerning the ST-GP, the two distributions and the median error were slightly different (Wilcoxon and Kruskal-Wallis: \(p\)-values equal to 0.048 and 0.034, respectively). GLMNET showed the same learning and test fitness distributions but different median RMSE (Wilcoxon: \(p>0.05\); Kruskal-Wallis: \(p=0.041\)), whereas LSTM achieved different distributions with similar median. VE-GP was the only method that produced the same fitness distributions with the same median among the learning and test sets.