## 3 Results
### Feature Selection
This is a backward stepwise selection process, starting from all features up to 0.01% of them, with a step of 0.01% reduction. The importance of the features is the outcome of the Reciprocal Ranking. The optimal number of features for all species is arounf 2%. As a final step in the modeling procedure, the initial classifiers from Section 3.2 are again deployed, this time using the top features as defined by the process of feature selection. The test set is the same as before (20%), when XGBoost was used. The results are available on Tables 5 and 6. It is clear that feature selection improves all algorithms except Extra Trees Regression (improvement in only 3 species), Support Vector Regression (improvement in 6 species), and Linear Regression (improvement in 7 species). The improvement in all algorithms was of a considerable degree. Some of them had an exceptional increase in performance, like Linear Regression (42% improvement over baseline). The best models are again the same as in the initial comparison. These are XGBoost, Gradient Boosting and LightGBM, with XGBoost performing again slightly better on average. The results indicate that the ensemble feature selection used in this work is a consistent method, that improves almost all algorithms and by a high margin. Finally, a model has to be chosen to predict the probability of occurrence of each species in the whole Mediterranean, and that model will be XGBoost. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & \multicolumn{2}{c}{**ADA**} & \multicolumn{2}{c}{**DT**} & \multicolumn{2}{c}{**SVR**} & \multicolumn{2}{c}{**LR**} \\
**Species** & **Baseline** & **2\% Feats** & **Baseline** & **2\% Feats** & **Baseline** & **2\% Feats** \\ \hline _E. encrasicalus_ & 0.2432 & 0.1710 & 0.2784 & 0.2318 & 0.3490 & 0.3446 & 0.3724 & 0.1962 \\ _S. pilchardus_ & 0.1916 & 0.1542 & 0.2621 & 0.1551 & 0.3488 & 0.2552 & 0.3670 & 0.1904 \\ _S. aurita_ & 0.2587 & 0.1995 & 0.3283 & 0.2484 & 0.3432 & 0.3589 & 0.3528 & 0.3720 \\ _S. colins_ & 0.1924 & 0.1427 & 0.2377 & 0.1729 & 0.2420 & 0.2585 & 0.2662 & 0.1604 \\ _S. scombrus_ & 0.1351 & 0.1002 & 0.1694 & 0.1139 & 0.2164 & 0.1756 & 0.2191 & 0.1026 \\ _S. smaris_ & 0.1427 & 0.1153 & 0.1670 & 0.1165 & 0.1649 & 0.1377 & 0.1816 & 0.1208 \\ _T. thymmus_ & 0.0969 & 0.0611 & 0.116 & 0.0678 & 0.2305 & 0.1130 & 0.2397 & 0.0873 \\ _X. gladus_ & 0.0879 & 0.0596 & 0.1127 & 0.0647 & 0.2209 & 0.1203 & 0.2227 & 0.0978 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Test set performance of the algorithms used in Section 3.2. (continued from Table 5). \begin{table}
\begin{tabular}{l c c c c c} \hline \hline  & \multicolumn{2}{c}{**Validation**} & \multicolumn{2}{c}{**Test**} \\
**Species** & **Baseline** & **2\% Feats** & **Baseline** & **2\% Feats** & **Baseline** & **2\% Feats** \\ \hline _E. encrasicalus_ & 0.2369 & 0.2075 & 0.1503 & 36.5\% \\ _S. pilchardus_ & 0.1807 & 0.1562 & 0.1287 & 28.8\% \\ _S. aurita_ & 0.2548 & 0.2218 & 0.1877 & 26.3\% \\ _S. colins_ & 0.1783 & 0.1596 & 0.1302 & 26.9\% \\ _S. scombrus_ & 0.1254 & 0.1152 & 0.0738 & 41.1\% \\ _S. smaris_ & 0.1221 & 0.1153 & 0.0869 & 28.8\% \\ _T. thymmus_ & 0.0862 & 0.0812 & 0.0471 & 15.3\% \\ _X. gladus_ & 0.0794 & 0.0730 & 0.0464 & 41.5\% \\ \hline \hline \end{tabular}
\end{table} 
Table 4: XGBoost performance of each species. Baseline is the RMSE score of the test set before feature selection. Validation is the 10-fold cross validation RMSE using the top 2% features. Test is the RMSE score of the test set using the top 2% of features. Improvement is the test’s improvement over baseline.