# A Survey of Machine Learning Methods for Time Series Prediction
## 5 Experimental Results and Discussion
### Results  
#### 5.3.1 Overall Model Performance  
The first objective of this paper is to identify the models that exhibit the best overall performance across a variety of time series prediction tasks analyzed in the 79 surveyed studies. The models can broadly be categorized into two classes: Tree-Based Machine Learning (TBML) models and Deep Learning (DL) models. Figure 3 illustrates the comparative performance of these classes based on the First Place Aggregation (FPA) and Weighted Rank Aggregation (WRA) metrics. TBML models marginally outperform DL models as a class, achieving the best performance (FPA) in 54.55% of tasks studied, with a WRA score of 0.6910. In comparison, DL models perform best (FPA) in 52.70% of tasks and achieve a WRA score of 0.6486. It is important to note that ties for the highest-performing model contribute to a combined percentage exceeding 100%. On average, TBML models outperform DL models by approximately 2.5%. However, this comparison at the class level provides only a surface-level understanding of the performance landscape. A more granular analysis is needed to assess the performance of subclasses within each category. Figure 4 breaks down the performance of model subclasses within the TBML and DL categories for the FPA and WRA metrics. Notably, the \"Attention\" subclass, which exclusively comprises Transformer models, emerges as the best-performing model in three  
Figure 3: FPA and WRA score distribution for each class, including the comparative performance (%) between the two classes. tasks, but is only evaluated in five studies. While this result is intriguing and suggests a promising avenue for further research, the limited sample size warrants caution when interpreting these findings. Consequently, this paper does not place a primary focus on attention-based models. From the detailed subclass analysis, it is evident that the most effective algorithms for time series prediction are Recurrent Neural Network (RNN) models, primarily comprised of LSTM, GRU, and general RNN architectures, and the Specialized Tree-Based Models (SPTB) category, which includes XGBoost, LightGBM, and CatBoost. RNNs demonstrate a superior performance, ranking as the best-performing models in 61.36% of studies and  
Figure 4: FPA and WRA scores for each model. The “Model Use” legend indicates the total number of comparisons for each model, with specific counts labeled above the bars. achieving a WRA score of 0.7330. They are closely followed by SPTB models, which perform best in 50% of studies and achieve a WRA score of 0.6328. To further refine this understanding, the subclasses are broken down into individual models. Figures 5 and 6 present the performance of each model based on the FPA and WRA metrics, respectively. Among individual models, CatBoost emerges as the best performer across both metrics, followed by Transformers, LSTMs, and LightGBM. However, it is important to note that like Attention-Based Transformer models, the robustness of CatBoost's results may be limited, as it was the best-performing model in only two out of the three studies where it was evaluated. The LSTM model, on the other hand,  
Figure 5: FPA scores for each model. The “Model Use” legend indicates the total number of comparisons for each model, with specific counts labeled above the bars. Figure 6: WRA scores for each model. The “Model Use” legend indicates the total number of comparisons for each model, with specific counts labeled above the bars. demonstrates a strong and consistent performance, achieving FPA of 58.97% and a WRA score of 0.7222. With 39 instances of evaluation, its results are more statistically robust. LightGBM also performs well, with FPA of 58.82% and a WRA score of 0.6608, based on 17 studies. In contrast, the two weakest-performing models were MLPs and general FFNNs. Neither model achieved the highest rank in any study, and their WRA scores, 0.2265 for MLP and 0.1667 for FFNN, were the lowest among all evaluated models.