# A Survey of Machine Learning Methods for Time Series Prediction
## 5 Experimental Results and Discussion
### Analysis  
This section synthesizes the biggest takeaways from the experimental results, highlighting key insights, overarching trends, and limitations observed across the surveyed studies. Comparative Performance of Tree-Based and Deep Learning Models: The experimental results indicate that TBML models and DL models exhibit comparable performances overall. Among their respective subcategories, SPTB models and RNNs emerge as the best performers, further underscoring why these models dominate much of the research focus in this paper. However, TBML models demonstrate a slight overall edge due to the stronger performance of their weakest algorithms. Notably, RF and GBDT outperform the weakest DL models. At the individual algorithm level, CatBoost, LSTM, and LightGBM stand out as the best-performing models. While CatBoost has limited representation in the surveyedpapers, it is highly similar to LightGBM and XGBoost--both of which had much greater representation and demonstrated a strong performance--further supporting the conclusion that CatBoost is a top-performing model. On the other hand, attention-based architectures (e.g., Transformers) also show strong potential, but unlike CatBoost, there are no directly comparable algorithms in this category with more extensive research in the surveyed papers. This lack of broader representation underscores the need for the further validation of Transformers through more illustrative studies. Domain-Specific Performance: Performance analysis across application domains reveals distinct advantages for specific model groups:  
* TBML Models: these outperform in tasks related to energy and utilities, transportation and urban mobility, anomaly detection, and miscellaneous applications;
* DL Models: these outperform in tasks related to environmental and meteorological predictions, structural and mechanical health monitoring, and financial/market trend forecasting;
* SPTB Models: these outperform in tasks related to transportation and miscellaneous applications, while RNN models dominate in environmental, healthcare, and finance-related tasks;
* RNN Models: these outperform in tasks related to Environmental and Meteorological Predictions, Water and Air Quality, Structural and Mechanical Health Monitoring, Stock Market/Finance/Market Trends, and Healthcare and Biomedical Predictions. It is important to note that the dataset includes relatively small representations for some categories (ranging from 3.85% to 21.8% of the total papers). As a result, the findings presented here do not present definitive evidence that any one type of model is dominant within a given domain. Instead, they highlight interesting patterns and trends that merit further investigation and validation with larger datasets. However, there are several explanations that can account for the performance differential between models. TBML models excel in areas with tabular, noisy, missing, and categorical data, making them especially suitable for the domain of utilities, transportation, urban mobility, and anomaly detection, which often involve structured, tabular datasets that may have many sparse features. TBML models are especially equipped for anomaly detection as they are able to use the tree splitting criteria to capture feature interactions without needing much explicit feature engineering. Similarly, SPTB algorithms excel in transportation and miscellaneous applications as they are optimized for speed and do not require extensive feature engineering, making them compatible for a wide array of miscellaneous applications. DL models outperform in environmental/meteorological predictions and mechanical health monitoring due to their ability to effectively capture unstructured and spatial-temporal patterns in the data. Additionally, DL models perform better in financial/market trend forecasting as these models are better able to handle lagged effects and temporal dependencies, with less feature engineering than TBML models.