# A Survey of Machine Learning Methods for Time Series Prediction
## 5 Experimental Results and Discussion
### Analysis
Specifically, RNN models excel in a wide range of tasks including Water and Air Quality, Environmental and Meteorological Predictions, Structural and Mechanical Health Monitoring, Stock Market/Finance/Market Trends, and Healthcare/Biomedical Predictions due to their unique ability to remember long-term dependencies. Patterns in these domains can span over days, weeks, and even months, and RNN models like LSTM are able to remember this information through memory gates without manual feature engineering. Dataset Size: An analysis of the influence of the dataset size on the relative performance of machine learning models for time series prediction reveals some interesting trends. In ML tasks with the least amount of data, DL models perform comparably to TBML models overall. However, RNN models demonstrate a significant advantage over SPTB models. As the dataset size increases, DL models begin to show a slight edge over TBMLmodels in the lower mid-sized range (2173-7800 samples), although the gap narrows in the upper mid-sized range (7800-35,712 samples). Across this range, RNN models continue to outperform SPTB models, though the margin diminishes, reflecting SPTB models' growing robustness with increasing data. For larger datasets, TBML models gain a clear advantage over DL models, with their performance gap widening at the largest dataset range (206,573-11,275,200 samples). This trend demonstrates the scalability of TBML models, particularly SPTB models, in handling vast amounts of data. The reason that DL and specifically RNN models are able to perform better on smaller datasets than TBML models is they are better able to handle latent representations if sequential inputs are strong, even on small amounts of data. TBML models lack inducitve bias for sequence learning and thus may stuggle on small datsets if not given the appropriately engineered sequential features. Conversely, TBML models, specifically SPTB models, outperform DL models on larger datsets as these algorithms are much more computationally efficient at dealing with large amounts of data while remaining robust to noise, missing values, or irrelevant features, allowing them to quickly capture complex patterns. DL models are comparatively computationally expensive, harder to optimize, and require careful preprocessing for noisy data. When DL models are not given adequate training resources or are not carefully optimized, they consistently underperform relative to their TBML counterparts. It is worth pointing out that these generalizations do not hold true in all circumstances. Study [15] showcases that DL models such as LSTM struggle with overfitting and generalization, especially on small datasets, due to the nature of the model architecture (there are a larger number of weights and deviation terms for learning). The easiest way to overcome this limitation is to expand the dataset, as shown in study [16]. If there are no other data available, then DL models may fall short to other TBML methods. Study [67] demonstrates that in situations where no extra data are available, gradient-boosted TBML techniques show a superior generalization ability and thus lead to higher prediction accuracy. Temporal Resolution: This study examines the impact of time interval lengths on the performance of ML models for time series prediction. However, no consistent pattern or trend emerges across the time intervals. DL models perform better in some intervals, while TBML performs better in others. As such, there is no common thread that unites these results across all temporal resolutions studied. The reason for this may indicate that other aspects of the data composition or domain-specific tasks are more important and impactful on the model performance, leaving weak-to-nonexistent trends for temporal resolution differences. Influence of Research Focus: A notable finding is the inherent bias in performance outcomes depending on the research focus of the paper. DL models significantly outperform TBML models in papers focused on DL techniques, and vice versa for TBML-focused research.