# A Survey of Machine Learning Methods for Time Series Prediction
## 8 Future Work  
There are several areas of future work that researchers could focus on to expand the insights of this paper, especially in areas of growing importance in the field of DL. One of these areas that was briefly touched upon in this paper and that warrants future research is Transformer-based architectures. The initial research indicates that transformers excel in long-range dependency modeling and thus perform well in the reviewed papers [28; 33; 70; 74; 77], showcasing a performance on par with the best-performing models in the literature. As there are only five research papers included in this study, future research should investigate the validity of this by exploring more recent papers where transformer models are being applied to time series applications. One of the biggest challenges that \begin{table}
\begin{tabular}{c c c} \hline \hline
**Dataset Size** & **Best-Performing Model Class** & **Best-Performing Model Subclass** \\ \hline Small (0-2173) & TBML/DL & RNN \\ Small/Medium (2173–7800) & DL & SPTB/RNN \\ Medium (7800–35,712) & TBML & RNN \\ Medium/Large (35,712) & TBML & SPTB/RNN \\ Large (206,573-11,275,200) & TBML & SPTB \\ \hline Task Category & Best-Performing Model Class & Best-Performing Model Subclass \\ \hline Energy and Utilities & TBML & SPTB \\ Environmental and Meteorological & DL & RNN \\ Agriculture and Crop Management & TBML & SPTB \\ Water and Air Quality & TBML & RNN \\ Transportation and Urban Mobility & TBML & SPTB \\ Structural and Mechanical Health Monitoring & DL & RNN \\ Stock Market, Finance, and Market Trends & DL & RNN \\ Healthcare and Biomedical Predictions & TBML & RNN \\ Anomaly Detection & TBML & SPTB/RNN \\ Other & TBML & SPTB \\ \hline Time Interval & Best-Performing Model Class & Best-Performing Model Subclass \\ \hline
1 min & TBML/DL & RNN \\
5, 10 min & DL & RNN \\
15, 30 min & DL & RNN \\
1, 4 h & TBML & SPTB \\
1 day & TBML/DL & RNN \\
1 week, 8 day, 15 day, 16 day & TBML & SPTB \\
1 month & DL & RNN \\ \hline Computational Efficiency & Best-Performing Model Class & Best-Performing Model Subclass \\ \hline Training Time & TBML & SPTB \\ Inference time & TBML & SPTB \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Summary table of best-performing models for each data characteristic. faces attention-based architectures is their significant computational cost. An interesting area to explore in addressing this is pre-trained transformer models [93]. These models can be pre-trained on large collections of unrelated time series data, enabling the improved performance of training on datasets across domains. Other areas that future researchers could focus on include Diffusion Models and Neural Ordinary Differential Equation Models. Differential Equation Models have shown success in text, image, and video applications and recently have started to be applied to time series forecasting use cases (for a comprehensive review of diffusion models for time series applications, see [94]). Future research should focus on combatting the exceptionally high computational cost associated with the Diffusion Models applied to time series applications while maintaining high accuracy. Similarly, Neural Ordinary Differential Equations offer a modeling framework that provides a principled approach for forecasting continuous time series data by combining neural networks with the mathematics of differential equations (for a comprehensive review of Neural Ordinary Differential Equations for time series applications, see [95]). Neural Ordinary Differential Equations involve solving differential equations at a training and inference time which can be computationally very expensive. Future research in this field should focus on reducing these computational demands by developing more efficient solvers to increase the practicality of this DL architecture for time series forecasting applications. Conceptualization, T.H. and K.R. ; methodology, T.H. ; software, T.H. ; validation, T.H. ; formal analysis, T.H. ; investigation, T.H. ; resources, T.H. ; data curation, T.H.