# Forecasting Multivariate Time Series
### Forecasting
#### 2.5.1 Univariate Forecasting  
Univariate forecasting involves predicting future values of a single variable based on historical data from consecutive time intervals. The choice of forecasting technique depends on whether the time series exhibits seasonality and trend components. Two main types of univariate forecasting methods exist: statistical methods and advanced machine learning-based methods. Statistical methods include autoregression (AR), moving average (MA), autoregressive moving average (ARMA), autoregressive integrated moving average (ARIMA), and simple exponential smoothing. Moving average techniques such as simple moving average, exponential moving average, and cumulative moving average are common. For time series with seasonality and trend, options include seasonal autoregressive integrated moving average (SARIMA), seasonal autoregressive integrated moving average with exogenous regressors, and Holt-Winters exponential smoothing (HWES), with ARIMA and SARIMA being the most successful. ARIMA combines autoregression, moving average, and differencing to achieve stationarity. Proper identification of ARIMA parameters (p, d, q) is crucial, typically performed by fitting models with different orders using criteria like the Akaike information criterion (AIC) to select the best order. The AIC measures model performance relative to the number of parameters. The autocorrelation function (ACF), and partial autocorrelation function (PACF) cannot be used to identify reliable values for p and q [43]. Residual analysis is essential in ARIMA modeling to ensure residuals resemble white noise, indicating no autocorrelation. The Ljung-Box test is commonly used to assess residual autocorrelation, where \(p\)-values above a threshold suggest no significant autocorrelation. Time series forecasting has garnered significant attention in machine learning and deep learning. [PERSON] also considers these ML-based methods of forecasting. The literature presents various techniques for this purpose, including Long Short-Term Memory (LSTM), support vector regression (SVR), the multi-layer perceptron (MLP), decision trees, random forest, the recurrent neural network (RNN), and its variant the Gated Recurrent Unit (GRU). Among these techniques, LSTM stands out, known for its effectiveness in handling sequential data. However, traditional RNNs face challenges like vanishing and exploding gradients, leading to longer training times and lower precision. These issues are addressed by LSTM and GRU, which are more efficient in managing gradient problems. GRU differs from RNNs in its use of update and reset gates to control information flow and address gradient issues. These gates determine how much preceding information is retained or discarded, enhancing the model's efficiency. Variants of GRU, including bidirectional versions, are utilized in the proposed framework. LSTM, another variant of RNN, comprises multiple LSTM units, each equipped with cell, input, output, and forget gates. These gates play a crucial role in the LSTM model, facilitating the sequential processing of data and prediction generation. This framework employs various configurations of neural networks for forecasting, including Vanilla LSTM, Stacked LSTM, Bidirectional LSTM, and CNN-LSTM. Vanilla LSTM consists of one hidden layer and one output layer, while Stacked LSTM involves multiple hidden layers. Bidirectional LSTM processes data in both forward and backward directions and then combines the interpretations. CNN-LSTM integrates a CNN to understand sequence parts and then passes them to LSTM for interpretation, forming a hybrid model. Comparing GRU and LSTM, no clear winner emerges across all performance parameters. GRU executes faster with fewer parameters and less memory usage, while LSTM demonstrates greater precision, especially on larger datasets. MLP serves as a foundational artificial neural network, featuring fully connected feedforward layers suitable for classification and regression tasks.