## 4 Empirical Results
### Experimental Design
\begin{table}
\begin{tabular}{l c c c} \hline \hline
**Layer** & **Hyperparameter** & **Search Space** & **Optimal Value** \\ \hline \multirow{5}{*}{1 st Layer (S\&P500)} & LSTM Units & [32; 48; 64; 80; 96; 112; 128] & 32 \\  & Dense Units & [32; 48; 64; 80; 96; 112; 128] & 48 \\  & Epochs & \{5, 10, 15, 20\} & 5 \\  & Batch Size & \{16, 32, 64\} & 64 \\ \hline \multirow{5}{*}{2 nd Layer (VIX + Attention) Attention Heads} & Dense Units & [32; 48; 64; 80; 96; 112; 128] & 48 \\  & LSTM Units & \{32, 48, 64, 80, 96, 112, 128\} & 80 \\ \cline{1-1}  & Attention Heads & \{2, 4, 6, 8\} & 6 \\ \cline{1-1}  & Epochs & \{5, 10, 15, 20\} & 15 \\ \cline{1-1}  & Batch Size & \{16, 32, 64\} & 32 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Hyperparameter search space and optimal values for the first and second layers. dencies. The self-attention mechanism enables dynamic learning of relationships across all time steps in a sequence. However, its computational complexity grows quadratically with the sequence length (\(O(N^{2})\)), making it computationally expensive for long time series. Moreover, in domains like financial time series where data are often limited, such complex models are prone to overfitting, and their theoretical advantages may not translate into practical performance gains. Likewise, hybrid CNN-LSTM models can capture both local patterns and temporal dynamics. Nevertheless, their structural complexity and the extensive hyperparameter tuning they require make fair and reproducible comparisons difficult, especially under constrained computational resources. Given these limitations, we adopt a VMD-Cascaded LSTM architecture with integrated Multi-Head Attention, instead of employing full Transformer or CNN-LSTM models. In our framework, the original time series is first decomposed using VMD, which separates the signal into multiple IMFs that represent different frequency components. These IMFs are then modeled using a Cascaded LSTM structure to hierarchically learn temporal dependencies. Multi-Head Attention is incorporated to allow the model to focus dynamically on informative time steps during prediction. This architecture captures the core benefits of attention mechanisms while maintaining suitability for nonstationary, data-constrained time-series environments. As such, the proposed model achieves a balanced trade-off between computational efficiency and forecasting performance. These models are chosen to systematically analyze the impact of three key components:  
* The effect of VMD on predictive performance;
* The influence of hyperparameter tuning across layers;
* The contribution of the Attention mechanism in improving volatility forecasting. The baseline models, LSTM1 and LSTM2, tests the performance of standard LSTM architectures without VMD. VMD LSTM models evaluate how feature decomposition affects prediction accuracy, differentiating between configurations that use only VMD S&P 500 data and those incorporating VMD VIX components. Finally, the VMD-Cascaded LSTM models assess the performance gains from cascading multiple LSTM layers and applying the Attention mechanism. The details of these models are as follows:  
* **LSTM1**: Predicts VIX using S&P 500 data with first-layer hyperparameter tuning. This model serves as a baseline LSTM model without VMD, helping to assess the effectiveness of feature extraction from raw S&P 500 data. \begin{table}
\begin{tabular}{l l} \hline \hline
**Component** & **Configuration** \\ \hline Optimizer & Adam \\ Learning Rate & 0.001 \\ Dropout (1 st Layer) & 0.2 \\ L2 Regularization (Dense Layer) & Yes \\ Early Stopping & Patience = 5, monitor = val\_loss \\ Batch Size & 64 (1 st Layer), 32 (2 nd Layer) \\ Epochs & 5 (1 st Layer), 15 (2 nd Layer) \\ LSTM Units (1 st Layer) & 32 \\ Dense Units (1 st Layer) & 48 \\ LSTM Units (2 nd Layer) & 80 \\ Dense Units (2 nd Layer) & 48 \\ Multi-Head Attention Heads & 6 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Model configuration of the proposed VMD-Cascaded LSTM with Attention architecture.