# MM-iTransformer: A Multimodal Approach to Economic Time Series Forecasting with Textual Data
## 4 Experiment
### Experimental Results
The DM test results show statistically significant improvements in forecasting accuracy when textual data are integrated. Specifically, the Gold-price dataset yields a DM statistic of 2.030 with a \(p\)-value of 0.043, and the Forex dataset exhibits a DM statistic of 3.010 with a \(p\)-value of 0.003. These results indicate that the improvements in predictive performance \begin{table}
\begin{tabular}{c c} \hline \hline
**Dataset** & **DM Test Statistic (\(p\)-Value)** \\ \hline Gold-price & 2.030 (0.043 *) \\ \hline Forex & 3.010 (0.003 **) \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Diebold–Mariano (DM) test results comparing the forecasting performance of MM-iTransformer vs. iTransformer. Significance levels: * \(p<0.05\), ** \(p<0.01\). \begin{table}
\begin{tabular}{l c c c c c c c c c} \hline \hline \multirow{2}{*}{**Models**} & \multicolumn{4}{c}{**Forex**} & \multicolumn{4}{c}{**Gold-Price**} \\ \hline  & **MSE** & **MAE** & **RMSE** & **MAPE** & **MSE** & **MAE** & **RMSE** & **MAPE** \\ \hline MM-iTransformer (Ours) & 0.019 & 0.094 & 0.137 & 0.160\% & 0.057 & 0.185 & 0.239 & 0.182\% \\ iTransformer [32] & 0.022 & 0.102 & 0.147 & 0.189\% & 0.065 & 0.207 & 0.254 & 0.196\% \\ ARIMA [40] & 0.030 & 0.092 & 0.174 & 0.889\% & 0.105 & 0.279 & 0.325 & 2.786\% \\ Random Walk Model [41] & 0.020 & 0.094 & 0.142 & 9.495\% & 0.110 & 0.288 & 0.331 & 19.727\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Performance comparison of different models on Forex and Gold-price datasets. The underlined values indicate the best performance for each metric. \begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**Forex**} & \multicolumn{4}{c}{**Gold-Price**} \\ \hline  & **MSE** & **MAE** & **RMSE** & **MAPE** & **MSE** & **MAE** & **RMSE** & **MAPE** \\ \hline With random text & 0.021 & 0.099 & 0.146 & 0.201\% & 0.069 & 0.215 & 0.263 & 0.205\% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Ablation study on the impact of textual data temporal alignment on forecasting accuracy. with the MM-iTransformer, which incorporates textual data, are statistically significant compared to the iTransformer, which relies solely on the historical price data. These findings provide statistical evidence that supports the hypothesis that textual integration enhances the ability of forecasting models to capture complex market dynamics more effectively than models relying on time series data alone. This validation confirms the relevance and efficacy of incorporating textual data into economic forecasting models, reinforcing the value of multimodal approaches in enhancing prediction accuracy. Analysis of Historical Window Lengths: To assess model performance stability, we experimented with various historical window lengths ranging from 36 to 384 days on the Gold-price and Forex datasets. The results indicate that shorter windows typically result in lower loss, with a 96-day window yielding optimal performance as shown in Figures 5 and 6. Extending the window beyond 96 days led to increased errors from outdated and less relevant data, which is detrimental for short-term forecasting. The sizes of our datasets--1352 samples for Gold-price and 2347 for Forex--also guided our choice of window length to avoid overfitting. Larger windows risk models memorizing specific data features instead of generalizing from underlying patterns, reducing their effectiveness on unseen data. Furthermore, we did not opt for smaller windows, as they could miss significant economic cycles and trends crucial for robust predictions. The chosen 96-day window provides a balanced approach, capturing essential market dynamics without the noise and overfitting associated with longer windows. This balance facilitates efficient learning and reliable forecasting, making it well suited for our dataset constraints. Example Forecasting Results: Figure 7 presents a detailed comparison of forecasting outcomes from the MM-iTransformer and iTransformer models applied to Forex datasets.