# MM-iTransformer: A Multimodal Approach to Economic Time Series Forecasting with Textual Data
## 4 Experiment
### Experimental Results  
This study compares the MM-iTransformer framework with the iTransformer, ARIMA, and the Random Walk Model using the Forex and Gold-price datasets. We primarily focus on the Mean Squared Error (MSE), as the models are trained using this metric. Additionally, we evaluate their predictive capabilities using Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Mean Absolute Percentage Error (MAPE). The results, presented in Table 2, show the following:  
1. Comparison between MM-iTransformer and iTransformer: The MM-iTransformer integrates textual data with historical price data and demonstrates superior performance over the iTransformer. On the Forex dataset, it reduces the MSE from 0.022 to 0.019, showing a 13.64% reduction. For the Gold-price dataset, the MSE is reduced from 0.065 to 0.057, a reduction of 12.31%. These results highlight the significant benefits of incorporating textual data into forecasting models, enhancing the MM-iTransformer's accuracy and robustness. 2. Single-variable and Multi-variable Prediction Performance: MM-iTransformer excels in both single-variable (Gold-price) and multi-variable (Forex) predictions, affirming its capability to utilize textual data effectively. This inclusion provides additional predictive signals, enabling the model to capture market dynamics more accurately than using historical price data alone. 3. Comparison with Classical Models: Compared to classical models, both the MM-iTransformer and iTransformer exhibit superior performance, with MM-iTransformer consistently achieving the lowest MSE across datasets. However, the iTransformer does not consistently outperform the Random Walk Model on the Forex dataset, possibly due to difficulties in adapting to abrupt market changes--a scenario where the Random Walk Model, which assumes price continuity, may have an advantage. Figure 4: (**Left**): Line graph showing performance for the MM-iTransformer model using a historical window of \(L=96\) days across various forecast horizons (\(T=4,8,12,24,36,48,60,96\) days). (**Right**): Detailed tabular representation of predictive performance metrics (MSE, MAE, RMSE, MAPE). In summary, the MM-iTransformer framework marks a significant advancement in economic forecasting by leveraging multimodal data, particularly through the integration of textual information. This study confirms the potential of advanced deep learning techniques to revolutionize financial market predictions by incorporating diverse data types. Ablation study: To assess the impact of temporally aligned news on prediction accuracy, we conduct an ablation study by randomizing the order of news texts within the time series window. This change disrupts the contextual relevance of the news data, simulating scenarios where the financial news is not aligned with historical market prices. By removing this alignment, we could isolate and examine the specific effect of the temporal relevance of textual inputs on forecasting performance. As shown in Table 3, using randomized news texts shows a clear decline in performance compared to temporally aligned news. In the Gold-price dataset, the MSE increases to 0.069, and for the Forex dataset, it increases to 0.021 when random news is used. This indicates that non-contextual, misaligned news does not contribute meaningfully to prediction accuracy and may even reduce it. The observed rise in error rates underscores the critical role of temporal alignment, affirming that relevant, timely news data significantly enhance forecasting accuracy. **Statistical Significance Analysis of Textual Data Integration:** To rigorously ascertain the impact of integrating textual data into economic forecasting models, we utilize the Diebold-Mariano (DM) test to statistically evaluate the enhancements observed with the MM-iTransformer compared to the traditional iTransformer. The DM test results are summarized in Table 4, where the DM statistics for both the Gold-price and Forex datasets are presented.