# DR-Transformer: A Multi-Features Fusion Framework for Tropical Cyclones Intensity Estimation
## 4 Experiment
### Intensity Estimation and Prediction  
Our transformer model aggregates temporal correlations from the input sequence. Hence, we changed the length \(T\) of the input sequence to search for the best estimation and prediction results. For estimating intensity at \(T\), we utilized \(T\) images and \(T-1\) estimated intensity before \(T\). In addition, our prediction experiments were divided into 3-h, 6-h, 12-h, and 24-h forecasts. Specifically, we masked the attention weight matrix in the transformer encoder for predictions. \[Estimate:(x_{t-24h},\widehat{y}_{t-24h}),...,(x_{t})\Rightarrow \widehat{y}_{t}\] \[3 hourforecast:(x_{t-21h},\widehat{y}_{t-21h}),...,(x_{t}, \widehat{y}_{t})\Rightarrow\widehat{y}_{t+3h}\] \[6 hourforecast:(x_{t-18h},\widehat{y}_{t-18h}),...,(x_{t}, \widehat{y}_{t})\Rightarrow\widehat{y}_{t+6h}\] \[12 hourforecast:(x_{t-12h},\widehat{y}_{t-12h}),...,(x_{t}, \widehat{y}_{t})\Rightarrow\widehat{y}_{t+12h}\] \[24 hourforecast:(x_{t},\widehat{y}_{t})\Rightarrow\widehat{y}_{t+ 24h}\]  
The experiment results are presented in Table 4. When \(T=7\), TC estimation reaches the best result of 7.76 knots. Predictions obtain the best results when \(T=9\) and the best RMSEs of the 6 h prediction and 12 h prediction are 8.55 and 10.52, respectively. \begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Loss** & **MAE** & **RMSE** \\ \hline (a) & CE & 7.63 & 10.36 \\ (b) & DC & 7.44 & 9.71 \\ (c) & DC + RI & 6.79 & 8.88 \\ (d) & DC + RI + smooth & 6.87 & 9.16 \\ (e) & DC + RI + transformer & **6.06** & **7.76** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Ablation study on different components in DR-transformer framework. In the table, DC/RI/CE represents whether to use the corresponding loss and smooth/transformer represents the way we process temporal data. To demonstrate the effectiveness of our transformer, we compared it with LSTM and the smooth procedure. We set the length of the sequence to 7 in the transformer and LSTM. In the smooth procedure, we used a weighted average operation and the weights are 5, 4, 3, 2, and 1. The MAE and RMSE results on each category are shown in Figure 5. The transformer exhibited a better performance with both RMSE and MAE metrics in all categories than the other two methods. The box plots of these three methods are shown in Figure 5. In Figure 5, we use a red rectangle to mark some outliers and we can see that our transformer has fewer outliers than other methods. Additionally, our transformer has a smaller interquartile range, which means the estimation is more stable. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Approach**} & \multicolumn{2}{c}{\(\mathbf{T=3}\)} & \multicolumn{2}{c}{\(\mathbf{T=4}\)} & \multicolumn{2}{c}{\(\mathbf{T=5}\)} & \multicolumn{2}{c}{\(\mathbf{T=6}\)} & \multicolumn{2}{c}{\(\mathbf{T=7}\)} & \multicolumn{2}{c}{\(\mathbf{T=8}\)} & \multicolumn{2}{c}{\(\mathbf{T=9}\)} \\ \cline{2-13}  & **MAE** & **RMSE** & **MAE** & **RMSE** & **MAE** & **RMSE** & **MAE** & **RMSE** & **MAE** & **RMSE** & **MAE** & **RMSE** & **MAE** & **RMSE** \\ \hline ours (3 h-predict) & 6.46 & 8.53 & 6.45 & 8.52 & 6.41 & 8.44 & 6.58 & 8.84 & 6.42 & 8.51 & 6.47 & 8.54 & 6.30 & **8.09** \\ ours (6 h-predict) & 7.31 & 9.80 & 7.00 & 9.35 & 7.11 & 9.46 & 7.14 & 9.67 & 6.93 & 9.30 & 7.04 & 9.93 & 6.53 & **8.55** \\ ours (12 h-predict) & - & - & - & 9.23 & 12.65 & 8.92 & 12.41 & 8.57 & 11.85 & 8.66 & 11.91 & 7.65 & **10.52** \\ ours (24 h-predict) & - & - & - & - & - & - & - & - & - & 12.12 & **17.16** \\ ours (estimate) & 6.13 & 7.96 & 6.08 & 7.82 & 6.11 & 7.91 & 7.92 & 8.04 & 6.01 & **7.76** & 6.16 & 7.93 & 6.79 & 8.67 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Estimation and prediction RMSE and MAE of our method. In our prediction experiment, the length of the input time series must be greater than the predicted length. Therefore, we put a ‘-’ in any position to represent an unpredictable situation. Figure 5: (**a,b**) are the comparison of MAE, RMSE respectively. And (c) is the box plot of estimated errors.