# DR-Transformer: A Multi-Features Fusion Framework for Tropical Cyclones Intensity Estimation
###### Abstract  
Convolutional neural networks have achieved great success in analyzing potential features inside tropical cyclones (TCs) using satellite images for intensity estimation. However, due to the high similarity of visual features in TC images, it is still a challenge to learn the accurate mapping between TC images and numerical intensity. Existing works mainly focus on the visual features of a single TC, ignoring the impact of intensity continuity and time evolution among TCs on decision making. Therefore, we propose a DR-transformer framework for temporal TC intensity estimation. Inside DR-transformers, a novel DR-extractor can extract Distance-consistency(DC) and Rotation-invariance (RI) features between TC images, and therefore can better learn the contours, structures, and other visual features of each TC image. DC features can reduce the estimation error between adjacent intensities, and RI features can eliminate feature deviation caused by shooting angles and TC rotation. Additionally, a transformer with a DR-extractor as the backbone is applied to aggregate the temporal correlation in a series of TC images, which can learn the evolution from intensity to the visual features of TC. Experiments show that the final result, an RMSE of 7.76 knots, outperforms the baseline, and is better than any previously reported method trained on the TCIR dataset. intensity estimation; metric learning; transformer; tropical cyclone 2021 1 (1) (202  
by different angles to reduce the variance of estimation, which ignored multi-view features of the image itself. Specifically, we describe our idea in Figure 1, where DC is short for distance-consistency, and RI is short for rotation-invariance. As shown in Figure 1a, existing methods reduce number of the negative samples, which caused samples with a larger intensity difference to have a relatively small distance difference. For example, if the intensity is 120, it can be misclassified into 80, which is unacceptable. However, in Figure 1b, it shows a perfect mapping between image and intensity; even if misclassified, it will only be done so to adjacent categories. Additionally, since TC is constantly rotating, it is important to extract rotation-invariance features. In Figure 1c, TCs are misclassified at certain camera angles, while their rotation-invariance features can be correctly classified. Here, we give two definitions. DC: distance-consistency--the equal proportion relationship between the sample intensity and sample distance in feature space. RI: rotation-invariance--the same TC images with different angles have the same representation in the feature space. Motivated by the above, we formulated TC intensity estimation from a metric learning perspective and propose a DR-transformer framework with a DR-extractor and a transformer. The DR-extractor is a CNN model based on a joint loss of both Distance-consistency (DC) loss and Rotation-invariance (RI) loss. DC loss learned features by restraining the ratio, which is defined as the quotient of the differences in distance and in intensity. RI loss is used to extract the rotation-invariance feature according to the characteristics of continuous rotation of TC and differences in shooting perspectives. For extracting the RI feature, we ensured the feature embedding extracted from an image and its random-rotated counterpart were close. Finally, we applied a transformer [5] to aggregate the temporal correlation from a series of TC images, which can learn about the relationship between the change in intensity and that of TC contour, structure, and other visual features. We further feed the extracted feature of our DR-transformer framework into a nearest neighbor (NN) classifier for estimating and predicting intensity. Extensive experiments on the TCIR [4] dataset demonstrate the effectiveness of our proposed method. We summarize our contributions as follows:  
* We propose a distance-consistency loss function, which is a general loss function, especially suitable for constructing image features with continuous labels.