# DR-Transformer: A Multi-Features Fusion Framework for Tropical Cyclones Intensity Estimation
## 3 Methods
### Image-to-Intensity Feature Learning
#### 3.1.1 Distance Consistency Feature Learning  
TC intensity in the dataset is a series of scalars and is different from traditional classify-based labels, which are discrete. If we treat continuous labels as discrete class labels or relative labels, we will neglect their numerical relations. Therefore, inspired by [PERSON] [16], we proposed DC loss based on (N + 1)-tuplet in order to make full use of numerical relation. With the help of DC loss, we can construct a DC embedding space in which the distance between samples in the embedding space is directly proportional to that in the label space. Figure 2: The overall pipeline of the proposed DR-transformer framework. During the inference stage, a series of the image is fed into the DR-extractor to generate the feature vectors sequentially. Meanwhile, we encode the intensity (estimated from classifier) to a one-hot vector. Then, the transformer takes one-hot vectors and feature vectors as input. Finally, the outputs of the transformer pass through a classifier to estimate and predict the intensity. The training details of DC loss are shown in Figure 3. To obtain DC loss, we constructed a (N + 1)-tuplet of training samples with an anchor \(a\) and \(N\) neighbors randomly sampled from the remaining ones. Then we fed them into the CNN to obtain its embedding \(f(x_{i},\theta)\), hereinafter abbreviated to \(f_{i}\). Note that embedding \(f\) should be \(L_{2}\) normalized because without such a normalization, their magnitudes tend to diverge. Then, for samples \(i\) and \(j\), we defined pairwise ratios of feature embedding distance and label distance as  
\[r_{ij}=\frac{D(f_{i},f_{j})}{D(y_{i},y_{j})} \tag{2}\]  
where \(D(\cdot)\) indicates Euclidean distance. Finally, our distance-consistency loss can be formulated as  
\[\ell_{DC}=-\sum_{i=1}^{N}\log\frac{r_{ai}}{\sum_{j=1}^{N}r_{aj}} \tag{3}\]  
where \(r_{ai}/\sum_{j=1}^{N}r_{aj}\) is the normalization of _rai_ and \(a\) represents the anchor. Note that (3) reaches the minimum value while ratio \(r_{a1}=r_{a2}=...=r_{aN}\). Our loss aims to compare the ratio, which can reflect the consistency between label distance and feature embedding distance, to decide optimizing direction. Ideally, we hope the ratio between image pairs is almost equivalent and this property can be explained through its gradients, which are given by  
\[\frac{\partial\ell_{DC}}{\partial f_{i}}=\frac{f_{i}-f_{a}}{D(f_{i},f_{a})} \cdot\ell_{i}^{\prime} \tag{4}\]  
\[\frac{\partial\ell_{DC}}{\partial f_{a}}=-\sum_{i=1}^{N}\frac{\partial\ell_{ DC}}{\partial f_{i}} \tag{5}\]  
where \(f_{i}-f_{a}\) is a vector and \(\ell_{i}^{\prime}\) is a scalar value computed by  
\[\ell_{i}^{\prime}=1-\frac{r_{ai}}{\frac{1}{N}\sum_{j=1}^{N}r_{aj}} \tag{6}\]  
As shown in (4) and (5), the optimization of \(f_{i}\) depends on both vector \(f_{i}-f_{a}\) and scalar \(\ell_{i}^{\prime}\). The former represents the direction between the embedding vectors and the latter compares different distance ratios with the mean value in a mini-batch to decide the optimizing direction to be the same or opposite. As shown in (6), if the ratio \(r_{ai}\) is less than the mean of all the ratios, the scalar \(\ell_{i}^{\prime}\) is negative. Apparently, the distance of samples \(i,a\) that are close in label space will be pulled closer. In the training stage, we utilized methods such as [14] to construct a mini-batch. We randomly select \(N\) samples as neighbors and \(M\) samples as anchors to construct mini-batch. Each anchor is combined with the \(N\) neighbors to form (N + 1)-tuple because we find that all of the tuplets sharing the same neighbors lead to a more definite convergence  
Figure 3: Training details of DC loss. We construct a (N + 1)-tuplet of training samples with an anchor and N neighbors and feed them to DR-extractor. Finally, we calculated the DC loss of feature embeddings and intensities. status. Therefore, there are M tuplets in a mini-batch in total.