## 3 Methodology of TIN
### Contrastive Learning Model  
The success of the GPT series [59] in natural language processing (NLP) proves that learning from web-scale data can yield better results than good manual annotation [60]; however, this strategy is ignored in the computer vision (CV) field. CLIP establishes a link between image and text via a visual encoder and a text encoder, and then trains the model on 400 million pairs of image and text-in-text pairs of network data, with the aim of learning the representation of images from text. This very large training dataset offers CLIP the powerful ability to learn features and provides it with a zero-shot nature, making it unparalleled in its adaptability to the environment and allowing for good results to be obtained by direct inference on different datasets. The two encoders process text and image data separately, with the text encoder using Transformer and the image encoder using two models, ResNet or Vision Transformer (ViT). The visual encoder accepts an image as input and the text encoder accepts a text sentence as input, e.g., The image is a [label]\({}^{\prime}\). The subsequent process can be divided into several steps. First, the image encoder and text encoder are used to map an image \(I=\{l_{i}\}_{i=1}^{N}\) and all label templates \(T=\{T_{i}\}_{i=1}^{N}\) to different spatial dimensions, where N denotes the batch size, i.e., \(f_{\text{v}}=E_{i}^{I}(I_{i})\in\mathbb{R}^{d_{\text{v}}}\), \(f_{\text{t}}=E_{i}^{T}(T_{i})\in\mathbb{R}^{d_{\text{t}}}\). The features of the two modules are mapped to the same embedding space and normalized through the dimension matrices \(D_{\text{v}}\in\mathbb{R}^{d_{\text{v}}\times d}\) and \(D_{\text{t}}\in\mathbb{R}^{d_{\text{t}}\times d}\):  
\[V=\frac{D_{\text{v}}^{T}f_{\text{v}}}{||D_{\text{f}}^{T}f_{\text{v}}||},\quad T =\frac{D_{\text{t}}^{T}f_{\text{t}}}{||D_{\text{t}}^{T}f_{\text{t}}||} \tag{1}\]  
Figure 1: Overview of our TIN template. We transmitted the original image and label templates to the vision-language backbone of the long-tailed data. Visual attention was used to assess long-distance dependencies, and the relevance of each channel was determined and reweighted to improve the visual features. The gradients were reweighted according to the label frequencies when the head class obtained several gradients. Finally, the accuracy of the tail class considerably improved while the accuracy of the head class slightly decreased.  
where \(V\) and \(T\) represent the normalized visual and textual features of the same dimension, respectively, and \(N\) represents the number of image-text pairs in the batch. Then, we optimize the two encodes via a contrastive learning loss, which can be formulated as:  
\[\begin{split}\mathcal{L}_{v-t}=&\mathcal{L}_{vis}+ \mathcal{L}_{text}\\ =&-\frac{1}{N}\sum_{i}^{N}\log\frac{exp\big{(}V_{i}^{T }T_{i}/\tau\big{)}}{\sum_{j=1}^{N}exp\big{(}V_{i}^{T}T_{j}/\tau\big{)}}\\ &-\frac{1}{N}\sum_{i}^{N}\log\frac{exp\big{(}T_{i}^{T}V_{i}/\tau \big{)}}{\sum_{j=1}^{N}exp\big{(}T_{i}^{T}V_{j}/\tau\big{)}}.\end{split} \tag{2}\]  
The overall training goal was to maximize the high similarity score and decrease the mismatch score when coping with matching losses from two separate directions. \(\mathcal{L}_{vis}\) and \(\mathcal{L}_{text}\) denote the loss of the visual and language sides, respectively, and \(\tau\) is the temperature parameter.