## 3 Methodology of TIN
### Loss Function  
The head class accounts for most of the samples in the long-tailed dataset, whereas the tail class accounts for only a minor portion. Therefore, the samples from the head category impose enormous negative gradients on the tail categories during training, swamping the positive gradients from the tail categories themselves. The classifier tends to deliver low responses to the tail categories to minimize the training loss incurred due to the uneven learning process. For the single-label classification problem, we revisited the concept and formulation of the cross-entropy loss function. Each sample can only have one label in some tasks, such as the ImageNet image classification task or the MNIST handwriting digit recognition dataset [18]. For a single sample, the loss function is calculated as follows:  
\[L_{ce}=-\sum_{i=1}^{n}y_{i}log\hat{y_{i}},\quad\hat{y_{i}}=\frac{e^{z_{i}}}{ \sum_{j=1}^{n}e^{z_{j}}}, \tag{5}\]  
where \(n\) is the label, \(y_{i}\) is the one-hot encoding, \(z=[z_{1},z_{2},\cdots,z_{n}]\) is the logit of each category, and \(\hat{y_{i}}\) is the predicted probability. \[\frac{\partial L_{ce}(\mathbf{z})}{\partial z_{i}}=\hat{y_{i}}-1 \tag{6}\]  
\[\frac{\partial L_{ce}(\mathbf{z})}{\partial z_{j}}=\hat{y_{j}} \tag{7}\]  
When \(i\) is the head label, and \(j\) is the tail label, the classifier applies a higher penalty to class \(j\) under the action of the loss function, and the prediction probability becomes massively skewed toward the head class. Figure 3: The standard large kernel convolution can be separated into three operations to decrease the number of computations, e.g., a depthwise convolution (DW-C), a depthwise division convolution (DW-D-C), and a pointwise convolution (PW-C). In the figure, blue is the center point, and green represents the range of convolution. By decreasing the weights of uncommon negative samples, the seesaw loss equalizes the gradient [32]. In this paper, we simplify the seesaw loss, and the formula is as follows:  
\[ \begin{split} L_{\textit{seesaw}}(\mathbf{z})=-\sum_{i=1}^{n}y_{i}log \hat{y_{i}},\\ \hat{y_{i}}=\frac{e^{z_{i}}}{\sum_{j\
eq 1}^{n}M_{ij}e^{\hat{z_{j}}}+ e^{z_{i}}+eps},\\ \frac{\partial L_{\textit{seesaw}}(\mathbf{z})}{\partial z_{j}}=M_{ ij}\frac{e^{z_{j}}}{e^{z_{i}}}\hat{y_{i}}\end{split}  \tag{8}\]  
where \(m_{ij}\) is the adjustment factor; \(eps=1\times 10^{-6}\) is used to prevent NaN values. The negative sample gradient of sample \(i\) to sample \(j\) can be expressed as Equation (9). Mitigation Factor: According to the collected samples, the seesaw loss can continually adjust the samples in each category. This work supplies the corresponding amount of data as prior knowledge to speed up model training. As shown in Equation (9), the adjustment factor decreases the penalty for negative samples in a type when the number of samples of type \(i\) is greater than the number of samples of type \(j\); otherwise, the adjustment factor is set as 1. The exponent \(p\) is a hyper-parameter that adapts the magnitude of mitigation. \[M_{ij}= \begin{cases}1,&\quad if\ N_{i}\leqslant N_{j}\\ \left(\frac{N_{j}}{N_{i}}\right)^{p},&\quad if\ N_{i}>N_{j}\end{cases}  \tag{10}\]  
During the ResNet training process on the clamp dataset [11], we can determine the distribution of the cumulative gradients of positive and negative samples imposed on each class of classifiers, as shown in Figure 4. The positive-to-negative sample gradient ratio is close to 1.0 for the rust category and is very small for the rare rust label. As the quantity of samples decreases, the classification accuracy dramatically decreases, impacting the model performance. However, the seesaw loss can obtain better outcomes by more effectively balancing the positive and negative gradient ratios. Figure 4: Under the traditional cross-entropy loss, the head class’s positive and negative sample gradients are close to 1.0 while the sparse tail class is tiny, which will cause an accuracy bottleneck.