# Contrastive Enhanced Learning for Multi-Label Text Classification
## 3 Materials and Methods
### Contrastive Global Representation Enhancement
#### 3.3.2 Contrastive Global Representation Learning for Long-Tail Labels  
In MLTC, a common challenge is the long-tail distribution of labels, where a few labels (head labels) have a large number of associated samples, while many others (tail labels) have only a few. This imbalance often leads to models that are biased towards head labels, resulting in poor performance on tail labels, which are typically more challenging to predict. To address this issue, we introduce a method called contrastive global representation learning, designed to enhance the model's sensitivity to tail label data by leveraging the relationships between head and tail labels. We aim to further enhance the expressiveness of \(G_{i}\) through this method.  
The core idea is to utilize the global representations of different samples within a minibatch as positive samples in a contrastive learning framework. By doing so, we aim to transfer the robust features learned from head labels to improve the representation and classification of tail labels. Given a minibatch of size \(b\), let \((X_{i},Y_{i})\) represent a specific instance within this minibatch, where \(X_{i}\) is the input document and \(Y_{i}\) is the corresponding set of labels. The global representation of this instance is denoted by \(G_{i}\). For each instance \(i\), we consider the remaining instances in the minibatch \(P=\{\big{(}X_{j},Y_{j}\big{)}|j\in\{1,2,...,b\},j\
eq i\}\). For one sample, \(\big{(}X_{j},Y_{j}\big{)}\in P,\text{ let }G_{j}\) represent its global representation.  
Inspired by the work in [20], we formulate the contrastive loss, \(L_{pair}\), as follows:  
\[L_{pair}=\sum_{i=1}^{b}\sum_{j=1,j\
eq i}^{b}-\beta_{ij}log\frac{e^{-(G_{i} \cdot\ C_{j})/\varphi}}{\sum_{k\in\{1,2,...,b\},k\
eq i}e^{-(G_{i}\cdot\ C_{ k})/\varphi}}. \tag{4}\]  
Here, \(\varphi\in\mathbb{R}^{+}\) is a temperature factor that controls the smoothness of the distribution, and the symbol \(\cdot\) denotes cosine similarity, which is used to measure the similarity between the global representations of different instances. The parameter \(\beta_{ij}\) is a weighting factor that accounts for the similarity between samples based on their labels. It is defined as follows:  
\[C_{ij}=Y_{i}^{\text{T}}Y_{j},\text{ }\beta_{ij}=\frac{C_{ij}}{\sum_{k\in\{1,2,...,b\},k\
eq i}C_{ik}} \tag{5}\]  
Here, \(C_{ij}\) calculates the number of identical labels between samples \(i\) and \(j\). The weighting factor \(\beta_{ij}\) thus reflects the degree of label overlap between the two samples, with a higher value indicating a greater similarity. By incorporating this weighting factor into the loss function, we ensure that the contrastive learning process places more emphasis on samples with similar labels, thereby reinforcing the learning of shared features between head and tail labels. Essentially, \(L_{pair}\) computes, in a weighted fashion, the contrastive loss for every pair of global representations in the batch as positive pairs, and then sums their losses. Therefore, it is computed from the perspective of the entire batch.  
This contrastive enhancement method serves a dual purpose. Firstly, it allows the model to leverage the rich information present in documents associated with head labels to improve the representation of documents with tail labels. This transfer of information is particularly beneficial in scenarios where tail labels are under-represented, as it enables the model to learn more robust features for these challenging cases. Secondly, by enforcing similarity between documents with shared labels, the model becomes better at capturing the subtle nuances that distinguish different labels, leading to more accurate classification outcomes.  
After the dual contrastive enhancement processes of positive label representation alignment and contrastive global representation learning, we enhance \(G_{i}\) to \(G_{i}^{\prime}\), which will ultimately be used for classification.