# Contrastive Enhanced Learning for Multi-Label Text Classification
## 3 Materials and Methods
### Contrastive Global Representation Enhancement
#### 3.3.1 Positive Label Representation Alignment  
In MLTC, the effectiveness of label information utilization is crucial for accurate classification. To fully leverage label information, it is essential that the global representation, \(G_{i}\), resides in the same space as the label embedding space, \(Y_{i}^{*}\). This alignment ensures that the model can effectively integrate label information when making classification decisions based on \(G_{i}\) and \(Y_{i}^{*}\).  
To achieve this, we propose a method called positive label representation alignment, which strengthens the connection between the global representation and the label embeddings. This approach not only enhances the utilization of label information but also promotes a unified representation space where the global and label-specific features coexist harmoniously, leading to more effective decision making during classification.  
Let \(A_{i}=\{Y_{m,i}^{*}\big{|}\,y_{m,i}=1,m\in\{1,2,\ldots,L\}\}\) represent the set of positive sample embeddings, where the embeddings vary depending on the input document, \(i\). The global representation of the document is denoted by \(G_{i}\). To encapsulate the label information effectively, we define a comprehensive positive label representation, \(H_{pos,i}\), as the average of the positive label embeddings:  
\[H_{pos,i}=\frac{1}{|A_{i}|}\sum_{Y_{m,i}^{*}\in A_{i}}Y_{m,i}^{*} \tag{2}\]  
Here, \(|A_{i}|\) represents the number of positive labels associated with document \(i\). This comprehensive positive label representation serves as a proxy for the overall label information associated with the document. By averaging the embeddings, we obtain a more stable and representative vector that encapsulates the collective information of all positive labels. As shown in Figure 1, the global representation, \(G_{i}\), of the document is derived from the [CLS] token output of the BERT model, which captures a contextualized summary of the entire input, integrating both the document and label tokens. Next, we aim to approximate this comprehensive positive label representation, \(H_{pos,i}\), with the global representation, \(G_{i}\), by enhancing their similarity.  
Since cosine similarity is in the range of \([-1,1]\), with larger values indicating higher similarity, directly using it is not aligned with our optimization direction. Therefore, we use cosine distance, defined as \(1-\cosine similarity\), to ensure positive values and to align the optimization direction with the overall loss function. The similarity measure is defined as follows:  
\[L_{diff,i}=1-G_{i}\cdot H_{pos,i}. \tag{3}\]  
Here, the symbol \(\cdot\) also denotes cosine similarity. As \(L_{diff,i}\) is minimized, the global representation, \(G_{i}\), and the comprehensive positive label representation, \(H_{pos,i}\), align in space and become closer to each other. By using the same measurement strategy as in the contrastive learning phase, we maintain consistency in how the model evaluates vector relationships across different components. This spatial consistency is crucial for enabling the model to make accurate classification decisions, as it ensures that the global representation is well aligned with the label-specific information. This approach not only strengthens the connection between document and label representations, but also contributes to the overall effectiveness of the classification process by ensuring that all relevant information is integrated coherently.