# Contrastive Enhanced Learning for Multi-Label Text Classification
## 3 Materials and Methods
### Prediction  
After obtaining the enhanced global representation, \(G_{i}^{\prime}\), and the individual label representations, \(Y_{i}^{*}\), for a given sample, we proceed to combine these representations to predict the labels. Specifically, we add the enhanced global representation, \(G_{i}^{\prime}\), to each label representation, \(Y_{i}^{*}\), and then pass this combined vector, \(H\), through a series of fully connected layers to generate predictions:  
\[\hat{y}=sigmoid(WH). \tag{6}\]  
Here, \(W\) represents the trainable weight matrix of the fully connected layers, and \(H\) is the sum of both the global representation, \(G_{i}^{\prime}\), and the label representation, \(Y_{i}^{*}\). The sigmoid function is applied to the output to map the values into probabilities, effectively indicating the likelihood of each label being relevant to the document.  
We then use cross-entropy loss to measure the discrepancy between the predicted probabilities and the true label values, encouraging the model to produce accurate predictions. Specifically, let \(y_{ij}\) denote the true binary label value for the \(i\)-th document and the \(j\)-th label, where \(y_{ij}=1\) indicates that the label is relevant to the document and \(y_{ij}=0\) otherwise. The predicted probability of the \(j\)-th label being relevant to the \(i\)-th document is denoted by \(\hat{y}_{ij}\). The cross-entropy loss function, defined over a minibatch of size \(b\), is then as follows:  
\[L_{cls}=-\sum_{i=1}^{b}\sum_{j=1}^{L}\bigl{[}y_{ij}log(\hat{y}_{ij})+\bigl{(}1- y_{ij})log(1-\hat{y}_{ij})\bigr{]}. \tag{7}\]  
Finally, the overall loss function, which drives the training process, is a weighted combination of multiple components, each targeting a specific aspect of the model's performance. This loss function is applied to each minibatch during training, guiding the optimization process for every batch:  
\[L_{overall}=L_{cls}+\alpha_{1}L_{con}+\alpha_{2}L_{diff}+\alpha_{3}L_{pair} \tag{8}\]  
Here, \(L_{overall}\) and \(L_{pair}\) are defined over the entire minibatch, while the contrastive losses, \(L_{con}\) and \(L_{diff}\), are computed by summing their instance-level losses across all samples in the minibatch:  
\[L_{con}=\sum_{i=1}^{b}L_{con,i},\ L_{diff}=\sum_{i=1}^{b}L_{diff,i}. \tag{9}\]  
Thus, \(L_{overall}\) combines various types of losses. The following terms apply:  
* \(L_{cls}\) is the cross-entropy loss that focuses on improving classification accuracy.
* \(L_{con}\) is the contrastive loss designed to enhance label correlation by separating positive and negative label pairs in the embedding space.  
* \(L_{diff}\) is the loss that approximates the global representation to the comprehensive positive label representation, ensuring alignment in the embedding space.
* \(L_{pair}\) is the contrastive global representation learning loss, aimed at improving the sensitivity to tail labels by leveraging the relationships between head and tail labels.  
The terms \(\alpha_{1},\ \alpha_{2},\ \text{and}\ \alpha_{3}\) are hyperparameters that balance the influence of each component in the overall loss, allowing the model to jointly optimize for label prediction accuracy, label correlation, and representation alignment.