# Multi-Augmentation-Based Contrastive Learning for Semi-Supervised Learning
###### Abstract  
Semi-supervised learning has been proven to be effective in utilizing unlabeled samples to mitigate the problem of limited labeled data. Traditional semi-supervised learning methods generate pseudo-labels for unlabeled samples and train the classifier using both labeled and pseudo-labeled samples. However, in data-scarc scenarios, reliance on labeled samples for initial classifier generation can degrade performance. Methods based on consistency regularization have shown promising results by encouraging consistent outputs for different semantic variations of the same sample obtained through diverse augmentation techniques. However, existing methods typically utilize only weak and strong augmentation variants, limiting information extraction. Therefore, a multi-augmentation contrastive semi-supervised learning method (MAC-SSL) is proposed. MAC-SSL introduces moderate augmentation, combining outputs from moderately and weakly augmented unlabeled images to generate pseudo-labels. Cross-entropy loss ensures consistency between strongly augmented image outputs and pseudo-labels. Furthermore, the MixUP is adopted to blend outputs from labeled and unlabeled images, enhancing consistency between re-augmented outputs and new pseudo-labels. The proposed method achieves a state-of-the-art performance (accuracy) through extensive experiments conducted on multiple datasets with varying numbers of labeled samples. Ablation studies further investigate each component's significance.  
c 2024 17911.11.  
labeled and unlabeled input images, along with their augmented versions, as positive pairs. These consistency regularization-based methods in deep learning operate under the assumption that, even after applying data augmentation [35; 36; 37], the classifier should maintain consistent class probabilities for unlabeled samples, implying that the semantic content remains unchanged. The augmented versions of an input image should exhibit greater similarity to the original image compared to other unrelated images. To adhere to this assumption, researchers introduce perturbations to the input samples through data augmentation, thereby generating augmented samples that are similar to the original data.  
While the aforementioned algorithms have significantly contributed to improving learning accuracy in situations where labeled data are limited, they exhibit a notable decline in performance when confronted with a scarcity of labeled samples. This decline can be primarily attributed to the inability to fully leverage the informative content present in the unlabeled data, leading to an overreliance on the limited labeled samples.  
Different from approaches that typically enforce consistency between the model outputs of strongly augmented unlabeled images and weakly augmented unlabeled images (using pseudo-labels or soft labels), MAC-SSL introduces a moderate augmentation step, where the model outputs of moderately augmented unlabeled images collaborate with the model outputs of weakly augmented images to derive pseudo-labels for the unlabeled images. Subsequently, the consistency (contrastive loss) between the model outputs of strongly augmented images and the pseudo-labels is enforced. Furthermore, inspired by MixMatch [33], the MixUP [38] is adopted to combine the three different outputs of unlabeled images and the outputs of labeled images to obtain further augmentation, which ensures consistency (unsupervised loss) between the model outputs of the re-augmented images and the newly generated pseudo-labels derived from the mixed labels or pseudo-labels.  
The main contributions of this article are summarized as follows:  
1. A novel moderate augmentation technique is introduced, which is incorporated into two distinctive losses.
2. Numerous experiments demonstrate that MAC-SSL achieves state-of-the-art (SOTA) results across all standard benchmark datasets (Section 3.3).
3. The conducted ablation experiments illustrate the excellent performance of MAC-SSL (Section 3.4).