# Multi-Augmentation-Based Contrastive Learning for Semi-Supervised Learning
## 2 Method
### The Proposed MAC-SSL
It is computed as the mean squared error between the pseudo-label \(q^{u\prime}_{b}\) and the three model predictions \(p(q|u^{\prime\prime}_{wb})\), \(p(q|u^{\prime\prime}_{wb})\), and \(p(q|u^{\prime\prime}_{sb})\) as follows:  
\[\mathcal{L}_{u}=\frac{1}{\mu B}\sum_{b=1}^{\mu B}\big{(}\|q^{u\prime}_{b}-p(q \mid u^{\prime\prime}_{wb})\|_{2}^{2}+\|q^{u\prime}_{b}-p(q\mid u^{\prime \prime}_{mb})\|_{2}^{2}+\|q^{u\prime}_{b}-p(q\mid u^{\prime\prime}_{sb})\|_{2} ^{2}\big{)}, \tag{14}\]  
where \(u^{\prime\prime}_{wb}\), \(u^{\prime\prime}_{wb}\), \(u^{\prime\prime}_{sb}\in\mathcal{U}^{\prime\prime}_{w}\), \(\mathcal{U}^{\prime\prime}_{m}\), and \(\mathcal{U}^{\prime\prime}_{s}\) and \(p(q|u^{\prime\prime}_{wb})\), \(p(q|u^{\prime\prime}_{mb})\), and \(p(q|u^{\prime\prime}_{sb})\) are the model's predictions for \(u^{\prime\prime}_{wb}\), \(u^{\prime}_{mb}\), and \(u^{\prime\prime}_{sb}\), respectively. Figure 3: Diagram of MAC-SSL. Figure 4: Flowchart of the MAC-SSL process. The contrastive loss, \(\mathcal{L}_{c}\), is established between \(\mathcal{U}^{\prime}_{w}\), \(\mathcal{U}^{\prime}_{m}\), and \(\mathcal{U}^{\prime}_{s}\). The object in this case is to maintain a certain level of strength in the optimization of the model on \(\mathcal{U}^{\prime}_{s}\). Therefore, a threshold, \(\tau\), is employed to \(p^{u}_{b}\), and instead of converting \(p^{u}_{b}\) into pseudo-labels, the predicted distribution's state is directly utilized to constrain the model's output, \(p(q|u^{\prime}_{sb})\), which is also a predicted distribution, on \(\mathcal{U}^{\prime}_{s}\) using cross-entropy. \(\mathcal{L}_{c}\) can be calculated as follows:  
\[\mathcal{L}_{c}=\frac{1}{\mu B}\sum_{b=1}^{\mu B}f(\max(p^{u}_{b})\geq\tau)H( p^{u}_{b},p(q|u_{sb^{\prime}})), \tag{15}\]  
where \(u^{\prime}_{sb}\in\mathcal{U}^{\prime}_{s}\) and \(p(q|u^{\prime}_{sb})\) are the model's predictions for \(u^{\prime}_{sb}\), \(H\) denotes the cross-entropy between the distribution \(p^{u}_{b}\) and \(p(q|u^{\prime}_{sb})\), and \(f(\bullet)\) represents 1 if the inequality in the parentheses holds (and it is otherwise 0). By combining the aforementioned loss functions, the loss function of MAC-SSL is defined as follows:  
\[\mathcal{L}=\mathcal{L}_{x}+\mathcal{L}_{xm}+\lambda_{u}\mathcal{L}_{u}+ \lambda_{c}\mathcal{L}_{c}, \tag{16}\]  
where \(\lambda_{u}\) and \(\lambda_{c}\) are trade-off hyper-parameters that control the weights of the unsupervised loss \(\mathcal{L}_{u}\) and the contrastive loss \(\mathcal{L}_{c}\), respectively. This design takes into account the different characteristics of the data by comprehensively utilizing different types of data and loss functions, enabling the model to learn the features of the data more comprehensively and, thus, improve its performance. Specifically, the benefits of such comprehensive utilization are reflected in the following aspects:  
1. Integration of information from different types of data: This model combines labeled and unlabeled data, applying different types of loss functions to both. \(\mathcal{L}_{x}\) and \(\mathcal{L}_{xm}\) utilize the true labels of labeled data, \(\mathcal{L}_{u}\) utilizes pseudo-labels of unlabeled data, and \(\mathcal{L}_{c}\) performs contrastive learning on unlabeled data. By comprehensively utilizing these different types of data and loss functions, the model can learn the features of the data more comprehensively, thereby improving its performance. 2. Enhancing the model's generalization ability: Integrating different types of loss functions can improve the model's generalization ability. \(\mathcal{L}_{x}\) and \(\mathcal{L}_{xm}\) help the model learn accurate classification decisions on labeled data, while \(\mathcal{L}_{u}\) and \(\mathcal{L}_{c}\) can help the model utilize information from unlabeled data to enhance its generalization ability. 3. Strengthening the model's understanding of the data: Different types of loss functions allow the model to understand the data from different perspectives.