# Cras: Curriculum Regularization and Adaptive Semi-Supervised Learning with Noisy Labels
## 3 Proposed Method  
In this section, we introduce CR to enhance warm-up performance in the presence of noisy labels, and AWL, which adjusts to the bias of the learning data for adaptive SSL. The CRAS method incorporates both CR and AWL. By implementing CR in the warm-up phase of sample selection and incorporating AWL into the loss function in the SSL phase, we emphasize that the CRAS framework can be easily integrated with existing methods such as DivideMix. Algorithm 1 presents the pseudocode for CR, and Figure 1 provides an overview of the CRAS method. ```
0: batch size \(B\), training data \(N=\{(\mathbf{x}_{i},\mathbf{y}_{i}):i\in(1,...,B)\}\), temporal ensembling momentum \(\beta(0\leq\beta\leq 1)\), weight averaging momentum \(\gamma(0\leq\gamma\leq 1)\), regularization parameter \(\lambda\), mixup hyperparameter \(\alpha\), confidence threshold \(\tau\), network parameters \(\Theta_{1},\Theta_{2}\)
1:\(\mathbf{t}_{1},\mathbf{t}_{2}=\mathbf{0}_{[B\times\mathbb{C}]},\mathbf{0}_{ [B\times\mathbb{C}]}\)\(\triangleright\) set initial average predictions
2:\(\Theta_{1},\Theta_{2}=\mathbf{0},\mathbf{0}\)\(\triangleright\) set initial average weights
3:while\(e\leq\) WarmupEpoch do
4:\(\hat{u}_{i}=-1:i\in(1,...,B)\)\(\triangleright\) set initial predictions for all data
5:for\(m\) in [1,2]do
6:\(\bar{B}=\text{mixup}(B,\alpha)\)\(\triangleright\) apply mixup augmentation to the mini-batch
7:\(\bar{\Theta}_{m}=\gamma\bar{\Theta}_{m}+(1-\gamma)\Theta_{m}\)\(\triangleright\) apply weight averaging
8:for\(b\) in \(B\)do
9:\(\mathbf{t}_{b}^{(e)}=\beta\mathbf{t}_{b}^{(e)}+(1-\beta)\mathbf{p}_{b}^{(e)}\)\(\triangleright\) apply temporal ensembling
10:for\(c=1\) to \(C\)do
11: Calculate \(\sigma_{c}\) using (1) and (10)\(\triangleright\) compute estimated learning effect
12:if\(\max\sigma_{c}^{(c)}=\sum_{b=1}^{B}1(\text{argmax}\,\hat{u}_{b}=-1)\)then
13:\(\beta_{c}=\sigma_{c}^{(e)}/\max\{\max_{c}\,\sigma^{(e)},B-\sum_{c}^{C}\sigma _



### Curriculum Regularization

Methods such as DivideMix, which combine sample selection with SSL, necessitate a network warm-up period for dividing the training data. DivideMix exploits the memorization effect to warm up the network with all the training data before dividing them into clean and noisy sets, while preventing the network from fitting noisy labels. The pre-noisy-label-fitting network is then used to divide the training data. However, when the number of samples per class is small, or the noise rate is high, the model may fit the noisy labels before completing adequate training. Our proposed CR is based on ELR+, which functions as a robust warm-up technique for noisy labels. ELR+ uses a regularization term that uses targets generated from the predictions made during the early-learning stage of the network that represent the true class; these targets cancel the loss gradient for samples for which the network's prediction and the label do not match. DNNs have the property that they learn easy samples first and then hard samples later, even when trained on a dataset that contains only correct labels [13]. Thus, the network outputs low-confidence predictions for hard samples. In ELR+, low-confidence targets are set for hard samples, canceling the loss gradient for these samples and hindering learning. CR takes a different approach from that of FixMatch: it sets a threshold for targets instead of pseudolabels. By performing regularization using only reliable targets (those that exceed the threshold), it solves the problem of the suppressed learning of hard samples. However, setting a high threshold during the warm-up stage results in no targets exceeding the threshold, leading to regularization not being applied and the network adapting to noisy labels. To solve this problem, it is necessary for the threshold to be tailored to the learning situation of the samples. Moreover, it has been reported that the learning difficulty of samples varies between classes [46]. The learning status of the network is affected by this classwise learning difficulty.