# Cras: Curriculum Regularization and Adaptive Semi-Supervised Learning with Noisy Labels
## 3 Proposed Method
FlexMatch performs curriculum pseudolabeling (CPL), which uses classwise thresholds that are set according to the learning difficulty, as formalized in (3), to select pseudolabels. Inspired by CPL, CR performs target selection in a manner that considers classwise learning difficulty. In the warm-up phase, if learning difficulty were measured using a threshold \(\tau\), similar to that of (1), a low threshold would be set for all classes because of the small number of targets that exceed the threshold, and regularization

Figure 1: Overview of the proposed CRAS method. The CR loss function \(\mathcal{L}_{CR}\) defined in (11) serves as a warm-up technique, training two networks. During the division phase, GMMs are employed at each epoch to divide the per-sample loss distribution into labeled and unlabeled sets, following the DivideMix strategy. In the figure, the labeled and unlabeled datasets used for model A are denoted by \(\mathcal{X}^{(A)}\) and \(\mathcal{U}^{(A)}\), respectively. Similarly, the labeled and unlabeled datasets used for model B are denoted by \(\mathcal{X}^{(B)}\) and \(\mathcal{U}^{(B)}\), respectively. In the SSL phase, the AWL function defined in (15) is incorporated into the MixMatch algorithm and controlled by adaptive weights \(\lambda_{a}\) in (14), which account for data bias. would apply to all samples. In CR, the above problem is solved without using additional parameters by setting a threshold as formalized in the following equation; this threshold measures the learning difficulty from the average predictions within the mini-batch. \[\tau_{avg}=\frac{1}{B}\sum_{b=1}^{B}\max_{c}\mathbf{t}_{b}^{(k)}. \tag{10}\]

The \(\tau\) in (1) is replaced by \(\tau_{avg}\), and the loss function for CR is expressed as follows. \[\mathcal{L}_{CR} =\mathcal{L}_{CE}+\frac{1}{B}\sum_{b=1}^{B}\mathcal{F}(\mathbf{t} _{b}^{(k)})\log\Big{(}1-\left\langle\mathbf{p}_{b},\mathbf{t}_{b}^{(k)}\right\rangle \Big{)}, \tag{11}\] \[\mathcal{F}(\mathbf{t}_{b}^{(k)}) =1\Big{(}\max_{c}\mathbf{t}_{b}^{(k)}>\tau_{\mathrm{argmax}_{b}^{ (k)}}\Big{)}. \tag{12}\]

### Adaptive Weighted Loss for Data Bias

In the SSL phase, a small number of learning samples per class or a high noise rate may cause an imbalanced division of learning data, with a larger set of unlabeled samples and a smaller set of labeled samples. This imbalance can contribute to overfitting and consequently hinder the generalization capability of the model. DivideMix solves this problem by increasing the regularization strength, using a large value of \(\lambda_{u}\) in (8), when the noise rate is high. However, the sizes of both the labeled and unlabeled data sets fluctuate according to the sample selection per epoch. This challenge, unique to the sample selection methods employing noisy labels, has not been addressed in prior SSL methods [42; 44] or the methods introducing them [25]. We propose an AWL that adjusts to learning data imbalances. When the learning data become heavily skewed towards the unlabeled sample set, and the labeled sample set becomes small, a larger weight is assigned to improve the generalization performance. Moreover, by adjusting the weights according to the balance of the learning data as it varies during the learning process, it becomes possible to set appropriate weights even when the noise rate is unknown, without the need to change the hyperparameters. A simple function that assigns a large weight when the proportion of labeled samples \(r\) is small, such as \(r^{-2}\), can be considered. However, when \(r\) is small, the weight becomes too large, and the supervised loss in the cross-entropy term is ignored. Therefore, we designed and applied a scaling function to control the divergence of weights, as follows. \[\mathcal{W}(r^{(k)})=s(r^{(k)})\cdot(r^{(k)})^{-2}, \tag{13}\]

where \(s\) represents a scaling function \(s(x)=mx^{n/2-m^{2}\log_{10}x}\), \(m\) is a hyperparameter, \(n=\log_{10}C\), and \(C\) is the number of classes.