# DYMatch: Semi-Supervised Learning with Dynamic Pseudo Labeling and Feature Consistency
## 4 Experiments
### Experimental Settings  
The baseline models employed for the comprehensive performance comparison encompassed Mean Teacher [27], Pseudo-Label [16], MixMatch [7], ReMixMatch [8], UDA [18], FixMatch [9], FlexMatch [10], DoubleMatch [23], FeatMatch [22], Meta pseudo-labelling [19], SimMatch [47], Semi-Clustering [48], SoftMatch [11] and FreeMatch [12]. The Mean Teacher and Pseudo-Label, respectively, employed the consistency regularization method and the pseudo-labeling method, which have become widely used in the SSL approach. MixMatch and UDA both explore the impact of different data augmentation methods on SSL. Based on MixMatch, ReMixMatch suggests that achieving a balance in the distribution of prediction scores can significantly enhance the model's classification performance. FixMatch stands out as an attempt to combine consistency regularization and pseudo-labeling methods, resulting in good classification performance. DoubleMatch and FeatMatch innovatively refined the consistency regularization method, taking inspiration from FixMatch. Meanwhile, FlexMatch also enhanced the pseudo-labeling method upon the foundation laid by FixMatch. SoftMatch and FreeMatch take into account the impact of the model's training status for the confidence threshold. Meta pseudo-labeling introduced an innovative training diagram. SimMatch and Semi-Clustering strive to integrate methods from other fields, including self-supervised and deep clustering strategies, into FixMatch. The hyper-parameters of all baseline methods are consistent with those in their corresponding papers.  
To ensure a fair comparison, we followed the guidelines recommended in [49] and shared the same training pipelines for the same dataset, which include optimizer, learning rate decay schedule and the backbone module. In our experiments on all benchmark datasets, the model was trained employing the SGD optimizer. The momentum and weight decay in SGD were set to 0.9 and 1 \(\times\) 10\({}^{-3}\), respectively. The Nesterov method was not utilized in SGD. The learning rate was initialized at 0.02 and gradually decreased employing the cosine annealing scheduler. The Wider ResNet network [50] was employed as the backbone module. In detail, we employed Wider ResNet-28-2 for CIFAR-10, SVHN and the domain adaptation datasets. For the CIFAR-100 and STL-10 datasets, the Wider ResNet-28-8 and the Wider ResNet-37-2 networks are employed as the backbone, respectively. Moreover, we show the average and the standard deviation of the results for each model by trained three times on each number of labeled data on each dataset.  
DYMatch involves five hyper-parameters: the dimension of the queue in the dynamic local threshold estimation (\(Q\)), the momentum decay in the EMA method (\(\lambda\)), the dimension of the output feature dimension in linear projection module \(h(\cdot)\) (\(d\)) and the relative weight hyper-parameters (\(\mathcal{L}_{pl}\) and \(\mathcal{L}_{cr}\)). In the process of training, \(\lambda\) and \(Q\) were set as 0.999 and 100, respectively. In most cases, \(\lambda_{pl}\) and \(\lambda_{cr}\) were set to 2.0. However, when only a minimal amount of labeled data is available (i.e., 40 labeled data for CIFAR-10, SVHN and STL-10 datasets, 400 labeled data for CIFAR-100), \(\lambda_{pl}\) and \(\lambda_{cr}\) were adjusted to 3.0. This means that in these situations with limited labeled data, we should pay more attention to the constraints for the training on the unlabeled data. Moreover, the different backbones were employed for the different datasets. Thus, for CIFAR-10 and SVHN datasets, the \(d\) in \(h(\cdot)\) was set to 128. In CIFAR-100, the parameter \(d\) was set to 512, and for STL-10, the parameter \(d\) was set to 256. The labeled batch size \(N_{L}\) is set to 64. We set the \(\sigma\) as 7, which means that the unlabeled batch size \(N_{U}\) is set to 7 times of \(N_{L}\) for all benchmark datasets.