# DYMatch: Semi-Supervised Learning with Dynamic Pseudo Labeling and Feature Consistency
## 1 Introduction
The methods utilizing a high fixed threshold explicitly abandon the requirement for the quantity, concentrating solely on generating high-quality pseudo labels. However, when the labeled data are extremely rare, the challenge with the high fixed threshold lies in its difficulty in ensuring the quality of pseudo labels. Conversely, the stage-wise threshold adjustment method is equivalent to compromising the quality of pseudo labels in favor of increasing their quantity. Such a strategy is susceptible to introducing incorrect pseudo labels, leading to the cognitive bias during the training process. Therefore, a novel dynamic pseudo-labeling estimation method based on Gaussian mixture model is proposed in this paper. We assume that the prediction scores for a specific category in a training iteration are sampled from a Gaussian mixture distribution comprising two distributions, namely the positive and negative distributions. This assumption can be extended to the whole unlabeled data. By solving for this Gaussian mixture model, the model can assign each category a local threshold that aligns with the current training status. Specifically, for a given category, we consider the prediction scores of this category as the weighted sum of the positive and negative distribution, and employ the maximum likelihood estimation to predict the parameters of each Gaussian distribution. Consequently, the dynamic pseudo-label estimation method can adaptively generate the optimal local threshold at different training iterations so as to stabilize the quantity and quality of pseudo labels. Intuitively, during the early training stages, the dynamic pseudo-label estimation method employs a relatively low threshold to encourage the model to train on more unlabeled data and accelerate the convergence. As the classification performance improves and the prediction confidence increases, a higher threshold is generated to filter out incorrect pseudo labels, alleviating the cognitive bias of the model. Inspired by previous researches [22; 23], we recognize the significance of constraining the training process of feature representation in SSL tasks. Therefore, in DYMatch, we also propose a feature-correlation consistency regularization method, and its integration with the dynamic pseudo-label estimation method can effectively enhance the model'sclassification performance. Unlike other feature-constraint methods, the feature-correlation consistency regularization method is selectively applied to the unlabeled data with prediction scores exceeding the confidence threshold generated by the dynamic pseudo-label estimation method. Moreover, we implement feature-correlation consistency regularization and dynamic pseudo-label estimation at different layers in the model, thereby effectively avoiding the influence of the strong coupling between these two methods on the model's performance. The paper is structured as follows. Section 2 reviews the common pseudo-labelling and consistency regularization methods in SSL, and also introduces the relevant background on Gaussian mixture model and its applications in SSL. Section 3 details the motivation and implementation of the DYMatch. In Section 4, we present the results of the comparative experiments and ablation studies. The paper is ultimately summarized in Section 5.