# DYMatch: Semi-Supervised Learning with Dynamic Pseudo Labeling and Feature Consistency
## 1 Introduction  
In the field of computer vision, deep learning methods have consistently been at the forefront, and their superior performance heavily relies on the availability of enough accurately annotated labeled data [1, 2]. However, the acquisition of enough labeled data is a labor-intensive and expensive task. Now, there are various methods available to address such issues, such as weakly supervised learning [3, 4], zero-shot learning [5], and semi-supervised learning [6]. Among them, semi-supervised learning offers an effective approach by employing a few labeled data and a large amount of unlabeled data for training. It shows great potential in practical applications and can significantly reduce the dependence of training on laborious data collection and data annotation [7, 8, 9, 10, 11, 12]. The main challenge in SSL lies in effectively utilizing unlabeled data to improve the model's generalization performance [6]. And consistency regularization [13, 14, 15] and pseudo-labeling [16, 17] represent two most widely utilized paradigms designed for SSL, with their combined approaches also demonstrating good performance [9, 10, 18, 19]. The key idea underlying these methods is grounded in the assumption that the training process of the SSL model should follow the low-entropy assumption, that is, the model should maintain the consistency in predictions when different perturbations are applied to the same unlabeled data [6]. However, a potential limitation in employing the pseudo-labeling methods to address this problem is the requirement to set a fixed confidencethreshold [7; 8; 9; 18] or implement a stage-wise threshold adjustment method [20] for the selection of suitable pseudo labels. In pseudo-labeling methods designed with a fixed confidence threshold, the model selects unlabeled data with prediction scores surpassing the threshold for training and directly discard those falling below the threshold. For example, UDA [18] and FixMatch [9] employ a fixed high-confidence threshold to maintain the quality of the chosen pseudo labels. However, during the early training stages, a fixed high-confidence threshold (e.g., 0.99) may exclude too much unlabeled data from training, resulting in a large amount of unlabeled data being left unused. This approach overlooks the fact that, in the early training stages, the model has a limited ability to distinguish unlabeled samples, making it challenging to generate high-confidence predictions. An enhanced approach for the fixed threshold is to employ a stage-wise threshold adjustment strategy. For example, Dash [20] and AdaMatch [21] opt to gradually increase the threshold as the training progresses. The utilization of unlabeled data in such methods demonstrates an improvement compared to methods using a fixed confidence threshold. However, the performance of the stage-wise threshold adjustment strategy is directly influenced by the pre-defined hyper-parameters, and the setting of hyper-parameters is largely decoupled from the model's training process in most cases. This is especially notable in disregarding the varying difficulties associated with learning data from different categories. Furthermore, optimal hyper-parameters have to be researched on different datasets. FlexMatch [10] attempts to employ different local (class-based) thresholds for samples from different categories. Although the setting of local threshold takes into account the different learning status of the model for different categories, it is still derived from a pre-defined fixed global (dataset-based) threshold. In essence, these methods fundamentally ignore the crucial effect of the model's training process on the pseudo-label estimation, whether for global or local thresholds. Essentially, the core requirement for a pseudo-label estimation method is to maintain a balance between the quantity and quality of pseudo labels during the training process.