# DYMatch: Semi-Supervised Learning with Dynamic Pseudo Labeling and Feature Consistency
## 3 DYMatch
### Motivation of Dynamic Pseudo-Label Estimation Method  
The core idea of the pseudo-labeling method is to integrate more unlabeled data into the training set by employing appropriate confidence threshold, thereby enriching the distribution of the training set and effectively improving the classification and generalization performance of the model. In this section, inspired by [PERSON] et al. [12], we consider the model's prediction scores for a specific category as a binary classification problem. The analysis of this binary classification problem provides motivation for the design of dynamic pseudo-label estimation method. Figure 1: A diagram of the training process of unlabeled data in DYMatch. Firstly, the strongly and weakly augmentations are used to generate two different visions of the unlabeled data. The feature maps \(h\big{(}f\left(\widetilde{u}\right)\big{)}\) and \(f\left(\widetilde{u}^{\prime}\right)\) are used to compute the loss of the feature-correlation consistency regularization method. And the \(Q_{i}\) denotes the pseudo labels selected by the dynamic pseudo-label estimation method based on the Gaussian Mixture Model. Please see more details in our method section below. Assuming a binary classification problem, where the true distribution is a uniform mixture of two Gaussian distributions, that is, the sample set \(Xs\) contains positive samples (\(+1\)) and negative samples (\(-1\)). The input data \(X\) follows the following conditional distribution:  
\[X|\mathrm{Y} =-1\sim N\big{(}\mu_{1},\sigma_{1}^{2}\big{)} \tag{3}\] \[X|\mathrm{Y} =+1\sim N\big{(}\mu_{2},\sigma_{2}^{2}\big{)}\]  
Furthermore, we assume that \(\mu_{2}>\mu_{1}\). Given that binary classification models often employ a \(Sigmoid(\cdot)\) function as the output activation function, the prediction score of the model can be defined as:  
\[s(x)=\frac{1}{1+\exp^{-\beta(x-\frac{\mu_{1}+\mu_{2}}{2})}} \tag{4}\]  
where \(\beta\) is used to denote the current training status of the model. Typically, during the training process, the model should gradually become more confident for its classification performance, so \(\beta\) is a gradually increasing positive parameter. \((\mu_{1}+\mu_{2})/2\) is the Bayes optimal linear decision boundary. In the early training stages, \(\beta\) is relatively small. The decision boundary is around the input data \(x\), and \(s(x)\) may be closer to 0.5, indicating that the model fails to generate a confident prediction for the input data \(x\). As the model becomes more confident, \(\beta\) is expected to gradually grow during training, and the decision boundary gradually moves away from the input data \(x\). When \(x-(\mu_{1}+\mu_{2})/2\) is positive and large enough, \(s(x)\) tends towards 1. In this case, the input data \(x\) is viewed as a positive sample. And when \(x-(\mu_{1}+\mu_{2})/2\) is negative and its absolute value is large enough, \(s(x)\) will approach 0. At this time, the model assigns the input data \(x\) to the negative class. Based on the above analysis, we can employ a fixed threshold \(\eta\in(0.5,1)\) to partition the prediction scores. The input data \(x\) will be assigned as pseudo label 1, when \(x\) satisfies the following conditions:  
\[s(x)=\frac{1}{1+\exp^{-\beta(x-\frac{\mu_{1}+\mu_{2}}{2})}}>\eta \tag{5}\]  
And the Equation (5) can be rewritten as:  
\[x>\frac{\mu_{1}+\mu_{2}}{2}+\frac{1}{\beta}log(\frac{\eta}{1-\eta}) \tag{6}\]  
Likewise, \(x\) will be assigned as pseudo label \(-1\) if:  
\[s(x)=\frac{1}{1+\exp^{-\beta(x-\frac{\mu_{1}+\mu_{2}}{2})}}<1-\eta \tag{7}\]  
and the Equation (7) is equivalent to:  
\[x>\frac{\mu_{1}+\mu_{2}}{2}-\frac{1}{\beta}log(\frac{\eta}{1-\eta}) \tag{8}\]  
Finally, when \(1-\eta\leq s(x)\leq\eta\), the model cannot generate a high-confidence prediction for the input \(x\). Generally, the utilization of unlabeled data is directly affected by the confidence threshold \(\eta\). Specifically, as the confidence threshold \(\eta\) increases, the utilization of unlabeled data gradually gets lower.