# Assembly Quality Detection Based on Class-Imbalanced Semi-Supervised Learning
## 2 Class-Imbalanced Semi-Supervised Learning
### Certainty Driven Selection  
The deficiency of existing perturbation-based semi-supervised learning methods is that all the outputs are regularized without exception. A large part of the outputs can be unreliable due to the confirmation bias [26]. Confirmation bias results from incorrect predictions of unlabeled data used in subsequent training, increasing the confidence of wrong predictions and making the model resist new changes. In this case, maintaining the consistency regularization will result in the student model converging incorrectly. In the absence of labeled targets as supervision, evaluating the certainty of teacher predictions, and filtering out low-certainty samples is necessary to ensure that the consistency constraints only apply to high-certainty samples.  
The certainty driven selection method is shown in Figure 4. It is assumed that the teacher network has \(H\) layers, with the parameters set \(\theta^{t}=\{\Phi_{h}\}_{h=1}^{H}\) determined by limited random variables, and \(\Phi_{h}\) represents the parameters of layer \(h\). For sample \(x_{i}\), the predicted distribution \(q(\hat{y}_{i}^{t}|x_{i})\) of the teacher is approximated as [30]  
\[q(\hat{y}_{i}^{t}|\ x_{i})=\int p(\hat{y}_{i}^{t}|x_{i},\theta^{t})q(\theta^{t })d\theta^{t}, \tag{6}\]  
where \(p(\hat{y}_{i}^{t}|x_{i},\theta^{t})\) is the prediction probability based on the input data \(x_{i}\) and model parameters \(\theta^{t}\), and \(q(\theta^{t})\) is the posterior distribution of model parameters, which cannot be obtained directly but can be estimated by dropout variational inference. Dropout variational inference is a practical method for approximating large and complex models [31]. Dropout is applied to every weight layer both in the training and the testing phase. The inference is obtained by sampling the approximate posterior, also referred to as Monte Carlo dropout. In addition, [PERSON] et al. [32] believe that a prediction with high certainty should be consistent under randomly sampled subnetworks and random noise in inputs. Assuming that the set of sub-sampling results of \(K\) random enhancements of input data is \(Y=\left\{\hat{y}_{i,k}^{t}(x_{i},\theta_{k}^{t},\eta_{k}^{t})\right\}_{k=1}^{K}\), and the prediction variance (PV) [30] is used to measure the uncertainty of the prediction. The higher the variance, the higher the uncertainty:  
\[\begin{split}\text{U}(\text{x}_{i})&=PV=\sum\limits_ {c}Var[p(\tilde{y}_{i,1}^{t}=c\Big{|}\text{x}_{i},\theta_{1}^{t},\eta_{1}^{ \prime}),\cdots,p(\tilde{y}_{i,K}^{t}=c\Big{|}\text{x}_{i},\theta_{K}^{t},\eta _{K}^{t})]\\ &=\sum\limits_{c}(\frac{1}{k}\sum\limits_{k}(p(\tilde{y}_{i,k}^{ t}=c\Big{|}\text{x}_{i},\theta_{k}^{t},\eta_{k}^{t})-\mu_{c})^{2})\\ &\text{where }\mu_{c}=\frac{1}{k}\sum\limits_{k}p(\tilde{y}_{i,k}^{ t}=c\Big{|}\text{x}_{i},\theta_{k}^{t},\eta_{k}^{t})\end{split} \tag{7}\]For input data \(x_{i},i\in\{1,\cdots,B\}\) of the mini-batch, the uncertainty values of teacher predictions are \([U(x_{1}),\cdots,U(x_{B})]\). The inputs are sorted in ascending order of the uncertainty to form the ordered input set \(\{P_{1},\ldots,P_{B}\}\). The reliable input samples set \(M=\{P_{1},\ldots,P_{m}\}\) contains \(m\) lowest uncertainty samples chosen from the ordered input set and is used for the consistency constraints. \(m=\min(\beta e,B),e\) as the epoch, and \(\beta\) as the ramp-up coefficient. The number of samples selected by certainty will increase over time, as the teacher predictions will become more accurate during training.