# Dual-Training-Based Semi-Supervised Learning with Few Labels
## 4 Experiments
### Ablation Study  
This subsection investigates the impact of different components in our method. We first investigated the effectiveness of our proposed dual training strategy for model performance. The experiments were conducted on Cifar100 with 800 labeled data. Since our proposed dual training strategy includes training with soft labels and hard labels, to verify the effectiveness of the dual training strategy, we alternately evaluated the method with only soft labels and only hard labels on Cifar100 with 800 labeled data, which can be seen in Table 5. It should be noted that the dual training strategy trains the model with data from two different kinds of strong data augmentation, while training with single labels only employs data augmented once. In addition, in this paper, training with hard labels means that we only use high-confidence samples selected by threshold. The method achieved an accuracy of 50.96% with only hard labels. The accuracy rose to 51.98% when the model was trained with soft labels instead of hard labels. However, it also showed a significant margin between training with a single kind of label and the dual training strategy. Next, we compared the performance of the model with different dual training strategies. These results are also presented in Table 5, which shows that the model trained by a dual training strategy with both hard and soft labels can reach the highest accuracy of 54.34%, which is 0.55% and 1.97% higher than using only hard labels or only soft labels, respectively. As such, a dual training strategy with both hard and soft labels is beneficial for improving model performance. During the training process, we found that using soft labels to train the model can accelerate the convergence of the model, which can be seen in Figure 5. However, training only with soft labels increases the risk of the model learning incorrect knowledge, resulting in poor performance at the end of the training. Using hard labels makes the model learn less information at the beginning of training but promotes the model's focus on more confident samples, thus reducing the risk of learning incorrect knowledge. Hence, we combined the hard labels and soft labels to train our model and achieved the greatest accuracy. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{3}{c}{**SVHN**} \\ \cline{2-4}  & **250 Labels** & **500 Labels** & **1000 Labels** \\ \hline \(\pi\)-Model [42] & 42.66 \(\pm\) 0.91 & 53.33 \(\pm\) 1.39 & 65.90 \(\pm\) 0.03 \\ Mean-Teacher [18] & 42.70 \(\pm\) 1.79 & 55.71 \(\pm\) 0.53 & 67.71 \(\pm\) 1.22 \\ MixMatch [19] & 92.12 \(\pm\) 0.06 & 94.53 \(\pm\) 0.43 & 95.13 \(\pm\) 0.04 \\ FixMatch [16] & 95.45 \(\pm\) 0.07 & 95.73 \(\pm\) 0.15 & 95.94 \(\pm\) 0.10 \\ ICL-SSL [24] & 95.58 \(\pm\) 0.14 & 95.80 \(\pm\) 0.12 & 96.05 \(\pm\) 0.14 \\ \(p\)-Mix [25] & 97.41 \(\pm\) 0.02 & 97.39 \(\pm\) 0.01 & 97.45 \(\pm\) 0.01 \\ Ours & **97.42 \(\pm\) 0.06** & **97.62 \(\pm\) 0.06** & **97.71 \(\pm\) 0.08** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison with state-of-the-art methods in test accuracy (%) on SVHN dataset with 250, 500 and 1000 labeled data. The bold represents the best result. \begin{table}
\begin{tabular}{c c} \hline \hline
**Method** & **Test Accuracy (\%)** \\ \hline Single hard labels & 50.96 \\ Single soft labels & 51.98 \\ Dual training with hard labels & 53.79 \\ Dual training with soft labels & 52.37 \\ Dual training strategy & **54.34** \\ \hline \hline \end{tabular}
\end{table} 
Table 5: The effectiveness of the dual training strategy on Cifar100 with 800 labeled data. The bold represents the best result. In this study, we employed the distribution alignment to further improve the accuracy of pseudo-labels of unlabeled data.