Paper: Bi-Level Optimization for Pseudo-Labeling Based Semi-Supervised Learning (ICLR-2024)

Reviewer comment (Theory):
The convergence of the proposed method is not revealed either theoretically or empirically. I think both theoretical and empirical analysis can be done to analyze the convergence problem. Similar theoretical analysis of convergence can be found in many bi-level optimization papers, and introducing it can improve the paper. Besides, it is also important to present the convergence figure of parameters, i.e. pseudo-labels in this paper. For example, the empirical study of the convergence property can be referred to Shu et al. (2019).

Author response:
* **About convergence analysis**

Thanks for the suggestion. We added empirical convergence analysis to Appendix D of the revised paper.

Reviewer comment (Experiments):
Since the approach explicitly introduces the validation data sets in the training phase, it is not clear whether the comparison with the previous method is still fair. Previous methods at most used the validation set for tuning hyperparameters. Authors should explicitly discuss this issue.

Author response:
* **About validation data and fair comparison**

We **did not** introduce separate validation datasets. The term of “validation loss” in the paper refers to the outer loss, which is used to assess/validate the quality of pseudo-labels through the model parameters. It is computed on the whole training set—the labeled training data $\mathcal{D}^l$ and the unlabeled training data $\mathcal{D}^u$ for SSL (please see Eq.(3), Eq.(4) and Eq.(5)), which are available for all the SSL methods. We did not introduce validation datasets.  We used **the same data splits and inputs** as previous studies, ensuring fair comparisons.

Reviewer comment (Experiments):
Is the comparison with previous SSL methods still fair?

Author response:
* **About validation data and fair comparison**

We **did not** introduce separate validation datasets. The term of “validation loss” in the paper refers to the outer loss, which is used to assess/validate the quality of pseudo-labels through the model parameters. It is computed on the whole training set—the labeled training data $\mathcal{D}^l$ and the unlabeled training data $\mathcal{D}^u$ for SSL (please see Eq.(3), Eq.(4) and Eq.(5)), which are available for all the SSL methods. We did not introduce validation datasets.  We used **the same data splits and inputs** as previous studies, ensuring fair comparisons.

Reviewer comment (Experiments):
The pseudo-labels are simply initialized by the initial teacher model parameters and the update process is impacted by various factors, including the learning rate α, linear combination weight γ and the EMA based teaching models. Consequently, the update of the pseudo-labels tends to be slow. Are there alternative methods that can be used to initialize pseudo-labels to expedite the convergence of model training, such as employing warm-up with labeled data or using mean soft labels (1/num_classes)?

Author response:
As the initial pseudo-labels serve as the starting point for the bi-level optimization, it is beneficial to start with good initial pseudo-labels to accelerate the training process. That is the very reason we proposed to pre-train teacher and student models on the training data (labeled and unlabeled) using the Mean-Teacher approach and use them as the initial teacher and student models (please see the description under Eq.(1) in the paper). The pseudo-labels are then initialized by the pre-trained initial teacher model, which are much more informative than using mean soft labels.  One may use other pre-training methods to provide the initial pseudo-labels. But our choice of pre-training with the Mean-Teacher approach can conveniently provide the initial student and teacher model parameters as well.  In the experiments, our proposed method substantially outperforms the Mean Teacher method.

Reviewer comment (Presentation):
In the Algorithm 1, the update of $\theta^{t+1}_{S}$ in the last 3rd line is equivalent to the 7th line and it seems to be redundant.

Author response:
The 7th line in Algorithm 1 updates the model parameters $\underline{\text{on the current minibatch}}$ using $\underline{\text{pseudo-labels in the current step}}$, making the model ready to calculate the outer loss, while the last 3rd line from the end employs $\underline{\text{updated pseudo-labels}}$ for the final update on the model parameters $\underline{\text{on } \mathcal{D}^u}$ at the end of the iteration. As a result, each line serves a distinct function in the model's iterative refinement process.

Reviewer comment (Experiments):
In Table 1, there exists some unreasonable data, which is different from the original paper. For example, in ReMixMatch[3], the test error of 6.27±0.34 for 250 labels and 5.14±0.04 for 4000 labels in CIFAR-10 in original paper differs from the values presented in this paper. Is it because the experimental setup is different from the original paper? Furthermore, in BOPL, the test error for 250 labels is lower than that for 1000 labels in CIFAR-10, which seems to contradict common sense. If these results are accurate, it could imply that BOPL may not be the top-performing method under this setting. According to common expectations, SimMatch's [4] results at 1000 labels should fall within the range of 4.84 and 3.96, while BOPL's reported result of 5.12 seems less favorable by comparison.

Author response:
We cited the ReMixMatch’s results reported in the SimMatch paper since our experimental setup is the same as the SimMatch paper. 
BOPL  produced smaller errors with 250 labels than 1000 labels. It could be due to poor local optima. Our full approach, BOPL+ICT overcomes this issue, achieving errors 4.12 (for 250 labels) and 4.02 (for 1000 labels). 
As for comparing SimMatch with 1000, we compared our method with the reported results of SimMatch. SimMatch’s result for 1000 labels on CIFAR-10 was not reported.

Reviewer comment (Experiments):
Compared with existing SSL methods, it's possible that the experiments lack some widely used and important settings, particularly in more challenging conditions. It is suggested to validate the effectiveness of the BOPL approach under the setting of 40 labels in CIFAR-10, 400 labels in CIFAR-100 and 250 labels in SVHN, following the setups of FixMatch[5] and SimMatch[4]. Meanwhile, the results of BOPL+ICT are missing from the main text, please provide additional experimental data.

Author response:
We included additional experimental results as well as the BOPL+ICT results in Table 5 and Table 6 in the appendix. We have added the BOPL+ICT results in the main text of the revised paper.

Reviewer comment (Evaluation):
The approximation in Eq. 14 still requires two parameter updates to compute the gradient; this must be slow.

Author response:
Firstly, in Eq.(14), the computation of the parameters $\theta_S^+$ and $\theta_S^-$ actually only requires one **single** standard gradient calculation on the outer loss $\mathcal{L}\_\text{outer} $  for $\delta{\theta\_S^{t+1}}$. The extra cost for computing $\nabla\_{\widehat{\bf y}\_i} \mathcal{L}\_\text{outer}$  is only on computing two predictions on one  unlabeled instance ${\bf x}^u\_i$ using ${\bf \theta}\_S^+$ and ${\bf \theta}\_S^-$ respectively. Therefore, the extra computational cost over standard gradient descent is not significant. Secondly, direct computation of the target gradient $\nabla\_{\widehat{\bf y}\_i} \mathcal{L}\_\text{outer}$  is very difficult. The approximation in Eq.(14) provides a convenient solution in this scenario.

Reviewer comment (Presentation):
Presentation could be improved. Prop. 1 is just a nice expression for the chain rule - consider combining with proposition 2 or moving it to the Appendix? I also think the description in Alg. 1 could be simplified by writing only the steps critical to the algorithm, e.g. no need to write "compute loss L_{inner}" which doesn't really help with understanding. Or if the authors feel that it's needed, write the expression again in the algorithm description to make it self-contained.

Author response:
Thanks for the suggestions. We use two separate propositions to enhance clarity in illustrating the rationale behind the two separate key steps. Aligning the loss terms in the algorithm with their corresponding equations is also intended to ensure clarity and avoid confusion.

Reviewer comment (Novelty):
The novelty of this method seems incremental - while it's practically useful, the only real contribution is to write the parameters $\theta_S$ as a function of $Y$ with a cross-entropy / entropy loss, from which the method naturally follows. However it's not clear why this method results in improved performance.

Author response:
The proposed BOPL approach provides a novel perspective and an effective solution to semi-supervised learning by treating pseudo-labels as latent variables and formulating pseudo-labeling as a bi-level optimization problem. Bi-level optimization offers a systematic optimization framework for coherently learning two sets of inter-dependent variables through a pair of bi-level objectives.  Our bi-level optimization formulation allows us to optimize the pseudo-labels interdependently with the model parameters in a coherent and simultaneous manner, avoiding problems, such as cumulative errors and inconsistency, often encountering in separate-step optimization methods.  The effectiveness of our approach is demonstrated through extensive experiments on various benchmarks, showcasing its practical utility.

Reviewer comment (Reproducibility):
Is the pseudo-label updates implemented using expression in Eq. (14) under Prop. 2 to update the labels? It isn't mentioned for sure whether this is used; Equation 17 suggests that the labels are being updated with the true gradients rather than the approximation form in Eq. 14. If so, why do the authors describe the approximation?

Author response:
The pseudo-label updates are indeed implemented using the expression in Eq. (14) under Proposition 2 to update the labels. It is very difficult to directly compute the (true) gradient, which is the reason we proposed the approximation in Eq.(14).  The gradient term used in Eq. (17) is the term in Eq.(14). We added a sentence in the paper to confirm this.