Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: 1. This paper introduces Curriculum Learning to semi-supervised keypoint localization, which is an automatic pseudo-labeled data selection method. The method uses reinforcement learning to learns a series of dynamic thresholds. 

2. Besides, this paper proposes the cross-training strategy for pseudo-labeling to alleviate confirmation bias. 

3. The experiments shows that the proposed method can effectively improve the performance in different dataset and surpass other semi-supervised methods.

main_review: ##########################################################################

Strengths:
1. The task is practical and motivation is well. How to select the threshold of pseudo label is an important and complicated task.

2. Overall, the method of this paper is technically reasonable and novel. The paper first applies curriculum learning to semi-supervised keypoint localization and proposes use RL to search the best curriculum.

3. The experiments show that the proposed method is effective compared to other SSL methods. The ablation study validates the three parts of proposed method are all important and can improve performance (about 2%-5%).



##########################################################################

Weaknesses:
1. In my opinion, the time complexity of proposed algorithm is too high. The RL search process increases the training time by T*M times (=128 in this paper). The high complexity will make the model less scalable, such as in dataset size. The author maybe can explain about the current training cost and whether the scalability is indeed a problem.

2. The dataset in experiments in the article is somewhat simple and small in scale. If the mainstream datasets in human pose estimation such as COCO keypoint, or full MPII, can be used in experiments, the contributions would be more convincing.

3. The experimental comparison in Effect of parameter search is not sufficient, and the selected baseline method is weak.
The paper only compares Random Search and does not fully explain the details of this baseline. Does this baseline method also select the optimal strategy from candidates of the same size (T*M)?
In addition, considering that using RL to search is one of the main contributions of this paper, the author should consider comparing other stronger and more comprehensive baselines. For example, manually design many curriculums whose thresholds are gradually decreasing (on the epoch level) and select the curriculum with the best result.




##########################################################################

Minor comments:
1. The division and use of data sets are not particularly clear. Are the reported numbers all evaluated in testing set, and D_val in Eq.3 uses validation set? But the paper says that the MPII validation set is for evaluation. So what is the D_val in MPII.

2. In the paper, the symbol N represents both the keypoint network and the number of epochs, which is somewhat confusing.

3. In my opinion, the proposed method is not particularly related to the keypoint localization task. It will be better if this method can be applied to other tasks.

4. In addition, the author maybe can add discussion on the following papers about semi-supervised human pose estimation.
Rongchang Xie, Chunyu Wang, Wenjun Zeng, Yizhou Wang. An Empirical Study of the Collapsing Problem in Semi-Supervised 2D Human Pose Estimation. ICCV 2021.

##########################################################################

Updates:

Thanks for the authors' response. The response and new resutls address my main concern. I tend to accept this paper. 


summary_of_the_review: This paper proposes a novel and effective threshold selection method for semi-supervised keypoint localization. Meanwhile, I think there are some weaknesses in practicality. I currently choose borderline accept. The author may can explain about the weaknesses.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper introduces a method for semi-supervised keypoint localization based on pseudo-labeling with auto-curriculum learning. The Auto-curriculum learning approach learns a series of dynamic thresholds for automatic selection of high-quality pseudo-labeled examples for model retraining. The reinforcement learning (RL) framework, more specifically, the proximal policy optimization algorithm, is used to search for the optimal curriculum. The method is evaluated on four benchmarks in keypoint localization.
main_review: Strengths:
1. The authors provide both intuition and theoretical explanations of the proposed method supported by experimental results. 
2. Although all components are widely used techniques in the field, the application of RL to tackle pseudo-labeled sample selection for keypoint localization is novel.
3. The proposed method achieves the improvement upon the state-of-the-art on several benchmarks for human and animal body landmark localization and specifically in a low labeled data regime (5% of data is labeled).
4. Informative ablation studies and evaluation of the generalization ability on domain transfer.

Weaknesses:
1. The RL part of the approach has many moving parts and it would be beneficial to justify the choices of hyperparameters in the proximal policy optimization algorithm.
2. Inner-loop network training is executed multiple times during policy learning. How has the training complexity (time to convergence) increased compared to previous works? 

Questions:
1. Section 3.2: Is Eq(1) computed per image or per keypoint given that each image has K keypoints? If an image has some keypoints with the confidence above a threshold and some are below, does the image get selected for the training round?
2. How does the proposed pseudo-labeling strategy deal with not-visible keypoints in the image (e.g., a left eye and a left-wing of a bird are not visible if the bird is depicted from the right-side viewpoint)? 

summary_of_the_review: The paper is well-written and easy to follow. The proposed method is described in detail. The method is evaluated on four popular datasets for the keypoint localization task. The proposed method demonstrates superior performance especially in cases with only 5% of labeled data (out of no more than 10,000 examples). Ablation studies justify the design choices. The method, while combining existing techniques, is proven experimentally to be superior to the previous works and will add to the body of knowledge on keypoint localization.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: A semi-supervised learning method (PLACL) is proposed. This method employs a Pseudo-Labeling (PL) approach. It consists in iteratively (these iterations are called rounds):

1. predicting pseudo-labels to unlabeled data using the current model,
2. training a series of models from scratch using the labeled data and selections of pseudo-labeled data. The pseudo-label selection is performed using a series of thresholds (called curriculum) over the scores output by the model.

The authors propose an Auto-Curriculum Learning (ACL) strategy to automatically update the curriculum using a reinforcement learning approach (PPO2). The fact that the curriculum is updated at each round is called "Curriculum Residual Learning". They also employ a cross-training strategy to prevent the issue of confirmation bias.

The performances of PLACL are evaluated on a keypoint localization application on 5 different datasets. When the percentage of labeled data is very low (5% or 10%), PLACL outperforms the state of the art semi-supervised learning competitors.

main_review: ## Strengths

1. The idea of performing ACL using PPO2 is simple and elegant.
2. The paper is well written and easy to read.
3. The method is always either on par or outperforms the keypoint localization network competitors.

## Weaknesses/Questions

1. PLACL seems computationally very intensive. In the experiments M=8, R=6 and T=16, so it means that 736 networks are trained from scratch, isn't it ? If I am not mistaken I did not see any information concerning the training time of PLACL, type and number of GPUs used, and a comparison against sota methods. For instance, how does PLACL compare against SSKL in terms of training time, memory footprint, etc.?
2. The results that are reported in the experiments (Table 1 and 2) for the line "PLACL (ours)" correspond to a forward pass in your implementation of HRNet using which weights? In Alg. 2, the output is the "optimal curriculum". Does it mean that a final training is performed using the last threshold of the optimal curriculum? If that is the case, I suggest to add this step at the end of Alg.2 and return the weights of this network.
3. PLACL seems application agnostic. The results are always either on par or outperform the keypoint localization network competitors but the improvement w.r.t. SSKL for instance is not huge. I suggest to consider a second application to strengthen the paper.
4. In section 4.2, it is said "However, the analysis of their combined effects is outside the scope of this work.". If there exists methods (with codes available) that combine these effects, I suggest to include them in the experiments, even if they outperform PLACL.
5. In Alg.2, pseudo-labels are predicted using $N_{\omega^*}^{r-1}$, is it the network obtained at the previous round for j=M and t=T or do you train another network with ($\Gamma^{r-1})^*$ ? I think this question is related to my remark 2.
summary_of_the_review: The paper is interesting and well written but the novelty is limited (using PPO2 to update the thresholds), the results are not very impressive (especially compared to SSKL) and the evaluation is limited to a single application (keypoint localization) while PLACL is application agnostic. Thus I believe the paper is not ready for a publication at ICLR. Please answer my questions from the "main review".
review_title: null

Meta-review: This paper proposes a pseudo-labeled data selection method for semi-supervised pose estimation. The investigated task in this paper is practical and useful. The framework is well designed and reasonable, and extensive ablation studies are conducted to test the efficacy of the method. After discussion, all the reviewers recommend accept of this paper.
Decision: Accept (Poster)