Pseudo-Labeled Auto-Curriculum Learning for Semi-Supervised Keypoint Localization
Authors: Can Wang
Sheng Jin
Yingda Guan
Wentao Liu
Chen Qian
Ping Luo
Wanli Ouyang
Venue: ICLR 2022 Poster (2022)
Keywords: Keypoint Localization
Semi-Supervised Learning
Curriculum Learning

Abstract: Localizing keypoints of an object is a basic visual problem. However, supervised learning of a keypoint localization network often requires a large amount of data, which is expensive and time-consuming to obtain. To remedy this, there is an ever-growing interest in semi-supervised learning (SSL), which leverages a small set of labeled data along with a large set of unlabeled data. Among these SSL approaches, pseudo-labeling (PL) is one of the most popular. PL approaches apply pseudo-labels to unlabeled data, and then train the model with a combination of the labeled and pseudo-labeled data iteratively. The key to the success of PL is the selection of high-quality pseudo-labeled samples. Previous works mostly select training samples by manually setting a single confidence threshold. We propose to automatically select reliable pseudo-labeled samples with a series of dynamic thresholds, which constitutes a learning curriculum.Extensive experiments on five keypoint localization benchmark datasets demonstrate that the proposed approach significantly outperforms the previous state-of-the-art SSL approaches. 

Published as a conference paper at ICLR 2022
PSEUDO-LABELED AUTO-CURRICULUM LEARNING FOR SEMI-SUPERVISE KEYPOINT LOCALIZATION
Can Wang'
Sheng Jin
Yingda Guan
Wentao Liu'
Chen Qian'
Ping Luo² Wanli Ouyang²
'SenseTime Research and Tetras.Al {wangcan, SinshengJ@tetral aii
The University of Hong Kong Thee University of Sydney
(guanyingda, liuwentao,
www.nn/sneeneetemeeemmmmm com wanli. ouyangesydnney edu.au
pluoêcs . hkk. hk
ABSTRACT
Localizing keypoints of an object is a basic visual problem. However, supervised learning of keypoint localization network often requires a large amount of data, which is expensive and time-consuming to obtain. To remedy this, there is an ever-growing interest semi-supervised learning (SSL), which leverages a small set of labeled data along with large set of unlabeled data. Among these SSL approaches, ascudo-labeling (PL) is one of the most popular. PL approaches apply pscudo-labels to unlabeled data, and then train the model with combination of the labeled and pseudo-labeled data iteratively. The key to the success of PI. is the selection of high-quality pseudo-labeled samples. Previous works mostly select training samples by manually setting a single contidence threshold. We propose automatically select reliable pseudo-labeled samples with a series of dynamic thresholds, which constitutes a learning curriculum. Extensive experiments on six keypoint localization benchmark datasets demonstrate that the proposed approach significantly outperforms the previous state-of fate-of-ther oate--of-the-arr the-art SSL approaches.
INTRODUCTION
Keypoints (also termed as landmarks) popular representation of objects that precisely represent locations of object parts and contain concise information about shapes and poses. Example keypoints are "right shoulder" many visual tasks, including action recognition (Yan et al., 2018), fine-grained classification (Gavves et al,, 2013; 2015). pose tracking (Jin al., 2017; 2019) and re-identification (Zhao et al., 2017).
human body the "tail tip" of cat. Keypoint localization is the basis of
Keypoint localization has achieved great success with the advent of deep learning in recent years (Newell et al., 2016; Xiao et al., 2018; Duan al., 2019; Sun et al., 2019; Jin al., 2020a; Xu at, Geng et al., 2021; Li et al., 2021b). However, the success of deep networks relies on vast amounts of labeled data, which is often expensive and tine-consuming to collect. Semi-supervised learning (SSL) is one of the most important approaches for solving this problem. It leverages exten- sive amounts of unlabeled data in addition to sparsely labeled data to obtain gains in performance. Pseudo-labelin (PL) has become of the most popular SSL approaches due to its simplicity. PL-based methods iteratively add unlabeled samples into the training data by pscudo-labeling them with model trained combination of labeled and pseudo-labeled samples.
PL-based methods commonly require a predefined handpicked threshold (Lee et al., 2013; Oliver et al., 2018), to filter out low-confidence noisy predictions. However, a single fixed threshold does not take into account the dynamic capacity of the current model for handling noisy pseudo-labels, leading sub-optimal performance. In this work, we borrow ideas from Curriculum Learning (CL) (Bengio et al., 2009) and design our curriculum as a series of thresholds for PL, which is tuned according to the feedback from the model. CL a widely used strategy control the model training pace by selecting from easier to harder samples. With carefully designed curriculum, noticeable improvement is obtained. However, traditional CL methods suffer from hand-designed curricula, which heavily rely on expertise and detailed analysis for specific domains. Manual cur-
"Corresponding author.

---

Published as a conference paper at ICLR 2022
riculum design based on handcrafted criteria is always tedious and sub-optimal. Moreover, curricu- lum design (or threshold setting) is complicated. High-confidence pseudo-labels typically corre- spond to easier samples with clean labels, while low-confidence pseudo-labels correspond to harder samples with noisy labels. How to design a curriculum to balance the correctness, representative- ness, and difficulty of pseudo-labeled data is an open problem. This paper is devoted to tackling the aforementioned problem, i.e. how to automatically learn an optimal learning curriculum for pseudo-labeling a data-driven way. To this end, we propose a novel method, called Pseudo- Labeled Auto-Curriculun Learning (PLACL). PLACL formulates the curriculum design problem as a decision- decision-making problem and leverages the reinforcement learning (RL) framework to solve it.
Additionally. PL-based methods suffer from confirmation bias (Tarvainen & Valpola, 2017), also known as noise accumulation (Zhang et al., 2016), and concept drift (Cascante-Bonillyy This long-standing issue stems from the use of noisy or incorrect pseudo-labels in subsequent train- ing stages. As a consequence, the noise accumulates and the performance degrades as the learning process evolves over time. To mitigate this problem, we propose the cross-training strategy which alternatively performs pseudo-label prediction and model training on separate sub-datasets.
al., 2021).
We benchmark PLACL on six keypoint localization datasets, including LSPET (Johnson & Ever- ingham, 2011), MPII (Andriluka et al., 2014),
(Welinder et al., 2010), ATRW (Li et al., 2019c), MS-COCO (Lin et al., 2014), and AnimalPose (Cao et al., 2019). We empirically show that PLACL is general and can be applied to various keypoint localization tasks (human and animal pose estimation) and different keypoint localization networks. With a simple yet effective search paradigm, our method significantly boosts the keypoint estimation performance and achieves superior performance other SSL methods. We hope our method will inspire the community to rethink the potential of PL-based methods for semi-supervised keypoint localization.
Our main contributions can be summarized as follows:
We propose Pseudo-Labeled Auto-Curriculum Learning (PLACL). It is an an automatic pseudo-labeled data selection method, which learns a series of dynamic thresholds (or cur- riculum) via reinforcement learning. To the best of our knowledge, this is the first work that explores automatic curriculum learning for semi-supervised keypoint localization.
We propose the cross-training strategy for pseudo-labeling mitigate the long-standing problem of confirmation bias.
Extensive experiments on a wide range of popular datasets demonstrate the superiority of PLACL over the previous state-of-the-arr SSL approaches. In addition, PLACL is model- agnostic and can be casily applied to different keypoint localization networks.
2 RELATED WORKS
2.1 SEMI-SUPERVISED KEYPOINT LOCALIZATION
Keypoint localization focuses predicting the keypoints of detected objects, e.g. human body parts (Li et al., 2019b; Jin et al., 2020b), facial landmarks (Bulat & Tzimiropoulos, 2017), hand keypoints (Zimmermann & Brox, 2017) and animal poses (Cao et al., 2019). However, training a keypoint localization model often requires a large amount of data, which is expensive and time- consuming to collect. Semi-supervised keypoint localization is one of the most promising ways to solve this problem. Semi-supervised keypoint localization can be categorized into consistency reg- ularization based methods and pseudo-labeling based methods. Consistency regularization meth- ods (Honari et al., 2018; Moskvyak et al., 2020) assume that the output of the model should not be invariant to realistic perturbations. These approaches typically rely on modality-specific augmen- tation techniques for regularization. Pscudo-labeling methods (Ukita & Uematsu, 2018; Dong & Yang, 2019; Cao et al., 2019; Li & Lee, 2021) use labeled data to predict the labels of the unla- beled data, and then train the model in supervised way with combination of labeled and selected pseudo- labeled data. Our approach also builds upon pseudo-labeline methods. In contrast to previ- ous works, we propose to learn pseudo-labeled data selection via reinforcement learning.
2

---

Published as a conference paper at ICLR 2022
2.2 CURRICULUM LEARNING
Curriculum learning is firstly introduced by Bengio et al. (2009). It is a training strategy that trains machine learning models from easy to complex samples, imitating human education. The curricu- lum is often pre-determined by heuristics (Khan et al., 2011; I; Bengio et al., 2009; Spitkovsky et al., 2009). However, it requires expert domain knowledge and exhaustive trials to find a good currieu- lum suitable for specific task and its dataset. Recently, automatic curriculum learning methods are introduced to break through these limits. Popular ones include self-paced learning methods (Kumar et al., 2010; Jiang et al., 2014; Zhao et al., 2015) and reinforcement learning (RL) based meth- ods (Graves et al., 2017; Matiisn et al., 2019; Fan et al., 2018). Our approach can be categorized as RL-based methods. Unlike previous works that focus supervised learning, our approach is de- signed for the SSL paradigm. Our work is mostly related to Curriculum Labeling (Cascante-Bonill al., 2021). It adopts a hand-crafted curriculum based on Extreme Value Theory (EVT) facili- tate model training. In contrast, we propose an automatic curriculum learning approach by searching for dynamic thresholds for pseudo-labeling, In addition, the curriculum of (Cascante- Bonilla al., 2021) is coarse- coarse-grained on the round level, while our curriculum is fine-grained on the epoch level.
2.3 REINFORCEMENT LEARNING FOR AUTOML
Reinforcement learning (RL) has shown impressive results in range of applications. Well-known examples include game playing (Mnih et al., 2015; Silver et al., 2016; 2017) and robotics con- trol(Schulman et al., 2015; Lillicrap et al., 2016). Recent works have employed RL the AutoML, automating the design of a machine learning system, egg. searching for neural architectures (Zoph & Le, 2017; Zoph al., 2018; Baker et al., 2017; Pham et al., 2018), augmentation policies (Cubuk et al., 2019), activation functions (Ramachandran et al., 2017), loss functions (Li et al., 2019a; 2021a), and training hyperparameters (Dong et al., 2020). In contrast these works, we apply RL to the automatic selection of pscudo-labeled data in the context of pseudo-labeling.
PSEUDO-LABELED AUTO-CURRICULL LEARNING (PLACL)
3.1 OVERVIEW
Our PLACL algorithm is illustrated in Fig. 1. The training process consists of R seeff-training rounds and each round consists of N training epochs. (0) In the initial round (r = 0), we pre-train a keypoint localization network 0% the labeled data, where denotes the weights of the network. And for the r-th round, (1) The trained network O" is used to predict pseudo-labels for unlaheled data. (2) We adopt reinforcement learning (RL) automatically generate the learning curriculum. Specifically, our curriculum (1") consists of a series of thresholds for pscudo-labeled data selection.
..... 1" [nr ...; 75.. where 7 E [0, 1] is the threshold for each epoch i. (3) We then select reliable pseudo-labeled data by the searched curriculum. (4) We retrain new model (0S=1) using both the
labeled samples and selected pseudo-labeles samples. (5) This process repeated for R rounds.
3.2 PSEUDO-LABER SELECTION FOR SEMI-SUPERVID KEYPOINT LOCALIZATION
We denote the labeled dataset with N₁ samples ID, { (I)x)) where I; and Y, denote the i-th training image and its keypoint annotations (the x-y coordinates of K keypoints). The N, unlabeled images are denoted as D,, keypoint labels. Generally, we have [N]] < [Na).
Ni
(11) which are not associated with any ground-truth
Pseudo-labeling based method builds upon the general idea of self-training (McLachlan, 1975), where the keypoint localization network θ,, goes through multiple rounds of training. In the ini- tialization round, the model is first trained with the small labeled training set Drrran = Dy in a usual supervised manner. In subsequent rounds, the trained model is used to estimate labels for the unla- N.
beled data DD, = ( i=1] Here, we omit the superscript r for simplicity. Specifically, given an unlabeled image IT the trained keypoint localization network θ_, predicts K heatmaps. Each heatmap is 2D Gaussian centered the joint location, which represents the confidence of
3

---

Published as a conference paper at ICLR 2022
(5) Repeat
(0) Train
Retrain
%
0++1
6 602
0
labeled Samples
(1) Predict
So
(3) Select Samples
Unlabeled Samples
Psoudo Labeled Samples
(2) Curriculum Generation
Yi
Figure 1: Pseudo-Labele Auto- Cutt-Cuuriiulum Learning (PLACL). (0) In the initial round, the model O% is pre-trained on the labeled data. And for the r-th round, (1) the trained network O is used predict pseudo-labels for unlabeled data. (2) A learning curriculum consisting of series of thresholds (7f) is generated. (3) Reliable pseudo-labeled data is selected by the searched curriculum. (4) new model e is retrained using both the labeled samples and selected pseudo-labeled samples. (5) This process is repeated by re-labeling unlabeled data using the new model.
the k-th keypoint. The output pseudo-labeled keypoint location (Y,") is the highest response in the heatmap space. And the confidence score C (e., (T,)) is the response value at the keypoint location.
Then, pseudo-label selection process is adopted. Let g = [g,,..., gN..] C (0, 1). be a binary vector representing the selection of pseudo-labels, where g, denotes whether the keypoint prediction on I," is selected.
1, if > { O, otherwise
gi
where y (0,1) is the confidence threshold. Pseudo-labeled samples with higher confidence added to the training set.
Dtrain { Y; (-)) u (I (
where 9: =1}
Then the keypoint localization network is retrained with a combination of labeled and pseudo- labeled training data Drrain
3.3 CROSS-TRAINING STRATEGY
In this section, we introduce the cross-training strategy in our curriculum learning framework. In a typical self-training round, the model predicts noisy pseudo-labels which are used in subsequent training stages. Since the pseudo-label prediction is performed on dataset of known data (on which training performed), the noise accumulates in a positive feedback loop. This causes the long-standing issue of confirmation bias (Tarvainen & Valpola, 2017). To mitigate this problem, propose the cross-training strategy. Specifically, we randomly partition the unlabeled data D, into two complementary subsets (D and (2) The partition is fixed across all the training rounds. The self-training conducts alternatively for (D(") and (D) For {1...., R), when is odd, we use combination of (1) and D, to train the model, and the trained model predicts pseudo-labels for the next round on (2 When even, we a combination of (5(2) and Di to train the model, and perform pseudo-label prediction on TD(") Before each round, the model parameters are re-initialized with random weights following Cascante-Bonilla et al. (2021) to avoid noise accumulation.
3.4 CURRICULUM RESIDUAL LEARNING
Directly learning for rounds of curriculum parameters separately can be inefficient. Considering that the model is reinitialized and retrained in each self-training round, assume that the optimal
4

---

Published as a conference paper at ICLR 2022
curricula for different rounds should have some similar patterns. We propose greedy multi-step searching algorithm. For round r, we use the searched curriculum in the previous round r - 1 as the base curriculum to guide the searching of current curricula I". Inspired by ResNet (He et al., 2016), we propose the curriculum residual learning strategy. Formally, we learn a bias term Ar" around the base curriculum, i.e. r = (Ir=1)"= + Ar", where (I>-1)" means the searched optimal curriculum for round r - 1, and (1 is initialized with all zeros. We empirically find that this strategy accelerates the model convergence speed and achieves marginally better performance. To further reduce the search space, we choose every G epochs as an epoch group, which shares the same threshold parameters for pseudo-labeled data selection. In total, we have NG epoch groups, and the size of each epoch group is G. Therefore, the search space is reduced by a factor of G, where 10 in our implementation.
3.5 CURRICULUM SEARCH VIA REINFORCEMENT LEARNING
Our PLACL can be formulated as an optimization problem shown in Eq. 3. In the inner-loop, we optimize the weights w of the keypoint localization network Ow to minimize the training loss L (see Alg. 1). the outer-loop, we apply proximal policy optimization (PPO2) algorithm (Schulman et al., 2017) to search for the curriculum I' that maximize the evaluation metric & (e.g. PCK) on the validation set Dval (see Alg. 2).
max E(T) E(Ow- Dval; s.t. wT(T)= arg min L(Owi Drain T)- r
(3)
The training consists of R rounds. In each round r, the PPO2 search process consists T sampling steps. In each step, M sets of parameters are sampled independently from a truncated normal dis- tribution (Nakano al., 2012; Fujita & Maeda, 2018), Ar" ~ (or (1[, ofT, where je and I²I are the mean and covariance (ar is fixed to 0.2 practice). These sampled parameters are used to construct M different training curricula for training M keypoint localization networks separately. Then the mean of the distribution is updated by PPO2 algorithm according to the evaluation score of the M networks.
The objective function of PPO2 is formulated in Eq. 4.
Thu (ATT: ,0,0011)) THe [AT); 1L, 0²1)
THe (AT; wi, 0²1) Thp (AT P
J (n²) = B, min
(TT), CLIP
e, +e & (T;)
16 021)
(4)
where the function CLIP 1 e, + e) clips to be more than + and no less than 1-6 Following the common practice (Li et al., 2021a), , the mean reward is subtracted for better conver- gence. Č (Tz)= E (rz) m Σラ (T²) and the policy Thu is defined as the probability density function (PDF) of the truncated normal distribution. PPO2 enforces the probability ratio between old and policies मe. (AT7; "=",021)) /Tµę (ATj; T²T) stay within small interval to control the size of each policy update. We then compute the gradients and update the parameters by 1i41 + nh + V J (n²) with learning rate of > After T' sampling steps, we choose μe with the highest average evaluation score as (TT). And (IT)* is used as the base curriculum for the next round, And after rounds, our final optimal curriculum is obtained, r = [("',....,
Training details In the training phase, the keypoint localization network and the curriculum search policy are simultancously optimized. For the outer-loop. the PPO2 (Schulman et al., 2017) search procedure is conducted for T = 16 sampling steps, and in each step M = 8 sets of parameters (cur- riculum) are sampled. The clipping threshold is = 0.2, and nوا is updated with the learning rate of a 0.2. We empirically use R = 6 self- training rounds, and group size G = 10 for curriculum search. For the inner-loop, we follow the common practice (Sun et al., 2019; Contributors, 2020) to train the keypoint localization network with Mean-Squared Error (MSE) loss for N = 210 epochs per round. Adam (Kingma & Ba, 2015) with a learning rate of 0.001 is adopted. We reduce the learning rate by factor of 10 at the 170-th and 200-th epochs. Although the RL search process in- creases the training complexity, the total training cost is not too high (only 1.5 days with 32 NVIDIA Tesla V 10 GPUs). More detailed training settings for each task are provided in $4.2 and $4.3.
5

---

Published as a conference paper at ICLR 2022
Table 1: Keypoint localization with different percentage of labeled images. We report mean and standard deviation from three runs for different randomly sampled labeled subsets. Pseudo-labeling (PL) based methods are not evaluated for 100% of labeled data because there no unlabeled data to generate pseudo-labels for. The results marked with are from (Moskvyak et al., 2020).
Percentage of labeled images
Method
10%
20%
50%
100%
Dataset 1: LSPET
HRNet (Sun et al., 2019)
40.19=
45.17=1.15
55.22-
6..61=1.25
12.12-10.30
ELT (Honari et al., 2018)
1.56
47.22=0091
57.34=0.94
66.81=0.62
12.22-+0.13
Gen (Jakab et al,, 2018)
1.41
67.75=1.00
68.80=0.91
69.70=0.77
02,25
SSKL (Moskvyak et al., 2020)
0,99
69.56-0.66
71.85=0.33
72.59=0.56
74.29-0.21
PI." (Radosavovic et at., 2018)
37.36+1.88
42.05-1.68
48.86-1.23
64.45-0.96
CL (Cascante-Bonille et al,, 2021)
61.27+1.54
65.43-1.19
69.14-0.93
70.29+1.18
PLACL (Ours)
7..76=1.447
71.91=1.15
7..30=0.88
72.73=1.23
Dataset 2: MPII
HRNet" (Sun et al., 2019)
66.22++.660
69.18=1.03
7.883=0.87
7..73=0.35
81.11=0.115
ELT (Honari et al., 2018)
68.27=10.64
71.03=0.46
72.37
77.75=0.31
81.01
Gen (Jakab et al., 2018)
1.159=1.12
72.63=0.62
74.95=0.32
79.86=0.19
80.92-1.32
SSKL." (Moskvyak et al., 2020)
74.15++0.83
76.56-0.48
78.46-0.36
80.75-0.32
82.12+0.144
PL (Radosavovic et al,, 2018)
1.744=1.75
64.78=1.44
69.35=1.11
77.43=0.48
CL (Cascante-Bonilla et al., 2021)
1.56
73.15=0955
7..80=0992
77.49=0.35
PLACL (Ours)
7..83=1.41
78.36
79.68
80.81
Dataset 3: CUB-200-2011
HRNet (Sun et al., 2019)
85.77+0.38
88.62-0.14
90.18-022
92.60-0.28
93.62++0.13
ELT (Honari et al., 2018)
86.5410.34
89.48-0.25
90.86=0.13
92.26-0.06
93.77-0..1
Gen (Jakab el al., 2018)
88.37=0.40
90.38=0.22
91.31=0.21
92.79=0.14
03.62-
SSKL" (Moskvyak et al., 2020)
91.11=00.33
91.47=0.36
92.36=0.30
92.80=0.24
93.81 10.13
PL." (Radosavovic et al., 2018)
86.31=0045
89.51=0.32
90.88=0.28
92.78=0.27
CL (Cascante-Bonilla et al., 2021)
91.46=0041
92.35-0.34
92.74-0.27
92.97-0.21
PLACL (Ours)
93.01-0.33
93.28-0.29
03.5
03.8
Dataset 4: ATRW
HRNet (Sun et al., 2019)
69.22:10.87
77.55=0.84
86,41=0.45
92.17=0.18
94.44-1.11
ELT (Honari et al., 2018)
7..53=1.24
80.35=0.96
87.98-0.77
92.80=0.21
94.75+0.11
Gen (Jakab al., 2018)
89.5410.57
90.48=0.49
91.16=0.13
92.27=0.24
94.80-0.11
SSKL* (Moskvyak et al., 2020)
92.57-0.64
94.29-0.66
94.49=0.36
94.63=0.18
95.31
PI." (Radosavovic al., 2018)
67.97+1.007
75.26-0.74
84.69-0.57
92.15-0.24
CL (Cascante-Bonee Bonilll al., 2021)
8..01
89.13=0.94
92.34=0.51
93.57=0.26
PLACL (Ours)
0.83
9.85
94.85=0.48
95.01=0.17
Dataset 5: MS-COCO'2017
HRNet (Sun et al., 2019)
1.24
66.02=1.07
0.84
72.81-0.73
74.61+0.58
CL (Cascante- BonillB et al., 2021)
64.47=1.18
67.82-0.95
7..3
72.92-0.84
PLACL (Ours)
6..39-1.03
7..11=0.89
0.66
73.42-0.57
4 EXPERIMENTS
4.1 DATASETS AND EVALUATION METRICS
Datasets: To show the versatility of PLACL, we conduct experiments on diverse datasets. LSPET (Leeds Sports Pose Extended Dataset) (Johnson & Everingham, 2010; 2011) consists of im- ages people doing sports activities. We use 10,000 images from (Johnson Everingham, 2011) for training and 2,000 images from (Johnson & Everingham, 2010) for validation and testing. MPII Human Pose dataset (Andriluka et al., 2014) is a well-known benchmark for human pose estimation. The images are collected from YouTube videos, showing people doing daily human activities. We follow (Moskvyak et al., 2020) to use 10,000 random images from MPII rain i for training, 3,311 images from MPII train for validation and MPII val 1 for evaluation. CUB-200-2011 (Caltech- UCSD Birds-200-22 20102001) (Welinder et al., 2010) dataset is a well-known dataset for SSL. consists of 200 fine-grat -reinga bird species with 15 keypoint annotations. We follow (Moskvyak et al., 2020) to split dataset into training (100 categories with 5,864 images), validation (50 categories with 2,958 images) and testing (50 categories with 2,966 images). ATRW (Li et al., 2019c) dataset contains

---

Published as a conference paper at ICLR 2022
images of 92 Amur tigers captured from multiple wild zoos in challenging and unconstrained con- ditions. For each tiger, 15 body keypointss are annotated. The dataset consists of 3,610 images for training. 516 for validation, and 1,033 for testing. MS-COCO 2017 (Lin et al., 2014) is a popular large-scale benchmark for human pose estimation, which contains over 150,000 annotated people. We randomly select 500 images from COCO train inain for validation, the remaining training set (115k images) for training, and COCO val (5k images) for evaluation. We use this dataset to validate the applicability of our approach on large-scale data. AnimalPose (Cao et al., 2019) dataset contains 5,517 instances of five animal categories: dog, cat, horse, sheep, and cow. It consists of 2,798 im- ages for training, and the 810 images for validation and 1,000 images for testing. We use it to test the generalization and domain-transfer capacity of our proposed method.
Evaluation Metries: PCK (Probability of Correct Keypoint): A detected keypoint is considered
correct if the distance between the predicted and true keypoint is within certain threshold (ol), where or is a constant and is the longest side of the bounding box. We adopt PCK @0.1 (a = 0.1) for LSPET (Johnson & Everingham, 2011), CUB-200-2011 (Welinder et al., 2010), and ATRW (Li et al., 2019c) datasets. PCKh is adapted from PCK, where 1 is the head size that corresponds to 60% of the diagonal length of the ground-truth head box. We adopt PCKh@0.5 (a 0.5) for MPII (Andriluka et al., 2014) dataset. Standard AP (Average Precision) is another commonly used evaluation metric. It is based on object keypoint similarity (OKS), which measures the distance between predicted keypoints and ground-truth keypoints normalized by the scale of the object. We use mAP for AnimalPose (Cao et al., 2019) datasets.
4.2 COMPARISONS WITH THE STATE-OF-THE ART SSL APPROACHES
In Table 1, we compare with the supervised baseline (HRNet (Sun et al., 2019)) and other state- of-the-art SSL approaches. We experiment with different percentages of labeled images (5%, 10%, 20%, 50%, and 100%). For fair comparisons, all results are obtained using HRNet-w33 backbone with the input size of 256 x 256. We follow (Moskvyak al., 2020) to prepare datasets and exclude half body transforms, and testing tricks (post-processing. and flip testing).
Comparisons with consistency regularization methods. ELT (Honari et al., 2018) (equivariant landmark transformation) loss encourages the model to output keypoints that are equivariant to in- put transformations. Gen (Jakab et al., 2018) learns to extract geometry-relatee conditional image generation. SSKL (Moskvyak et al., 2020) learns the pose invariant keypoint representations with semantic keypoint consistency constraints. These consistency regularization methods have shown superior results over the supervised baseline, however, they inferior to PLACL method on all datasets and different percentages of labeled samples. Especially we show that PLACL is mostly effective for low data regimes. For example, in CUB-200-2011 1 dataset, PLACL with only 5% labeled data achieves better performance (93.01 vs 92.80) than SSKL with 50% la- beled data. And in LSPET dataset, we show that PLACL improves the performance of baseline by a large margin from 40.19 19 to 70.76 with 5% labeled images.
features through
Comparisons with pseudo-labeling method. We also compare with a pseudo-labeling (PI.) base- line (Radosavovic et al., 2018). Overall PLACL significantly outperforms the PL baseline on all datasets. As pointed out by Moskvyak et al. (2020), the vanilla PL approach does not perform well for the keypoint localization task with low data regime, due to the lack of an effective pseudo- label selection scheme. Instead, PLACL is able to automatically select high-quality pseudo-labeled samples, which is the key to the success of pseudo-labeling based methods.
Comparisons with curriculum-learninn et al., 2021) is a recently proposed approach that applies hund-crafted curriculum to facilitate training of SSL. We observe that our proposed PLACI significantly outperforms CL., which vali- dates the effectiveness of our proposed automatic curriculum learning.
method. Curriculum Labeling (CL) (Cascante-Bonil
Experiments on large-scale datasets. Inspired by Zhou et al. (2020), in order to decrease the RL curriculum search cost for the large-scale MS-COCO (Lin et al., 2014) and full MPII (Andriluka al., 2014) datasets, we use a light proxy task with reduced number of training samples (5k) for RL curriculum search. After the search procedure, re-train the keypoint networks with the searched curriculum on the full training set and evaluate them on the test set. Please refer to A.4 for more analysis about proxy tasks and A.5 for experiments on the full MPII (Andriluka et al., 2014) dataset.
7

---

Published as a conference paper at ICLR 2022
Group (G)
Test Accuracy: PCX
70
Test Accuracy
Fixed Thresheld
15%5
PLACL
CUB (5%6)
PLACL 16-15)
NPU
PLACL 16-00
PLACLIG
2
Number of Self- Rounds
Threshold
(a)
(b)
Figure 2: (a) Accuracy on different datasets (5% labeled data) with various rounds (R). (b) Com- parisons with different fixed thresholds (blue dots) and different group sizes G. Experiments conducted on LSPET dataset (5% labeled data).
4.3 EVALUATION OF GENERALIZATION CAPACITY
Generalization to different domains. We investigate the generalization ability of our proposed model on domain transfer. To this end, we conduct experiments AnimalPose (Cao et al., 2019) dataset in Table 2. Specifically, we follow (Cao et al., 2019) to choose one animal class (e.g. cat) as the target domain and the remaining four classes for the source domain. The images of the source do- main are fully annotated, while the images of the target domain are unlabeled. For fair comparisons, we adopt the AlphaPose model (Fang et al., 2017). which uses ResNet-101 (Xiao et al., 2018) as the backbone. All models are trained with the pose-labeled human dataset involved. The AlphaPose baseline model pre-trained on the human dataset and fine-tuned on the labeled source animal data. For pseudo-labeling based approaches (Inoue et al., 2018; Cao et al., 2019), the unlabeled target an- imal data used for pseudo-labeling. For domain adaptation approaches (Tzeng et al., 2015; Long et al., 2016), the unlabeled target data is used for domain transfer. Please refer to Cao et al. (2019) for details about the compared upproaches. We observe that our proposed approach consistently outperforms the previous state-of-the art methods cross cross-domain semi- cemi-supervised learning.
Table Dataset 6: AnimalPose. Evaluation of generalization capacity to the target unseen animal class, All results are obtained using the AlphaPose model (Fang al., 2017) with ResNet-101 (He et al., 2016) as the backbone. Results marked with ** are from Cao et al. (2019).
mAP for each class
Method
cat
dog sheep cow horse
AlphaPose Baseline* (Fang et al., 2017)
37.6
37.3
49.4
50.3
47.9
Dom Confusion" (Tzeng et al., 2015)
38.0 37.7
49.5
50.6
48.5
Residual Transfer" (Long et al., 2016)
37.8 38.2
49.1
50.8
48.6
CycleGAN+PLI (Inoue et al., 2018)
35.9 36.7
48.0
50.1
48.1
WS-CDA+PPLOO (Cao et al., 2019)
42.3 41.0
54.7
57.3
53.1
PLACL (Ours)
47.1 42.9
59.5
58.4
66.0
4.4 ANALYSIS
Number of self-training rounds. Along with the increasing self-training rounds (It), the quality of the pseudo-labels gradually improves (see Fig. 3) and the test accuracy increases (see Fig. 2a) until saturation. The experiments are conducted on multiple datasets with 5% labeled data. Interest- ingly, different datasets require different number of rounds to achieve optimal, because four-legged animals (ATRW (Li et al., 2019c)) have more pose variations than birds CCBB-200-201 (Welinder al., 2010)). We use R 6, because further increasing R does bring significant gains.
8

---

Published as a conference paper at ICLR 2022
Comparisons with pseudo-labeling with different fixed thresholds. As shown in Fig. 2b, compare our PLACL with 10 static thresholds from 0.0 to 0.9. We observe that PLACL clearly outperforms all these fixed threshold alternatives. Moreover, we find that the accuracy will be sig- nificantly affected by different thresholds (61.01 % PCK for y = 0.0 vs 67.35% PCK for y 0.7).
Choice of epoch group size. In Fig. 2b, we also compare the performance of different epoch group sizes (G). We empirically find that smaller will produce better performance, but at the cost of increased search space. We choose G 10 trade-off between accuracy and efficiency.
4.5 ABLATION STUDIES
In Table 3, we present ablation studies to measure the contribution of each component. All ablative experiments are conducted on LSPET (Johnson Everingham, 2011) dataset with 5% labeled data. PCK @0.1 1 is adopted as the evaluation metric.
Effeet of cross-training strategy. We find that without cross-training strategy the accuracy signif- icantly drops from 70.76 67.13, due to noise accumulation over time. Especially, find that there is little to no improvement after the first self-training round.
Effect of curriculum learning. We compare PLACL with the alternative that only searches for fixed threshold via RL. We find that using dynamic thresholds improves upon the theehol alternative by large margin, which validates the effectiveness of curriculum learning.
Effect of parameter search. We compare PPO2 (Schulman et al., 2017) search with Random Search (w/o PPO2 search). We randomly sampled T M curricula and pick out the best for comparisons. PPO2 search obtains much better performance (70.70 76 vs 65.42). This indicates that the searching problem is non-trivial and that searching algorithm is very effective.
We also compare with manually designed curricula whose thresholds are gradually decreasing on the epoch level. We tried five curricula with different decrease slopes and reported the best one as Manually Design. We observe that PLACL significantly outperforms the manually-designed curriculum baseline which validates the effectiveness of automatic curriculum search.
Table 3: Ablation studies on LSPET dataset with 5% labeled data.
Method
PCK @0.1
PLACL, w/o cross-training strategy
67.13
PLACL, w/o curriculum learning
68.51
PLACL. w/o PPO2 search (Random Search)
65.42
PLACL, w/o PPO2 search (Manually Design)
65.71
PLACL,, full method
70.76
CONCLUSIONS
We propose novel Pseudo-labeled Auto-Curriculum Learning (PLACL) for the task of semi- supervised keypoint localization. We propose to learn a curriculum to automatically select reliable pseudo-labels and propose cross-training strategy mitigate the confirmation bias problem. Ex- tensive experiments on 6 diverse datasets validate the effectiveness and versatility the proposed method. We believe that proposed approach is generic and we plan to investigate the applicabil- ity of PLACL on other visual tasks, such as object detection and semantic segmentation.
Acknowledgement. We thank Lumin Xu and Wang Zeng for their valuable feedback to the paper. Ping Luo is supported by the General Research Fund of HK No.27208720 and 17212120. Wanli Ouyang was supported by the Australian Research Council Grant DP200103223, FT210100228, and Australian Medical Research Future Fund MRFAI000085.

---

Published as a conference paper at ICLR 2022
REFERENCES
Mykhaylo Andriluka, Leonid Pishchulin, Peter Gchler, and Bernt Schiele. 2d human pose estimation: New benchmark and state the art analysis. In IEEE Conf. Comput. Vis. Pattern Recog,, 2014. 2, 6, 7, 16
Bowen Baker, Otkrist Gupta, Nikhil Naik, and Ramesh Raskar. Designing neural network architectures using reinforcement learning. Int. Conf. Learn. Represent,, 2017. 3
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum lcarning. In Int. Conf. Machine Learning.. pp. 41-48, 2009. 1,3
Adrian Bulat and Georgios Tzimiropoulos. How far are we from solving the 2d & 3d face alignment problem? In Int. Conf. Comput. Vis., pp.
2017. 2
Jinkun Cao, Hongyang Tang, Hao-Shu Fang, Xiaoyong Shen, Cowu Lu, and Yu- fu-Wing Tai. Cross-domain adaptation for animal pose estimation. In Int. Conf. Comput. Vis., pp. 9498-9507, 2019. 2, 7,8
Paola Cascante-Bonin Fuwen Tan, Yanjun Qi, and Vicente Ordonez. Curriculum labeling: Revisiting pseudo-
labeling for semi- sune-super learning. AAAI 2021. 2, 3, 4, 6, 16
MMPose Contributors. Openmmlab pose estimation toolbox and benchmark. https:://gtthbb.cm
open- mmm hammlabmmee htwpaooemmeee.eme.mmmee/eooeeeee 2020. 5
Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoe V Le. Autoaugment: Learning augmentation strategics from data. In IEEE Conf. Comput. Vis. Pattem Recog, pp. 113-123, 2019. 3
Xuanyi Dong and Yi Yang. Teacher supervises students how to learn from partially labeled images for facial landmark detection. In Int. Conf. Comput. Vis., pp. 783-792, 2019. 2
Xuanyi Dong, Mingxing Tan, Adams Wei Yu, Daiyi Peng, Bogdan Gabrys, and Quac V Lc. Autohas: Efficient hyperparameter and architecture search. arXiv preprint arXiv:2006.03653 2020. 3
Haodong Duan, Kwan- Yee Lin, Sheng Jin, Wentao Liu, Chen Qian, Wanli Ouyang. Trb: novel triplet representation for understanding 2d human body. In Int. Conf. Comput. Vis., 2019.
Yang Fan, Fei Tian, Tao Qin, Xiang-Yan Yang Li, and Tic- Yan Liu. Learning to teach. arXiv preprint 1xXiv:1805.03643 2018.
Hao-Shu Fang, Shuqin Xie, Yu- Wing Tai, and Cewu Lu. Rmpe: Regional multi-person pose estimation. In Int. Conf. Comput. Vis., pp. 2334 2243-2333322 2017. 8
Yasuhiro Fujita and Shin-ichi Maeda. Clipped action policy gradient. In Int. Conf. Machine Learning., pp- 1597-1606, 2018.
Efstratios Gavves, Basura Fernando, Cees GM Snoek, Amoll WM Smeulders, and Tinne Tuytelaars. Fine- grained categorization by alignments. In Int. Conf. Comput. Vis., pp. 1713-1720, 2013. 1
Efstratios Gavves, Basura Fernando, Cees GM Snoek, Amold WM Smeulders, and Tinne Tuytelaars. Local alignments for fine-grained categorization. Int. J. Comput. Vis., 111(2):11-212, 2015.
Zigang Geng, Ke Sun, Bin Xiao, Zhaoxiang Zhang, and Jingdong Wang. Bottom-up human pose estimation disentangled keypoint regression. In IEEE Conf. Comput. Vis. Pattern Recog,, 2021. 1
Alex Graves, Mare G Bellemare, Jacob Menick, Remi Munos, and Koray Kavukeuoglu. Automated curriculum learning for neural networks. Int. Conf. Machine Learning,, pp. 1311 1320. PMLR, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conf. Comput. Vis. Pattern Recog., 2016. 5,
Sina Honari, Pavlo Molchanov, Stephen Tyree, Pascal Vincent, Christopher Pal, and Jan Kautz. Improving landmark localization with semi semi-supervised learning. In IEEE Conf. Comput. Vis. Pattern Recog., pp. 1546- 1555, 2018. 2, 6,
Naoto Inoue, Ryosuke Furuta, Toshihiko Yamasaki, and Kiyoharu Aizawa. Cross-domain weakly- object detection through progressive domain adaptation. In IEBE Conf. Comput. Vis. Pattern Recog., pp. 5001-5009, 2018.
Tomas Jakab, Ankush Gupta, Hakan Bilen, and Andrea Vedaldi. Unsupervised learning of object landmarks through conditional image generation. In Adv. Neural Inform. Process. Syst., pp. 4020-4031, 2018. 6, 7
10

---

Published as a conference paper at ICLR 2022
Lu Jiang. Deyu Meng, Teruko Mitamura, and Alexander Hauptmann. Easy samples first: Self-pacee rerank- ing for zeao-e multimedia search. In ACM Int. Conf. Multimedia, pp. 547-556, 2014. 3
Sheng Jin, Xujie Ma, Zhipeng Han, Yue Wu, Wei Yang, Wentao Liu, Chen Qian, Wanli Ouyang. To- wards multi-person pose tracking: Bottom-up and top-down methods. Int. Conf. Comput. Vis. Worksh., volume 2, pp. 7, 2017.
Sheng Jin, Wentao Liu, Wanli Ouyang, and Chen Qian. Multi-person articulated tracking with spatial and temporal embeddings. In IEEE Conf. Comput. Vis. Pattern Recog,, 2019.
Sheng Jin, Wentao Liu, Enze Xie, Wenhai Wang. Chen Qian, Wanli Ouyang, and Ping Luo. Differentiable hierarchical graph grouping for multi-person pose estimation. In Eur. Conf. Comput. Vis., 2020a.
Sheng Jin, Lumin Xu, Jin Xu, Can Wang, Wentao Liu, Chen Qian, Wanli Ouyang, and Ping Luo. Whole-body human pose estimation the wild. Eur. Conf. Comput. Vis., 2020b.
Sam Johnson Mark Everingham. Clustered pose and nonlinear appearance models for human pose estima- tion. In Brit. Mach. Vis. Conf., 2010. 6
Sam Johnson and Mark Everingham. Learning effective human pose estimation from inaccurate annotation. In IEEE Conf. Comput. Vis. Pattern Recog,, 2011. 2, 6, 6,7,9 9
Faisal Khan, Bilge Mutlu, and Jerry Zhu, How do humans teach: On curriculum learning and teaching dimen- sion. In Ady. Neural Inform. Process. Syst., pp. 1449-1457, 2011. 1. 3
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Int. Conf. Learn. Represent., 2015. 5
M Kumar, Benjamin Packer, and Daphne Koller. Self- learning for latent variable models. Adv. Neural Inform. Process. Syst., 23:1189 189711977 2010. 3
Mong-Hyun Lee et al. Pseudo-label: The simple and efficient cemi-supereised learning method for deep neural networks. In Int. Conf. Machine Learning Worksh., volume 3, pp. 896, 2013. I
Chen Li and Gim Hee Lee. From synthetic to real: Unsupervised domain adaptation for animal pose estimation. In IEEE Conf. Comput. Vis. Pattern Recog., pp. 1482-1491, 2021. 2
Chuming Li, Xin Yuan, Chen Lin, Minghao Guo, Wci Wu, Junjie Yan, and Wanli Ouyang. Am-lfs: Automl for loss function search. In Int. Conf. Comput. Vis., pp. 8410-8419, 2019a. 3
Hao Li, Chenxin Tao, Xizhou Zhu, Xiaogang Wang, Gao Huang, and Jifeng Dai. Auto seg-loss: Searching metric surrogates for semantic segmentation. Int. Conf. Learn. Represent, 2021a. 3,5
Jiefeng Li. Can Wang, Hao Zhu, Yihuan Mao, Hao-Shu Fang, and Cewu Lu. Crowdpose: Efficient crowded scenes pose estimation and new benchmark. In IEEE Conf. Comput. Vis. Pattern Recog.. 2019b.
Jiefeng Li. Siyuan Bian, Ailing Zeng, Can Wang. Bo Pang, Wentao Liu, and Cewu Lu. Human pose regression with residual log-like likelihood estimation. In Int. Conf. Comput, Vis., 2021b. 1
Shuyuan Li, Jianguo Li, Weiyao Lin, and Hanlin Tang. Amur tiger re-identification in the wild. In Int. Conf. Comput. Vis. Worksh., 2019c. 2, 6, 7,8
Timothy P Lillicrap, Jonathan Hunt, Alexander Pritzel, Nicolas Heess, Tom Frez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. Int. Conf. Learn. Represent., 2016. 3
Tsung-Yi Yi Lin, M. Maire, Serge J. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick. Microsoft coco: Common objects context. In Eur. Conf. Comput. Vis., 2014. 2, 16
Mingsheng Long, Han Zhu, Jianmin Wang, and Michael Jordan. Unsupervised domain adaptation with residual transfer networks. Adv. Neural Inform. Process. Syst, 2016.
Tambet Matiisen, Avital Oliver, Taco Cohen, and John Schulman. Teacher-stule student curriculum learning. IEEE Transactions on Neural Networks and Learning Systems, 31(9):3732-377 2019. 3
Geoffrey J McLachlan. Iterative reclassification procedure for constructing an asymptotically optimal rule of allocation in discriminant analysis. Journal of the American Statistical Association, 706350))365-369, 1975.
11

---

Published as a conference paper at ICLR 2022
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Mare G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforeement learning. Nature, 518(7540):529 533, 2015. 3
Olga Moskvyak. Frederic Maire, Feras Dayoub, and Mahsa Baktashmotlagh. Semi-supervised keypoint local- ization. Int. Conf. Learn. Represent, 2020. 2, 6, 7, 16
Daichi Nakano, Shin-ichi Maeda, and Shin Ishii. Control a free-falling cat by policy-based reinforcement learning. In International Conference on Artificial Neural Networks, pp. 116-123. Springer, 2012. 5
Alejandro Newell, Kaiyu Yang, and Jia Deng. Stacked hourglass networks for human pose estimation. Eur. Conf. Comput. Vis., 2016.
Avital Oliver, Augustus Odena, Colin Raffel, Ekin Cubuk, and Ian Goodfellow. Realistic evaluation of deep semi-supervised learning algorithms. Adv. Neural Inform. Process. Syst., 2018.
Hicu Pham, Melody Guan, Barret Zoph, Quoe Lc, and Jeff Dean. Efficient neural architecture search via parameters sharing. In Int. Conf. Machine Learning,, pp. 4095-4104, PMLR, 2018. 3
Iijaa Radosavovic, Piotr Dollár, Ross Girshick, Georgia Gkioxari. and Kaiming He. Data distillation: Towards omni- onni-supereised learning. In JEEE Conf. Comput. Vis. Pattern Recog,, pp. 4119-411122 4128, 2018. 6,
Prajit Ramachandran, Barret Zoph, and Quoe V Le. Searching for activation functions. arXiv preprint
arXiv:1710.05941, 2017. 3
John Schulman, Sergey Levine, Picter Abheel, Michael Jordan, and Philipp Moritz, Trust region policy opti- mization. In Int. Conf. Machine Learning.. pp. 1889-1897. PMLR, 2015.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov, Proximal policy optimization algorithms. arXiv preprint arXiv: 1tt::1700 707.06347, 2017. 5,9
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwisser, Joannis Antonoglou, Voda Panneershelvam, Marc Lanctot, al. Mastering the game of go with deep neural networks and tree scarch. Nature, 529(7587):484 489, 2016. 3
David Silver, Julian Schrittiessrr Karen Simonyan, loannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowl- edge. Nature, 550(7676):354-355) 2017.
Valentin Spitkovsky, Hiyan Alshawi, and Daniel Jurafsky. Baby steps: How "less is more" in unsupervised dependency parsing. 2009. 3
Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. Deep high-resolution representation learning for human pose estimation. In IEEE Conf. Comput. Vis. Pattern Recog.. 2019. 5, 6, 7, 15, 16
Antti Tarvainen and Harri Valpola. Mean teachers are hetter role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Int. Conf. Learn. Represent, 2017. 2,4
Eric Tzeng, Judy Hoffman, Trevor Darrell, and Kate Saenko. Simultancous deep transfer across domains and tasks. In Int. Conf. Comput. Vis., pp. 4068-4076, 2015. 8
Norimichi Ukita and Yusuke Uematsu. Semi-and and Image Understanding, 170:67-78, 2018. 2
human pose estimation. Computer Vision
Welinder, S. Branson, T. Mita, C. Wah, F. Schroff, S. Belongie, and Perona. Caltech-UCS Birds 200.
Technical Report CNS ENS-TR-WN -00--T--20000 2010-001, California Institute of Technology, 2010. 2, 6, 7,8
Bin Xiao, Haiping Wu, and Yichen Wei. Simple baselines for human pose estimation and tracking. In Eur. Conf. Comput. Vis., 2018. 1, 8, 15
Lumin Xu, Yingda Guan, Sheng Jin, Wentao Liu, Chen Qian, Ping Luo, Wanli Ouyang, and Xiaogang Wang- Vipnas: Efficient video pose estimation via neural architecture search. In IEEE Conf. Comput. Vis. Pattern Recog.. 2021.
Sijie Yan, Yuanjun Xiong, and Dahua Lin. Spatial temporal graph convolutional networks for skeleton- based action recognition. In AAAI, 2018.
Feng Zhang, Xiatian 7hu, Hanbin Dai, Mao Ye, and Ce Zhu. Distribution-awarc coordinate representation for human pose estimation. In IEEE Conf. Comput. Vis. Pattern Recog, pp. 7093-7102, 2020. 15
12

---

Published as a conference paper at ICLR 2022
Zixing Zhang. Fabien Ringeval, Bin Dong, Eduardo Coutinho, Erik Marchi, and Bjöm Schüller. Enhanced semi-supervised learning for multimodal emotion recognition. In ICASSP, pp. 5185-5188 5189. IEEE, 2016. 2
Haiyu Zhao, Maoqing Tian, Shuyang Sun, Jing Shao. Junjie Yan, Shuai Yi, Xiaogang Wang. and Xiaoou Tang. Spindle net: Person re-identification with human body region guided feature decomposition and fusion. In IEEE Conf. Comput. Vis. Pattern Recog, pp. 1077-1085, 2017.
Qian Zhao, Deyu Meng, Lu Jiang. Qi Xie, Zongben Xu, and Alexander Hauptmann. Selff self-paced learning for matrix factorization. AAAI, 2015.
Dongzhan Zhou, Xinchi Zhou, Wenwei Zhang, Chen Change Loy, Shuai Yi, Xuesen Zhang, and Wanli Ouyang. Econas: Finding proxies for economical neural architecture search. IEEE Conf. Comput. Vis. Pattemn Recog. pp. 11396- 11404, 2020. 7, 16
Christian Zimmermann and Thomas Bιox. Learning to estimate hand pose from single rgb images. In Int. Conf. Comput. Vis., 4903 4911. 2017. 2
Barret Zoph and Quoe V Le. Neural architecture search with reinforcement learning. Int. Conf. Learn Represent, 2017. 3
Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc Le. Learning transferable architectures for scal- able image recognition. In JEEE Conf. Comput. Vis. Pattern Recog,, pp. 8697-8710, 2018. 3
13

---

Published as a conference paper at ICLR 2022
A APPENDIX
A.1 PSEUDO-CODE FOR PLACL ALGORITIMM
Here we present the pseudo-code for the proposed Pseudo-Labeled Auto-Curriculum Learning (PLACL) algorithm. In PLACL, the keypoint localization network and the curriculum policy are jointly optimized. In the inner-loop, we optimize the keypoint localization network as shown in Algorithm 1. In the outer-loop, the curriculum policy is updated according to the performance of the keypoint localization network, as shown in Algorithm 2.
Algorithm 1: Inner-loop network training
Inpul:
1. Labeled data ID; pscudo-labeled data DD,.
2. Number of training groups per round No:
3. Curriculum r:
Output: Obtained optimal network weights w (T).
Random initialization of network weights;
for g = 1 to NG do
Update current threshold 7/- [[g]
Compute data selection vector g (Eq. 1);
Construct the training set Devos (Eq. 2)
Update network weights w via back-propagttion:
return w (T))
Algorithm Pseudo-Labeled Auto-Curriculum Learning (PLACL)
Input:
1. Labeled data DD, unlabeled data D..
2. Initialized distribution μ and o².
3. Number of rounds R.
3. Searching steps T; Sampling number M.
4. Evaluation metric (e.g. PCK@0.1) g.
Output: Obtained the optimal curriculum I" and the final network O....
Initialize (T")* with all zeros.
Pre-train keypoint localization network 0% using labeled data DD,
for r R do
%% then
Predict keypoint pseudo-labels D(") with e/=
B. (())
else
Predict keypoint pscudo-labels D with O/=1;
D
D:
for
T do
for = 1 to M do
Sample parameter Ar Nuune [0.1] 02)) Ar,, 1I"
Get - -. via inner-loop network training using 12,, (Alg. 1): Compute the evaluation metric & (17,,) = 5(0% Drat)
Compute the objective function J (ι)) (Eq. 4);
Update wit1 + 1I aV J (p"):
(I")* arg max,;; Σn & (TT,) Vt 1..., T; Get Or, via network training using (Ir)* (Alg. 1); return [ [(T')',... ...., ([ and e²
14

---

Published as a conference paper at ICLR 2022
A.2 VISUALIZATION OF PSEUDO-LABELED SAMPLES
In order to provide a better illustration of how pseudo-labels evolve in self-training rounds, we visualize some pscudo-labeled samples for different datasets. We observe that the quality of pseudo- labels gradually improves with the increase of the self-training rounds.
Round 0
Round 1
Round 2
Round
Round
Figure 3: Visualization of how pseudo-labet, evolve in self-training rounds.
A.3 GENERALIZATION TO DIFFERENT KEYPOINT LOCALIZATION MODELS.
Table 4 shows the improvement when PLACLE is applied to the recent state-of-the-art keypoint local- ization models which vary in model architectures and training/testing techniques. The experiments conducted on LSPET dataset with 5% labeled images and 95% unlabeled images. We show that PLACL consistently improves the performance of the state-of-the-ant approaches by a large margin. PLACL does not require any knowledge of the keypoint localization models, making it easy to in practice.
Table 4:
Performance improvement of different keypoint localization methods by PLACL. Experi- ments are conducted LUB-200-2011 dataset (5% labeled data) with PCK @0.1 as the metric.
Method
Backbone
w/o PLACL
w/ PLACL
SimpleBaseline (Xiao al., 2018)
ResNet-50
79.16
93.51
SimpleBaseline (Xiao et al., 2018)
ResNet-101
81.34
93.66
SimpleBaseline (Xiao al., 2018)
ResNet-152
86.15
94.27
HRNet (Sun et al., 2019)
HRNet-w32
85.86
93.01
HRNet (Sun et al., 2019)
HRNet-w48
85.89
94.26
DARK (Zhang et al., 2020)
HRNet-w32
86.67
94.18
15

---

Published as a conference paper at ICLR 2022
A.4 ANALYSIS OF PROXY TASKS
There are lot of methods that target at improving the searching efficiency in literature, e.g. using reduced-training proxy tasks (input size, model size, training samples, and training epochs can be reduced (Zhou et al., 2020)). With these techniques, we are able to get orders of magnitude less computation cost, but can still match the performance. Therefore, we believe that the scalability is not a problem. For example, in order to further reduce the complexity, we use a light proxy task (with reduced training samples) for the RL search process. Specifically, we randomly select a small proportion of the training data (e.g. 5k images) for efficient curriculum search. After the search procedure, we re-train the keypoint networks with the searched curriculum on the full training set and evaluate them on the test set.
As shown in Table 5, we randomly select different number of training images (5k and 10k) for RL curriculum search. We find that reducing the number of training images by half (from 10k to 5k) does not decrease the final performance much, which validates the effect of using proxy tasks. Such strategy enables us to apply the proposed PLACL approach to large-scale datasets, such as MS-COCO (Lin et al., 2014) and the full MPII (Andriluka et al., 2014) datasets.
Table 5: We randomly select different number training images (#Images) for RL curriculum search, and re- re-train the keypoint networks with the searched curriculum on the full training set.
#Images
5%
10%
20%
50%
100%
LSPET
5k
70.72::1.49
71.93
72.2411.82
12.71=1.19
10k
1.47
71.91
7.880
72.73 1.23
MS-COCO'2017
69.3
10.11:10.89 7..64:00.66
0.54
10k
1.024
70.12:10.87
71.61
73.43
A.5 EXPERIMENTS ON THE FULL MPII DATASET
As shown in Table 6, we provide the results on the full MPII (Andriluka et al., 2014) dataset. Since the codes of SSKL (Moskvyak al., 2020) not publicly available, only compare with CL (Cascante-Bonil
et al., 2021) in the experiments. We see that our proposed PLACL consis- tently outperforms CL. especially for low labeled data regime (5% and 10%). Note that our results on the full MPII are obtained using reduced-training proxy tasks, i.e. we use 5K images for RL curriculum search, and re-train the model with the obtained curriculum on the full training set.
Table Comparisons with CL on the full MPII dataset.
Method
5%
10%
20%
50%
100%
Full MPII
HRNet (Sun et al., 2019) CL Messannee-Bonille et al., 2021)
18.00=1..35
81.89-10.94
82.94-0.67
88.34:40.55
89.76:10.17
60.38=1.33
83.0641..89
84.57=0.68
88.72:10.34
PLACL (Ours)
$2.21=1.33
05.5
86.24-0.33 0.56
09.16-+0.21
16


Reviews:
[{"review_id": "SjMEmgCFKiK", "reviewer": "Reviewer_UzKw", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strengths\n\n1. The idea of performing ACL using PPO2 is simple and elegant.\n2. The paper is well written and easy to read.\n3. The method is always either on par or outperforms the keypoint localization network competitors.\n\n## Weaknesses/Questions\n\n1. PLACL seems computationally very intensive. In the experiments M=8, R=6 and T=16, so it means that 736 networks are trained from scratch, isn't it ? If I am not mistaken I did not see any information concerning the training time of PLACL, type and number of GPUs used, and a comparison against sota methods. For instance, how does PLACL compare against SSKL in terms of training time, memory footprint, etc.?\n2. The results that are reported in the experiments (Table 1 and 2) for the line \"PLACL (ours)\" correspond to a forward pass in your implementation of HRNet using which weights? In Alg. 2, the output is the \"optimal curriculum\". Does it mean that a final training is performed using the last threshold of the optimal curriculum? If that is the case, I suggest to add this step at the end of Alg.2 and return the weights of this network.\n3. PLACL seems application agnostic. The results are always either on par or outperform the keypoint localization network competitors but the improvement w.r.t. SSKL for instance is not huge. I suggest to consider a second application to strengthen the paper.\n4. In section 4.2, it is said \"However, the analysis of their combined effects is outside the scope of this work.\". If there exists methods (with codes available) that combine these effects, I suggest to include them in the experiments, even if they outperform PLACL.\n5. In Alg.2, pseudo-labels are predicted using $N_{\\omega^*}^{r-1}$, is it the network obtained at the previous round for j=M and t=T or do you train another network with ($\\Gamma^{r-1})^*$ ? I think this question is related to my remark 2.", "summary_of_the_paper": "A semi-supervised learning method (PLACL) is proposed. This method employs a Pseudo-Labeling (PL) approach. It consists in iteratively (these iterations are called rounds):\n\n1. predicting pseudo-labels to unlabeled data using the current model,\n2. training a series of models from scratch using the labeled data and selections of pseudo-labeled data. The pseudo-label selection is performed using a series of thresholds (called curriculum) over the scores output by the model.\n\nThe authors propose an Auto-Curriculum Learning (ACL) strategy to automatically update the curriculum using a reinforcement learning approach (PPO2). The fact that the curriculum is updated at each round is called \"Curriculum Residual Learning\". They also employ a cross-training strategy to prevent the issue of confirmation bias.\n\nThe performances of PLACL are evaluated on a keypoint localization application on 5 different datasets. When the percentage of labeled data is very low (5% or 10%), PLACL outperforms the state of the art semi-supervised learning competitors.\n", "summary_of_the_review": "The paper is interesting and well written but the novelty is limited (using PPO2 to update the thresholds), the results are not very impressive (especially compared to SSKL) and the evaluation is limited to a single application (keypoint localization) while PLACL is application agnostic. Thus I believe the paper is not ready for a publication at ICLR. Please answer my questions from the \"main review\".\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "C108q_pp8_T", "reviewer": "Reviewer_QPpV", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. The authors provide both intuition and theoretical explanations of the proposed method supported by experimental results. \n2. Although all components are widely used techniques in the field, the application of RL to tackle pseudo-labeled sample selection for keypoint localization is novel.\n3. The proposed method achieves the improvement upon the state-of-the-art on several benchmarks for human and animal body landmark localization and specifically in a low labeled data regime (5% of data is labeled).\n4. Informative ablation studies and evaluation of the generalization ability on domain transfer.\n\nWeaknesses:\n1. The RL part of the approach has many moving parts and it would be beneficial to justify the choices of hyperparameters in the proximal policy optimization algorithm.\n2. Inner-loop network training is executed multiple times during policy learning. How has the training complexity (time to convergence) increased compared to previous works? \n\nQuestions:\n1. Section 3.2: Is Eq(1) computed per image or per keypoint given that each image has K keypoints? If an image has some keypoints with the confidence above a threshold and some are below, does the image get selected for the training round?\n2. How does the proposed pseudo-labeling strategy deal with not-visible keypoints in the image (e.g., a left eye and a left-wing of a bird are not visible if the bird is depicted from the right-side viewpoint)? \n", "summary_of_the_paper": "The paper introduces a method for semi-supervised keypoint localization based on pseudo-labeling with auto-curriculum learning. The Auto-curriculum learning approach learns a series of dynamic thresholds for automatic selection of high-quality pseudo-labeled examples for model retraining. The reinforcement learning (RL) framework, more specifically, the proximal policy optimization algorithm, is used to search for the optimal curriculum. The method is evaluated on four benchmarks in keypoint localization.", "summary_of_the_review": "The paper is well-written and easy to follow. The proposed method is described in detail. The method is evaluated on four popular datasets for the keypoint localization task. The proposed method demonstrates superior performance especially in cases with only 5% of labeled data (out of no more than 10,000 examples). Ablation studies justify the design choices. The method, while combining existing techniques, is proven experimentally to be superior to the previous works and will add to the body of knowledge on keypoint localization.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "-QVHhLpx5Ag", "reviewer": "Reviewer_6dEq", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "##########################################################################\n\nStrengths:\n1. The task is practical and motivation is well. How to select the threshold of pseudo label is an important and complicated task.\n\n2. Overall, the method of this paper is technically reasonable and novel. The paper first applies curriculum learning to semi-supervised keypoint localization and proposes use RL to search the best curriculum.\n\n3. The experiments show that the proposed method is effective compared to other SSL methods. The ablation study validates the three parts of proposed method are all important and can improve performance (about 2%-5%).\n\n\n\n##########################################################################\n\nWeaknesses:\n1. In my opinion, the time complexity of proposed algorithm is too high. The RL search process increases the training time by T*M times (=128 in this paper). The high complexity will make the model less scalable, such as in dataset size. The author maybe can explain about the current training cost and whether the scalability is indeed a problem.\n\n2. The dataset in experiments in the article is somewhat simple and small in scale. If the mainstream datasets in human pose estimation such as COCO keypoint, or full MPII, can be used in experiments, the contributions would be more convincing.\n\n3. The experimental comparison in Effect of parameter search is not sufficient, and the selected baseline method is weak.\nThe paper only compares Random Search and does not fully explain the details of this baseline. Does this baseline method also select the optimal strategy from candidates of the same size (T*M)?\nIn addition, considering that using RL to search is one of the main contributions of this paper, the author should consider comparing other stronger and more comprehensive baselines. For example, manually design many curriculums whose thresholds are gradually decreasing (on the epoch level) and select the curriculum with the best result.\n\n\n\n\n##########################################################################\n\nMinor comments:\n1. The division and use of data sets are not particularly clear. Are the reported numbers all evaluated in testing set, and D_val in Eq.3 uses validation set? But the paper says that the MPII validation set is for evaluation. So what is the D_val in MPII.\n\n2. In the paper, the symbol N represents both the keypoint network and the number of epochs, which is somewhat confusing.\n\n3. In my opinion, the proposed method is not particularly related to the keypoint localization task. It will be better if this method can be applied to other tasks.\n\n4. In addition, the author maybe can add discussion on the following papers about semi-supervised human pose estimation.\nRongchang Xie, Chunyu Wang, Wenjun Zeng, Yizhou Wang. An Empirical Study of the Collapsing Problem in Semi-Supervised 2D Human Pose Estimation. ICCV 2021.\n\n##########################################################################\n\nUpdates:\n\nThanks for the authors' response. The response and new resutls address my main concern. I tend to accept this paper. \n\n", "summary_of_the_paper": "1. This paper introduces Curriculum Learning to semi-supervised keypoint localization, which is an automatic pseudo-labeled data selection method. The method uses reinforcement learning to learns a series of dynamic thresholds. \n\n2. Besides, this paper proposes the cross-training strategy for pseudo-labeling to alleviate confirmation bias. \n\n3. The experiments shows that the proposed method can effectively improve the performance in different dataset and surpass other semi-supervised methods.\n", "summary_of_the_review": "This paper proposes a novel and effective threshold selection method for semi-supervised keypoint localization. Meanwhile, I think there are some weaknesses in practicality. I currently choose borderline accept. The author may can explain about the weaknesses.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a pseudo-labeled data selection method for semi-supervised pose estimation. The investigated task in this paper is practical and useful. The framework is well designed and reasonable, and extensive ablation studies are conducted to test the efficacy of the method. After discussion, all the reviewers recommend accept of this paper.
Decision: Accept (Poster)