P SEUDO -L ABELED A UTO -C URRICULUM L EARNING FOR S EMI -S UPERVISED K EYPOINT L OCALIZATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Localizing keypoints of an object is a basic visual problem.
However, supervised learning of a keypoint localization network often requires a large amount of data, which is expensive and time-consuming to obtain.
To remedy this, there is an ever-growing interest in semi-supervised learning (SSL), which leverages a small set of labeled data along with a large set of unlabeled data.
Among these SSL approaches, pseudo-labeling (PL) is one of the most popular.
PL approaches apply pseudo-labels to unlabeled data, and then train the model with a combination of the labeled and pseudo-labeled data iteratively.
The key to the success of PL is the selection of high-quality pseudo-labeled samples.
Previous works mostly select training samples by manually setting a single conﬁdence threshold.
We propose to automatically select reliable pseudo-labeled samples with a series of dynamic thresholds, which constitutes a learning curriculum.
Extensive experiments on six keypoint localization benchmark datasets demonstrate that the proposed approach signiﬁcantly outperforms the previous state-of-the-art SSL approaches.
I NTRODUCTION
Keypoints (also termed as landmarks) are a popular representation of objects that precisely represent locations of object parts and contain concise information about shapes and poses.
Example keypoints are ”right shoulder” on a human body or the ”tail tip” of a cat.
Keypoint localization is the basis of many visual tasks, including action recognition (Yan et al., 2018), face recognition (Bulat & Tzimiropoulos, 2017), ﬁne-grained classiﬁcation (Gavves et al., 2013; 2015) and reidentiﬁcation (Zhao et al., 2017; Miao et al., 2019).
Keypoint localization has achieved great success with the advent of deep learning in recent years.
However, the success of deep networks relies on vast amounts of labeled data, which is often expensive and time-consuming to collect.
Semi-supervised learning (SSL) is one of the most important approaches for solving this problem.
It leverages extensive amounts of unlabeled data in addition to sparsely labeled data to obtain gains in performance.
Pseudo-labeling (PL) has become one of the most popular SSL approaches due to its simplicity.
PL-based methods iteratively add unlabeled samples into the training data by pseudo-labeling them with a model trained on a combination of labeled and pseudo-labeled samples.
PL-based methods commonly require a predeﬁned handpicked threshold (Lee et al., 2013; Oliver et al., 2018), to ﬁlter out low-conﬁdence noisy predictions.
However, a single ﬁxed threshold does not take into account the dynamic capacity of the current model for handling noisy pseudo-labels, leading to sub-optimal performance.
In this work, we borrow ideas from Curriculum Learning (CL) (Bengio et al., 2009) and design our curriculum as a series of thresholds for PL, which is tuned according to the feedback from the model.
CL is a widely used strategy to control the model training pace by selecting from easier to harder samples.
With a carefully designed curriculum, noticeable improvement is obtained.
However, traditional CL methods suffer from hand-designed curricula, which heavily rely on expertise and detailed analysis for speciﬁc domains.
Manual curriculum design based on handcrafted criteria is always tedious and sub-optimal.
Moreover, curriculum design (or threshold setting) is complicated.
High-conﬁdence pseudo-labels typically correspond to easier samples with clean labels, while low-conﬁdence pseudo-labels correspond to harder samples with noisy labels.
How to design a curriculum to balance the correctness, representativeness, and difﬁculty of pseudo-labeled data is an open problem.
This paper is devoted to tackling the aforementioned problem, i .
e .
how to automatically learn an optimal learning curriculum for pseudo-labeling in a data-driven way.
To this end, we propose a novel method, called PseudoLabeled Auto-Curriculum Learning (PLACL).
PLACL formulates the curriculum design problem as a decision-making problem and leverages the reinforcement learning (RL) framework to solve it.
Additionally, PL-based methods suffer from conﬁrmation bias (Tarvainen & Valpola, 2017), also known as noise accumulation (Zhang et al., 2016), and concept drift (Cascante-Bonilla et al., 2021).
This long-standing issue stems from the use of noisy or incorrect pseudo-labels in subsequent training stages.
As a consequence, the noise accumulates and the performance degrades as the learning process evolves over time.
To mitigate this problem, we propose the cross-training strategy which alternatively performs pseudo-label prediction and model training on separate sub-datasets.
We benchmark PLACL on six keypoint localization datasets, including LSPET (Johnson & Everingham, 2011), MPII (Andriluka et al., 2014), CUB-200-2011 (Welinder et al., 2010), ATRW (Li et al., 2019b), MS-COCO Lin et al. (2014), and AnimalPose (Cao et al., 2019).
We empirically show that PLACL is general and can be applied to various keypoint localization tasks (human and animal pose estimation) and different keypoint localization networks.
With a simple yet effective search paradigm, our method signiﬁcantly boosts the keypoint estimation performance and achieves superior performance to other SSL methods.
We hope our method will inspire the community to rethink the potential of PL-based methods for semi-supervised keypoint localization.
Our main contributions can be summarized as follows:
• We propose Pseudo-Labeled Auto-Curriculum Learning (PLACL).
It is an an automatic pseudo-labeled data selection method, which learns a series of dynamic thresholds (or curriculum) via reinforcement learning.
To the best of our knowledge, this is the ﬁrst work that explores automatic curriculum learning for semi-supervised keypoint localization.
• We propose the cross-training strategy for pseudo-labeling to mitigate the long-standing problem of conﬁrmation bias.
• Extensive experiments on a wide range of popular datasets demonstrate the superiority of PLACL over the previous state-of-the-art SSL approaches.
In addition, PLACL is modelagnostic and can be easily applied to different keypoint localization networks.
R ELATED W ORKS 2. S EMI - SUPERVISED KEYPOINT LOCALIZATION
Keypoint localization focuses on predicting the keypoints of detected objects, e .
g .
human body parts (Lin et al., 2014; Jin et al., 2020), facial landmarks (Bulat & Tzimiropoulos, 2017), hand keypoints (Zimmermann & Brox, 2017) and animal poses (Li & Lee, 2021; Cao et al., 2019).
However, training a keypoint localization model often requires a large amount of data, which is expensive and time-consuming to collect.
Semi-supervised keypoint localization is one of the most promising ways to solve this problem.
Semi-supervised keypoint localization can be categorized into consistency regularization based methods and pseudo-labeling based methods.
Consistency regularization methods (Honari et al., 2018; Moskvyak et al., 2020) assume that the output of the model should not be invariant to realistic perturbations.
These approaches typically rely on modality-speciﬁc augmentation techniques for regularization.
Pseudo-labeling methods (Ukita & Uematsu, 2018; Dong & Yang, 2019;
Cao et al., 2019; Li & Lee, 2021) use labeled data to predict the labels of the unlabeled data, and then train the model in a supervised way with a combination of labeled and selected pseudo-labeled data.
Our approach also builds upon pseudo-labeling methods.
In contrast to previous works, we propose to learn pseudo-labeled data selection via reinforcement learning.
2. C URRICULUM L EARNING
Curriculum learning is ﬁrstly introduced by Bengio et al.
It is a training strategy that trains machine learning models from easy to complex samples, imitating human education.
The curriculum is often pre-determined by heuristics (Khan et al., 2011; Bengio et al., 2009; Spitkovsky et al., 2009).
However, it requires expert domain knowledge and exhaustive trials to ﬁnd a good curriculum suitable for a speciﬁc task and its dataset.
Recently, automatic curriculum learning methods are
[Figure]
introduced to break through these limits.
Popular ones include self-paced learning methods (Kumar et al., 2010; Jiang et al., 2014;
Zhao et al., 2015) and reinforcement learning (RL) based methods (Graves et al., 2017; Matiisen et al., 2019; Fan et al., 2018).
Our approach can be categorized as RL-based methods.
Unlike previous works that mainly focus on supervised learning, our approach is specially designed for the SSL paradigm.
Our work is mostly related to Curriculum
Labeling (Cascante-Bonilla et al., 2021).
It adopts a hand-crafted curriculum based on Extreme
Value Theory (EVT) to facilitate model training.
In contrast, we propose an automatic curriculum learning approach by searching for dynamic thresholds for pseudo-labeling.
In addition, the curriculum of (Cascante-Bonilla et al., 2021) is coarse-grained on the round level, while our curriculum is ﬁne-grained on the epoch level.
2. R EINFORCEMENT LEARNING FOR A
UTO ML
Reinforcement learning (RL) has shown impressive results in a range of applications.
Well-known examples include game playing (Mnih et al., 2015; Silver et al., 2016; 2017) and robotics control(Schulman et al., 2015; Lillicrap et al., 2016).
Recent works have employed RL to the AutoML, automating the design of a machine learning system, e .
g .
searching for neural architectures (Zoph & Le, 2017; Zoph et al., 2018; Baker et al., 2017; Pham et al., 2018), augmentation policies (Cubuk et al., 2019), activation functions (Ramachandran et al., 2017), loss functions (Li et al., 2019a; 2021), and training hyperparameters (Dong et al., 2020).
In contrast to these works, we apply RL to the automatic selection of pseudo-labeled data in the context of pseudo-labeling.
P SEUDO - LABELED A UTO -C URRICULUM L EARNING (PLACL) 3. O VERVIEW
Our PLACL algorithm is illustrated in Fig.
The training process consists of R self-training rounds and each round consists of N training epochs.
In the initial round ( r = 0 ), we pre-train a keypoint localization network Θ 0 ω on the labeled data, where ω denotes the weights of the network.
And for the r -th round, (1) The trained network Θ rω is used to predict pseudo-labels for unlabeled data.
(2) We adopt reinforcement learning (RL) to automatically generate the learning curriculum.
Speciﬁcally, our curriculum ( Γ r ) consists of a series of thresholds for pseudo-labeled data selection.
Γ r =
[ γ r 1 , . .
, γ rN ] , where γ ri ∈ [0 , 1] is the threshold for each epoch i .
(3) We then select reliable pseudo-labeled data by the searched curriculum.
(4) We retrain a new model ( Θ r +1 ω ) using both the labeled samples and selected pseudo-labeled samples.
This process is repeated for R rounds.
3. P SEUDO - LABEL S ELECTION FOR SEMI - SUPERVISED KEYPOINT LOCALIZATION
We denote the labeled dataset with N l samples as D l = (cid:110)(cid:0) I li , Y li (cid:1)(cid:12)(cid:12) N l i =1 (cid:111) ,where I i
and
Y
i denote the i -th training image and its keypoint annotations (the x-y coordinates of K keypoints).
The N u unlabeled images are denoted as D u = (cid:110)
( I ui )
|
N u
i =1 (cid:111) , which are not associated with any ground-truth keypoint labels.
Generally, we have | N l | (cid:28) | N u | .
Pseudo-labeling based method builds upon the general idea of self-training (McLachlan, 1975), where the keypoint localization network Θ ω goes through multiple rounds of training.
In the initialization round, the model is ﬁrst trained with the small labeled training set D train =
D l in a usual supervised manner.
In subsequent rounds, the trained model is used to estimate labels for the unlabeled data ˜ D
u = (cid:26)(cid:16)
I ui , ˜ Y ui (cid:17)(cid:12)(cid:12)(cid:12)
N u
i =1 (cid:27) .
Here, we omit the superscript r for simplicity.
Speciﬁcally, given an unlabeled image I ui , the trained keypoint localization network Θ ω predicts K heatmaps.
Each heatmap is a 2D Gaussian centered on the joint location, which represents the conﬁdence of the k -th keypoint.
The output pseudo-labeled keypoint location ( ˜ Y ui ) is the highest response in the heatmap space.
And the conﬁdence score C (Θ ω ( I ui )) is the response value at the keypoint location.
Then, pseudo-label selection process is adopted.
Let g =
[ g 1 , ..., g N u ] ⊆ { 0 , 1 } N u be a binary vector representing the selection of pseudo-labels, where g i denotes whether the keypoint prediction on I ui is selected.
[Equation]
where γ ∈ (0 , 1) is the conﬁdence threshold.
Pseudo-labeled samples with higher conﬁdence are added to the training set.
[Equation]
Then the keypoint localization network is retrained with a combination of labeled and pseudolabeled training data D train .
3. C ROSS - TRAINING STRATEGY
In this section, we introduce the cross-training strategy in our curriculum learning framework.
In a typical self-training round, the model predicts noisy pseudo-labels which are used in subsequent training stages.
Since the pseudo-label prediction is performed on a dataset of known data (on which training is performed), the noise accumulates in a positive feedback loop.
This causes the long-standing issue of conﬁrmation bias (Tarvainen & Valpola, 2017).
To mitigate this problem, we propose the cross-training strategy.
Speciﬁcally, we randomly partition the unlabeled data D u into two complementary subsets D (1) u and D (2) u .
The partition is ﬁxed across all the training rounds.
The self-training conducts alternatively for D (1) u and D (2) u .
For r = { 1 , . . .
, R } , when r is odd, we use a combination of ˜ D (1) u and D l to train the model, and the trained model predicts pseudo-labels for the next round on D (2) u .
When r is even, we use a combination of ˜ D (2) u and D l to train the model, and perform pseudo-label prediction on D (1) u .
Before each round, the model parameters are re-initialized with random weights following Cascante-Bonilla et al.
(2021) to avoid noise accumulation.
3. C URRICULUM R ESIDUAL L EARNING
Directly learning for R rounds of curriculum parameters separately can be inefﬁcient.
Considering that the model is reinitialized and retrained in each self-training round, we assume that the optimal curricula for different rounds should have some similar patterns.
We propose a greedy multi-step searching algorithm.
For round r , we use the searched curriculum in the previous round r − 1 as the base curriculum to guide the searching of current curricula Γ r .
Inspired by ResNet (He et al.,
, we propose the curriculum residual learning strategy.
Formally, we learn a bias term ∆Γ r around the base curriculum, i .
e .
Γ r =
(Γ r − 1 ) ∗ + ∆Γ r , where (Γ r − 1 ) ∗ means the searched optimal curriculum for round r − 1 , and (Γ 0 ) ∗ is initialized with all zeros.
We empirically ﬁnd that this strategy accelerates the model convergence speed and achieves marginally better performance.
To further reduce the search space, we choose every G epochs as an epoch group, which shares the same threshold parameters for pseudo-labeled data selection.
In total, we have N G epoch groups, and the size of each epoch group is G .
Therefore, the search space is reduced by a factor of G , where G = 10 in our implementation.
3. C URRICULUM S EARCH VIA R EINFORCEMENT L EARNING
Our PLACL can be formulated as an optimization problem shown in Eq.
In the inner-loop, we optimize the weights ω of the keypoint localization network Θ ω to minimize the training loss L (see Alg. 1).
In the outer-loop, we apply proximal policy optimization (PPO2) algorithm (Schulman et al., 2017) to search for the curriculum Γ that maximize the evaluation metric ξ ( e . g . PCK) on the validation set D val (see Alg. 2).
[Equation]
The training consists of R rounds.
In each round r , the PPO2 search process consists of T sampling steps.
In each step, M sets of parameters are sampled independently from a truncated normal distribution (Nakano et al., 2012; Fujita & Maeda, 2018), ∆Γ r ∼ N trunc[0 , 1] (cid:0) µ rt , σ 2 I (cid:1) , where µ rt and σ 2 I are the mean and covariance ( σ is ﬁxed to 0 .
2 in practice).
These sampled parameters are used to construct M different training curricula for training M keypoint localization networks separately.
Then the mean of the distribution is updated by PPO2 algorithm according to the evaluation score of the M networks.
The objective function of PPO2 is formulated in Eq. 4.
[Equation]
where the function CLIP( x, 1 − (cid:15), 1+ (cid:15) ) clips x to be no more than 1+ (cid:15) and no less than 1 − (cid:15) .
Following the common practice (Li et al., 2021), the mean reward is subtracted for better convergence.
˜ ξ (cid:0) Γ rj (cid:1) =
ξ (cid:0) Γ rj (cid:1) − 1 M (cid:80) Mj =1 ξ (cid:0) Γ rj (cid:1) and the policy π µ r is deﬁned as the probability density function (PDF) of the truncated normal distribution.
PPO2 enforces the probability ratio between old and new policies π µ r (cid:0) ∆Γ rj ; µ r , σ 2 I (cid:1) /π µ rt (cid:0) ∆Γ rj ; µ rt , σ
I (cid:1) to stay within a small interval to control the size of each policy update.
We then compute the gradients and update the parameters by µ rt +1 ← µ rt + α ∇ µ r J ( µ r ) with a learning rate of α > 0 .
After T sampling steps, we choose µ rt with the highest average evaluation score as (Γ r ) ∗ .
And (Γ r ) ∗ is used as the base curriculum for the next round.
And after R rounds, our ﬁnal optimal curriculum is obtained, Γ ∗ =
[(Γ 1 ) ∗ , . .
, (Γ R ) ∗ ] .
Training details In the training phase, the keypoint localization network and the curriculum search policy are simultaneously optimized.
For the outer-loop, the PPO2 (Schulman et al., 2017) search procedure is conducted for T = 16 sampling steps, and in each step M = 8 sets of parameters (curriculum) are sampled.
The clipping threshold is (cid:15) = 0 . 2 , and µ rt +1 is updated with the learning rate of α = 0 .
We empirically use R = 6 self-training rounds, and group size G = 10 for curriculum search.
For the inner-loop, we follow the common practice (Sun et al., 2019) to train the keypoint localization network with Mean-Squared Error (MSE) loss for N = 210 epochs per round.
Adam (Kingma & Ba, 2015) with a learning rate of 0 .
001 is adopted.
We reduce the learning rate by a factor of 10 at the 170-th and 200-th epochs.
Although the RL search process increases the training complexity, the total training cost is not too high (only 1.5 days with 32 NVIDIA Tesla
V100 GPUs).
More detailed training settings for each task are provided in §4.2 and §4.3.
[Table]
E XPERIMENTS 4. D ATASETS AND E VALUATION M ETRICS
Datasets: To show the versatility of PLACL, we conduct experiments on 5 diverse datasets.
LSPET (Leeds Sports Pose Extended Dataset) (Johnson & Everingham, 2010; 2011) consists of images of people doing sports activities. We use 10,000 images from (Johnson & Everingham, 2011) for training and 2,000 images from (Johnson & Everingham, 2010) for validation and testing.
MPII Human Pose dataset (Andriluka et al., 2014) is a well-known benchmark for human pose estimation.
The images are collected from YouTube videos, showing people doing daily human activities.
We follow (Moskvyak et al., 2020) to use 10,000 random images from MPII train for training, 3, images from MPII train for validation and MPII val for evaluation.
CUB-200-2011 (CaltechUCSD Birds-200-2011) (Welinder et al., 2010) dataset is a well-known dataset for SSL.
It consists of 200 ﬁne-grained bird species with 15 keypoint annotations.
We follow (Moskvyak et al., 2020) to split dataset into training (100 categories with 5,864 images), validation (50 categories with 2, images) and testing (50 categories with 2,966 images).
ATRW (Li et al., 2019b) dataset contains images of 92 Amur tigers captured from multiple wild zoos in challenging and unconstrained conditions.
For each tiger, 15 body keypoints are annotated.
The dataset consists of 3,610 images for training, 516 for validation, and 1,033 for testing.
MS-COCO’2017 (Lin et al., 2014) is a popular large-scale benchmark for human pose estimation, which contains over 150,000 annotated people.
We randomly select 500 images from COCO train for validation, the remaining training set (115k images) for training, and COCO val (5k images) for evaluation.
We use this dataset to validate the applicability of our approach on large-scale data.
AnimalPose (Cao et al., 2019) dataset contains 5,517 instances of ﬁve animal categories: dog, cat, horse, sheep, and cow.
It consists of 2,798 images for training, and the 810 images for validation and 1,000 images for testing.
We use it to test the generalization and domain-transfer capacity of our proposed method.
Evaluation Metrics: PCK (Probability of Correct Keypoint): A detected keypoint is considered correct if the distance between the predicted and true keypoint is within a certain threshold ( αl ), where α is a constant and l is the longest side of the bounding box.
We adopt PCK@0.1 ( α = 0.1) for LSPET (Johnson & Everingham, 2011), CUB-200-2011 (Welinder et al., 2010), and ATRW (Li et al., 2019b) datasets.
PCKh is adapted from PCK, where l is the head size that corresponds to 60% of the diagonal length of the ground-truth head box.
We adopt PCKh@0.5 ( α = 0.5) for
MPII (Andriluka et al., 2014) dataset.
Standard AP (Average Precision) is another commonly used evaluation metric.
It is based on object keypoint similarity (OKS), which measures the distance between predicted keypoints and ground-truth keypoints normalized by the scale of the object.
We use mAP for AnimalPose (Cao et al., 2019) datasets.
4. C OMPARISONS WITH THE S TATE - OF - THE -A RT SSL A PPROACHES
In Table 1, we compare with the supervised baseline (HRNet (Sun et al., 2019)) and other stateof-the-art SSL approaches.
We experiment with different percentages of labeled images (5%, 10%, 20%, 50%, and 100%).
For fair comparisons, all results are obtained using HRNet-w32 backbone with the input size of 256 × 256 .
We follow (Moskvyak et al., 2020) to prepare datasets and exclude half body transforms, and testing tricks (post-processing, and ﬂip testing).
Comparisons with consistency regularization methods.
ELT (Honari et al., 2018) (equivariant landmark transformation) loss encourages the model to output keypoints that are equivariant to input transformations.
Gen (Jakab et al., 2018) learns to extract geometry-related features through conditional image generation.
SSKL (Moskvyak et al., 2020) learns the pose invariant keypoint representations with semantic keypoint consistency constraints.
These consistency regularization methods have shown superior results over the supervised baseline, however, they are inferior to our
PLACL method on all datasets and different percentages of labeled samples.
Especially we show that
PLACL is mostly effective for low data regimes.
For example, in CUB-200-2011 dataset, PLACL with only 5% labeled data achieves better performance (93.01 vs 92.80) than SSKL with 50% labeled data.
And in LSPET dataset, we show that PLACL improves the performance of baseline by a large margin from 40.19 to 70.76 with 5% labeled images.
Comparisons with pseudo-labeling method.
We also compare with a pseudo-labeling (PL) baseline (Radosavovic et al., 2018).
Overall our proposed PLACL signiﬁcantly outperforms the pseudolabeled (PL) baseline on all datasets.
As pointed out by Moskvyak et al.
(2020), the vanilla pseudolabeling approach does not perform well for the keypoint localization task with a low data regime, due to the lack of an effective pseudo-label selection scheme.
Previous PL approaches generally require a relatively large ( > 50% ) labeled dataset to generate reliable pseudo-labels.
Instead, PLACL is able to automatically select high-quality pseudo-labeled samples, which is the key to the success of pseudo-labeling based methods.
Comparisons with curriculum-learning method.
Curriculum Labeling (CL) (Cascante-Bonilla et al., 2021) is a recently proposed approach that applies a hand-crafted curriculum to facilitate training of SSL.
We observe that our proposed PLACL signiﬁcantly outperforms CL, which validates the effectiveness of our proposed automatic curriculum learning. Experiments on large-scale datasets.
Inspired by Zhou et al.
(2020), in order to decrease the RL curriculum search cost for the large-scale MS-COCO (Lin et al., 2014) and full MPII (Andriluka et al., 2014) datasets, we use a light proxy task with reduced number of training samples (5k) for RL curriculum search. After the search procedure, we re-train the keypoint networks with the searched [Figure] s t
A cc u r a c y
[Figure]
curriculum on the full training set and evaluate them on the test set.
Please refer to A.4 for more analysis about proxy tasks and A.5 for experiments on the full MPII (Andriluka et al., 2014) dataset.
4. E VALUATION OF GENERALIZATION CAPACITY
Generalization to different domains.
We investigate the generalization ability of our proposed model on domain transfer.
To this end, we conduct experiments on AnimalPose (Cao et al., 2019) dataset in Table 2.
Speciﬁcally, we follow (Cao et al., 2019) to choose one animal class (e.g. cat) as the target domain and the remaining four classes for the source domain.
The images of the source domain are fully annotated, while the images of the target domain are unlabeled.
For fair comparisons, we adopt the AlphaPose model (Fang et al., 2017), which uses ResNet-101 (Xiao et al., 2018) as the backbone.
All models are trained with the pose-labeled human dataset involved.
The AlphaPose baseline model is pre-trained on the human dataset and ﬁne-tuned on the labeled source animal data.
For pseudo-labeling based approaches (Inoue et al., 2018; Cao et al., 2019), the unlabeled target animal data is used for pseudo-labeling.
For domain adaptation approaches (Tzeng et al., 2015; Long et al., 2016), the unlabeled target data is used for domain transfer.
Please refer to Cao et al.
for details about the compared approaches.
We observe that our proposed approach consistently outperforms the previous state-of-the-art methods on cross-domain semi-supervised learning.
[Table]
[Table]
4. A NALYSIS
Number of self-training rounds.
Along with the increasing of self-training rounds ( R ), the quality of the pseudo-labels gradually improves (see Fig. 3) and the test accuracy increases (see Fig. 2a) until saturation.
The experiments are conducted on multiple datasets with 5% labeled data.
Interest- ingly, different datasets require a different number of rounds to achieve optimal, because four-legged animals (ATRW (Li et al., 2019b)) have more pose variations than birds (CUB-200-2011 (Welinder et al., 2010)).
We use R = 6 , because further increasing R does not bring signiﬁcant gains.
Comparisons with pseudo-labeling with different ﬁxed thresholds.
As shown in Fig.
2b, we compare our PLACL with 10 static thresholds from 0 . 0 to 0 .
We observe that PLACL clearly outperforms all these ﬁxed threshold alternatives.
Moreover, we ﬁnd that the accuracy will be signiﬁcantly affected by different thresholds (61.01% PCK for γ = 0 .
vs 67.35% PCK for γ = 0 . 7 ).
Choice of epoch group size .
In Fig.
2b, we also compare the performance of different epoch group sizes ( G ).
We empirically ﬁnd that smaller G will produce better performance, but at the cost of increased search space.
We choose G = 10 to trade-off between accuracy and efﬁciency.
4. A BLATION S TUDIES
In Table 3, we present ablation studies to measure the contribution of each component.
All ablative experiments are conducted on LSPET (Johnson & Everingham, 2011) dataset with 5% labeled data.
PCK@0.1 is adopted as the evaluation metric.
Effect of cross-training strategy.
We ﬁnd that without cross-training strategy the accuracy significantly drops from 70 .
76 to 67 . 13 , due to noise accumulation over time.
Especially, we ﬁnd that there is little to no improvement after the ﬁrst self-training round.
Effect of curriculum learning.
We compare PLACL with the alternative that only searches for a ﬁxed threshold via RL.
We ﬁnd that using dynamic thresholds improves upon the ﬁxed-threshold alternative by a large margin, which validates the effectiveness of curriculum learning.
Effect of parameter search.
We compare PPO2 (Schulman et al., 2017) search with Random
Search (w/o PPO2 search).
We randomly sampled T × M curricula and pick out the best one for comparisons.
PPO2 search obtains much better performance ( 70 .
76 vs 65 . 42 ).
This indicates that the searching problem is non-trivial and that our searching algorithm is very effective.
We also compare with manually designed curricula whose thresholds are gradually decreasing on the epoch level.
We tried ﬁve curricula with different decrease slopes and reported the best one as Manually Design .
We observe that PLACL signiﬁcantly outperforms the manually-designed curriculum baseline which validates the effectiveness of automatic curriculum search.
[Table]
C ONCLUSIONS
In this paper, we propose a novel Pseudo-labeled Auto-Curriculum Learning (PLACL) for the task of semi-supervised keypoint localization.
We propose to learn a curriculum to automatically select reliable pseudo-labels and propose cross-training strategy to mitigate the conﬁrmation bias problem.
Extensive experiments on 6 diverse datasets validate the effectiveness and versatility of the proposed method.
The ideas of pseudo-labeling and consistency regularization are complementary and we will explore the combination of these two techniques in the future.
We believe that our proposed approach is generic and we plan to investigate the applicability of PLACL on other visual tasks, such as object detection and semantic segmentation.
A A PPENDIX A. P SEUDO - CODE FOR PLACL ALGORITHM
Here we present the pseudo-code for the proposed Pseudo-Labeled Auto-Curriculum Learning (PLACL) algorithm.
In PLACL, the keypoint localization network and the curriculum policy are jointly optimized.
In the inner-loop, we optimize the keypoint localization network as shown in
Algorithm 1.
In the outer-loop, the curriculum policy is updated according to the performance of the keypoint localization network, as shown in Algorithm 2.
Pre-train keypoint localization network Θ using for r = 1 to R do if r %2 =
= 1 then
Predict keypoint pseudo-labels ˜ D
(1) u with Θ r − 1 ω ∗ ; ˜ D u =
˜ D (1) u else
Predict keypoint pseudo-labels ˜ D
(2) u with Θ r − 1 ω ∗ ; ˜ D u = ˜ D (2) u for t = 1 to T do for j = 1 to M do
Sample parameter ∆Γ rj,t ∼ N trunc µ rt σ
I (cid:1) ; Γ
= ∆Γ +
Get Θ rω ∗ ,j,t via inner-loop network training using Γ rj,t (Alg. 1) ;
Compute the evaluation metric ξ (cid:0) Γ rj,t (cid:1) =
ξ (Θ rω ∗ j t ; D val ) ;
Compute the objective function J ( µ ) (Eq. 4);
Update µ rt +1 ← µ rt + α ∇ µ r J ( µ r ) ; (Γ r ) ∗ = arg max µ rt (cid:80) Mj =1 ξ (cid:0) Γ rj,t (cid:1) , ∀ t =
T ;
Get Θ via network training
; return
A. V ISUALIZATION OF P SEUDO -L ABELED S AMPLES
In order to provide a better illustration of how pseudo-labels evolve in self-training rounds, we visualize some pseudo-labeled samples for different datasets.
We observe that the quality of pseudolabels gradually improves with the increase of the self-training rounds.
[Figure]
Round
Round
Round
A. G ENERALIZATION TO
Table 4 shows the improvement when PLACL is applied to the recent state-of-the-art keypoint localization models which vary in model architectures and training/testing techniques.
The experiments are conducted on LSPET dataset with 5% labeled images and 95% unlabeled images.
We show that
PLACL consistently improves the performance of the state-of-the-art approaches by a large margin.
PLACL does not require any knowledge of the keypoint localization models, making it easy to use in practice.
[Table]
A. A NALYSIS OF PROXY TASKS
There are a lot of methods that target at improving the searching efﬁciency in literature, e . g .
using reduced-training proxy tasks (input size, model size, training samples, and training epochs can be reduced (Zhou et al., 2020)).
With these techniques, we are able to get orders of magnitude less computation cost, but can still match the performance.
Therefore, we believe that the scalability is not a problem.
For example, in order to further reduce the complexity, we use a light proxy task (with reduced training samples) for the RL search process.
Speciﬁcally, we randomly select a small proportion of the training data ( e .
g .
5k images) for efﬁcient curriculum search.
After the search procedure, we re-train the keypoint networks with the searched curriculum on the full training set and evaluate them on the test set.
As shown in Table 5, we randomly select different number of training images (5k and 10k) for
RL curriculum search.
We ﬁnd that reducing the number of training images by half (from 10k to 5k) does not decrease the ﬁnal performance much, which validates the effect of using proxy tasks.
Such a strategy enables us to apply the proposed PLACL approach to large-scale datasets, such as
MS-COCO (Lin et al., 2014) and the full MPII (Andriluka et al., 2014) datasets.
[Table]
A. E XPERIMENTS ON THE FULL MPII DATASET
As shown in Table 6, we provide the results on the full MPII (Andriluka et al., 2014) dataset.
Since the codes of SSKL (Moskvyak et al., 2020) are not publicly available, we only compare with
CL (Cascante-Bonilla et al., 2021) in the experiments.
We see that our proposed PLACL consistently outperforms CL, especially for low labeled data regime (5% and 10%).
Note that our results on the full MPII are obtained using reduced-training proxy tasks, i
[Table]
e .
we use 5K images for RL curriculum search, and re-train the model with the obtained curriculum on the full training set.
[Table]