Semi-Supervised
Semantic Segmentation

background• DataFully-Supervisedlabeled images

Semi-Supervisedlabeled images+unlabeled images

Semi-Supervised Segmentation• Mainstream Methods
Mean-Teacher

studentEMA

Self Training

Supteacher
features:
1. Joint training.
2. Exponential Moving Average to update teacher.

features:1. serial training
2. multi-stage training procedure
3. Intermediate results

Semi-Supervised Segmentation• Mean-Teacher
Classic CPS:

representative works:
Mean-Teacher [NeurIPS 2017]
CCT[CVPR 2020]
Consistency method
CutMix-Seg[BMCV 2020]
CutMix SDA in SSSSGCT[ECCV 2020]
CPS[CVPR 2021]

Features:1. Cross Pseudo-label supervision.
2. use CutMix SDA to further boost performance.

AEL[NeurIPS 2021]
U2PL[CVPR 2022]

Semi-Supervised Segmentation• Self-Training

representative works:Naïve Student [ECCV 2020]
Noisy Student [CVPR 2020]
DSBN [ICCV 2021]
ST++ [CVPR 2022]
research status:
1. few attention
2. Limited improvement
3. Weak performance
Compared with MT kinds.

U2PL• motivation
Previously works select the highly confident predictions as the pseudo ground-truth.
Weakness: most pixels may be left unused due to unreliability.
-> make sufficient use of unlabeled images
Intuition
PersonBicycle
MotorbikeCarSofa
Airplane

0.60
0.18
0.14
0.03
0.03
0.02

1. An unreliable prediction may get
confused among the top classes, however,it should be confident about the pixel notbelonging to the remaining classes.
2. Such a pixel can be convincingly treatedas a negative sample to those mostunlikely categories. -> Contrastive
Learning

U2PL

Reliable/Unreliable split

U2PL
Reliable criterion quantify𝑝!" ∈ 𝑅 # , softmax probabilities generated by the segmentation
head of the teacher model for the ith unlabeled image at pixel j

DPA -- Dynamic Partition Adjustment

Anchor, Positive and negative
PRT -- Probability Rank Threshold
Anchor pixels (representation)
for class c

Negative samples

Labeled images:(a) not belongs to class c;(b) similar to class c.
𝛿$ = 0.3
Positive samples

Unlabeled images:(a) be unreliable;(b) probably not belongs to class c;(c) not belongs to most unlikely classes.

U2PL

InfoNCE loss

Experiments

Experiments
Results on Pascal VOC

Results on Cityscpaes

Ablation StudyAblation StudyAnalysis
Feature space visualization

ST++• motivation
1. construct a strong baseline of self-training via injecting strong dataaugmentations (SDA) on unlabeled images.
alleviate overfitting noisy labels,
decouple similar predictions between the teacher and student

2. [Novelty] selective re-training via prioritizing reliable unlabeled images.
incorrect pseudo labels are prone to accumulate and degrade the performance.

ST++Select and Prioritize Reliable Images
Observation:
positive correlation between the segmentation performance and the evolving stability of
produced pseudo masks during the supervised training phase.

How to evaluate stability of pseudo masks?
Unlabeled image 𝑢! ,'K checkpoints 𝑇""%&

saved during training,

Predict pseudo masks 𝑀!"'"%&

Experiments
Results on Pascal VOC val set

Experiments
Results on cityscapes val set

Results on Pascal VOC val set

Data Augmentation Strategies
Effectiveness of SDA

Reliable Effectiveness
Reliable selection
improve pseudo mask
quality.
Need compare withretraining with all
unlabeled data.

Ablation Study
Ablation on different re-training methods