Predicting Segmentation “Easiness” from the Consistency
for Weakly-Supervised Segmentation
Wataru Shimoda and Keiji Yanai
The University of Electro-Communications, Tokyo, Japan

Proposed Method

Objective
Weakly supervised segmentation- Use only image-level annotation

PersonFirst step
- Obtain segmentation mask by existing weakly supervised segmentation method.
Second step
- Mine better results by our proposed method
Third step
- Train full supervised segmentation model with mined supervisionhorseCar

Weakly supervised segmentation

Weakly supervisedannotation

Fully supervisedannotation

Contributions- Improved the method by Simonyan et al. [1] greatly- Achieved state-of-the-art in weakly-supervised
segmentation with PASCAL VOC 2012

Results(noisy)

BackgroundDistinct class specific saliency map(DCSM) [ECCV 2016]- Visualization using subtraction forclass specific map- Improved method of Simonyan et al.
[ICLRWS 2014]- The method adapts visualization for
multi class objects.
- Achieved high score on weakly
supervised segmentation.
Simple to complex(STC) [PAMI 2016]- Calculate seed area in advance- Train full supervised modelwith the seed
- Repeat this process.
（Self-pased learning）- Kind of EM-algorithm-> initialization is crucial.

miningFull supervised model (Deeplab)
ImgSegmentation

Supervision

Pred[Shimoda et al. ECCV 2016]

Experiments
Top5 retrieval results obtained by our proposed method on Pascal VOC 2012 dataset- Most of mined segmentation seeds are close to the ground truth.
- The results of sofa, chair and table include some noise. In general segmentation results ofthese class show low performance, hence the retrieved results are affected by the low
quality of prediction directly[Wei et al. PAMI 2016]

Estimating “Easiness”
Reason of fault of segmentation using visualization- Accuracy of classification- Various object size
- Gap between the classification task and the segmentation task
Estimation of “Easiness” using consistency(1)Consistency of visualization- The subtraction on DCSM generates clear class specific maps.
- We used defference between
visualization with subtraction and witout subtraction- This takes into account the classification result and image
complexity.
(2)Consistency of receptive field- On the inference of pixel surrounding pixel information is very
important.
- The receptive field size is related to the segmentation accuracystrongly.
- We used the change of receptive field on difference of input image
size

Pascal VOC 2012 validation set
Comnbination of “Easiness” with data augmentation

Result on Pascal VOC 2012 test set
Full supervised

O2PSDS
Deeplab

Year
2012
2014
2015

methodUsing additional annotations
Point annotation
VideoAcc
47.6
52.6
71.6
YearAcc

2016
2017

46.0
58.7

2015
2014
2016
2016
2016

24.9
33.8
45.1
51.2
51.7

2017
2017

52.8
55.7

Using only the image level label

Global pooling
Simonyan (Backward)
Ours (Backward)
STC (re-training)
SEC (re-training +unsupervised)
Ours (Backward + re-training)
Adversarial Erasing*We used the same networkarchitecture with deeplab model fortraining full supervised model.

Conclusion
Trade off exists between the number of training data and quality of training data.
The segmentation accuracy can be boost by data selection with data augmentation.