  Being able to segment unseen classes not observed during training is an
important technical challenge in deep learning, because of its potential to
reduce the expensive annotation required for semantic segmentation. Prior
zero-label semantic segmentation works approach this task by learning
visual-semantic embeddings or generative models. However, they are prone to
overfitting on the seen classes because there is no training signal for them.
In this paper, we study the challenging generalized zero-label semantic
segmentation task where the model has to segment both seen and unseen classes
at test time. We assume that pixels of unseen classes could be present in the
training images but without being annotated. Our idea is to capture the latent
information on unseen classes by supervising the model with self-produced
pseudo-labels for unlabeled pixels. We propose a consistency regularizer to
filter out noisy pseudo-labels by taking the intersections of the pseudo-labels
generated from different augmentations of the same image. Our framework
generates pseudo-labels and then retrain the model with human-annotated and
pseudo-labelled data. This procedure is repeated for several iterations. As a
result, our approach achieves the new state-of-the-art on PascalVOC12 and
COCO-stuff datasets in the challenging generalized zero-label semantic
segmentation setting, surpassing other existing methods addressing this task
with more complex strategies.
