Venue: ICLR 2018 conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper presents an alternative adversarial loss function for image segmentation, and an additional loss for unlabeled images.

+ well written
+ good evaluation
+ good performance compared to prior state of art
- technical novelty
- semi-supervised loss does not yield significant improvement
- missing citations and comparisons

The paper is well written, structured, and easy to read.
The experimental section is extensive, and shows a significant improvement over prior state of the art in semi-supervised learning.
Unfortunately, it is unclear what exactly lead to this performance increase. Is it a better baseline model? Is the algorithm tuned better, or is there something fundamentally different compared to prior work (e.g. Luc 2016).

Finally, it would help if the authors could highlight their technical difference compared to prior work. The presented adversarial loss is similar to Luc 2016 and "Image-to-Image Translation with Conditional Adversarial Networks, Isola etal 2017". What is different, and why is it important?
The semi-supervised loss is similar to Pathak 2015a, it would help to highlight the difference, and show experimentally why it matters.

In summary, the authors should highlight the difference to prior work, and show why the proposed changes matter.
review_title: review

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposed an approach for semi-supervised semantic segmentation based on adversarial training. Built upon a popular segmentation network, the paper integrated adversarial loss to incorporate unlabeled examples in training. The outputs from the discriminator are interpreted as indicators for the reliability of label prediction, and used to filter-out non-reliable predictions as augmented training data from unlabeled images.  The proposed method achieved consistent improvement over existing state-of-the-art on two challenging segmentation datasets.

Although the motivation is reasonable and the results are impressive, there are some parts that need more clarification/discussion as described below.

1) Robustness of discriminator output:
The main contribution of the proposed model is exploiting the outputs from the discriminator as the confidence score maps of the predicted segmentation labels. However, the outputs from the discriminator indicate whether its inputs are from ground-truth labels or model predictions, and may not be directly related to ‘correctness’ of the label prediction. For instance, it may prefer per-pixel score vectors closed to one-hot encoded vectors. More thorough analysis/discussions are required to show how outputs from discriminator are correlated with the correctness of label prediction.     

2) Design of discriminator
I wonder if conditional discriminator fits better for the task. i.e. D(X,P) instead of D(P). It may prevent the model generating label prediction P non-relevant to input X by adversarial training, and makes the score prediction from the discriminator more meaningful. Some ablation study or discussions would be helpful.

3) Presentations
There are several abused notations; notations for the ground-truth label P and the prediction from the generator S(X) should be clearly separated in Eq. (1) and (4). Also, it would better to find a better notation for the outputs from D instead of D^(*,0) and D^(*,1).  
Training details in semi-supervised learning would be helpful. For instance, the proposed semi-supervised learning strategy based on Eq. (5) may be suffered by noise outputs from the discriminator in early training stages. I wonder how authors resolved the issues (e.g. training the generator and discriminator are with the labeled example first and extending it to training with unlabeled data.) 

review_title: No title

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper describes techniques for training semantic segmentation networks. There are two key ideas:

- Attach a pixel-level GAN loss to the output semantic segmentation map. That is, add a discriminator network that decides whether each pixel in the label map belongs to a real label map or not. Of course, this loss alone is unaware of the input image and would drive the network to produce plausible label maps that have no relation to the input image. An additional cross-entropy loss (the standard semantic segmentation loss) is used to tie the network to the input and the ground-truth label map, when available.

- Additional unlabeled data is utilized by using a trained semantic segmentation network to produce a label map with associated confidences; high-confidence pixels are used as ground-truth labels and are fed back to the network as training data.

The paper is fine and the work is competently done, but the experimental results never quite come together. The technical development isn’t surprising and doesn’t have much to teach researchers working in the area. Given that the technical novelty is rather light and the experimental benefits are not quite there, I cannot recommend the paper for publication in a first-tier conference.

Some more detailed comments:

1. The GAN and the semi-supervised training scheme appear to be largely independent. The GAN can be applied without any unlabeled data, for example. The paper generally appears to present two largely independent ideas. This is fine, except they don’t convincingly pan out in experiments.

2. The biggest issue is that the experimental results do not convincingly indicate that the presented ideas are useful.
2a. In the “Full” condition, the presented approach does not come close to the performance of the DeepLab baseline, even though the DeepLab network is used in the presented approach. Perhaps the authors have taken out some components of the DeepLab scheme for these experiments, such as multi-scale processing, but the question then is “Why?”. These components are not illegal, they are not cheating, they are not overly complex and are widely used. If the authors cannot demonstrate an improvement with these components, their ideas are unlikely to be adopted in state-of-the-art semantic systems, which do use these components and are doing fine.
2b. In the 1/8, 1/4, and 1/2 conditions, the performance of the baselines is not quoted. This is wrong. Since the authors are evaluating on the validation sets, there is no reason not to train the baselines on the same amount of labeled data (1/8, 1/4, 1/2) and report the results. The training scripts are widely available and such training of baselines for controlled experiments is commonly done in the literature. The reviewer is left to suspect, with no evidence given to the contrary, that the presented approach does not outperform the DeepLab baseline even in the reduced-data conditions.

A somewhat unflattering view of the work would be that this is another example of throwing a GAN at everything to see if it sticks. In this case, the experiments do not indicate that it did.
review_title: not enough for a first-tier conference

Meta-review: The paper presents a reasonable idea, probably an improved version of method (combination of GAN and SSL for semantic segmentation) over the existing works. Novelty is not ground-breaking (e.g., discriminator network taking only pixel-labeling predictions, application of self-training for semantic segmentation---each of this component is not highly novel by itself). It looks like a well-engineered model that manages to get a small improvement with a semi-supervised learning setting. However, given that the focus of the paper is on semi-supervised learning, the improvement from the proposed loss (L_semi) is fairly small (0.4-0.8%).
Decision: Reject