Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors argue that the unlabeled data with pseudo labels is more beneficial to the feature extractor than the mask predictor in a semantic segmentation framework. Based on this, they introduce the Gentle Teaching Assistant model which learns from pseudo labels and (only) transfers the feature representation to the student model.


strengths_and_weaknesses: ### Strengths:
+ A novel gentle teaching assistant model that utilizes pseudo labels to aid feature representation learning.
+ The paper is well motivated and the logic is easy to follow.
+ Extensive ablation studies are provided.


### Weaknesses:
- Considering that GTA only uses pseudo labels, it is unclear to me why GTA is able to transfer better feature representation to the student model.
- The presentation could be improved. The method section could be more concise (see the questions).
- Not enough qualitative results (see the questions).

Post rebuttal:
I'd keep the current rating based on the rebuttal. Although the authors provided reasonable answers to the Q1 (similar concerns also appears in Reviewer sFdH), the implementation is a bit heuristic and need further discussions in the revised paper. 

questions: * Why not utilize both labeled data and pseudo labels to train GTA? It does not make sense to me that fewer data can learn better feature representation.
* How to initialize GTA, is it initialized randomly and trained from scratch?
* Is it necessary to maintain the similarlity between the parameters of GTA and the student model? If the two models are very different from each other, does it make sense to transfer the knowledge through EMA?
* More qualitative results (maybe in the supp) can help improve the paper. Currently, there is only one figure of visualization result in the supplementary and main text.
* Missing ablation study of EMA hyper-parameters.
* In Table 5, is the difference between ours and the original EMA that only the decoder parameters are updated? If so, maybe it’s better to express it as EMA (all parameters) and EMA (decoder).

Typos:
* line 109, is y^u a typo?
* eq1: there is no k in p^u_{i, j}, although we can guess that k is kth term in p^u_{i, j},
* eq3: why the CE loss is defined on data and pseudo labels rather than prediction and labels?
* table 1: does 10582 should be 1464?


limitations: The authors point out one limitation that the proposed method performs better on high-quality labeled data.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper tackles the problem of semi-supervised learning (limited labeled data and a higher amount of unlabeled data) in the context of semantic segmentation. The authors adopt an auxiliary teaching assistant as an extension of the classical teacher-student paradigm. The proposed GTA module is trained solely on the unlabeled data using predictions from the original teacher. GTA facilitates feature representation learning that is further passed down to the student network without impacting the mask predictor from incorrect pseudo labels (trained supervised). Results are showcased on popular benchmarks, such as PASCAL VOC 2012 and Cityscapes. 

strengths_and_weaknesses: Strengths:
* Although the proposed extension of the classic teacher-student paradigm is quite simple and straightforward, it is novel and effective
* The paper tackles an interesting topic for the research community
* The method performs considerably better w.r.t previous publications on popular benchmarks
* Extensive experiments with recent relevant related work and ablation studies are also a plus
* Algorithm 1, alongside Figure 1 and 2 complements the text beautifully and further clarify the procedure and enhance the authors' contribution

Weaknesses:
* Some design decisions were not fully explained - e.g. using solely pseudo labels for training GTA
* Some experiments are incomplete - detailed in the "Questions" section
* There are some doubts regarding the experimental setup and reported numbers in the tables (not clear if the methods are reimplementations or original numbers from the papers - this should be specified in the caption of the tables) - since it's stated in the paper that the comparisons are fair.

Others:
* L107 - Typo - "Semi-Supervised Semantic _Segmentation_"
* L124 - Another relevant, even pioneer relevant work for the Teacher-Student framework is [1] which definitely deserves mention.

[1] Croitoru, Ioana, Simion-Vlad Bogolin, and Marius Leordeanu. "Unsupervised learning from video to detect foreground objects in single images." Proceedings of the IEEE International Conference on Computer Vision. 2017.
questions: * Why not use the labeled data in training the GTA (supervised loss), wouldn't that yield better feature representations? Even if it were implemented as an alternating supervised and unsupervised training procedure for GTA it would have been an interesting experiment. 

* The dataset splits in Tables 1 and 2 are very confusing. Why not keep the same splits for both labeled and augmented training sets? In Table 2 what is the proportion of labeled (fine) versus unlabeled (coarse-SBD) data? 

* If the Student network is trained only on the labeled data, wouldn't this mean that the Student always sees fewer samples than the other two NNs, how would then the training procedure work? Algorithm 1 does not clarify this part and I suspect you can do this with different batch sizing, but did not find the batch size for any of the models. Maybe clarify this part a bit.

* The authors suggest that training the student with faulty pseudo labels would hinder results. How about training GTA supervised and the Student semi-supervised? Did the authors experiment with this as well, any thoughts on what would the outcome be?

* Do all NNs (GTA, Teacher, Student) have the same type of feature extractor (ResNet-101 pre-trained on ImageNet) and mask predictor (DeepLabv3+)? How about the methods in Tables 1, 2, and 3? 

* I do not see a clear advantage of applying the re-weighting mechanism (higher weight on higher confidence) after removing the pixels from the pseudo labels with low confidence. That proposed procedure won't solve the miscalibration or over-confidence phenomenon (detailed L40 - 46). This fact is also confirmed by the results in Table 7 (performance drop after applying re-weighting, which gets corrected after applying Laplace Smoothing). What regions benefit from applying this procedure? Did you see a gain in smaller objects or finer details? A more extensive analysis would have helped.

* In the supplementary material the authors stated that they warm up all the NNs for 1 epoch on labeled data - what are the implications of not doing so? And why just 1 epoch? Just for fair comparisons?

* It would have been interesting to see what is the maximum performance the proposed method could achieve given all the labeled and unlabeled data from any of the benchmarks. I do not see this experiment in any table.

* Suggestion: The "SupOnly" naming is kind of confusing since it's not purely supervised learning right (using only labeled data)? Isn't it the standard Teacher-Student paradigm which is also semi-supervised? Also, stick to one naming convention - Teacher-Student and SupOnly are used interchangeably - Table 4 and Table 5.
limitations: * The authors have addressed some of their method's limitations (addressing lower performance gains on the augmented sets) but qualitative results and comparisons with other methods and the coarse labels the authors refer to would have maybe a stronger argument - even if they were added as supplementary material.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper addresses the semi-supervised semantic segmentation problem. A teaching assistant model is incorporated into the teacher-student mutual learning framework, which disentangles the effects of pseudo labels on feature extractor and mask predictor and protects the student model from the negative influences caused by unreliable pseudo labels in the mask predictor. Results on benchmark datasets show competitive performance against previous methods. 
strengths_and_weaknesses: The proposed method is simple yet effective. The idea of gentle teaching assistant is novel. Although decomposing the segmentation task into feature extraction and mask prediction is not new, disentangling the effects of pseudo labels for the tasks in new and interesting. The paper is well written and easy to read.
questions: The authors seemed not taken the liberty to establish statistical significance of the experimental results. Did the results the average of several experiments with different random seeds? They do not but promise to release the code upon the paper accepted. 
limitations: Yes, the authors have briefly discussed the limitations of the proposed method in the end of the experimental section.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes a semi-supervised method for semantic segmentation via a teacher-student framework. Unlike the standard approach that the teacher model generates pseudo-labels for unlabeled data, while the student model takes both labeled and pseudo-labeled data as training signals, the authors observe that pseudo-labels can be noisy and directly using them to train the student model may not be optimal. Therefore, the paper proposes to use another assistant model that takes pseudo-labels and only transfers feature representations to the student model via EMA. In addition, the paper also adopts a weighted pseudo-labeling scheme to train the assistant model. Experiments are conducted on PASCAL VOC and Cityscapes to demonstrate the usefulness of introducing the assistant model.
strengths_and_weaknesses: **Strength**
- The paper is written well and is easy to follow
- The proposed method with the assistant model is simple and effective, which can reduce the issue of noisy pseudo-labels
- The proposed feature transmission from the assistant model to the student model can be a generally useful tool for other tasks
- Experiments show good performance improvements on two datasets

**Weakness**

Some hyperparameters and technical details are not explained very well
- What is the \tau value in Eq. (6)?
- How do the authors select the threshold for pseudo-labeling, i.e., is \gamma a constant or varied and how?

Experimental results:
- Table 4-7 are conducted on a smaller labeled set (183 images). However, I wonder how Table 4-7 would look like when there are more labeled data, e.g., >1000/2000 labeled images, which can be a more practical situation in real applications.
- In Table 6, the teacher model performs better than the student model. It's interesting to know whether it is the same case for all experimental settings. If so, what is the intuition behind it? The authors may have more discussions on this.

Missing references:
- Adversarial Learning for Semi-Supervised Semantic Segmentation, BMVC'18
- Semi-supervised semantic segmentation with high-and low-level consistency, PAMI'19
- Semi-Supervised Semantic Image Segmentation With Self-Correcting Networks, CVPR'20
- Semi-Supervised Semantic Segmentation With Pixel-Level Contrastive Learning From a Class-Wise Memory Bank, ICCV'21
questions: Please see the above comments in Weakness.
limitations: Limitations are briefly discussed at the end of the paper. However, the statements may not be correct. The authors state that the improvement in Table 2 is smaller than the one in Table 1, but their compared methods are not the same though. For example, the improvement over CutMix in Table 1 and 2 are actually similar. Also, when the labeled data is around 700 or 1400, the final results in Table 1 and 2 are similar. Although the labeled data can be noisy in the augmented set as the authors mentioned, I would suggest the authors to discuss the limitation from other perspectives.
review_title: null

Meta-review: metareview: The paper was reviewed by four expert reviewers in the field. The initial ratings were three weak accept and one weak reject. 

In the response to reviewer sFdH (who gave Weak reject), the authors clarify all the questions from the reviewer, including using labeled data in GTA, the data split, training details, advantages of re-weighting mechanism. While the reviewer sFdH did not acknowledge the rebuttal, the AC believes that these questions have been sufficiently addressed. 

Given the novel approach, extensive quantitative evaluation, and clear writing, the AC agrees with the three reviewers and recommends to accept.
confidence: Certain
Decision: Accept