Venue: MICCAI 2022 Challenge FLARE

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The authors apply noisy student model to achieve efficient abdominal organ segmentation. In addition to the model construction, they also made a series of data pre-processing and post-processing to lower the computational cost. The reported results show that the proposed method achieves better performance and lower cost than the original nn-UNet. 

Advantages:
1. The paper is written fluently and is easy to understand.
2. Authors optimize the entire working flow such that the proposed method has faster inference speed and lower computational cost.
3. The quantitative comparison is very detailed.

Disadvantage:
1. The novelty of this method is relatively weak. It seems the authors just use the noisy student model without proposing new ideas or methods.
2. Authors mentioned that they ensembled 5 teacher models but did not mention the ensembling method and the ablation study on the teacher model is not clear. For example, why use 5 teachers? What is the performance of the teachers?
review_title: Review for Noisy nnU-Net

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This paper exploits an ensemble of teacher networks to provide pseudo-labels for training a noisy nnU-Net student. To reduce computational costs, this paper further uses a small architecture and resampling scheme. Experimental results show that the proposed method can effectively save computational costs and improve segmentation performance.




Pros:  

+ This paper adapts nnU-Net to semi-supervised learning setting by using self-training with an ensemble of teacher models and a noisy student model of
the same network architecture.

+ This paper achieves good performance and effectively reduces computational costs.

+ The ablation studies shows the effectiveness of design choices used in the proposed methods.

Cons: 

- The introduction section presents several challenges such as shape variations, abnormalities, and distribution shifts. However, these challenges are not the focus of this paper as the method section does not clearly clarify how these challenges are addressed. A better way is to focus on the unlabeled data and the efficiency in FLARE 22.

- In the Introduction, "With some small architecture and preprocessing adaptations". But the pre-processing in Sec. 2.1 is too short to present the preprocessing adaptations. So I would suggest including some more details, e.g., how to do percentile clipping.

- In Tab. 3, please explain a little bit about what p_{sample} and P_{channel} are.
review_title: A efficient and effective method

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: 
**Summary**
This paper combine nnU-Net with noisy student to use unlabeled data and design a low-footprint nnU-Net to achieve efficient inference.

**Pros**
* Well-written paper with clear method description and complete experimental results.
* Simple but effective method to make low footprint nnU-Net student yields significant improvement over default nnU-Net but consume less GPU memory and inference 10× faster.

**Problems**
* The experiments are conducted on only 20 held-out cases, it is better to report the results on validation leaderboard (50 cases) to compare with other participants methods directly. 
* For better visualization of Fig2, it is better to adjust the `level` and `window` to be consistent and all images should have the same image size.
review_title: A nice try to combine nnU-Net with nosiy student

final_score: rating: 7: Good paper, accept
review_content: The paper showed their points clearly, and writing in such manner that it stick with their list of contributions. That's being said, the structure of paper is coherence. However, I have some advice about content arrangement.

Abstract: Author articulate the challenges and the method abundantly. But, about the results, I feel it is sketchy. Maybe need to include quantitative results, like model achieve # DSC, using # GPU memory, inference speed reduce from # time to # time by factor of two. 

In the method, "We propose a method based on the nnU-Net framework [5]". And in 2.2 Proposed Method, "The proposed noisy nnU-Net student is inspired by knowledge distillation [4]". I guessing author is trying to say that we propose noisy nnU-Net student inspired by KD[4]
, nnU-Net framework[5]. If it articulate in this manner, it's more coherence and easy to read.

I did not spot any grammar or other issues in this paper further

review_title: Points are clear generally speaking, just need some minor changes 

Meta-review: metareview: Nice paper. Please address the reviewers' comments in the revised manuscript
confidence: 5: The area chair is absolutely certain
Decision: Minor Revision