Venue: MICCAI 2022 Challenge FLARE

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: A baseline model is proposed using augmented 3D-UNet and adopted semi-supervised method–Mean Teacher, to evaluate the FLARE2022 validation cases quantitatively. The proposed method achieves an average dice similarity coefficient (DSC) of 0.60. 

Strengths: An average method with an average mean DSC.

Weaknesses: The manuscript is not prepared in a good way.

Comments for authors:

1: At least three relevant keywords should be added

2: If you follow the preprocessing method from some other work, kindly mention that clearly.

3: Please make sure the figure you draw is by yourself rather than copying (in case you copied).

4: There is no clear methodology, and it seems to have many missing connections; please pay attention to that.

5: The caption for Table 2 should be on top of the table.

6: The Figures and Tables are split/floated throughout the manuscript and not in an arranged order. Similarly, the same issues for the conclusion and acknowledgment sections. It is suggested to properly order all the things such as text, figures, and tables in the relevant places.

8: We are also not sure about the computational time of your method by mentioning that the average running time is 0.986 s per case in the inference phase, and the maximum used GPU memory is 2011 MB. Please revisit the computational time as well.

7: Weak English. Therefore, it is suggested to check the whole manuscript in this regard.

8: Need to add an ablation study.

Note that this manuscript needs extensive changes in terms of all suggestions and should be polished. Otherwise, it will lead to rejection in the next round of revision. 

review_title: A baseline model is proposed using augmented 3D-UNet and adopted semi-supervised method–Mean Teacher, to evaluate the FLARE2022 validation cases quantitatively. However, extensive changes are required in the manuscript.

final_score: rating: 5: Marginally below acceptance threshold
review_content: The authors designed a whole-volume-based semi-supervised segmentation framework using a two-stage network and the Mean Teacher Method. Also in terms of efficiency, using a method to improve the speed in the inference stage.

The authors performed extensive ablation studies on the selection and comparison of methods. This makes the paper more logical and readable.

There are some problems, which must be solved before it is considered for publication. If the following problems are well-addressed:
* Do not copy other paper figures.
* The caption section of the figure needs to be supplemented with a more detailed description.
* In Section 2.2, The details of the method need to be briefly described. How your network uses the Mean Teacher Method.
* In Section 4, You should supplement the results of the validation without using semi-supervision.
review_title: More details need to be described

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper's structure and grammar are good. It's very easy to read about. However, there are some issues here.

About Citation: please include citations for existing work used in your paper, such as, 3D-Unet, semisupervised method–Mean Teacher, etc,.

The paper mentioned using of 3D-Unet and semisupervised method–Mean Teacher. But the contributions of the paper are not stated out. In other words, there are some new ideas or just make combination of existing works.

There no need to mention the future work in results section. if feel it needed so, please put it in future works section specifically.  (see 4.2 Segmentation efficiency results)

In Conclusion section, "Our method performs high generalization and robustness on most organs,such as liver, kidney and spleen in terms of DSC scores". There is no related works, benchmarks or reasons to draw such conclusion. You can compare related works or benchmarks, to say you have higher generalization and robustness. Or you can do an analysis about quantitative results, and then draw the conclusion. In other words, please explain why  "Our method performs high generalization and robustness on most organs, such as liver, kidney and spleen in terms of DSC scores".  
review_title: need to citate existing works used in paper. If can, please articulately state the contributions of the paper. Articulate the reason of conclusion 

final_score: rating: 3: Clear rejection
review_content: **Suggest improvements:**

- Do not copy figures directly from other works. 
- No clear definition of 'abnormal data' is mentioned in the pre-processing stage. How to define it?
- The inference stage is extremely fast, but the mDSC of baseline(supervised training) is about 0.8 or more in this competition, while it's 0.6 in this work, even improved by the Mean Teacher method. So, how's the performance of the baseline? It's not shown in Table 3. It needs to be improved so much.
- This work has no discussion and figures for well-segmented and challenging cases.
- Tables and formulas on page 7-9 are not mentioned and explained.
review_title: The method is fast enough, but its performance needs to improve.

final_score: rating: 7: Good paper, accept
review_content: 1. The description of the network structure needs to be improved.
2. The image resolution is too low for Fig.1.
3. The results are convinced and well proved.
review_title: clear description of the method

final_score: rating: 2: Strong rejection
review_content: Authors apply Mean Teacher method into 3D U-Net model to leverage the unlabeled data. In fact, the 3D U-Net is adapted from last year solution with new training technique.

This paper lacks of effort and is unfinished, it has poor word choices, misdirecting sentences,  and contains many editting errors.

Some outlined errors and suggested improvement:
- Page 2: what is "temporal ensembling" problem ? should be detailed
- Figure 1, 2 should be redrawn instead of using those from the original paper
- Figure 3 and 4 have wrong visualization settings and good/bad cases should be selected and described
- On section 2.2, page 2, authors mention that their solution is based on the previous winner of FLARE21, and have coarse-to-fine manner. However, in the pipline (figure 1, 2) and the training protocols they dont mention the second stage. Moreover, in section 4, page 6, they write that “we will make improvement … by designing two-stage coarse-and-fine framework” indicating that they dont implement it in this paper. —> Conflicts
- Missing comparison between "using unlabeled data" and "not using unlabeled data" 
- Many typos and odd word choices, wrong collocations in page 6 and 7. Sentences are confusing to the reviewer
- Errors in page 7 and 8 (after the achknowledgement), what are those equations and tables 4→ 8 ?
review_title: This paper lacks of effort and is unfinished, it has poor word choices, misdirecting sentences,  and contains many editting errors.

Meta-review: metareview: Figure 1 suspected plagiarism
Reviewers raise many concerns and suggestions. Please address all comments in the revised manuscript.
confidence: 5: The area chair is absolutely certain
Decision: Major Revision