Venue: MIDL 2023 Conference

Reviews:

final_score: rating: 4: Weak accept
review_content: summary: This work adapted Student-Friendly Teacher (SFT) training framework (Park et al., 2021) for Magnetic Resonance Image
Reconstruction application. Two datasets have been used to evaluate the proposed method and the performance was measured by PSNR and SSIM. The SFT-KD method showed marginal improvement compared to 5 standard KD frameworks.
strengths: + The manuscript was well-written and Figure 2 was well-illustrated. And the detailed explanations in the appendix were appreciated.
+ The experiments were comprehensive, including 2 MRI datasets, 5 standard KD frameworks, and 2 performance metrics.
weaknesses: - The novelty of the proposed SFT-KD-Recon is very limited compared to the original SFT method.
- The performance improvement SFT-KD-Recon is very marginal compared to the standard KD frameworks in all experiments.
detailed_comments: Please refer to the "Questions To Address In The Rebuttal" Section.
recommendation: Oral
review_title: null

final_score: rating: 4: Weak accept
review_content: summary: The paper presents a student-friendly teacher training approach for MRI reconstruction, by combining knowledge distillation (KD) and deep cascaded CNN to let the teacher network be aware of the student network. The teacher network and the student network are jointly trained to make sure the representations of them can be aligned. The experimental results demonstrate that the proposed KD strategy can significantly improve the performance of the existing KD methods.
strengths: The authors adopted KD to improve the performance of deep cascaded CNN, which is a good strategy for the MRI reconstruction task, and the experimental comparisons with multiple KD methods indicate the superiority of the student-friendly teacher training approach.
weaknesses: The meaning of using KD for the MRI reconstruction task is not very clear, indeed, the experimental results show the effectiveness of the proposed strategy, but the authors do not compare the proposed method with the existing MRI reconstruction methods, which cannot sufficiently verify the effectiveness of the proposed method. In addition, the proposed SFT-KD-Recon is similar to the SFT (Park et al., 2021), and the innovation in this paper is weak.
recommendation: Poster
review_title: null

final_score: rating: 2: Weak reject
review_content: summary: This paper is an extended application of the student-friendly teacher training method (NeurIPS’21, https://openreview.net/forum?id=0xs40KGnsq3). Here, the authors apply this spirit to the MRI reconstruction task. The main spirit is jointly training the teacher model with the student model, so that the teacher model can be more effectively adapted during the KD process. The method is validated on two datasets.
strengths: A strength is similar to their reference NeurIPS paper – the problem may be important in KD. 
Overall, the paper is well-organized and well-written.
The method is validated on two datasets and shows some improvement.

weaknesses: The novelty seems limited. I believe that just replacing the original method with the reconstruction architecture is not so interesting to MIDL readers. Considering that this paper is a direct application of the student-friendly teacher training method (NeurIPS’21), authors should further provide insightful discussion and highlight the challenging part when applying this method to MRI reconstruction to show that it is not a trivial combination/application. Otherwise, readers could not gain enough insights into this paper and may be more interested in the original paper.

Requirement of the student architecture. In the case that student and teacher architectures are heterogeneous, how will the method here perform? The common KD problem does not require the prior of student model, yet, here we should know the student model in advance. Will this limit the application in practice? Authors may need some experiments on this.

What is the KD in Figure 4 standing for? You mention several KD baselines.

A significance test is needed. The improvement is very marginal. Therefore, it is hard to discern whether this training is interesting to the MRI reconstruction community since we should define the student model in advance, which seems not practical. I mean, in previous knowledge distillation, we have a well-trained teacher model, and we can define a shallow student model when we need to compress. Then, we can just perform the KD process. Yet, this method requires the student model during training, i.e., if I have a different compression ratio requirement, I should even retrain the teacher with the new student architecture. If the improvement is very marginal, it is little interesting to developers considering the computation cost.

recommendation: []
review_title: null

Meta-review: metareview: This work presents a student-friendly teacher (SFT) training approach for MRI reconstruction by using knowledge distillation (KD). The structure and presentation of this paper are clear. Although all reviewers raised concerns on the novelty, these concerns were well dispelled in the rebuttal. This work is valuable in MRI reconstruction  due to its promising ability in knowledge distillation.

Pros:
- KD is a useful strategy for MRI reconstruction
- The experiments of validating this work is convincing
- The proposed method can address heterogeneous cases

All reviewers rated weak reject before the rebuttal due to their main concerns on novelty, but two of three changed their rating to weak accept after the rebuttal. Thanks to the detailed and convincing rebuttal from authors, I recommend to accept this paper.
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)