Venue: MIDL 2022 Conference

Reviews:

final_score: rating: 4: Weak accept
review_content: summary: This paper describes an approach to semi-supervised segmentation of images using cross teaching between a standard convolutional neural network (CNN) and a transformer network. These two approaches are potentially complementary since CNNs are composed of localized convolutions, while transformer networks compute embeddings derived from self-attention that can involve long-range interactions. The two networks are combined in a relatively straightforward cross teaching approach and the approach is evaluated on a public data set of  cardiac MRI (ACDC). The results are compared to a number of baseline semi-supervised approaches, as well as ablative variations of the proposed technique.  This is a nice paper overall that makes a novel contribution to the segmentation literature, but is likely not that impactful to the specific application of cardiac imaging.
strengths: - The idea of using a transformer and a CNN in cross teaching semi-supervised paradigm is clever.
- Multiple techniques are used for comparison. 
- The results seem to show good performance compared to baseline techniques.
weaknesses: - The application to cardiac MRI is a bit of an afterthought. There is nothing in the technique that is tailored to the application, and the details of how the data set is used are lacking.
- There are multiple aspects of the experiments that are not well explained.
detailed_comments: - The ACDC data set is actually a 4D data set. I believe the labels are provided at end systole and end diastole. It is not clear which labels the paper are using and whether the rest of the cine frames that do not have labels are being used or ignored.
- The experiments describe how many labeled results are used but I could not find how many unlabeled images were used for semi-supervised training. Is it 2 * (70 - the number of cases)?  Please clarify.
- In Table 1, are the fully supervised results using 7 cases since that is stated in the caption? I believe limited supervision is using 7 cases and fully supervised is using 70 but that should be made explicit. 
- It is not clear what the differences are in the results in Tables 1 and the 7 case results of Table 2.
- Why are the transformer outputs ignored at test-time? How does it perform compared to the CNN? Is that the starred results in the table? It is confusing because the starred results refer to many different methods.
- References are difficult to review since the bibliography format lists the first name of the author at the beginning, but the sorting is based on the last name of the author. 


recommendation: Oral
review_title: null

final_score: rating: 3: Borderline
review_content: summary: A simple but efficient cross-teaching framework using the existing CNN model and transformer is proposed to tackle the semi-supervised medical image segmentation. Though it is an incremental work by introducing the hybrid transformer and convolution structure, the performance achieves the SOTA compared with other basslines.
strengths: The extensive experiments show the performance superiority of hybrid networks in medical segmentation compared with other baselines. Ablation studies are also well-analyzed. The code will be publicly released for reproduction. The paper is well-organized and easy to follow.
weaknesses: It is better to discuss more recent works.
[1]. Recurrent Mask Refinement for Few-Shot Medical Image Segmentation
[2]. Coarse-to-fine adversarial networks and zone-based uncertainty analysis for NK/T-cell lymphoma segmentation in CT/PET images.
[3]. Every Annotation Counts: Multi-Label Deep Supervision for Medical Image Segmentation.

The novelty is limited by introducing the existing transformer (swin transformer) and U-net structure. As the main novelty, the cross-teaching strategy is also simple and has not changed a lot compared with other semi-supervised learning.

The transformer has a powerful ability to extract global and long-range knowledge. But it suffers from memory and computational cost. It is better to provide the time and GPU memory of the network both in the training and test stage.

The structure of the proposed network is 2D network which is blind to the spatial relationships among the slices. It is better to verify the superiority of the proposed network than other 3D models.

recommendation: Poster
review_title: null

final_score: rating: 2: Weak reject
review_content: summary: This paper concerns the problem of limited training in CNNs and transformers in medical image segmentation.  It proposes a semi-supervised method for cross-teaching between a CNN and a transformer.  It uses labeled and unlabeled data and trains both CNN (UNet) and transformer (Swin-UNet) to take advantage of their local and long-range perspectives, and shares information between them.  It outperforms eight existing semi-supervised methods on a public benchmark.  
strengths: This is a very simple approach, easy to understand
It addresses the problem of limited training, which is a ubiquitous problem in medical image analysis.  
It uses the positive aspects of both CNN and transformer to good effect

weaknesses: The exclusive use of ACDC for such a simple and generally applicable approach leaves one wondering about other tasks. While impressive in its ability to beat the other methods, its potential widespread use remains in question. None of the results compare favorably to the current ACDC leaderboard (and this comparison is not made by the authors).  
There have been many improvements to the basic UNet architecture over the years. 
The method is compared only to other semi-supervised methods.  

detailed_comments: Proprieties should be properties
Low-bound should be lower-bound 
The details describing the semi-supervision aspect of the experiments are sketchy.  In particular, it is noted that 140 images from 70 patients are used for training and 60 images from 30 patients are used for validation.  By “validation” do the authors mean “testing”? Is cross-validation used (it is not mentioned)?   How much fully labeled data are used versus no labels?  Where is the semi-supervised aspect?  
Why resize the images?  What happens to the labels as the result of resizing (as there must be interpolation, which is problematic with labels)?  
“when combing previous” should be “when comparing to previous” 
“this work attempts to introduce” should be “this work introduces” 

recommendation: Poster
review_title: null

final_score: rating: 3: Borderline
review_content: summary: This paper proposed a cross-teaching scheme between CNN and Transformer for medical image segmentation. In addition to classical supervised learning, the prediction of CNN will be used for regulating Transformer, and the prediction of Transformer used for regulating CNN for unlabeled data. As a whole framework, it belongs to semi-supervised learning methods. The experiments were conducted on the public benchmark dataset ACDC.
strengths: + The method is easy to follow and implement. The underlying motivation of the cross-teaching is clear.
+ The baseline choice and comparison are rich.
+ The statistical significance of the results is reported.
+ Both quantitative and qualitative results are presented.
+ The public codebase of this work seems strong and comprehensive.
weaknesses: - The results were only validated on one dataset. The generalizability of the framework to other segmentation tasks remains unclear.
- The ablation study is incomplete. Several variables in comparison were not controlled.
- The experimental settings were not clearly stated.
- The computational cost and training speed of the proposed method vs. existing methods were not mentioned in the paper.
detailed_comments: Table 1 and Table 2 look crowded; I suggest making standard deviations smaller font sizes.
recommendation: Poster
review_title: null

Meta-review: metareview: This paper proposed a simple framework by combining CNN and Transformer by cross teaching for semi-supervised medical image segmentation. Experiments on cardiac application demonstrate better performance over other methods. Overall, this paper is well organized and the results are convincing. The majority of reviewers (3/4) has recommended weak accept. Based on own reading, I also recommend accept, although comparison results with other SOTA methods such as 3D models can be added in the final version.
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)