Venue: MIDL 2023 Conference

Reviews:

final_score: rating: 5: Strong accept
review_content: summary: This paper proposes an "Inherent Consistent Learning" (ICL) framework to work on scarce labeled data and numerous unlabeled data They introduce 2  modules based on cross-attention mechanism:  Supervised Semantic Proxy Adaptor (SSPA) and Unsupervised Semantic Consistent Learner.

Tested extensively on 3 large public cohorts.   



strengths: Excellent paper very clearly written with extensive appendix material and method tested extensively on 3 large public cohorts.
Nice visual illustrations for several use-cases.
Clear tables of results. 
weaknesses: Limitations:
- Report training times for the different SOTA and the proposed method
- Unsupervised hyper-parameters have very different values for the 3 use cases. Any insight on their tuning strategy and why this is the case?
recommendation: Oral
review_title: null

final_score: rating: 5: Strong accept
review_content: summary: The paper presents a novel way of inherent consistent learning for accurate semi-supervised medical image segmentation.
In particular, they introduce two external modules. One supervised semantic proxy adaptor (SSPA) and an unsupervised semantic consistent learner (USCL). Both modules are based on attention mechanism to align the semantic category representations of labeled and unlabeled data as well as updating the global semantic representations over the entire training set.
The evaluation is very extensive comparing it against the best state-of-the-art methods. Moreover, the methods presented in this paper have much better accuracy in most of the experiments, especially when the annotated data is extremely limited. 


strengths: The evaluation is extensively performed which gives confidence on the results.
The quality results are also very nice in comparison to other methods, even in difficult tasks such as automatic segmentation of the RV.
The ability of learning to segment from such a few number of labeled datasets (3 vs 140) while achieving > 82% of dice score while keeping the lowest variance is impressive.
The fact of using a plug-and-play scheme makes the method useful for other architectures.
Moreover the paper is really well-written. 
I think it would be very useful for the community to have this paper published.
weaknesses: A possible weakness might be the non-publication of the code. It is a nice paper and the results are very promising. It would be great if the code would be made available for the community.
Another possible weakness might be the choice of the labelled datasets. It would be great to see whether the results repeat when cross-evaluating with different labelled datasets in order to avoid overfitting. (For example, to see whether 3 other labelled datasets would get the same results on ACDC).
detailed_comments: The caption of Figure 1 doesn't explain much what A and B are even though this is explained in the text.
recommendation: Oral
review_title: null

final_score: rating: 4: Weak accept
review_content: summary: Label efficient segmentation is an important problem in medical image segmentation. This work presents a new semi-supervised segmentation strategy that uses limited labeled data with large amounts of unlabelled data to learn discriminatory representations. This is achieved using cross-attention between the labeled and unlabelled representations. The queries (semantic proxies) thus learned are then used to improve segmentation performance. Experiments on three datasets (one in Appendix) show large performance improvements compared to other semi-supervised learning frameworks, especially in the low labeled regimes.

strengths: * The proposed use of cross-attention between labeled and unlabeled representations to improve semantic discrimination is an interesting idea. Coupled with the unsupervised consistency learner, enables learning of useful semantic representations at different scales. 

* The experimental evaluation is thorough, and shows clear performance improvements at small label regimes, across all the datasets. 

* The qualitative discussions and ablation studies presented in Sec 3.5 are very insightful; that the learnt semantic proxies are so clearly separated is highly interesting.

* The literature review is comprehensive and the paper is clearly written.
weaknesses: * **Variability of labeled data**: In most of the semi-supervised learning settings, the selection of the labels that perhaps have most useful information can make a difference in the number of labeled examples to be used. While the results across the board are better compared to other baselines, an evaluation of the robustness of these performance improvements over different subsets of labeled data can show the usefulness of the learned semantic proxies. 

* **Influence of number of unlabeled data points**: Another interesting insight one could obtain is by studying the influence of the number of unlabeled data points used for the unsupervised training. 


 
detailed_comments: See comments above.
recommendation: Oral
review_title: null

final_score: rating: 4: Weak accept
review_content: summary: This paper explores learning in low data settings. The proposed method [Inherent Consistent Learning (ICL)] is evaluated on two public datasets [AMIOS, ACDC] against a robust set of state of the art alternatives. The methods show substantive improvement when learning on extremely limited datasets. Interestingly, benefits decrease relative to standard baselines (Unit) as the number of scans increases to the full number available. Overall, the proposed innovations are highly relevant for low-shot learning. 

Please note that no material beyond the references was reviewed per MIDL policy. 
strengths: + Very strong validation approach across metrics evaluated, structures evaluated, and statistics shown. 
+ Clear writing and mathematical presentation. 
+ Reproducibly defined loss functions. 
+ The approach is quite generalizable toward other 
weaknesses: - The text of the paper is over-optimistic in terms of impact and scope. The scope is good, but many statements are overly definitive. For example: "To solve the above issue..." and "Extensive experimental results...".

- The novelty of the component networks is good, but not transformative. 
recommendation: Oral
review_title: null

Meta-review: metareview: Due to high cost of medical image annotations, semi-supervised medical image segmentation using limited labeled data is an important problem. The paper presents a novel way of inherent consistent learning for accurate semi-supervised medical image segmentation using  limited labeled data and large amounts of unlabelled data. Two modules based on cross-attention mechanism are introduced: Supervised Semantic Proxy Adaptor (SSPA) and Unsupervised Semantic Consistent Learner. Both modules are based on attention mechanism to align the semantic category representations of labeled and unlabeled data as well as updating the global semantic representations over the entire training set. All reviewers agreed that the paper is clearly written and  evaluations are extensive on 3 large public cohorts. Results indicated improved accuracy especially when the annotated data is extremely limited. All reviewers suggested acceptance of the paper, and I agree with the reviewers.
confidence: 5: The area chair is absolutely certain
Decision: Accept (Oral)