Venue: MIDL 2023 Conference

Reviews:

final_score: rating: 3: Borderline
review_content: summary: This paper presents SegPrompt for addressing the task of kidney stone classification using endoscope images. The network relies on a pre-trained U-Net to produce segmentation maps, and leverage the segmentation maps as prompts to improve classification performance. The method is evaluated on a collected dataset, and results look promising. 
strengths: The paper is well written, and it's easy for reviewer to get the idea and technical details of the method. The built dataset can be a valuable resource to the community if released. The idea is clear and interesting. The performance of the method is promising in comparison with the competitors. 
weaknesses: While the idea is interesting, there are several issues regarding the method and experiments. 

First, one clear downside of the proposed approach is that it heavily relies on segmentation annotations. This may limits the usage of the method in clinical practise, and actually will prevent the extension of the method to other medical tasks, as authors mention in the conclusion part. In addition, it is not clear how are the segmentation annotations collected. 

Second, some experimental details should be clarified. Regarding the training/validation sets, what are the numbers of training / validation samples? In addition, I am actually thinking which set are the experiments results obtained from. Is it from the validation set or actually from a leave-out testing set? 

Third, regarding the pipeline of the method, I am curious how about to directly train a segmentation network for joint segmentation and classification. What's the advantage of the proposed two-stage method agains such a simplified one-stage baseline.  
recommendation: Poster
review_title: null

final_score: rating: 4: Weak accept
review_content: summary: This paper seems to leverage developments in transformers to kidney stone classification, complementing tag information with segmentation maps of the stones.

The method is evaluated on a private small dataset of around 1500 images.

Due to my limited knowledge on the topic of transformers, for now I am sticking to a conservative borderline (the results seem simply _bizarre_ to me at the moment, I hope this is clarified during rebuttal). But I intent to close the gap in my knowledge during the rebuttal.
strengths: - Seemingly very good results
- Seemingly solve the problem at hand (no further research required, the trained model can be deployed in a clinical setting)
- The method seems to be quite data efficient, leveraging a pre-trained model to fine-tune to the task
weaknesses: - The paper is rather difficult to follow if one is not up to date with the transformers litterature
- The reporting of the results is not clear, results by fold would be useful, and clarifying if the authors are reporting train or validation peformances would help
detailed_comments: Misc:
- Page 3: "Iy is designed toimprove" -> "It is designed to improve"
- \text should be used to format words in equations (such as $z_s^i = m^i + p^i_s + r \text{ for } i=1,...,l_m$ in Eq. (2).
recommendation: Poster
review_title: null

final_score: rating: 3: Borderline
review_content: summary: This paper proposes a deep learning framework for the classification of kidney stones subtypes using segmentation maps to help the subsequent classification of kidney stones.
The proposed method is evaluated on a quality filtered selection of images from laparoscopic surgery videos in terms of accuracy, F1 score, Precision and Recall
strengths: The experiments appear overall well designed with care for cross-validation and indication of performance variability
The results are carefully interpreted with insight on the meaning of the findings and interesting points made

weaknesses: Motivation is not really clear as the reader only learns at the setting of the experiments what are the two subtypes of kidney stones to classify but there is no explanation as to why these two subtypes need to be separated.
There are some statements of outperformance but these are not supported by any statistical test. Having a ROC AUC for the classification performance would have been useful
The authors state the relevance of the proposed method when learning with fewer examples but there are no experiments to support that aspect of the paper. 
detailed_comments: There is a general slight lack of clarity in the paper probably due to the ordering in which the information is sometimes provided. It is not clear for instance how the additional images are properly "leveraged" - Do they require kidney segmentation if not the full annotation?
recommendation: Poster
review_title: null

Meta-review: metareview: The authors present a DL method for classifying kidney stones subtypes. They employ segmentation maps as additional prompts.

Overall, to me this paper remain quite borderline. Overall, the concerns of the reviewers seem to revolve around clarify of the paper and motivation, and the experimental setup (e.g. lack of certain baselines, some confusion over the over-fitting comments, etc). Upon the rebuttal, it seems like the problem remains narrow, and while the authors properly justify not being able to run certain baselines due to missing code/data, the insights we are left to take form the paper are narrow (specific task, specific domain, requiring manual segmentations for a classification problem, etc). Nevertheless, for this particular task, the paper warrants some interesting discussion.
 
I am a tad confused about the discussion around transformers (with reviewer CYQw), as the authors seem to defer their choice of transformers to previous literature, e.g. with comments like "transformers have been proven effective for many image recognition tasks.". This is very hand-wavy -- transformers are complicated machines that have only been shown to be superior in other very specific domains. Even in general computer vision, where transformers are certainly 'hot/popular' literature, most careful analysis show that CNNs are as good (e.g. see convnext paper). In medical imaging, I am not familiar with literature that has actually shown them to be superior, although they are being often examined. I have no concerns with authors using transformers, but the justification and answer to reviewer seem insufficient. 

Overall, I would say the contribution is narrow but perhaps offers some interesting topics of discussion. I recommend a poster acceptance.
confidence: 3: The area chair is somewhat confident
Decision: Accept (Poster)