Venue: MIDL 2022 Conference

Reviews:

final_score: rating: 1: Strong reject
review_content: summary: The authors present a transformer-based approach for lesion segmentation in automated breast ultrasound (ABUS) images.  The approach is based on the SegFormer method, extended with "region aware components".  In essence, these region aware components are box ROIs in which an extra attention layer is applied.  The location and size of the ROIs is not learned, but derived from the data.
strengths: The method is based on SoA technologies and the dataset is large and interesting (albeit private).  The literature overview is good.  The authors have performed a few relevant ablation studies (although the evaluation has a few weaknesses).
weaknesses: A severe problem with the evaluation is that apparently, hyperparameter tuning was performed on the test set.  What the authors call "the first ablation study" is in fact not an ablation study, but hyperparameter tuning, and from Table 1, it appears that this was performed on the same test set that the final results are reported on.  The text also does not mention otherwise.  Unfortunately, this means that the observed "improvements" could be a matter of chance, and a new test set (that is not used for another round of tuning) would be required to prove otherwise.

Furthermore, the main contribution is of questionable quality: Region priors should better be learned, not hand-crafted. The proposed approach is just taking bounding boxes of regions from the training set (that is, one would hope that the analysis was performed on the training set only, because this is not described).

I am not too familiar with ABUS, so I will give the authors the benefit of the doubt concerning the question whether region priors can make sense, or whether positioning variability makes this dangerous anyhow.  Given my general US knowledge and the image appearance, though, I would expect a good deep learning setup, such as a transformer-based architecture, to be able to learn such priors from the images themselves, in which the imaging boundaries can be seen.  That approach would appear much more robust to me.

No statistical significance tests were mentioned.
detailed_comments: You highlight that RASAB is a pluggable component, because it generates the same *size* as the input (which is actually achieved by placing the smaller ROI output within a copy of the input).  Is there any good argument for why the output for the ROI and the background should have "compatible" features?  If you rely on the model to just learn to produce these, have you tried to check that this works (by inspecting the RASAB output for ROI borders, either manually or automatically)?

Is it relevant to mention that the experiments were run on a server with four RTX3090?  Please specify whether the experiments were using multiple GPUs, otherwise just mention the GPU they were run on.

Semantic segmentation is indeed voxel classification, but why is that an "improvement" over image classification?  (IMO it's related, but just a different task. People interested in image classification would not necessarily find that segmentation models bring only improvements.)

I suggest to use small spaces (\,) before units (cc,mm), but no space after the thousands-comma.
recommendation: []
review_title: null

Meta-review: metareview: This paper describes an architecture to segment tumors in breast ultrasound images. The authors have responded well to the comments and concerns raised by the reviewers and made subtantial improvements to the paper. The evaluation problem on the test set has been partly addressed retroactively. 
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)