Venue: MICCAI 2021 Workshop COMPAY

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: In this manuscript the author outlines a way to assess the uncertainty of a segmentation model. This is a relevant topic. While the authors describes in some detail the method based on Bayesian neural networks, the precise link with uncertainty is hard to assess. The authors provide a fair amount of equation but none is explained. The only result comes in the form of one figure (fig 3) that provides a color coding of the aleatoric and epistemic uncertainty. The result is also surprising in that the highest level of uncertainty are roughly in the center of the segmentation where you would expect it near the edges. No interpretation of the result is given.

While I feel this is a very interesting topic, I am not convinced of the execution in the work (or to be more precise: the description is insufficient), plus, that this work deals with MRI scans while the workshop is on computational pathology.
review_title: the author outlines a way to assess the uncertainty of a segmentation model

final_score: rating: 5: Marginally below acceptance threshold
review_content: The authors describe a Bayesian Neural Network approach to segment brain tumor in MRI scans. The authors want to address the important task to not only provide a segmentation result, but also a segmentation uncertainty measure.

Although I agree uncertainty measures are a very important topic, I do see several issues with the current manuscript. The main issues are that first, the main idea of uncertainty description does not seem to be captured in the results, other than with a short sentence that color refers to uncertainty. But what are the implications? Where is the model uncertain? What is the statistics of uncertainty (is the model almost always confident)? In the end, the authors report DICE and IoU results, instead of uncertainty, and therewith do not answer their own question (I don't see any quantitative result for uncertainty as implicated in the title). Second, the paper seems to be compiled in a rush and lacks on a minimum quality (see below).
- To me, the manuscript makes a poorly written impression, with several grammar issues, and some sentences being hard to understand. I would recommend the authors to proofread their manuscript.
- Formulas are not explained (e.g. Formula 1)
- Figure 1 is not explained
- In the Results, the Train size is not explained
- What are the comparing Dice and IoC numbers for brain tumor segmentation Dice of 64.3 and IoC of 55.8 does not seem to be quite high. Here, the uncertainty could help, as it would tell us if the wrongly detected regions are uncertain or certain.

I do find the topic interesting and important, but I think this paper needs more work to gain its well deserved quality and impact.

review_title: I recommend a major revision of the manuscript.

Decision: Reject