Venue: MIDL 2022 Conference

Reviews:

final_score: rating: 2: Weak reject
review_content: summary: The paper presents a continuous and differentiable function that maps lesion segmentation prediction to lesion count probability. The main motivation is that normal lesion counting (connected-component-based) is not differentiable and therefore cannot be used during training of a neural network. The authors perform quite a lot of experiments on a Gadolinium-enhancing (Gad) lesions MRI dataset.
strengths: - The method description is easy to follow and written nicely.
- The code is available. 
- The paper gives a very detailed introduction to previous works. 
- The authors performed a large number of experiments to give insights into the method
weaknesses: Due to the large number of experiments, no results are presented detailed enough in the paper itself. To understand the results and the paper, it is necessary to read most of the appendix (that is as long as the paper itself). 
-	Additionally, the focus of the paper gets lost due to all these experiments.
-	The discussion including limitations of the presented method is completely missing. 
-	No examples are shown in a figure demonstrating the results and cases in which one or the other method performs better.
-	The captions of the figures are not sufficient to understand the figures (especially not without the corresponding description in the text)

-	Even after reading the paper, I do not understand why the benefit of the proposed method (maybe because of the paper structure?). For a threshold of \tau = 0.5, it doesn’t make a difference which counting method I am using, so why should I use the more complex one? The segmentation networks are trained with \tau =0.5 or expect that and if it works with that, why do I want to be more robust against that parameter? Also, for the multi-task learning, I don’t see a benefit of the proposed method. Maybe there is one, but then please highlight it more and focus on it!

Open Questions: 
-	What happens if only one voxel has a high probability in a cluster? That would mean that R_i = 1, but a lesion consists of more than one voxel.
-	Does assumption 3 really holds? The existence of lesions is not independent..

detailed_comments: Please rename the "Experiments" section to "Experiments and Results" section.
recommendation: Poster
review_title: null

final_score: rating: 3: Borderline
review_content: summary: The authors propose a differentiable mapping function from lesion segmentation mask to lesion count distribution to enable end-to-end lesion counting training. They use Bayes theorem to approximate each individual lesion probabilities and aggregate them into a joint count distribution with a Poisson-binomial distribution by assuming that the existence of each lesion is independent. Experiments have been performed on a large scale Gad lesion dataset and the proposed method outperforms traditional method in post-hoc setup. In the count-based learning setup, the proposed method shows better calibration performance compared to the baseline methods.
strengths: 1. The problem is well-motivated that lesion counts information couldn’t be used for supervising in deep learning due to the discontinuity of the integers.
2. The proposed method is based on many domain-specific knowledge and the mathematical derivations are valid.
3. A large-scale dataset has been used for performance evaluation.

weaknesses: 1. Although in the post-hot counting setup the proposed method outperforms the traditional one, it’s not a deep learning approach where the mapping function is non-parametric which does not fit the scope of MIDL.
2. In the count-based learning setup, the performance of the proposed method is similar to the multi-head one.

recommendation: Poster
review_title: null

Meta-review: metareview: The authors have addressed the questions raised by the reviewers and have clearly improved the manuscript. 
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)