Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes to solve a classification task by jointly solving a problem of atlas building for each specific class. Thus the classifier network uses the shape features that are used to deform the templates as well as the features learned directly by the classifier. The trained model exhibits good classification performances as well as a good representation of the data (sharp templates). 
strengths_and_weaknesses: Strengths : the paper is well written , with solid mathematical basis and the proposed concept of joint learning is simple to grasp. The results are solid with extensive comparisons with baseline methods in various aspects : classification performance, as well as representation by deformable templates Two datasets are chosen, the former being toy examples and the latter being related to neuroimaging MRIs.

Weaknesses : the main contribution is to solve jointly two tasks instead of solving them sequentially (two step method). This makes it a fairly limited  contribution for this conference. Another weakness is that during inference, the choice of the atlas is not known and therefore comparison of the input image with the deformed atlas cannot be done. Besides, the paper assumes that each class can be well represented by the deformation of a single template (unimodal distribution)
questions: -The authors should comment on the use of Unet (including skip connections) as opposed to (variational) autoencoders in order to generate the shape features. 

- How is the proposed approach sensitive to the rigid transform between the template and image ? How is this taken into account in the proposed framework ?

- At inference time, the choice of the template to be towards the image is unknown. How to combine the classification network and the shape reconstruction network to get  the registered  template ?

- The authors should try to provide explanations about the slight improved performance for the generation of atlas since the atlas generation method is similar to the ones proposed before
limitations: - The authors do not provide a set of potential limitations of their work. 
- Limitations of the work include : 
i) the proposed approach is only valid for same modality images since simple similarity criteria (SSD) are used 
ii) It represents shape as the deformation of a single template which is only valid for a simple unimodal distribution of shapes 
iii) it does not allow to estimate the template deformation at inference time
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper presents a deep learning approach that improves the classification performance using shape priors that are jointly trained with the classification learning task without the need for pre-extracted shapes.  The core assumption is that class-specific shape information can be encoded by matching the input image to a representative atlas, in this paper this atlas is learned. The efficacy of the proposed method is showcased on synthetic and real medical data.

strengths_and_weaknesses: Strengths:
- Jointly learning atlases and sample-specific shape representation to boost classification performance is a novel idea.
- The use of deformation-based representation allows for detailed shape descriptors and avoids pre-extracting shapes from images.
- Implicit incorporating shape information within image classifiers should improve the robustness of the classifiers to intensity variations.
- The method provides explicit classification interpretations in terms of geometric features that derive the classification task.
- The method shows classification improvements in addition to interpretability. 

Weaknesses:
- Although claimed, it is not clear how would the proposed method be generalized to other representations (e.g. point-based models).
- The robustness to variations in image intensity is not demonstrated in the experiments.
- The method seems to only operate on aligned samples.

questions: - How would the proposed method be used for / adapted to point-based shape models?
- Are the images (training and testing) assumed to be roughly aligned? how is this accomplished? how sensitive is the method wrt misalignments in training? and how misalignment in testing can be handled?
- The learned multiple atlases of the brain data are not shown.  In figure 6, only one atlas is shown. Shouldn’t we expect an atlas per class?
- It is not clear if the heat maps in Figure 5 show shape-specific changes that are indicative of dementia, Geo-SIC is different from others, however, there is no discussion (or detailed figures) provided to demonstrate the specificity of these heatmaps to the dementia problem.
limitations: No. Authors are encouraged to discuss the limitations of the methods and delineate potential negative societal impact.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The article presents an approach to image classification using deformable atlases. This not only improves the classification itself but also makes it possible to better explain the decisions made by the models. In addition, the authors show that this approach is computationally less expensive than some previous approaches.
For the introduction of the atlas information into the deep learning model, they first train an encoder-decoder capable of predicting the atlas from the input images. In parallel, with the embeddings generated by the first one, they train a second network that will be in charge of making the predictions. 

strengths_and_weaknesses: Strengths:
*.- It appears to be a robust approach.
*.- For similar (or somewhat better) results to previous versions it has a lower computational cost.

Weaknesses:
*.- Equations (1-4) (basic for the construction of the loss functions) would require a somewhat more detailed explanation to facilitate the comprehensibility of the proposal. (Maybe it could be added in the annexes). 
*.- The models are not sufficiently explained. It seems that there are two loss functions that are applied alternatively depending on the model being trained. It is also not clear whether the error propagates to all models or only to the one in the phase currently being trained, freezing the rest. 

questions: *.- Improve the explanation of the equations used to build the loss functions. Include them in the annexes if you need extra space.
*.- References to Tables 1 and 2 are wrong.
*.- A comparison of the computational cost between the base models (AlexNet, CNN, ...) and the models with Geo-SIC should be presented.
*.- Due to the fact that the CNN base model is not a standard one, a better description of it should be added. 

limitations: The limitations of the model are correctly addressed.
review_title: null

Meta-review: metareview: Although there were a couple of initial questions/concerns about certain aspects of the paper, all reviewers appreciated the approach, the quality of presentation and the empirical results. After reading all responses by the authors, my impression is that all questions have been answered satisfactorily during the rebuttal period. Hence, I do recommend acceptance of this paper.
confidence: Certain
Decision: Accept