Venue: MIDL 2022 Conference

Reviews:

final_score: rating: 5: Strong accept
review_content: summary: In the manuscript, the authors propose KeypointMorph, an unsupervised end-to-end learning-based image registration framework that relies on automatically detecting corresponding keypoints. Their core insight is that matching keypoints between images can be used to obtain the optimal transformation via a differentiable closed-form expression. They use this observation to drive the unsupervised learning of anatomically-consistent keypoints from images. They apply their method to the estimation of affine registration on multi-modal brain MRI scans. The obtained registration is robust to large misalignments and yields better interpretability since the keypoints reveal which parts of the image are driving the final alignment. They demonstrate registration accuracy that surpasses current state-of-the-art methods. 
strengths: 
- Unsupervised end-to-end learning-based framework. 
- Smart and novel approach.
- Robust to large misalignments.
- Interpretable. 
- Good registration accuracy.
- The method can be designed to be equivariant under image translations and/or symmetric with respect to the input image ordering.
- The strategy leads to consistent keypoints, even across modalities.
- They have plans to make their code available. 
weaknesses: - Table A.1. include also the time for training.
- Evaluation is somehow limited. It relies on the Dice coefficient. 
- It is not clear what is the application for large misalignments.
- It is not evident how to extend the method to compute non-linear deformations. 

recommendation: Oral
review_title: null

final_score: rating: 4: Weak accept
review_content: summary: This paper introduces an approach to learn corresponding keypoints for use in multi-modal linear registration. Experiments were conducted on 3D brain MRI for T1, T2 and PD images, and the effectiveness of the approach in resolving synthetic transformations was demonstrated. The approach has clear benefits compared to optimisation based approaches and other learned approaches in terms of resolving large rotations and translations. Moreover, the extraction of keypoints provides an additional measure of interpretability. Given additional experimentation, this approach may indeed represent a useful step forward.
strengths: The use of learned keypoints for image registration has a clear benefit in terms of interpretability of the results, and explaining why a particular registration was chosen. Moreover, one could imagine a procedure where users select keypoints that they believe are meaningful to embed some amount of expert knowledge.

The other strength of this approach is that given keypoint locations, there are closed form approaches to derive globally optimal rigid, similarity of affine transformations.

The paper was clearly written, and figures 1 and 3 provide some nice illustrations of how the method works and inspecting the results. 
weaknesses: One oddity was figure 2 does not consider a plot for changes in scale, only translation and rotation. One would anticipate that such an approach to learning keypoints may suffer more when considering changes in scale - so this is important to investigate. I could not see this discussed at any point in the paper.

As a colour blind reader, I found figure 2 very difficult to read without referring to the text/table in the appendix. I would suggest thicker lines in the legend and the use of different symbols for different methods. 

It would also be useful to see how keypoint extraction (and associated alignment) is affected by noise/blurriness.

At testing time - are the different transformations (e.g. rotations/translations and scaling) applied separately or jointly? Given the description in appendix E, it looks like they are independently applied. It would be helpful to see quantitative results for images that have been translated, rotated and scaled.
detailed_comments: Given that you are only applying similarity transformations to the data - why are you not using the closed form estimation of a similarity transformation rather than an affine?

I think it would have been very helpful to show an illustration of what poor key points look like - to diagnose why they are unhelpful. 
recommendation: Oral
review_title: null

final_score: rating: 3: Borderline
review_content: summary: The authors propose an unsupervised approach for the problem of image keypoint registration. They claim that the proposed approach is interpretable, robust to misalignment, and incorporates image symmetries. 

The deep-learning part of the solution is supervised (to the best I could understand) and is in charge of finding the keypoints. This proposed approach is entirely depicted in Fig. 1 in the paper. To perform the experimental assessments, the authors used the IXI brain MRI pulic data set.  A comparison with two other methods is carried out.
strengths: - The paper is well written and mathematically soundy. I had no difficulties to follow the majority of the ideas of the work (although some ideas are passed sometimes in a confusing way);
- A very extensive survey of the literature, covering well recent works in the field of image registration;
- Appendices bring some interesting qualitative analysis.

weaknesses: Although the authors make some claims throughout the paper, it seems that some of them lack adequate motivations or rationales. This can be noted when, for example, the authors state that "the model is interpretable since the keypoints that drive the alignment can be visualized". Another example can be found at the end of page 3: "In contrast, our focus is on robust image registration via corresponding
keypoints; these prior works aim explicitly to obtain keypoints that are repeatable under different viewpoints and/or image acquisition conditions. Nevertheless, we build on previous ideas and introduce a framework that output matched keypoints in 3D space regardless of the initial position of the image." This is particularly confusing and questionable, and the authors miss to include a robust comparison with these other approaches or even motivate their statements.

I understood that the keypoints to be matched in the registration phase are found in a supervised way "Fixed and moving 3D images are passed through the same keypoint detection network composed of convolutional layers and a final center-of-mass (CoM) layer that predicts keypoints useful for registration", by using a CNN architecture. Only the registration stage is performed in an unsupervised fashion. Nevertheless, in Section 6, one can find something different: "a robust deep learning-based affine registration framework that employs unsupervised keypoint extraction". Did I miss something?

I think that the experimental analyses suffer from more comparative methods. The authors only used a traditional method and a Spatial Transformer Network. Considering Occam's razor, simpler methods could be considered for comparative analysis, such as the ones based on handcrafted features (like SIFT or SURF) and HOG descriptors, or even supervised CNN-based methods. Anyway, it seems to me that it is hard that the results could adequately support the initial claims in the present form.

It could be interesting to show some results of the matching part of the proposed method.




recommendation: Poster
review_title: null

Meta-review: metareview: This paper presents a well-received novel idea: unsupervised keypoint detection for affine registration, including inter-modality image registration cases. The proposed method is practical to time-consuming and often less reliable cases with large misalignments. Reviewers’ critiques and suggestions were mainly about more clarifications of the evaluation part. They were well answered in the rebuttal, and authors are expected to include those modifications in the final manuscript as mentioned.
confidence: 5: The area chair is absolutely certain
Decision: Accept (Poster)