Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper proposes a novel unsupervised topological difference detection algorithm. The model is based on a conditional variational auto-encoder and detects topological anomalies with regards to a reference alongside the registration
step. In terms of the experiments, as there exists no standard dataset of annotated topological differences for image registration, this paper validates the proposed topological difference detection algorithm on a proxy task of unsupervised anomaly detection in images.
main_review: Pros:

1. This paper explores a interesting problem: the alignment of domains whose topology is not the same. It is worth exploring as in many real applications, especially in biomedical domain, the anatomical/topological differences are very common and will potentially introduce bias in downstream analysis, which are routinely ignored.

2. The proposed model is able to detect topological inconsistencies with a purely registration-driven framework. 

3. By decomposing the joint likelihood into a likelihood for each pixel of the image, the proposed method could not only detect images that contain topological differences under the assumption that these images will overall have a lower likelihood, but also the position of outliers in the image.

Cons:

1. My concerns mainly remain in the experiment part. It's true that there exists no standard dataset of annotated topological differences for image registration. Instead the paper uses a proxy task of unsupervised anomaly detection in images to validate the effectiveness of the proposed method. But there is a gap between 'topological difference' and 'anamaly', which is not persuasive enough to demonstrate the efficacy of detecting topological differences. Also, the most strong baseline the paper chose is a paper published in 2010 (Li and Wyatt [26]), and more existing state-of-the-art anomaly detection algorithms should be compared to demonstrate the effectiveness of the proposed method. 
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: In this paper, the authors present an unsupervised topology difference detector based on geometric alignment. The method is based on a conditional variational autoencoder. It allows pixel-wise outlier detection. The method is evaluated on the task of brain tumor detection and compared to other unsupervised segmentation/registration methods and a supervised segmentation baseline. The results show that the presented method performs best compared to other unsupervised segmentation/registration methods.
main_review: The paper is well written and very detailed. Anomaly detection using VAE is a well-known method, but mostly in the context of image reconstruction. Using a VAE for outlier detection in the context of image registration is very interesting. The mathematical definition and the experiment description are very detailed and good to understand.

Comments:

1) There is no information about the registration accuracy. How does it influence the quality of outlier detection?
limitations_and_societal_impact: Yes, the authors have addressed the limitations of their work.
review_title: null

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: The paper presents an anomaly detection that specifically targeting topological differences. This is achieved by training an image registration neural network that warps control images to the target and highlighting differences. A prior term is modified to improve the alignment. The method is validated on BraTS, ABIDE, and OASIS. 
main_review: Here are my feedbacks:

1. The title, abstract, and introduction all emphasize topological differences, but it is unclear to me how the proposed method can specifically pick topological error over other types of outliers, both in theory and in practice. The warping is no longer guaranteed to be diffeomorphic, as also acknowledged by the authors "This drops the common topology assumption for the transformation, as transformations parameterized this way are not necessarily invertible anymore", so I'm totally confused here. This concern is also reflected in the results (Fig 3), where the signal largely resides in cortical gray matter and brain boundary in addition to the tumor region. It means the method is picking locations with high variation across subjects, not necessarily related to topology. I think the authors may want to define 'topological error' in a more stringent sense because a tumor does not necessarily break topology (again depending on what do you mean by topology here). 

2. Formulation of the warping model is questionable as it reduces to an elastic formulation rather than fluid diffeomorphism. It is unclear why $\alpha$ and $\beta$ in the prior can improve the model (especially in terms of topology). Eq. 4 indicates adding $\beta$ would penalize the magnitude of deformation, which might not be reasonable for brain alignment across subjects. Also unclear is the inference of $\alpha$ and $\beta$, which are hard parameters of the prior (instead of random variables). It is questionable to fit that prior to the data (then it is no longer a prior) while the fitting objective is intertwined with the learning of variational function $q$ (Eq. 7). How do $\alpha$ and $\beta$ come into the original generative process? I have a feeling the optimization no longer corresponds to the ELBO of the original likelihood.

3. Could the authors confirm that the disease cohort (i.e., AD and autism) is excluded from ABIDE and OASIS? because those subjects should also be considered as having brain injury, i.e., not in the space of healthy brains. In line with this concern, the method can detect 'abnormal regions' in the control cohort, which is surprising (Fig. 4).  This is potentially due to blending AD patients into the control cohort, or due to the method only detecting regions with large variation across subjects rather than just focusing on topology.

4. VAE-based anomaly detection is a crowded field. The comparison therefore is inadequate as it only considers An and Cho 2015. More recent methods like EGBAD and anoGAN should be at least be considered here
limitations_and_societal_impact: The inference of the model should be made clearer and the the evaluation should be expanded.
review_title: null

Meta-review: This paper addresses the problem of *topology-aware registration* and presents a first approach towards this goal. 

Reviews are quite mixed (even after the rebuttal phase) and the majority of reviewers perceived this work more like an *anomaly detection* approach rather than a means to an end, i.e., topology-aware registration. This seems to be primarily due to the proxy experiment of anomaly detection. Nevertheless, due to the non-existence of a suitable benchmark dataset to assess the ultimate objective of this work, this is a totally reasonable way to go in my point of view.

As to missing comparisons to recent state-of-the-art anomaly detection approaches, I do think that the authors make a fair point in their rebuttal: yes, one could compare to various anomaly detection methods, but this would, in essence, be a comparison against methods that cannot necessarily detect topological differences (which is the very task the presented method specifically addresses).

Overall, after reading the paper and carefully considering the reviews and the responses, I do not think that the changes required to address the reviewer's concerns are so major that they warrant rejection (e.g., the definition of topology, or clarification of voxel-wise losses). The required changes/adaptations are, in fact, quite minor and the concise comments by the authors already largely clarify the issues. Overall, I am recommending acceptance, based on the remarks outlined above. 
Decision: Accept (Poster)