Spot the Difference: Detection of Topological Changes via Geometric Alignment
Authors: Steffen Czolbe
Aasa Feragen
Oswin Krause
Venue: NeurIPS 2021 Poster (2021)
Keywords: Image registration
Geometric transformations
Variational Autoencoder
VAE
Topology

Abstract: Geometric alignment appears in a variety of applications, ranging from domain adaptation, optimal transport, and normalizing flows in machine learning; optical flow and learned augmentation in computer vision and deformable registration within biomedical imaging. A recurring challenge is the alignment of domains whose topology is not the same; a problem that is routinely ignored, potentially introducing bias in downstream analysis. As a first step towards solving such alignment problems, we propose an unsupervised algorithm for the detection of changes in image topology. The model is based on a conditional variational auto-encoder and detects topological changes between two images during the registration step. We account for both topological changes in the image under spatial variation and unexpected transformations. Our approach is validated on two tasks and datasets: detection of topological changes in microscopy images of cells, and unsupervised anomaly detection brain imaging.

Spot the Difference: Detection of Topological Changes via Geometric Alignment
Steffen Czolbe Department of Computer Science University of Copenhagen scrdscedi.. kuk dk
Aasa Feragen DTU Compute
Technical University of Denmark a a Eharrddtudk
Oswin Krause
Department of Computer Science
University of Copenhagen oswin. krause ku. dk
Abstract
Geometric alignment appears in a variety of applications. ranging from domain adaptation, optimal transport, and normalizing flows in machine learning; optical flow and learned augmentation computer vision and deformable registration within biomedical imaging. A recurring challenge is the alignment of domains whose topology is not the same; a problem that routinely ignored, potentially introducing bias in downstream analysis. As first step towards solving such alignment problems, we propose an unsupervised algorithm for the detection changes in image topology. The model is based on a conditional variational auto- encoder and detects topological changes between two images during the registration step. We account for both topological changes in the image under spatial variation and unexpected transformations. Our approach is validated on two tasks and datasets: detection of topological changes in microscopy images of cells, and unsupervised anomaly detection brain imaging.
Introduction
Geometric alignment is fundamental component of widely different algorithms, ranging from domain adaptation (7). optimal transport (40) and normalizing flows (35) 42 in machine learning; optical flow [1] and Iearned augmentation (20) in computer vision, and deformable registration
53). A recurring challenge is the alignment of domains whose topology is not the same. W When the objects to be aligned are probability distributions (35). this appears when distributions have different numbers of modes whose support is separated into separate connected components. When the objects to be aligned are scenes or natural images, the problem occurs with occlusion temporal changes [5T]. In biomedical image registration, the problem is very common and happens when the studied anatomy differs from "standard" anatomy (36). Despite being extremely common, this problem routinely ignored accepted inevitable, potentially introducing bias in downstream analysis.
within biomedical imaging [5] 15
19 39
We study two cases from biomedical image registration. One is the alignment of image slices to reconstruct a 3d volume, where changes in topology between slices introduce challenges in post- processing (Figure The other is the registration of brain MRI scans, where tumors give common examples of anatomies that are topologically different from healthy brains. In deformable image registration, "moving image" is mapped via nonlinear transformation to make it similar as possible to a "target" image, enabling matching local features or transferring information from one
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Image pair
Unsupervised prediction
Topological change
Figure 1: Left: Example of topological changes between two adjacent slices of human blood cells imaged via serial block-face scanning electron microscopy [41]. We aim to detect the change of topology caused by emerging organelle within the cell (highlighted by the red arrow) while accounting for non-linear deformations of the image introduced by natural shape changes between slices. Right: Heatmap of the likelihood of topological changes predicted by unsupervised model.
image to another. It is common to numerically stabilize the estimation of the transformation by constraining the predicted transformation to be diffcomorphic, that is, bijective and continuously differentiable in both directions. In particular, diffeomorphic transformations are homeomorphic, which implies that a common topology is assumed across all images [13 15). This topology is often provided by common template image Immplate from which all other images are obtained via the transformation from the group of diffeomorphisms G. Under this common topology assumption, the set of all images is given by
I cemplate + Φ E (
Topological differences in biomedical images can be caused by variety of processes. For instance, image slices obtained from volume do not all contain the same elements. Tumor growth or the removal surgical tissue can alter the topology of an image. Various processes can lead to the replacement or deformation of organic tissue, which cannot be mapped to the original image. We choose to model these topological differences as the inability to obtain one image from the other via homeomorphic transformation of the image domain. Since, within image registration, transformations are assumed to be continuously differeeniable, we are effectively modelling topological differences between pairs images via the failures of diffeomorphic image registration in aligning them.
As most registration algorithms align images based on intensity, c.g. minimizing mean squared error (MSE), these tissue changes make difficult map images correctly. The strong local deformations required to deal with the non-diffeomorphic part of the image inevitably also deform the surrounding area, leading distorted transformation fields in topologically matching parts of the image (36). These transformation fields adversely affect downstream tasks, for example indicating false size changes in adjacent regions.
Previous work on aligning topologically inconsistent domains. Attempting to relax the same- image assumption induced by fully diffeomorphic transformations not new. In the context of organs sliding against each other, several approaches exist, most of which rely on pre-annotating the sliding 46). with a few extensions to un-annotated
boundary using organ segmentation 66 10 22
37
43
images 38 45
When topological holes created removed in the domain, for example through tumors, patholo- gies, surgical resections, the loss function used for registration can be locally weighted or masked 26 29 30), or an artificial insection can be grown to correct anatomies [36). These approaches rely on annotation of the topological differences, which have to be provided manually or by segmentation. An exception given by Li and Wyatt [30], which detects changes in topology from the difference between the aligned images. This depends crucially on the ability to find good diffeomorphic registration outside the anomaly, which difficult all the while the applied transformation still diffcomorphic.

---

An alternative approach to registering topologically inconsistent images is to inpaint the difference in the source images to obtain a topologically consistent quasi-normal image. Then standard registration methods can be used on the altered images. Quasi-normal images can be obtained through low- rank and sparse matrix decomposition
principle component analysis [T6] 18). denoising
VAEs [52]. or learning of a blended representation [17). Registration with the quasi-normal approach retains the diffcomorphic properties of the transformation but does not register the topologically inconsistent areas of the images.
Our contribution. We propose an unsupervised algorithm for the detection of changes in image topology. To this end, we train conditional variational autoencoder for predicting image-to-image aligmment, obtaining per-target-pxel probability of being obtained from the moving image via diffeomorphic transformation. We combine semantic loss function trained to extract contextual information (8, with a learnable prior of transformations [9), allowing us to incorporate both the reconstruction error, as well as knowledge about the expected transformation strength.
We test the validity of our approach on novel dataset of cell slices with annotated topological changes and on the proxy task of unsupervised brain-tumor detection. We also validate our approach by investigating spatial "topological inconsistency likelihood", and showing that this likelihood is higher in regions where topologiaal inconsistencies are known to be common. Our model is able to detect topologiaal inconsistencies with a purely iiiistatioo-driver framework, and thus provides the first step towards an end-to-end registration model for images with topological discrepancies. The implementation available at hwtp://wwweeeeeeeee/o/og/ccanaaa
2 Background
2.1 Notation of images and transformations
We view an image interchangeably as two different structures. First, it is continuous function
I: Q - CRC where Ω = [0. 1) is the domain of the image, and C' the number of channels. This function image representation 1 where e is an index over the channels and I = (I, ........ I Second, this pixel grid is accompanied by graph structure that encodes the neighbourhood of each pixel. In this view, the set of neighbours of a pixel with index & (for example the 4 n-eiighbourhood of a pixel the image grid) is referred to ((k) and [N)( is the number of neighbours. The neighborhoods of a pixel gives rise to a graph which can be described via the graph laplacian A Rexn with A [N(]] and Ag. -1 when pixel k' E (k)),, and zero otherwise.
be approximated by grid of pixels with positions 37-- E 9 leading to the
I(xx).
Applying a spatial transformation & R + R to image is written J = I o 4 which can be seen as its own image with domain G2 = [0, 1 with pixel coordinates y/s E G2 and J. = (I))) The transformation & can be seen as a vector field on the image domain which assigns each pixel in J a position on I and thus it can be parameterized as pixel grid (b()) d I,..., D at the pixel coordinates of J using ((tx) = Vk + 4.. To make this choice of coordinate system clear, we will refer transformation that moves a pixel position from the domain G2 to the corresponding pixel in domain Or as Φ whenever it is not clear from the context. If Φ is diffeomorphism, it can alternatively parameterized by a vector field V on the tangent space around the identity, where the mapping between the tangent space and the transformation is given by &- exp(V). which amounts to integration over the vector field
2.2 Variational registration framework
It is possible to phrase the problem of fitting a registration model in terms of variational inference, using an approach similar to conditional variational autoencoders [47). Here, we summarize the approach taken by 99 31 For D-dimensional image pair (I,J), J). we assume that J is generated from by drawing transformation from prior distribution p [], apply it to I and then add pixel-wise noise:
Proise [J (T) dΦ
This includes the common topology assumption implicitly via p(T[I), which is typically chosen to produce invertible transformations depending only on the topology of I, well as the noise
3

---

model which does not assume systematic changes between J and I. This model can be learned using variational inference using a proposal distribution g( ( [ with evidence lower bound (ELBO)
(1) In contrast to variational autoencoders, the decoder is given by the known application of & Thus, the degrees of freedom in this model are in the choice of the encoder, prior, and the noise distribution.
I[] 2 T. [log Puoise (J o Φ)] K
Dalca et al. [9] proposed parameterize O vector field Vi(1) on the tangent space, which turns application of Φ eX)(V) into sampling an image with a spatial transformer module (24). As a prior for this parameterization, they chose a prior independent of
II N (v(d) 10,A-1) d=1
p())
where we used the implicit identification of + and V and the precision matrix is chosen as the Graph Laplacian the neighbourhood graph (see notation). Using encoder that for each pixel the KL divergence is derived as proposes q( (Vi I. = [N4k U)k (1)).
D
KL ([)]))))P(((I)) 2
ΣΣ log d=1 k=1
(4) k +IN()))
(m/ - -4(00 -1 LEN(K)
E
const. (2)
It is worth noting that this equation is invariant under translations of μ. This invariance manifests in -effciefi of A and as a result, const infinite. Thus, sampling from the prior and bounding the objective is impossible. Still training with this term works in practice as images are usually pre-aligned with an affine transformation and thus translations are close to zero. We will present slightly modified approach, rectifying the missing eigenvalue.
3 Detection of topological differences
The variational approach for learning the distribution of transformations introduced before optimizes an ELBO on log [(]]( This information is enough to detect images that contain topological differences under the assumption that these images will overall have lower likelihood. However, in our application, we need not only to detect the existence but also the position of outliers in the image. For this, we have ensure that log [I] can be decomposed into likelihood for each pixel of the image. It is immediately obvious by inspection of the ELBO [ together with the KL-Divergence (2). that the lower bound on logp([]) p[][ can be decomposed into pixel-wise terms if log Phoise (J [I 4) can be decomposed as such. To enforce this, we will introduce a general form of error function, which can be decomposed and includes the MSE as a special case. For this, we first map the images and to feature maps over the pixel positions l via a mapping fa (I) E IR and define the loss as:
Pnoise [ )) II (TI E,E,)
where y E RXx is a diagonal covariance matrix with variances learned during training.
The ability to decompose the likelihood is not enough for meaningful metric, as we have to ensure that each term is calculated in the correct coordinate system. This depends the parameterisation and regularisation of Φ. In the approach by Dalca et al. [9] the parameterization the tangent space and consequently the prior is also this space. Since the connection between & and V is given by integration of the vector field, decomposing (2) for a single pixel l will produce estimates based the local differential of the transformation, but will take the full path with starting and endpoints into account. Thus, correct cost assignments require integration of (2) over the computed path, which expensive and suffers from severe integration inaccuracies. Instead, we will use an alternative approach, where we parameterize & directly as a vector field on the image domain. Transformations parameterized this way are not necessarily invertible anymore, yet smoothness is still encouraged by the prior.
of O is defined on
Learnable prior
Using this parameterization, extend the approach by Dalca et al. and introduce a parameterized prior on &y that is learned simultancously with the model:
p p
II N (l))) 10, (0. Aas = as + n d=1
11
(4)

---

The expected variations and translations between transformation vectors are governed by a and B. Unlike most works in image registration, we do not treat these as tuneable hyperparameters, but instead view them as unknowns to be fitted to the data during training similar to [28 49 For efficient learning, we use an estimate for the optimal values for a, 3 over a batch of samples during training, and use running average at test time. A detailed explanation is given in supplementary material A.
The second term of 4 ensures that Ass is invertible, by adding a multiple of the eigenvector = (l...., 1)² It can be verified easily that A1 O. Unlike adding multiple of the identity matrix to A, adding the missing eigenvalue does not modify the prior in any other way than regularizing the translations. Further, it ensures that the KI. divergence of the resulting matrix be quickly computed up to constant as and 3 do not modify the same eigenvalues. Recomputing the KL-divergence for n transformation vectors in D dimensions leads
2
n
2 KL (g(*) I, J) P) = -nn D)g aoga- Doog 3 3
Σ
(d) 1k
71
D
+ log uk + (a)N(())
(d) n2 k
+ a
Σ
14 (d)
+ const (5)
IEN(K)
Decomposed error metric We define our pixel-wise error measure for topological change detection based on the ELBO [ with KL-divergence 55 follows, where compute μ4
(d) and uk via the
proposal distribution ((IIIJ) and pick & (d) =14 (d).
(d)
D n
[J = - log N (fa o 2 Σn} +
B14
ΣΣ
M
(d)
n
d=1
i=1
Σ - log ok (1)) + (a)N(())+
n2
B
2k
Σ IEN(R)
(1) (d) 1k 11 )
(6)
We will treat the loss over all pixels L(J = (JJ),, ..... Ln (J[I)) as another image with domain and pixel coordinates the same as J. This measure is not symmetric. The prior distribution does not treat the distributions JJ and q( {$ J. equally. If Φ maps a line in J to an area in I, this will incur large visible feature along the line due to violating the smoothness assumption encoded in the prior. On the other hand, if an area in J gets mapped to a line in I, the overall error contribution is smoothed out over the area. To rectify this issue, we will compute a bidirectional
measure [JII = I(J][T) + L[]])) o II-J, where P the same as the one used to compute [J] For this measure it holds that if 2 = Dr we have Laym (I]J) = Laym (J[I) 2 _ up
to interpolation errors caused by the finite coordinate grid.
Topological outlier detection Laym detects topological changes between two images. However, for evaluation on the Brain dataset, we are interested in topological outliers. Outliers can be detected using Lym by contrasting the observed deviations with the observed deviations within a larger set of control images C. This leads to the score
(J)) EEEc [4yy ''Ee [Lyy (I|K)] P1-s]].
4 Evaluation
We evaluate our approach on two tasks. In the first, we measure prediction agreement with annotated topological changes on dataset of cell slices. For this, we introduce the first dataset with annotated topological differences for image registration (see Section 4.1.. which allows us to significantly
expand on the evaluation strategies of prior work 226 29 30 In the second task, we adapt our approach to anomaly detection in order to detect brain tumors on slices of MRI images.
On the change detection task, we use our model prediction of Laym directly. On the anomaly detection task, we use the score 77. which subtracts the average scores over healthy patients for each pixel.
We compare our model to the following baselines:
5

---

1. Two unsupervised approaches for topological change detection:
Li and Wyatt's s (30] intensity difference and image gradient-based approach using a deterministic registration model
obtain the transformations.
Using the same model, we devise a method based on the Jacobian Determinant of the transfor- mation field | We expect strong stretching or shrinkage in areas of topological mismatch, which we measure using use the score log (Idet.
We adapt both approaches to the task of tumor detection by subtracting the average scores over healthy patients, analogous to 7
2. The approach by An and Cho [T] for unsupervised anomaly detection in images is based on the lo- cal reconstruction error of variational autoencoder. The error score is ||J J deec(ene(J]) I| where eJc(J) maps J to the mean of the variational proposal distribution and dec is the corresponding learned decoder. As the score does not use registration, we cannot use equation (7)
3. A supervised segmentation model trained for segmenting topological changes based on two input images on the cell dataset, and tumor segmentation based on a single input image on the brain dataset. Since this model requires annotated data, withhold 75% of the annotated volumes for training and evaluate the segmentation model only on the remaining samples.
In both tasks, we measure the pixel-wise agreement of the models with the annotated ground-truth using the receiver operating characteristic curve (ROC curve) and compare the area under the curve (AUC) between the models. AUC estimates are bootstrapped the subject level to obtain error estimates.
As additional evaluations, present qualitative examples and investigate whether brain regions with known topological variability get assigned higher scores in our model. For this we compute the pairwise average score Lsym over multiple healthy subjects and register them all to brain atlas using ELK sym (I K) o P1-AAtas]- We group the scores by their position on the brain atlas into partitions: cortical surfaces, subcortical regions, and ventricles.
4.1 Tasks and Data
Topology change detection in Cells Serial block face scanning electron microscopy (SBEM) is a method to obtain three-dimensional images from small biological samples. An image is taken from the face of the block, after which a thin section is cut from the sample to expose the next slice. A challenge is the accurate reconstruction of the volume, neighboring slices differ by both natural deformations and changes in topology. Natural deformations can be introduced by shape-changes of objects between the slices, and deformations of the sample due the physical cutting. Changes in topology occur due to objects present in one slice but not the other, and tears of the physical sample induced by the cutting.
We evaluate our method on the detection of topological changes between neighboring slices of human platelet cells recorded with SBEM. We use the pre-segmented dataset by Quay et al. [41] as a base. In the dataset, image slices are affinely pre-aligned and manually segmented into classes. Afterwards, for the validation and test set, we annotated changes in the topology of the segmentation masks. Using this approach, not all instances of topological changes in the image can be annotated as the segmentation maps merge several types of cell components into single class. The data is cropped into patches of 256 x 256 pixels and we use 9 patches of 50 slices for training, 4 patches of 24 slices for validation, and 5 patches of 24 slices for test (3 patches for the supervised approach due to the training-test split of annotated data).
Brain tumor detection Individual brains offer range of topological differences, especially in the
presence of tumors. Further, inter-subject differences are found at the cortical surface, where the sulci vary significantly (48). and near ventricles, which can either be open cavities, partially closed (36]. We quantitatively evaluate our method on the proxy task of detecting brain tumors. Tumors change the morphology of the brain and can thus be detected indirectly via the large transformations they cause. For this, we first train our model using a dataset of healthy images from the control group and then use (7) to obtain a score for topological outlier detection. For the control set, we combine TI weighted MRI scans of the healthy subjects from the ABIDE I [11 ABIDE II 11 and OASIS3 (27 studies.
ICC BY-NC-SA
http:///ww.ooooooeeeeeeee///roee www.iiennnsssnnneeeeeeeeeeeee 3//.

---

For the tumor set we use MRI seans from the BraTS2020 brain tumor segmentation challenge 3 34]. which have expert-annotates tumors. We use the Tl weighted MRI scans, and combine labels of the classes necrotic/cystic and enhancing tumor core into single tumor class. All datasets are anonymized, with no protected health information included and participants gave informed consent to data collection.
We perform standard pre-processing on both brain datasets, including intensity normalization, affine spatial alignment, and skull-stripping using FreeSurfer [14). From each 3D volume, we extract a center slice of 160 224 pixels. Scans with preprocessing errors are discarded, and the remaining images of the control dataset are split 2381/149/162 for train/validation/tes.. Of the tumor dataset, 84 annotated images with tumors larger than 5em2 along the slice are used for evaluation (17 for the
supervised approach due to the training-tes, split of subjects).
4.2 Model and training
All models evaluated are based on a U-Net (44) architecture, except An and Cho []. which we implement using spatial VAE following the previously published adaptation to Brain-scans by Venkatakrishnan et al. [50]. The networks consist of encoder and decoder stages of 64, 128, 256 channels for all registration models, and 32, 64, 128, 256 channels for the segmentation and VAE models. Each stage consists of batch normalization (23) and a convolutional layer.
In our approach, we U-Net to model p[] The output of the last decoder stage is fed through separate convolution layers with linear activation functions to predict the transformation mean and log-scaled variance. Throughout the network, we use LeakyReLu activation functions. The generator step I & is implemented by a parameterless spatial transformer layer (24). During training of our model, we use the analytical solution for prior parameters a, 3 (supplementary material, Eq. 8), averaged over the mini-bate batch of 32 image pairs. For validation and test, we use the running mean recorded during training. The diagonal covariance of the reconstruction loss Ey is treated as trainable parameter.
For all datasets, we use data augmentation with random affine transformations of the training images. For training, the optimization algorithm is ADAM (25) with a learning rate of 10-4, Regularization of all models is performed by applying an L2-penalty to the weights with a factor of 0.01 for the cell dataset and 0.0005 for the brains. We train each model on a single TitanRTX GPU, with maximum training times of 1 day for the cells and 4 days for the brains. Hyperparameters: The network by Venkatakrishnan et al. [50] has o = chosen from {0.1, 10). based on reconstruction loss on validation set. The deterministic registration model was trained using À 0.1 as in [8] For (30] , the parameters a of the Gaussian derivative kernel and hyper-parament K where chosen to maximize the AUC score, selecting
6, K 2 out of 11....,9??
For the reconstruction loss, we compare two different loss functions. The first using the MSE as in 99 30) The second is a semantic similarity metric similar to (8). To obtain the semantic image descriptors, we train a U-net with 32, 64, 64 channels for image segmentation, using the manual annotations of the cell set and automatically created labels obtained with FreeSurfer (14) for the brain control images. Notably, the segmentation models used for the loss have not been trained on images or pairs containing topological changes or tumors. From this network, we extract the features of the first three stages and use them as a 160-channel feature map in the loss ( For both the MSE and the semantic loss, we learn the variance parameters while training the variational autoencoder.
4.3 Results
The ROC curves of all trained models on the cell and brain tasks can be seen in Figure 2 For both tasks, the supervised model performed best (AUC 0.90, 0.95), while our proposed approach with semantic loss performed best among the unsupervised models (AUC 0.88, 0.80). The unsupervised approach for topological change detection by Li and Wyatt (30) (AUC 0.75, 0.70) performed overall best among the baselines, but worse than our method. The unsupervised anomaly detection method by An and Cho [1 (AUC 0.72, 0.67) performed well at detecting brain tumors, but worse at detecting topological changes in the cell images. Using the Jacobian determinant (AUC 0.75, 0.62) performed well on the cell images but worse on the brain tumor detection task. Our approach using MSE (AUC 0.72, 0.61) performed worse than the other methods on both tasks.
7

---

Cells
Brains
Method
AUC (Cells)
AUC (Brains)
1.0
True Positive Rate
Ours (Sem. Loss)
.877 = .003
.797 + .005
Ours (MSE)
.712 = .003
.611 + .009
Li and Wyatt
.755 + .009
.697 .006
0.4
Jac. Det.
.752 + .005
.616 .011
0.2
An and Cho
.718 .004
.670 + .01
Supervised Seg.
.999 = .004
.946 + .025
0.0
0.2
0.4
0.6
0.8
1
0.2
0.4 0.6 0.8
False Positive Rate
Figure 2: Receiver operating characteristic curves (ROC) and area under the curve (AUC) for detecting topological changes on the cell and brain datasets. We test models of our method for unsupervised topological change detection, trained with semantic loss function and the MSE in the reconstruction term, and compare against unsupervised baselines from image registration (Li and Wyatt [30], Jacobian Determinant) and unsupervised anomaly detection (An and Cho (T). For reference, we also include a supervised segmentation model, which has been trained on the ground truth annotations.
I
J
Lsym IJ[)
Figure 3: Topological differences detected by our method, cell dataset. Neighboring slices I, J in rows and 2. Heatmaps of the likelihood of topological differences detected with laym in row Heatmaps overlayed image J to comparison. Annotated topological differences used for evaluation outlined red. Note that only a subset topological anomalies present annotated in dataset.
When analyzing the ROC curves, our model performed best among the unsupervised models for all false positive rates, while the supervised model is the best overall. Finally, even though both models share the same trained model, the score used by Li and Wyatt (30) performed better than scoring using the Jacobian determinant on the brain tumor detection task, while on the cell dataset, both approaches performed the same.
We show qualitative results on the cell dataset in Figure In row 3, we see that Lyym detected annotated areas topological change (contoured red),
more certain at detecting changes in areas with high intensity difference. In many cases, the model assigns likelihood of topological changes to areas that have not been annotated the dataset, such as the merging cell boundary in column 2 or many small changes in the cell interior in column 4.

---

Sym (JII)
Q(J)
I
J
SEM
MSE
SEM
MSE
Figure 4: Topological differences detected by our method, brain dataset. Structurally normal brain in column brain with tumor in column Heatmaps of the likelihood of topological differences detected with Lyym in columns 3, 4. Likelihood of topological differences caused by the structural anomaly filtered by Eq. in columns Contour the ground truth brain tumor in red. Heatmaps overlayed image J to ease comparison.
50
Laym
0
-50
Cortical Surface
Ventridees
Subcortical Structures
Figure 5: Left: Heatmap of average location of topological differences among the control group. predicted by the semantic model, averaged with Ι,, Lymm (I|K) o L using a brain atlas as reference image. Center: We use morphological operations to split the atlas into cortical surface (blue), ventricles (orange) and sub-cortical structures (green). Right: Likelihood of topological differences occurring in each region. Boxplot with median, quartiles, deciles.

---

Qualitative results on the brain data are presented in Figure 4 When looking at columns 3 and 4, we see that Loym detected notable areas with high changes in topology compared to the reference image I. This includes the ventricles (rows 2.3), the cortical areas with the sulci (all rows) as well as tumor areas (rows 1,3.5). There was a clear difference in the behaviour between semantic loss and MSE as the semantic loss highlights broader regions of the surface. When comparing the outlier-detection measure (J) in columns 5 and 6, we can see that our approach filtered most of the ventricles and sulci leaving area around most tumor regions. Notable exceptions are rows 2 and 4, where the tumor area was not highlighted, as well as row I where only part of the tumor was detected.
In Figure 5 we show the average topological change score on healthy subjects. We see on the brain image and the box plot, that the cortical surfaces and ventricles get assigned higher scores than the subcortical structures.
5 Discussion and conclusion
In this work, have introduced a novel approach for the detection of topological changes. We evaluated our approach qualitatively and compared it quantitatively to previous approaches using both novel dataset with purpose-made annotations and on unsupervised segmentation proxy task. On both tasks, our approach performed best among the unsupervised methods, but could not reach the performance of the supervised method.
An unsupervised method is useful in practice, as annotations of topological changes are rarely available. While results are not pixel exact, they indicate where a registration algorithm must be used more carefully to obtain a valid registration. The results on the cell dataset align well with the annotations, and many of the false positives appear to be caused by incomplete annotation of the data. This is also reflected in the reported ROC-curves, which show that our model outperforms the supervised segmentation model at false positive rates larger than 0.5. The results obtained on the tumor segmentation proxy task are reinforced by the distribution of scores obtained on healthy patients in different parts of the brain. The high likelihood of topological differences in ventricles found agrees with previous work (36] and the higher scores in cortical surfaces reflect the fact, that the sulci of the cortical surface exhibit high variability between subjects 77]. which was previously difficult to quantify.
Our results also show that using semantic loss function is advantageous compared the MSE in this task, as all MSE based methods performed worse than our approach using the semantic loss. This is likely because the contrast between some anatomical areas is quite small and thus missed by the MSE. In contrast, the semantic loss incorporates more texture information and thus is capable of differentiating between areas of similar intensity but different semantics. However, particularly on the brain example, even the semantic approach misses tumors close to the cortex. We hypothesize, that this is in part caused by the similar appearance of tumors and grey matter, in part by the semantic model not being trained on tumors, and in part due to the cortical area containing high topological variation among the control group as well.
On the brain dataset, our unsupervised results for the method by An and Cho [ are in line with previously reported results comparable dataset [50). However, our supervised results are not comparable to the results published for the BRATS challenge, as we selected a subset of data for training and only used structural MRI images, discarding the other modalities. On the cell dataset, no other work on topology change or outlier detection is available.
Our study has several limitations. We only investigate registrations in 2D and topological differences might vanish if the whole 3D volume is considered. The transformations obtained by our unsupervised method differ from strongly regularised methods, as the hyperparameter-les learned prior under- regularises in order to maximize the likelihood of topological match during training. Conversely, the poor performance of the Jacobian determinant might be due to strong regularisation for good performance in image registration as we used the hyperparameters as found in [8).
In conclusion, our approach serves as the first step for unsupervised annotation of topological changes in image registration. Our approach is fully unsupervised and hyperparameter-fiees making it a prospective building block in an end-to-end topology-avee aware image registration model.
10

---

Acknowledgements
This work was funded by the Novo Nordisk Foundation (grants no. NNF200C0062606 and NNF170C0028360) and the Lundbeck Foundation (grant no. R218-2016-883),
The human platelet SBEM data and segmentations change annotations are ours. The Brain tumor data was provided by the BraTS challenge. The Brain control data was provided in part by OASIS Principal Investigators: T. Benzinger. D. Marcus, J. Morris; NIH P50 AG00561, P30 NS09857781, PO1 AG026276, POI AG003991, ROI AG043434, UL1 TR000448, RO1 EB009352. AV-45 doses provided by Avid Radiopharmaceutical,, wholly-owned subsidiary of Eli Lilly.
provided by Matthew Quay, the topological
References
[1] Jinwon An and Sungzoon Cho. Variational Autoencoder based Anomaly Detection using Reconstruction Probabiläy. Tech. rep. 2015.
[2] Vincent Arsigny, Olivier Commowick, Xavier Pennec, and Nicholas Ayache. "A log- -ugleueldeann framework for statistics on diffeomorppisms" In: International Conference on Medical Image Computing and Computer- Assisted Intervention. Springer Verlag, 2006, pp. 924-931.
[3] Spyridon Bakas, Hamed Akbari, Aristeidis Sotiras, Michel Bilello, Martin Rozycki, Justin Kirby, John B. Freymann, Keyvan Farahani, and Christos Davatzikos. "Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features".
In: Scientific Data 4 (2017).
[4] Spyridon Bakas et al. "Identifying the Best Machine Learning Algorithms for Brain Tumor Seg- mentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge". In: arXiiv 124 (2018).
[5] Guha Balakrishnan, Amy Zhao, Mert R. Sabuncu, John Guttag, and Adrian V. Dalca. "Vox- elMorph: A Learning Framework for Deformable Medical Image Registration", In: IEEE Transactions on Medical Imaging 38.8 (2019), pp. 1788-1800.
[6] Xiang Chen, Nishant Ravikumar, Yan Xia, and Alejandro F Frangi. "A Deep Discontinuity- Preserving Image Registration Network". In: International Conference on Medical Image Computing and Computer-Assister Intervention. 2021.
[7] Nicolas Courty, Remi Flamary, Devis Tuia, and Alain Rakotomamonjy. "Optimal Transport for Domain Adaptation".. In: IEEE Transactions on Pattern Analysis and Machine Intelligence 39.9 (2017), pp. 1853-1865.
[8] Steffen Czolbe, Oswin Krause, and Aasa Feragen. "Semantic similarity metrics for learned image registration". In: Proceedings of Machine Learning Research (2021).
[9] Adrian V. Dalca, Guha Balakrishnan, John Guttag, and Mert R. Sabuncu. "Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration". In: Medical Image Computing and Computer Assisted Intervention (2018), pp. 729-738.
[10] V Delmon, S Rit, R Pinho, and D Sarrut. "Registration of sliding objects using direction dependent B-splines decomposition Registration sliding objects using direction dependent B-splines decomposition
In: Phys. Med. Bio 58.5 (2013), pp. 1303-1314.
[11] Adriana Di Martino, Chao- Gan Yan. Qingyang Li, et al. "The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism". In: Molecular psychiatry 19.6 (2014), pp. 659-667.
[12] Adriana Di Martino et al. "Enhancing studies the connectome in autism using the autism brain imaging data exchange " In: Scientific Data 4.1 (2017), pp. 1-15.
[13] Mirza Faisal Beg, Michael Miller, Alain Trouvétrouv, and Laurent Younes. "Computing Large Deformation Metric Mappings via Geodesic Flows of Diffeomorphisms" In: International Journal of Computer Vision 61.2 (2005), pp. 139-157.
[14] B Fischl. FreeSurfer. Neurolmage, 62 (2), 774-781 2012.
[15] Ulf Grenander and Michael Miller. "Computational anatomy: An emerging discipline". In: Quarterly of applied mathematics 56 (1998), pp. 617-694.

---

[16] Xu Han, Roland Kwitt, Stephen Aylward, Spyridon Bakas, Bjoern Menze, Alexander Asturias, Paul Vespa. John Van Horn, and Mare Niethammer. "Brain extraction from normal and approach". In: Neurolmage 176
pathological images: A joint SeA/Image-Reconstruee (2018), pp. 431-445.
[17] Xu Han, Zhengyang Shen, Zhenlin Xu, Spyridon Bakas, Hamed Akbari, Michel Bilello, Christos Davatzikos, and Mare Niethammer. "A Deep Network for Joint Registration and Reconstruction of Images with Pathologies". In: 11th International Workshop on Machine Learning in Medical Imaging. Springer Nature, 2020, pp. 342-352.
[18] Xu Han, Xiao Yang. Stephen Aylward, Roland Kwitt, and Mare Niethammer. "Efficient registration of pathological images: A joint
approach". In: 4th
International Symposium on Biomedical Imaging 2017 (2017), pp. 10-14.
[19] Lasse Hansen and Mattias P. Heinrich. "Tackling the Problem of Large Deformations in Deep Learning Based Medical Image Registration Using Displacement Embeddings". In: Medical Imaging with Deep Learning (2020).
[20] Søren Hauberg, Oren Freifeld, Anders Boesen, Lindbo Larsen, John W Fisher, li Lars, and Kai Hansen. "Dreaming More Data: Class-dependent Distributions over Diflcomorphisms for Learned Data Augmentation". In: Artificial Intelligence and Statistics. Vol. 41. PMLR, 2016,
pp. 342-350.
[21] Berthold KP Horn and Brian G Schunck. "Determining optical flow". In: Artificial intelligence (1981), pp. 185-203.
[22] Rui Hua, Jose M. Pozo, Zeike A. Taylor, and Alejandro F. Frangi. "Multiresolution eXtended Free-Form Deformations (XFFD) for non-rigid registration with discontinuous transforms". In: Medical Image Analysis 36 (2017), pp. 113-122.
[23] Sergey loffe and Christian Szegedy. "Batch normalization: Accelerating deep network training by reducing internal covariate shift". In: International Conference on Machine Learning. International Machine Learning Society (IMLS), 2015, pp. 448-456.
[24] Max Jaderberg. Karen Simonyan, Andrew Zisserman, and Koray Kavukcuoglu. "Spatial Transformer Networks". In: Advances in neural information processing systems (2015).
[25] Diederik Kingma and Jimmy Lei Ba. "Adam: A method for stochastic optimization". In: International Conference on Learning Representations. 2015.
[26] Dongjin Kwon, Mare Niethammer, Hamed Akbari, Michel Bilello, Christos Davatzikos, and Kilian M. Pohl. "PORTR: Pre-operative and post-recurrence brain tumor registration". In:
IEEE Transactions on Medical Imaging 33.3 (2014), pp. 651-667.
[27] Pamela J LaMontagne, Tammie L S Benzinger, John C Morris, al. "OASIS-3: Longitudinal Neuroimaging, Clinical, and Cognitive Dataset for Normal Aging and Alzheimer Disease", In: medRxiv (2019).
[28] Dong Bok Lee, Dongchan Min, Seanie Lee, and Sung Ju Hwang. "Meta-GMVAE: Mixture of Gaussian VAE for Unsupervised Meta-Learning". In: International Conference on Learning Representations. 2020.
[29] Xiaoxing Li, Xiaojing Long, Christopher Wyatt, and Paul Laurienti. "Registration of Images with Varying Topology Using Embedded Maps". In: IEEE Transactions on Medical Imaging 31.3 (2012). pp. 749-765.
[30] Xiaoxing Li and Chritopher Wyatt. "Modeling topological changes in deformable registration". In: 2010 7th IEEE International Symposium on Biomedical Imaging. 2010, pp. 360-363.
[31] Lihao Liu, Xiaowei Hu, Lei Zhu, and Pheng-Ann Heng. "Probabilistic Multilayer Regulariza- tion Network for Unsupervised 3D Brain Image Registration". In: International Conference on Medical Image Computing and Computer-Assisted Intervention. 2019, pp. 346-354.
[32] Xiaoxiao Liu, Mare Niethammer, Roland Kwitt, Matthew McCormick, and Stephen Aylward. "Low-Rank the Resoue- Atlas-Based Analyses the Presence of Pathologies". In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 8675 LNCS.PART (2014), pp- 97-104.
[33] Xiaoxiao Liu, Mare Niethammer, Roland Kwitt, Nikhil Singh, Matt McCormick, and Stephen Aylward. "Low-Rank Atlas Image Analyses in the Presence of Pathologies". In: IEEE Trans- actions on Medical Imaging 34. 12 (2015), pp. 2583-2591.
[34] Bjoern H. Menze et al. "The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)". In: IEEE Transactions on Medical Imaging 34.10 (2015), pp. 1993-2024.
12

---

[35] Didrik Nielsen, Priyank Jaini, Emiel Hoogeboom, Ole Winther, and Max Welling. SurVAE Flows: Surjections to Bridge the Gap berween VAEs and Flows. Tech. rep. 2020, pp. 12685- 12696.
[36] Rune Kok Nielsen, Sune Darkner, and Aasa Feragen. "TopAwaRe: Topology-Aware Regis- tration". In: International Conference on Medical Image Computing and Computer-Assisted Intervention (2019), pp. 364-372.
[37] Danielle F. Pace, Stephen R. Aylward, and Mare Niethammer. "A locally adaptive regulariza- tion based on anisotropic diffusion for deformable image registration of sliding organs". In: IEEE Transactions on Medical Imaging 32.11 (2013), pp. 21 2126.
[38] Barthomiej W. Papiez, Mattias Heinrich, Jérome Fehrenbach, Laurent Risser, and Julia A. Schnabel. "An implicit sliding-motion preserving regularisation via bilateral filtering for deformable image registration". In: Medical Image Analysis 18.8 (2014), pp. 1299-1311.
[39] Sarah Parisot, William Wells, Stéphane Chemouny, Hugues Duffau, and Nikos Paragios. "Concurrent tumor segmentation and registration with uncertainty-based sparse non-uniform graphs". In: Medical Image Analysis 18.4 (2014), pp. 647-659.
[40] Gabriel Peyré and Marco Cuturi. "Computational optimal transport". In: Foundations and Trends in Machine Learning 11.5-6 (2019), pp. 1-257.
[41] Matthew Quay. Zeyad Emam, Adam Anderson, and Richard Leapman. "Designing deep neural networks to automate segmentation for serial block-face electron microscopy". In: International Symposium on Biomedical Imaging. Vol. 20r8-A IEEE Computer Society, 2018, pp. 405-408.
[42] Danilo Jimenez Rezende and Shakir Mohamed. "Variational Inference with Normalizing Flows". In: International conference machine learning. PMLR, 2015, pp. 1530-1538.
[43] Laurent Risser, François Xavier Vialard, Habib Y. Baluwala, and Julia A. Schnabel. "Piecewise- diffeomorphic image registration: Application to the motion estimation between 3D CT lung images with sliding conditions". In: Medical Image Analysis 17.2 (2013), pp- 193.
[44] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. "U-net: Convolutional networks for biomedical image segmentation". In: International Conference on Medical Image Computing and Computer-Assister Intervention. Vol. 9351. Springer Verlag, 2015, pp. 234-241.
[45] Dan Ruan, Selim Esedoğlu, and Jeffrey A. Fessler. "Discriminative sliding preserving regular- ization in medical image registration". In: Proceedings 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, ISBI 2009. 2009, pp. 430-433.
[46] Alexander Schmidt-Richberg Jan Ehrhardt, René Werner, and Heinz Handels. "Fast explicit diffusion for registration with direction -eeeedent regularization". In: Biomedical Image Registration 7359 (2012), pp. 220-228.
[47] Kihyuk Sohn, Xinchen Yan, and Honglak Lee. "Learning Structured Output Representation using Deep Conditional Generative Models". In: Advances in Neural Information Processing Systems. Vol. 28. 2015, pp. 3483-3491.
[48] Elizabeth R. Sowell, Paul M. Thompson, David Rex, David Kornsand, Kevin D. Tessner, Terry L.. Jernigan, and Arthur W. Toga. "Mapping sulcal pattern asymmetry and local cortical surface gray matter distribution in vivo: Maturation in perisylvian cortices". In: Cerebral Cortex 12.1 (2002), pp. 17-26.
[49] Arash Vahdat and Jan Kautz. "NVAE: A Deep Hierarchical Variational Autoencoder". In: Advances in Neural Information Processing Systems (2020).
[50] Abinav Ravi Venkatakrishnan, Seong Tae Kim, Rami Eisawy, Franz Pfister, and Nassir Navab. "Self-Supervised ontto--Distriuution Detection in Brain CT Scans". In: Medical Imaging Meets NeurIPS Workshop (2020).
[51] Yang Wang, Yi Yang, Zhenheng Yang, Liang Zhao, Peng Wang, and Wei Xu. "Occlusion Aware Unsupervised Learning Optical Flow" In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018, pp. 4884 4893.
[52] Xiao Yang. Xu Han, Eunbyung Park. Stephen Aylward, Roland Kwitt, and Marc Nietham- mer. "Registration of Pathological Images". In: Simulation and synthesis in medical imaging (Workshop) 9968 (2016), p. 97.
[53] Xiao Yang. Roland Kwitt, Martin Styner, and Mare Niethammer. "Quicksilver: Fast predictive image registration A deep learning approach". In: Neurolmage 158 (2017), pp. 378-396.
13


Reviews:
[{"review_id": "x5jePx5ZE4i", "reviewer": "Reviewer_daqa", "summary": "The paper presents an anomaly detection that specifically targeting topological differences. This is achieved by training an image registration neural network that warps control images to the target and highlighting differences. A prior term is modified to improve the alignment. The method is validated on BraTS, ABIDE, and OASIS. ", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Here are my feedbacks:\n\n1. The title, abstract, and introduction all emphasize topological differences, but it is unclear to me how the proposed method can specifically pick topological error over other types of outliers, both in theory and in practice. The warping is no longer guaranteed to be diffeomorphic, as also acknowledged by the authors \"This drops the common topology assumption for the transformation, as transformations parameterized this way are not necessarily invertible anymore\", so I'm totally confused here. This concern is also reflected in the results (Fig 3), where the signal largely resides in cortical gray matter and brain boundary in addition to the tumor region. It means the method is picking locations with high variation across subjects, not necessarily related to topology. I think the authors may want to define 'topological error' in a more stringent sense because a tumor does not necessarily break topology (again depending on what do you mean by topology here). \n\n2. Formulation of the warping model is questionable as it reduces to an elastic formulation rather than fluid diffeomorphism. It is unclear why $\\alpha$ and $\\beta$ in the prior can improve the model (especially in terms of topology). Eq. 4 indicates adding $\\beta$ would penalize the magnitude of deformation, which might not be reasonable for brain alignment across subjects. Also unclear is the inference of $\\alpha$ and $\\beta$, which are hard parameters of the prior (instead of random variables). It is questionable to fit that prior to the data (then it is no longer a prior) while the fitting objective is intertwined with the learning of variational function $q$ (Eq. 7). How do $\\alpha$ and $\\beta$ come into the original generative process? I have a feeling the optimization no longer corresponds to the ELBO of the original likelihood.\n\n3. Could the authors confirm that the disease cohort (i.e., AD and autism) is excluded from ABIDE and OASIS? because those subjects should also be considered as having brain injury, i.e., not in the space of healthy brains. In line with this concern, the method can detect 'abnormal regions' in the control cohort, which is surprising (Fig. 4).  This is potentially due to blending AD patients into the control cohort, or due to the method only detecting regions with large variation across subjects rather than just focusing on topology.\n\n4. VAE-based anomaly detection is a crowded field. The comparison therefore is inadequate as it only considers An and Cho 2015. More recent methods like EGBAD and anoGAN should be at least be considered here", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The inference of the model should be made clearer and the the evaluation should be expanded.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3 hours"}, "extra_text": {}}, {"review_id": "oFsfpy0znd", "reviewer": "Reviewer_bTUB", "summary": "In this paper, the authors present an unsupervised topology difference detector based on geometric alignment. The method is based on a conditional variational autoencoder. It allows pixel-wise outlier detection. The method is evaluated on the task of brain tumor detection and compared to other unsupervised segmentation/registration methods and a supervised segmentation baseline. The results show that the presented method performs best compared to other unsupervised segmentation/registration methods.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well written and very detailed. Anomaly detection using VAE is a well-known method, but mostly in the context of image reconstruction. Using a VAE for outlier detection in the context of image registration is very interesting. The mathematical definition and the experiment description are very detailed and good to understand.\n\nComments:\n\n1) There is no information about the registration accuracy. How does it influence the quality of outlier detection?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, the authors have addressed the limitations of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1.5"}, "extra_text": {}}, {"review_id": "V3UPxFgX4H5", "reviewer": "Reviewer_qinK", "summary": "This paper proposes a novel unsupervised topological difference detection algorithm. The model is based on a conditional variational auto-encoder and detects topological anomalies with regards to a reference alongside the registration\nstep. In terms of the experiments, as there exists no standard dataset of annotated topological differences for image registration, this paper validates the proposed topological difference detection algorithm on a proxy task of unsupervised anomaly detection in images.", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n\n1. This paper explores a interesting problem: the alignment of domains whose topology is not the same. It is worth exploring as in many real applications, especially in biomedical domain, the anatomical/topological differences are very common and will potentially introduce bias in downstream analysis, which are routinely ignored.\n\n2. The proposed model is able to detect topological inconsistencies with a purely registration-driven framework. \n\n3. By decomposing the joint likelihood into a likelihood for each pixel of the image, the proposed method could not only detect images that contain topological differences under the assumption that these images will overall have a lower likelihood, but also the position of outliers in the image.\n\nCons:\n\n1. My concerns mainly remain in the experiment part. It's true that there exists no standard dataset of annotated topological differences for image registration. Instead the paper uses a proxy task of unsupervised anomaly detection in images to validate the effectiveness of the proposed method. But there is a gap between 'topological difference' and 'anamaly', which is not persuasive enough to demonstrate the efficacy of detecting topological differences. Also, the most strong baseline the paper chose is a paper published in 2010 (Li and Wyatt [26]), and more existing state-of-the-art anomaly detection algorithms should be compared to demonstrate the effectiveness of the proposed method. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper addresses the problem of *topology-aware registration* and presents a first approach towards this goal. 

Reviews are quite mixed (even after the rebuttal phase) and the majority of reviewers perceived this work more like an *anomaly detection* approach rather than a means to an end, i.e., topology-aware registration. This seems to be primarily due to the proxy experiment of anomaly detection. Nevertheless, due to the non-existence of a suitable benchmark dataset to assess the ultimate objective of this work, this is a totally reasonable way to go in my point of view.

As to missing comparisons to recent state-of-the-art anomaly detection approaches, I do think that the authors make a fair point in their rebuttal: yes, one could compare to various anomaly detection methods, but this would, in essence, be a comparison against methods that cannot necessarily detect topological differences (which is the very task the presented method specifically addresses).

Overall, after reading the paper and carefully considering the reviews and the responses, I do not think that the changes required to address the reviewer's concerns are so major that they warrant rejection (e.g., the definition of topology, or clarification of voxel-wise losses). The required changes/adaptations are, in fact, quite minor and the concise comments by the authors already largely clarify the issues. Overall, I am recommending acceptance, based on the remarks outlined above. 
Decision: Accept (Poster)