Spot the Difference: Detection of Topological Changes via Geometric Alignment
Year: 2021

Abstract:
Geometric alignment appears in a variety of applications, ranging from domain adaptation, optimal transport, and normalizing flows in machine learning; optical flow and learned augmentation in computer vision and deformable registration within biomedical imaging. A recurring challenge is the alignment of domains whose topology is not the same; a problem that is routinely ignored, potentially introducing bias in downstream analysis. As a first step towards solving such alignment problems, we propose an unsupervised algorithm for the detection of changes in image topology. The model is based on a conditional variational autoencoder and detects topological changes between two images during the registration step. We account for both topological changes in the image under spatial variation and unexpected transformations. Our approach is validated on two tasks and datasets: detection of topological changes in microscopy images of cells, and unsupervised anomaly detection brain imaging.

heading: 1 Introduction
text: Geometric alignment is a fundamental component of widely different algorithms, ranging from domain adaptation [7], optimal transport [40] and normalizing flows [35, 42] in machine learning; optical flow [21, 51] and learned augmentation [20] in computer vision, and deformable registration within biomedical imaging [5, 15, 19, 39, 53]. A recurring challenge is the alignment of domains whose topology is not the same. When the objects to be aligned are probability distributions [35], this appears when distributions have different numbers of modes whose support is separated into separate connected components. When the objects to be aligned are scenes or natural images, the problem occurs with occlusion or temporal changes [51]. In biomedical image registration, the problem is very common and happens when the studied anatomy differs from "standard" anatomy [36]. Despite being extremely common, this problem is routinely ignored or accepted as inevitable, potentially introducing bias in downstream analysis.
We study two cases from biomedical image registration. One is the alignment of image slices to reconstruct a 3d volume, where changes in topology between slices introduce challenges in postprocessing (Figure 1). The other is the registration of brain MRI scans, where tumors give common examples of anatomies that are topologically different from healthy brains. In deformable image registration, a "moving image" is mapped via a nonlinear transformation to make it as similar as possible to a "target" image, enabling matching local features or transferring information from one
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
image to another. It is common to numerically stabilize the estimation of the transformation by constraining the predicted transformation to be diffeomorphic, that is, bijective and continuously differentiable in both directions. In particular, diffeomorphic transformations are homeomorphic, or topology-preserving, which implies that a common topology is assumed across all images [13, 15]. This topology is often provided by a common template image Itemplate, from which all other images are obtained via the transformation Φ from the group of diffeomorphisms G. Under this common topology assumption, the set of all images is given by
I = {Itemplate ◦ Φ|Φ ∈ G} .
Topological differences in biomedical images can be caused by a variety of processes. For instance, image slices obtained from a volume do not all contain the same elements. Tumor growth or the removal of surgical tissue can alter the topology of an image. Various processes can lead to the replacement or deformation of organic tissue, which cannot be mapped to the original image. We choose to model these topological differences as the inability to obtain one image from the other via a homeomorphic transformation of the image domain. Since, within image registration, transformations are assumed to be continuously differentiable, we are effectively modelling topological differences between pairs of images via the failures of diffeomorphic image registration in aligning them.
As most registration algorithms align images based on intensity, e.g. minimizing mean squared error (MSE), these tissue changes make it difficult to map images correctly. The strong local deformations required to deal with the non-diffeomorphic part of the image inevitably also deform the surrounding area, leading to distorted transformation fields in topologically matching parts of the image [36]. These transformation fields adversely affect downstream tasks, for example indicating false size changes in adjacent regions.
Previous work on aligning topologically inconsistent domains. Attempting to relax the sameimage assumption induced by fully diffeomorphic transformations is not new. In the context of organs sliding against each other, several approaches exist, most of which rely on pre-annotating the sliding boundary using organ segmentation [6, 10, 22, 37, 43, 46], with a few extensions to un-annotated images [38, 45].
When topological holes are created or removed in the domain, for example through tumors, pathologies, or surgical resections, the loss function used for registration can be locally weighted or masked [26, 29, 30], or an artificial insection can be grown to correct anatomies [36]. These approaches rely on annotation of the topological differences, which have to be provided manually or by segmentation. An exception is given by Li and Wyatt [30], which detects changes in topology from the difference between the aligned images. This depends crucially on the ability to find a good diffeomorphic registration outside the anomaly, which is difficult all the while the applied transformation is still diffeomorphic.
An alternative approach to registering topologically inconsistent images is to inpaint the difference in the source images to obtain a topologically consistent quasi-normal image. Then standard registration methods can be used on the altered images. Quasi-normal images can be obtained through lowrank and sparse matrix decomposition [32, 33], principle component analysis [16, 18], denoising VAEs [52], or learning of a blended representation [17]. Registration with the quasi-normal approach retains the diffeomorphic properties of the transformation but does not register the topologically inconsistent areas of the images.
Our contribution. We propose an unsupervised algorithm for the detection of changes in image topology. To this end, we train a conditional variational autoencoder for predicting image-to-image alignment, obtaining a per-target-pixel probability of being obtained from the moving image via diffeomorphic transformation. We combine a semantic loss function trained to extract contextual information [8], with a learnable prior of transformations [9], allowing us to incorporate both the reconstruction error, as well as knowledge about the expected transformation strength.
We test the validity of our approach on a novel dataset of cell slices with annotated topological changes and on the proxy task of unsupervised brain-tumor detection. We also validate our approach by investigating a spatial "topological inconsistency likelihood", and showing that this likelihood is higher in regions where topological inconsistencies are known to be common. Our model is able to detect topological inconsistencies with a purely registration-driven framework, and thus provides the first step towards an end-to-end registration model for images with topological discrepancies. The implementation is available at github.com/SteffenCzolbe/TopologicalChangeDetection.

heading: 2 Background

heading: 2.1 Notation of images and transformations
text: We view an image I interchangeably as two different structures. First, it is a continuous function I : ΩI → R C , where ΩI = [0, 1] D is the domain of the image, and C the number of channels. This function can be approximated by a grid of n pixels with positions xk ∈ ΩI leading to the image representation I (c) k , where c is an index over the channels and Ik = (I (1) k , . . . , I (C) k )
T = I(xk). Second, this pixel grid is accompanied by a graph structure that encodes the neighbourhood of each pixel. In this view, the set of neighbours of a pixel with index k (for example the 4-neighbourhood of a pixel on the image grid) is referred to as N(k) and |N(k)| is the number of neighbours. The neighborhoods of a pixel gives rise to a graph which can be described via the graph laplacian Λ ∈ Rn×n with Λk,k = |N(k)| and Λk,k′ = −1 when pixel k ′ ∈ N(k), and zero otherwise.
Applying a spatial transformation Φ : RD → RD to an image is written as J = I ◦ Φ, which can be seen as its own image with domain ΩJ = [0, 1]
D with pixel coordinates yk ∈ ΩJ and Jk = I(Φ(yk)). The transformation Φ can be seen as a vector field on the image domain which assigns each pixel in J a position on I and thus it can be parameterized as a pixel grid Φ (d) k , d = 1, . . . , D at the pixel coordinates of J using Φ(yk) = yk +Φk. To make this choice of coordinate system clear, we will refer to a transformation that moves a pixel position from the domain ΩJ to the corresponding pixel in domain ΩI as ΦJ→I, whenever it is not clear from the context. If Φ is a diffeomorphism, it can alternatively be parameterized by a vector field V on the tangent space around the identity, where the mapping between the tangent space and the transformation is given by Φ = exp(V ), which amounts to integration over the vector field [2].

heading: 2.2 Variational registration framework
text: It is possible to phrase the problem of fitting a registration model in terms of variational inference, using an approach similar to conditional variational autoencoders [47]. Here, we summarize the approach taken by [9, 31]. For a D-dimensional image pair (I,J), we assume that J is generated from I by drawing a transformation Φ from a prior distribution p(Φ|I), apply it to I and then add pixel-wise noise:
p(J|I) =
∫
pnoise(J|I ◦ Φ)p(Φ|I) dΦ
This includes the common topology assumption implicitly via p(Φ|I), which is typically chosen to produce invertible transformations depending only on the topology of I, as well as the noise
model which does not assume systematic changes between J and I. This model can be learned using variational inference using a proposal distribution q(Φ|I,J) with evidence lower bound (ELBO)
log p(J|I) ≥ Eq(Φ|I,J) [log pnoise(J|I ◦ Φ)]−KL(q(Φ|I,J)‖p(Φ|I)) . (1)
In contrast to variational autoencoders, the decoder is given by the known application of Φ to I. Thus, the degrees of freedom in this model are in the choice of the encoder, prior, and the noise distribution. Dalca et al. [9] proposed to parameterize Φ as a vector field V (d) k on the tangent space, which turns application of Φ = exp(V ) into sampling an image with a spatial transformer module [24]. As a prior for this parameterization, they chose a prior independent of I
p(Φ) = D ∏
d=1
N ( V (d) | 0,Λ−1 ) ,
where we used the implicit identification of Φ and V and the precision matrix Λ is chosen as the Graph Laplacian over the neighbourhood graph (see notation). Using an encoder that for each pixel proposes q(V (d) k |I,J) = N (µ (d) k , v (d) k ), the KL divergence is derived as
KL ( q(Φ|I,J)‖p(Φ|I) ) = 1
2
D ∑
d=1
n ∑
k=1
− log v (d) k +|N(k)|v (d) k +
∑
l∈N(k)
(
µ (d) k − µ (d) l
)2
+const . (2)
It is worth noting that this equation is invariant under translations of µ. This invariance manifests in rank-deficiency of Λ and as a result, const is infinite. Thus, sampling from the prior and bounding the objective is impossible. Still training with this term works in practice as images are usually pre-aligned with an affine transformation and thus translations are close to zero. We will present a slightly modified approach, rectifying the missing eigenvalue.

heading: 3 Detection of topological differences
text: The variational approach for learning the distribution of transformations introduced before optimizes an ELBO on log p(J|I). This information is enough to detect images that contain topological differences under the assumption that these images will overall have a lower likelihood. However, in our application, we need not only to detect the existence but also the position of outliers in the image. For this, we have to ensure that log p(J|I) can be decomposed into a likelihood for each pixel of the image. It is immediately obvious by inspection of the ELBO (1) together with the KL-Divergence (2), that the lower bound on log p(J|I) can be decomposed into pixel-wise terms if log pnoise(J|I ◦Φ) can be decomposed as such. To enforce this, we will introduce a general form of error function, which can be decomposed and includes the MSE as a special case. For this, we first map the images I and J to feature maps over the pixel positions k via a mapping fk(I) ∈ R F and define the loss as:
pnoise(J|I ◦ Φ) = n ∏
k=1
N (fk(J)|fk(I) ◦ Φ,Σf ) , (3)
where Σf ∈ R F×F is a diagonal covariance matrix with variances learned during training.
The ability to decompose the likelihood is not enough for a meaningful metric, as we have to ensure that each term is calculated in the correct coordinate system. This depends on the parameterisation and regularisation of Φ. In the approach by Dalca et al. [9] the parameterization V of Φ is defined on the tangent space and consequently the prior is also on this space. Since the connection between Φ and V is given by integration of the vector field, decomposing (2) for a single pixel k will produce estimates based on the local differential of the transformation, but will not take the full path with starting and endpoints into account. Thus, correct cost assignments require integration of (2) over the computed path, which is expensive and suffers from severe integration inaccuracies. Instead, we will use an alternative approach, where we parameterize Φ directly as a vector field on the image domain. Transformations parameterized this way are not necessarily invertible anymore, yet smoothness is still encouraged by the prior.
Learnable prior Using this parameterization, we extend the approach by Dalca et al. [9] and introduce a parameterized prior on Φk that is learned simultaneously with the model:
p(Φ) =
D ∏
d=1
N (
Φ(d) | 0,Λ−1αβ
)
, Λαβ = αΛ + β
n2 ✶✶
T (4)
The expected variations and translations between transformation vectors are governed by α and β. Unlike most works in image registration, we do not treat these as tuneable hyperparameters, but instead view them as unknowns to be fitted to the data during training similar to [28, 49]. For efficient learning, we use an estimate for the optimal values for α, β over a batch of samples during training, and use a running average at test time. A detailed explanation is given in supplementary material A.
The second term of (4) ensures that Λαβ is invertible, by adding a multiple of the eigenvector ✶ = (1, . . . , 1)T . It can be verified easily that Λ✶ = 0. Unlike adding a multiple of the identity matrix to Λ, adding the missing eigenvalue does not modify the prior in any other way than regularizing the translations. Further, it ensures that the KL divergence of the resulting matrix can be quickly computed up to a constant as α and β do not modify the same eigenvalues. Recomputing the KL-divergence for n transformation vectors in D dimensions leads to
2KL (q(Φ|I,J)‖pαβ(Φ)) = −(n− 1)D logα−D log β + β D ∑
d=1
(
1
n
n ∑
i=1
µ (d) i
)2
+
D ∑
d=1
n ∑
k=1
− log v (d) k +
(
α|N(k)|+ β
n2
)
v (d) k + α
∑
l∈N(k)
(
µ (d) k − µ (d) l
)2
+ const (5)
Decomposed error metric We define our pixel-wise error measure for topological change detection based on the ELBO (1) with KL-divergence (5) as follows, where we compute µ (d) k and v (d) k via the proposal distribution q(Φ|I,J) and pick Φ (d) k = µ (d) k :
Lk(J|I) = − logN (fk(J)|fk(I) ◦ Φ,Σf ) + βµ
(d) k
n2
D ∑
d=1
n ∑
i=1
µ (d) i
+
D ∑
d=1
− log v (d) k +
(
α|N(k)|+ β
n2
)
v (d) k + α
∑
l∈N(k)
(
µ (d) k − µ (d) l
)2
. (6)
We will treat the loss over all pixels L(J|I) = (L1(J|I), . . . , Ln(J|I)) as another image with domain and pixel coordinates the same as J. This measure is not symmetric. The prior distribution does not treat the distributions q(Φ|I,J) and q(Φ|J, I) equally. If ΦJ→I maps a line in J to an area in I, this will incur a large visible feature along the line due to violating the smoothness assumption encoded in the prior. On the other hand, if an area in J gets mapped to a line in I, the overall error contribution is smoothed out over the area. To rectify this issue, we will compute a bidirectional measure Lsym(J|I) = L(J|I) +L(I|J) ◦ΦI→J, where ΦI→J is the same as the one used to compute L(J|I). For this measure it holds that if ΦJ→I = Φ −1 I→J, we have Lsym(I|J) = Lsym(J|I) ◦ ΦJ→I up to interpolation errors caused by the finite coordinate grid.
Topological outlier detection Lsym detects topological changes between two images. However, for evaluation on the Brain dataset, we are interested in topological outliers. Outliers can be detected using Lsym by contrasting the observed deviations with the observed deviations within a larger set of control images C. This leads to the score
Q(J) = EI∈C [Lsym(J|I)− EK∈C [Lsym(I|K)] ◦ ΦI→J] . (7)

heading: 4 Evaluation
text: We evaluate our approach on two tasks. In the first, we measure prediction agreement with annotated topological changes on a dataset of cell slices. For this, we introduce the first dataset with annotated topological differences for image registration (see Section 4.1), which allows us to significantly expand on the evaluation strategies of prior work [26, 29, 30]. In the second task, we adapt our approach to anomaly detection in order to detect brain tumors on slices of MRI images.
On the change detection task, we use our model prediction of Lsym directly. On the anomaly detection task, we use the score (7), which subtracts the average scores over healthy patients for each pixel.
We compare our model to the following baselines:
1. Two unsupervised approaches for topological change detection:
• Li and Wyatt’s [30] intensity difference and image gradient-based approach using a deterministic registration model [5] to obtain the transformations.
• Using the same model, we devise a method based on the Jacobian Determinant of the transformation field |JΦ|. We expect strong stretching or shrinkage in areas of topological mismatch, which we measure using use the score log(|detJΦ|) 2.
We adapt both approaches to the task of tumor detection by subtracting the average scores over healthy patients, analogous to (7).
2. The approach by An and Cho [1] for unsupervised anomaly detection in images is based on the local reconstruction error of a variational autoencoder. The error score is ‖J−dec(enc(J))‖2, where enc(J) maps J to the mean of the variational proposal distribution and dec is the corresponding learned decoder. As the score does not use registration, we cannot use equation (7).
3. A supervised segmentation model trained for segmenting topological changes based on two input images on the cell dataset, and tumor segmentation based on a single input image on the brain dataset. Since this model requires annotated data, we withhold 75% of the annotated volumes for training and evaluate the segmentation model only on the remaining samples.
In both tasks, we measure the pixel-wise agreement of the models with the annotated ground-truth using the receiver operating characteristic curve (ROC curve) and compare the area under the curve (AUC) between the models. AUC estimates are bootstrapped on the subject level to obtain error estimates.
As additional evaluations, we present qualitative examples and investigate whether brain regions with known topological variability get assigned higher scores in our model. For this we compute the pairwise average score Lsym over multiple healthy subjects and register them all to a brain atlas using EI,K [Lsym(I|K) ◦ ΦI→Atlas]. We group the scores by their position on the brain atlas into partitions: cortical surfaces, subcortical regions, and ventricles.

heading: 4.1 Tasks and Data
text: Topology change detection in Cells Serial block-face scanning electron microscopy (SBEM) is a method to obtain three-dimensional images from small biological samples. An image is taken from the face of the block, after which a thin section is cut from the sample to expose the next slice. A challenge is the accurate reconstruction of the volume, as neighboring slices differ by both natural deformations and changes in topology. Natural deformations can be introduced by shape-changes of objects between the slices, and deformations of the sample due to the physical cutting. Changes in topology occur due to objects present in one slice but not the other, and tears of the physical sample induced by the cutting.
We evaluate our method on the detection of topological changes between neighboring slices of human platelet cells recorded with SBEM. We use the pre-segmented dataset by Quay et al. [41] as a base. In the dataset, image slices are affinely pre-aligned and manually segmented into 7 classes. Afterwards, for the validation and test set, we annotated changes in the topology of the segmentation masks. Using this approach, not all instances of topological changes in the image can be annotated as the segmentation maps merge several types of cell components into a single class. The data is cropped into patches of 256× 256 pixels and we use 9 patches of 50 slices for training, 4 patches of 24 slices for validation, and 5 patches of 24 slices for test (3 patches for the supervised approach due to the training-test split of annotated data).
Brain tumor detection Individual brains offer a range of topological differences, especially in the presence of tumors. Further, inter-subject differences are found at the cortical surface, where the sulci vary significantly [48], and near ventricles, which can either be open cavities, or partially closed [36]. We quantitatively evaluate our method on the proxy task of detecting brain tumors. Tumors change the morphology of the brain and can thus be detected indirectly via the large transformations they cause. For this, we first train our model using a dataset of healthy images from the control group and then use (7) to obtain a score for topological outlier detection. For the control set, we combine T1 weighted MRI scans of the healthy subjects from the ABIDE I [11]1, ABIDE II [12] and OASIS3 [27] studies.
1CC BY-NC-SA 3.0, https://creativecommons.org/licenses/by-nc-sa/3.0/
For the tumor set we use MRI scans from the BraTS2020 brain tumor segmentation challenge [3, 4, 34], which have expert-annotated tumors. We use the T1 weighted MRI scans, and combine labels of the classes necrotic/cystic and enhancing tumor core into a single tumor class. All datasets are anonymized, with no protected health information included and participants gave informed consent to data collection.
We perform standard pre-processing on both brain datasets, including intensity normalization, affine spatial alignment, and skull-stripping using FreeSurfer [14]. From each 3D volume, we extract a center slice of 160× 224 pixels. Scans with preprocessing errors are discarded, and the remaining images of the control dataset are split 2381/149/162 for train/validation/test. Of the tumor dataset, 84 annotated images with tumors larger than 5cm2 along the slice are used for evaluation (17 for the supervised approach due to the training-test split of subjects).

heading: 4.2 Model and training
text: All models evaluated are based on a U-Net [44] architecture, except An and Cho [1], which we implement using as a spatial VAE following the previously published adaptation to Brain-scans by Venkatakrishnan et al. [50]. The networks consist of encoder and decoder stages of 64, 128, 256 channels for all registration models, and 32, 64, 128, 256 channels for the segmentation and VAE models. Each stage consists of a batch normalization [23] and a convolutional layer.
In our approach, we use a U-Net to model p(Φ|I,J). The output of the last decoder stage is fed through separate convolution layers with linear activation functions to predict the transformation mean and log-scaled variance. Throughout the network, we use LeakyReLu activation functions. The generator step I ◦Φ is implemented by a parameterless spatial transformer layer [24]. During training of our model, we use the analytical solution for prior parameters α, β (supplementary material, Eq. 8), averaged over the mini-batch of 32 image pairs. For validation and test, we use the running mean recorded during training. The diagonal covariance of the reconstruction loss Σf is treated as a trainable parameter.
For all datasets, we use data augmentation with random affine transformations of the training images. For training, the optimization algorithm is ADAM [25] with a learning rate of 10−4. Regularization of all models is performed by applying an L2-penalty to the weights with a factor of 0.01 for the cell dataset and 0.0005 for the brains. We train each model on a single TitanRTX GPU, with maximum training times of 1 day for the cells and 4 days for the brains. Hyperparameters: The network by Venkatakrishnan et al. [50] has σ = 1 chosen from {0.1, 1, 10}, based on reconstruction loss on validation set. The deterministic registration model was trained using λ = 0.1 as in [8]. For [30], the parameters σ of the Gaussian derivative kernel and hyper-parameter K where chosen to maximize the AUC score, selecting σ = 6, K = 2 out of {1, . . . , 9}2.
For the reconstruction loss, we compare two different loss functions. The first is using the MSE as in [9, 30]. The second is a semantic similarity metric similar to [8]. To obtain the semantic image descriptors, we train a U-net with 32, 64, 64 channels for image segmentation, using the manual annotations of the cell set and automatically created labels obtained with FreeSurfer [14] for the brain control images. Notably, the segmentation models used for the loss have not been trained on images or pairs containing topological changes or tumors. From this network, we extract the features of the first three stages and use them as a 160-channel feature map in the loss (3). For both the MSE and the semantic loss, we learn the variance parameters while training the variational autoencoder.

heading: 4.3 Results
text: The ROC curves of all trained models on the cell and brain tasks can be seen in Figure 2. For both tasks, the supervised model performed best (AUC 0.90, 0.95), while our proposed approach with semantic loss performed best among the unsupervised models (AUC 0.88, 0.80). The unsupervised approach for topological change detection by Li and Wyatt [30] (AUC 0.75, 0.70) performed overall best among the baselines, but worse than our method. The unsupervised anomaly detection method by An and Cho [1] (AUC 0.72, 0.67) performed well at detecting brain tumors, but worse at detecting topological changes in the cell images. Using the Jacobian determinant (AUC 0.75, 0.62) performed well on the cell images but worse on the brain tumor detection task. Our approach using MSE (AUC 0.72, 0.61) performed worse than the other methods on both tasks.
When analyzing the ROC curves, our model performed best among the unsupervised models for all false positive rates, while the supervised model is the best overall. Finally, even though both models share the same trained model, the score used by Li and Wyatt [30] performed better than scoring using the Jacobian determinant on the brain tumor detection task, while on the cell dataset, both approaches performed the same.
We show qualitative results on the cell dataset in Figure 3. In row 3, we see that Lsym detected annotated areas of topological change (contoured in red), but is more certain at detecting changes in areas with high intensity difference. In many cases, the model assigns a likelihood of topological changes to areas that have not been annotated in the dataset, such as the merging cell boundary in column 2 or many small changes in the cell interior in column 4.
SEM MSE SEM MSEI J Lsym(J|I) Q(J)
Qualitative results on the brain data are presented in Figure 4. When looking at columns 3 and 4, we see that Lsym detected notable areas with high changes in topology compared to the reference image I. This includes the ventricles (rows 2,3), the cortical areas with the sulci (all rows) as well as tumor areas (rows 1,3,5). There was a clear difference in the behaviour between semantic loss and MSE as the semantic loss highlights broader regions of the surface. When comparing the outlier-detection measure Q(J) in columns 5 and 6, we can see that our approach filtered most of the ventricles and sulci leaving an area around most tumor regions. Notable exceptions are rows 2 and 4, where the tumor area was not highlighted, as well as row 1 where only part of the tumor was detected.
In Figure 5, we show the average topological change score on healthy subjects. We see on the brain image and the box plot, that the cortical surfaces and ventricles get assigned higher scores than the subcortical structures.

heading: 5 Discussion and conclusion
text: In this work, we have introduced a novel approach for the detection of topological changes. We evaluated our approach qualitatively and compared it quantitatively to previous approaches using both a novel dataset with purpose-made annotations and on an unsupervised segmentation proxy task. On both tasks, our approach performed best among the unsupervised methods, but could not reach the performance of the supervised method.
An unsupervised method is useful in practice, as annotations of topological changes are rarely available. While our results are not pixel exact, they indicate where a registration algorithm must be used more carefully to obtain a valid registration. The results on the cell dataset align well with the annotations, and many of the false positives appear to be caused by incomplete annotation of the data. This is also reflected in the reported ROC-curves, which show that our model outperforms the supervised segmentation model at false positive rates larger than 0.5. The results obtained on the tumor segmentation proxy task are reinforced by the distribution of scores obtained on healthy patients in different parts of the brain. The high likelihood of topological differences in ventricles found agrees with previous work [36] and the higher scores in cortical surfaces reflect the fact, that the sulci of the cortical surface exhibit high variability between subjects [7], which was previously difficult to quantify.
Our results also show that using a semantic loss function is advantageous compared to the MSE in this task, as all MSE based methods performed worse than our approach using the semantic loss. This is likely because the contrast between some anatomical areas is quite small and thus missed by the MSE. In contrast, the semantic loss incorporates more texture information and thus is capable of differentiating between areas of similar intensity but different semantics. However, particularly on the brain example, even the semantic approach misses tumors close to the cortex. We hypothesize, that this is in part caused by the similar appearance of tumors and grey matter, in part by the semantic model not being trained on tumors, and in part due to the cortical area containing high topological variation among the control group as well.
On the brain dataset, our unsupervised results for the method by An and Cho [1] are in line with previously reported results on a comparable dataset [50]. However, our supervised results are not comparable to the results published for the BRATS challenge, as we selected a subset of data for training and only used structural MRI images, discarding the other modalities. On the cell dataset, no other work on topology change or outlier detection is available.
Our study has several limitations. We only investigate registrations in 2D and topological differences might vanish if the whole 3D volume is considered. The transformations obtained by our unsupervised method differ from strongly regularised methods, as the hyperparameter-less learned prior underregularises in order to maximize the likelihood of a topological match during training. Conversely, the poor performance of the Jacobian determinant might be due to a strong regularisation for good performance in image registration as we used the hyperparameters as found in [8].
In conclusion, our approach serves as the first step for unsupervised annotation of topological changes in image registration. Our approach is fully unsupervised and hyperparameter-free, making it a prospective building block in an end-to-end topology-aware image registration model.

heading: Acknowledgements
text: This work was funded by the Novo Nordisk Foundation (grants no. NNF20OC0062606 and NNF17OC0028360) and the Lundbeck Foundation (grant no. R218-2016-883).
The human platelet SBEM data and segmentations were provided by Matthew Quay, the topological change annotations are ours. The Brain tumor data was provided by the BraTS challenge. The Brain control data was provided in part by OASIS Principal Investigators: T. Benzinger, D. Marcus, J. Morris; NIH P50 AG00561, P30 NS09857781, P01 AG026276, P01 AG003991, R01 AG043434, UL1 TR000448, R01 EB009352. AV-45 doses were provided by Avid Radiopharmaceuticals, a wholly-owned subsidiary of Eli Lilly.

Review summaries:
This paper proposes a novel unsupervised topological difference detection algorithm. The model is based on a conditional variational auto-encoder and detects topological anomalies with regards to a reference alongside the registration step. In terms of the experiments, as there exists no standard dataset of annotated topological differences for image registration, this paper validates the proposed topological difference detection algorithm on a proxy task of unsupervised anomaly detection in images.

In this paper, the authors present an unsupervised topology difference detector based on geometric alignment. The method is based on a conditional variational autoencoder. It allows pixel-wise outlier detection. The method is evaluated on the task of brain tumor detection and compared to other unsupervised segmentation/registration methods and a supervised segmentation baseline. The results show that the presented method performs best compared to other unsupervised segmentation/registration methods.

The paper presents an anomaly detection that specifically targeting topological differences. This is achieved by training an image registration neural network that warps control images to the target and highlighting differences. A prior term is modified to improve the alignment. The method is validated on BraTS, ABIDE, and OASIS.