<figure><img src="image_1.png" /></figure>

HAL
open science

# A Survey of Topology and Geometry-Constrained Segmentation Methods in Weakly Supervised Settings

Ke Chen, Noémie Debroux, Carole Le Guyader

► To cite this version:

Ke Chen, Noémie Debroux, Carole Le Guyader. A Survey of Topology and Geometry-Constrained Segmentation Methods in Weakly Supervised Settings. Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging, Springer International Publishing, pp.1-46, 2021, 10.1007/978-3-030-03009-4_85-1 . hal-04873352

HAL Id: hal-04873352

https://uca.hal.science/hal-04873352v1

Submitted on 22 Jan 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A Survey of Topology and Geometry-Constrained Segmentation Methods in Weakly Supervised Settings

Ke Chen, Noémie Debroux and Carole Le Guyader

**Ke Chen**

Computational Mathematics Group, Department of Mathematical Sciences, University of Liverpool Mathematical Sciences Building, Liverpool L69 7ZL, United Kingdom, e-mail: K.Chen@liverpool.ac.uk

**Noémie Debroux**

Pascal Institute, Image Guided Therapy team, axis Cardio-Vascular Interventional Therapy and Imaging, University of Clermont Auvergne, France, e-mail: noemie.debroux@uca.fr

**Carole Le Guyader**

Normandie Univ, Institut National des Sciences Appliquées de Rouen, Laboratory of Mathematics, 76000 Rouen, France, e-mail: carole.le-guyader@insa-rouen.fr
<figure><img src="image_3.png" /></figure>
# Contents

<table><tr><td colspan="2">A Survey of Topology and Geometry-Constrained Segmentation Methods in Weakly Supervised Settings</td><td>1</td></tr><tr><td colspan="2">Ke Chen, Noémie Debroux and Carole Le Guyader</td><td></td></tr><tr><td>1</td><td>Introduction</td><td>4</td></tr><tr><td>2</td><td>Geometrical Constraints</td><td>7</td></tr><tr><td>2.1</td><td>Characterisation of geometrical constraints</td><td>8</td></tr><tr><td>2.2</td><td>Model 1 – A simple variational model</td><td>9</td></tr><tr><td>2.3</td><td>Model 2 – A moving band model</td><td>9</td></tr><tr><td>2.4</td><td>Model 3 – A dual level model</td><td>9</td></tr><tr><td>2.5</td><td>Model 4 – The use of moments constraint for segmentation</td><td>10</td></tr><tr><td>2.6</td><td>Model 5 – Convex segmentation models</td><td>11</td></tr><tr><td>2.7</td><td>Model 6 – Convex models based on geodesic distances</td><td>12</td></tr><tr><td>2.8</td><td>Other possible models</td><td>13</td></tr><tr><td>3</td><td>Topological Prior Knowledge</td><td>14</td></tr><tr><td>3.1</td><td>Topology prescription</td><td>16</td></tr><tr><td>3.2</td><td>Regularisation enforcement on the evolving front</td><td>24</td></tr><tr><td>4</td><td>Joint Segmentation and Registration Models</td><td>28</td></tr><tr><td>4.1</td><td>Motivations</td><td>28</td></tr><tr><td>4.2</td><td>Overview of existing methods</td><td>30</td></tr><tr><td>4.3</td><td>A mixed segmentation/registration model based on a nonlocal characterisation of weighted total variation</td><td>32</td></tr><tr><td>5</td><td>Other Related Models</td><td>38</td></tr><tr><td>5.1</td><td>Optimal flow frameworks</td><td>38</td></tr><tr><td>5.2</td><td>Shape priors</td><td>38</td></tr><tr><td>5.3</td><td>Deep learning models</td><td>38</td></tr><tr><td>5.4</td><td>Multi-modal problems</td><td>38</td></tr><tr><td>6</td><td>Conclusion</td><td>39</td></tr><tr><td colspan="2">References</td><td>39</td></tr></table>

**Abstract** Incorporating prior knowledge into a segmentation process —whether it be geometrical constraints such as landmarks to overcome the issue of weak boundary definition, shape prior knowledge or volume/area penalization, or topological
prescriptions in order for the segmented shape to be homeomorphic to the initial one or to preserve the contextual relations between objects —, proves to achieve more accurate results, while limiting human intervention. In this contribution, we intend to give an exhaustive overview of these so called weakly/semi-supervised segmentation methods, following three main angles of inquiry: inclusion of geometrical constraints (landmarks, shape prior knowledge, volume/area penalization, etc.), incorporation of topological constraints (topology preservation enforcement, prescription of the number of connected components/holes, regularity enforcement on the evolving front, etc.), and lastly, joint treatment of segmentation and registration that can be viewed as a special case of cosegmentation.

# 1 Introduction

Image segmentation is an essential step in image processing on the way to make image analysis automatic, aiming to reproduce the ability of human beings to track down significant patterns and automatically gather them into relevant and identified structures with respect to features such as color, shape or orientation ([160]). More specifically, image segmentation consists in identifying meaningful constituents of a given image (e.g., homogeneous regions, shapes, edges, textures, etc.) for quantitative analysis or visualization purposes. Due to its countless applications among which, object detection, complexity reduction, scene parsing, image montage, colorization, organ reconstruction, tumor detection, computer-aided diagnosis, therapy planning to name a few (see [160] for an exhaustive overview), a lot of research has been carried out during the last three decades.

Not all image shapes and patterns extracted provide useful information. The usefulness is relative to applications. For instance, an automatic car mainly cares about stationary and moving objects on its path, not objects (such as buildings or trees) that are far away, while in medical imaging, a specialist on liver diseases is not primarily interested to see patterns in lungs and abdomen. Therefore in this practical sense, only models that have topology and geometry constraints built in to extract patterns of interest are really valuable, while other generic segmentation models capable of identifying all objects are not helpful.

Although simple to state, this task is nevertheless challenging and ill-posed as emphasized by Zhu et al. in the comprehensive segmentation survey [160]:

(i) First, owing to the polysemy of the word *object* and because interpretation is intrinsically subjective: different human beings may have different views of what an object is. The definition of an *object* encompasses several acceptations according to human perception: it can be something material —a thing—, a periodic pattern, an overall structure (e.g., a forest, the sea), or even a sub-part of a given object (e.g., a tumor in a brain MRI image);

(ii) Second, due to the difficulty in computerizing/reproducing the human vision system, capable of synthesizing (interpolating) the observed data into a continuous whole (—human tends to merge elements taking on shared similarities, to com-
plete missing data, to favor continuous contours, etc. —), whereas most images
in computers are represented by low-level characteristics reflecting mainly lo-
cal properties and failing thus to capture the global (continuous) nature of the
observed object.

These two elements together make the evaluation of segmentation techniques still
an open question.

An exhaustive classification of segmentation methods into three main categories
is provided in [160] and ordered according to the level of supervision or user in-
volvement, combined with a description/analysis of each methodology as follows:

(i) Global models: unsupervised methods which consist in partitioning a given image into meaningful constituents based only on low-level features (e.g., intensity levels, curvature, etc.) with no human intervention, and without any training data and a priori knowledge of the object model. These unsupervised methods are themselves subdivided into two groups : discrete methods —setting in which the image is considered as a fixed discrete grid —, including clustering-based approaches ([39, 43, 107, 117]) and graph-based methods ([57, 128]), and continuous methods ([18, 28, 30, 33, 79, 89, 101, 102, 109, 132, 8, 143, 142, 145]) —framework in which the image is seen as a continuous surface, avoiding thus the grid bias artifacts inherent to discrete methods and producing visually more pleasing results —, methods involving edge-based models and region-based ones;

(ii) Local models: semi/weakly supervised methods which incorporate a small amount of high-level information and as such, are usually interactive and require human expertise and intervention to better match human perception. This class of methods is partitioned into two sub-classes : (a) interactive methods that rely on a small amount of prior information provided by the user (e.g., labels of a few pixels as initial constraints) and that encompass three groups of methodology: contour tracking approaches ([109, 100, 92, 74, 96, 147, 84, 85]), label propagation approaches ([22, 68, 113, 11]) and local optimization approaches ([75, 47]); (b) image cosegmentation ([121]), well-suited for large scale image dataset and which consists in identifying common objects in a set of images.

(iii) Learning models: fully supervised methods (refer to [160, Section 4] and [62] for an overview): they consist in training a segmentation algorithm thanks to fully annotated data —all pixels are labeled as either boundary or non boundary —and then segmenting an unknown image. They reach high performance but the labeling is very expensive. However, more and more datasets are now available (see [160] for a list of them) with the explosion of machine learning-based algorithms and increasing computer abilities in the past few years.

In line with this classification, this chapter aims to focus on the second class of weakly
supervised methods and more specifically, on interactive approaches (although the
joint segmentation/registration models depicted below might be viewed as special
instances of cosegmentation). The study entails the following three focal areas that
can be envisioned as three distinct types of a priori knowledge included in the
segmentation process and that structure the rest of the paper:
(i) Geometrical constraints to define local objects such as incorporation of landmarks to overcome the issue of weak boundary definition or inclusion of shape prior knowledge (Section 2);

(ii) Prescription of topological constraints in order for the final shape to be homeomorphic to the initial one, to comply with a pre-defined topology, or in order for the evolving segmenting curve to exhibit fine regularity properties (Section 3);

(iii) Combined segmentation/registration models. Registration is at the crux of a wide range of applications as stressed in [37, 99, 108, 130]: shape tracking; fusion of anatomical images from Computerized Tomography (CT) or Magnetic Resonance Imaging (MRI) images, with functional images from Positron Emission Tomography (PET), Single-Photon Emission Computed Tomography (SPECT) or Functional Magnetic Resonance Imaging (fMRI), also called multi-modality fusion to facilitate intervention and treatment planning; computer-aided diagnosis and disease follow-up; surgery simulation; atlas generation to integrate anatomic, genetic and physiological observations from multiple patients into a common space and to conduct statistical analysis; radiation therapy; assisted/guided surgery; anatomy segmentation; computational model building and image subtraction for contrast enhanced images. Given two images called Template and Reference, registration consists in determining an optimal diffeomorphic deformation $\varphi$ mapping the Template into the Reference. This task is depicted in Figure 1 taken from [110]. As structure/salient component/shape/geometrical feature matching and intensity distribution comparison rule registration, it sounds relevant to intertwine the segmentation and registration tasks into a single framework. In this approach (Section 4), we make full use of segmentation from one image and the ability of registration to correlate different images (even across modalities).
<figure><img src="image_4.png" /><figcaption>Fig. 1: Mapping of a 2D slice of mouse brain gene expression data to its counterpart in an atlas.</figcaption></figure>

## 2 Geometrical Constraints

We now present the first class of methods building models based on a given set of geometrical constraints.

The usefulness of identifying and extracting a single object of interest among others included in a given image has been recognised for many years. The pioneering work of [79] discussed how to build spring-like forces, based on specifying the subsets (seed points) or sub-regions within object boundary, between control points of a snake in the energy functional to push out the snake out of a local minimum. A closely related idea of providing seed points is used in the live wire works such as [13] and [36] and in various references therein.

Below we focus more on variational frameworks. The first class of methods we present in this section aims to segment local image objects that are characterised by a set $\mathcal{M}$ of landmarks or markers included in the image set $\Omega$ defined by $\mathcal{M} = \{x_i \in \Omega | i = 1, \dots, m\}$. Only the objects that are closest to $\mathcal{M}$ will be segmented by those models, as illustrated in Figure 2.
<figure><img src="image_5.png" /><figcaption>Fig. 2: Geometrical constraints and segmentation. Note the initial $u$ is arbitrary.</figcaption></figure>

## 2.1 Characterisation of geometrical constraints

The discrete set $\mathcal{M}$ of markers needs to be converted into a representative function that can be included into a variational setting. This function, as used in [10, 115, 158, 131, 118], was defined as a Euclidean distance in [67, 84]

$$d(x, y) = \prod_{i=1}^{m} \left( 1 - \exp\left(-\frac{(x - x_i)^2}{2\sigma^2} - \frac{(y - y_i)^2}{2\sigma^2}\right) \right),$$

where $\sigma$ is some scaling constant. Here $0 \le d \le 1$ satisfies $d \approx 0$ near $\mathcal{M}$ and $d \approx 1$ away from it. An evolution equation of the form

$$\frac{\partial \phi}{\partial t} = |\nabla \phi| \nabla \cdot \left( d(x, y) g(|\nabla z(x, y)|) \frac{\nabla \phi(x, y, t)}{|\nabla \phi(x, y, t)|} \right)$$

is then derived (—z denoting the considered image —) to drive the initial level set curve to the desirable $\phi$ zero level line defining the intended object. Here $g$ is an edge detector function that helps the contour evolve by ensuring that the front stops propagating when localised on meaningful contours. Extended this model to a variational framework, several models were then considered.

**Initialisation.** In the models shown below, the level set $\phi$ is initialised automatically (i.e. no further user intervention is required) using the polygon formed by the marker points as displayed in Fig.2(c) for 3 markers, depending on the convexity of the underlying model; we may place a small circle near the markers if the number
of marker points is less than 3. However, when the underlying model is convex, initialization can be made with an arbitrary contour.

## 2.2 Model 1 – A simple variational model

The first and yet simple extension was done by Badshah-Chen (2010) in [10] by extending the Chan-Vese model [33] to incorporate geometric constraints so that noisy images can be better segmented than in [67] as follows

$$ \min_{\phi, c_1, c_2} \int_{\Omega} dg(|\nabla z|)|\nabla H(\phi)|d\Omega + \int_{\Omega} (\lambda_1 H(\phi)|z - c_1|^2 + \lambda_2(1 - H(\phi))|z - c_2|^2) d\Omega. $$

A weakness of [10] lies in the fact that the obtained global solution often contains neighbouring objects, which can be only avoided if one terminates iterations early. In some cases, the model reaches the same global segmentation result as the Chan-Vese solution.

## 2.3 Model 2 – A moving band model

One way to overcome the problem is to ‘know’ when to stop the algorithm or when a model reaches the object boundary (which may not correspond to the local minimiser of [10] model). The idea by Zhang-Chen-Gould-Yu (2014) in [158] is to evolve the initial curve from a polygon in a band fashion so that we would not step over the object boundary. The proposed model takes the form of

$$ \inf_{\phi, c_1, c_2} \int_{\Omega} \left( \lambda_1 H(\phi) |z - c_1|^2 b(\phi, \gamma_{in}, \gamma_{out}) + \lambda_2 (1 - H(\phi)) |z - c_2|^2 b(\phi, \gamma_{in}, \gamma_{out}) \right) d\Omega + \int_{\Omega} dg(|\nabla z|) |\nabla H(\phi)| d\Omega, $$

where $b(\phi, \gamma_{in}, \gamma_{out}) = H(\phi - \gamma_{in}) H(\gamma_{out} - \phi)$ defines a narrow band of varying and adaptive widths $\gamma_{in}, \gamma_{out}$. The idea of ‘not stepping over’ the desirable $\Gamma$ when $\phi = 0$ is achieved by an adaptive searching algorithm based on checking local intensity variations [158]. Of course, the use of a varying band domain in the formulation leads to a highly non-convex model, making it hard to develop a theory.

## 2.4 Model 3 – A dual level model

To overcome the issue raised in [10], alternatively, the idea of Rada-Chen (2012) in [115] is to compute the Chan-Vese solution by a level set function $\phi_G$ and also to
compute the desirable solution (of an object only) via an embedded level set function $\phi_L$, resulting in the dual level set formulation

$$ \min_{\phi_G, \phi_L, c_1, c_2} \left( \mu_L \int_{\Omega} dg(|\nabla z|) |\nabla H(\phi_L)| H(\phi_G + \gamma) d\Omega \right. \\ \left. + \mu_G \int_{\Omega} g(|\nabla z|) |\nabla H(\phi_G)| d\Omega \right. \\ \left. + \int_{\Omega} (\lambda_{1G} H(\phi_G) |z - c_1|^2 + \lambda_{2G} (1 - H(\phi_G)) |z - c_2|^2) d\Omega \right. \\ \left. + \int_{\Omega} (\lambda_{1L} H(\phi_L) |z - c_1|^2 + \lambda_{2L} (1 - H(\phi_L)) H(\phi_G) |z - c_1|^2) d\Omega \right. \\ \left. + \int_{\Omega} \lambda_{3L} |z - c_2|^2 (1 - H(\phi_L))(1 - H(\phi_G)) d\Omega \right) $$

where the first component balanced by $\mu_L$ essentially selects the desirable solution $\phi_L$ from the global solution $\phi_G$, while the last term weighted by $\lambda_{3L}$ helps separate the 'true' background from the foreground intensity. The latter is because both objects included in $\phi_G$ and the desirable object included in $\phi_L$ often have the same intensity $c_1$, i.e. objects included in $\phi_G$ but not desirable to our selection are quite different from the true background intensity $c_2$. Parameter $\gamma$ is an integer included to increase the search domain (by a small band) in the computation of the weighted length of $\Gamma_L$. Its introduction increases the model robustness until final convergence. The above model was found extremely robust for selective segmentation but its implementation doubles the amount of work one normally needs because of the use of two level set functions instead of the usual one.

## 2.5 Model 4 – The use of moments constraint for segmentation

To stay with one level set function and yet to overcome the drawback of getting redundant objects beyond set $\mathcal{M}$, a useful idea is to impose the so-called moment constraints. The early work [9] of 0th order moment (or area constraint) uses

$$ \min_{\Gamma} \oint_{\Gamma} g(z) ds + \frac{\mu}{A_p^2} \left( \int_{R_{\Gamma}} d\Omega - A_p \right)^2 \int_{R_{\Gamma}^C} g(z) d\Omega $$

where $A_p$ is an area prior (given), $g(z) = g(|\nabla z|)$ is an edge detector, $R_{\Gamma}$ is the domain inside the closed curve $\Gamma$, while $R_{\Gamma}^C$ is the outside domain. A level set reformulation is

$$ \min_{u} \left( \int_{\Omega} g(z) |\nabla H(u)| d\Omega + \frac{\mu}{A_p^2} \left( \int_{\Omega} H(u) d\Omega - A_p \right)^2 \int_{\Omega} g(z) H(-u) d\Omega \right) $$
where $u > 0$ in domain $R_Γ$, $H(-u)$ indicates the outside domain (note the typo in the definition of level set function $u$ in [9] which stated wrongly $u < 0$ in $R_Γ$). The above reformulation was surveyed in [106] which unfortunately replaced the latter term $\int_{\Omega} g(z)H(-u)d\Omega$ by $\int_{\Omega_{in}} g(z)d\Omega = \int_{\Omega} g(z)H(u)d\Omega$ which is a major typo.

High order moments were considered in [81]. Denoting by $u$ the indicator function, i.e. 1 inside object and 0 outside, the proposal in [81] for area constraint is

$$ \min_{u} E(u) = \int_{\Omega} f u \, d\Omega + \int_{\Omega} g |Du| \, d\Omega + \lambda \left( \int_{\Omega} u d\Omega - A_{p} \right)^{2}, $$

where the last term to impose the area $A_p$ of the targeted object is a simple version of a more general constraint

$$ a_1 \le \int_{\Omega} u \, d\Omega \le a_2. $$

Here $f$ is taken as the log likelihood ratio for observing $z(x, y)$ at a point $(x, y)$ given that $(x, y)$ is part of the background or the object. Higher order moments refer to tensors of high order e.g. centroid (first order), covariance (second order). Note that the set defined by the inequality is convex which could potentially be explored.

The model [116] by Rada-Chen (2013) builds the area constraint into selection

$$ \min_{\phi, c_1, c_2} \int_{\Omega} dg(|\nabla z|) |\nabla H(\phi)| d\Omega + \int_{\Omega} (\lambda_1 H(\phi) |z - c_1|^2 + \lambda_2 (1 - H(\phi)) |z - c_2|^2) d\Omega \\ + \nu \left( \int_{\Omega} H(\phi) d\Omega - A_1 \right)^2 + \nu \left( \int_{\Omega} (1 - H(\phi)) d\Omega - A_2 \right)^2, $$

where $A_1$ is the area defined by the polygon formed by markers in set $\mathcal{M}$ assuming $m \ge 3$ while $A_2 = \text{mes}(\Omega) - A_1$ is the area outside this polygon.

## 2.6 Model 5 – Convex segmentation models

The work of [32] proposes a convex relaxation idea for the Chan-Vese model that is nowadays widely used, where the relaxation consists in replacing $\{0, 1\}$ by $[0, 1]$ after substituting $H(\phi)$ by $u$. This idea is emphasised by [131] in Spencer-Chen (2015) who propose a convex selective segmentation model

$$ \min_{0 \le u \le 1} \int_{\Omega} dg(|\nabla z|) |\nabla u| d\Omega + \int_{\Omega} (\lambda_1 u |z - c_1|^2 + \lambda_2 (1 - u) |z - c_2|^2) d\Omega \\ + \theta \int_{\Omega} P_d u d\Omega $$
where $P_d$ is the scaled distance to the polygon $\mathcal{P}$ formed by $\mathcal{M}$ and specifically $P_d = 0$ if $(x, y)$ belongs to the polygon $\mathcal{P}$, encouraging $u = 0$ in $\Omega \setminus \bar{\mathcal{P}}$. The model works well only if $\theta$ is appropriately chosen which may not be always easy to do.

## 2.7 Model 6 – Convex models based on geodesic distances

A deep idea to explore more the marker set $\mathcal{M}$ is to design a new distance function $d(x, y)$ that takes into account several factors: the set $\mathcal{M}$ itself as before, large edges of image $z$, the previously used Euclidean distance $P_d$ and possible anti-markers (to define a set $\mathcal{AM}$ of points that are definitely not in the intended object). Then a geodesic distance encompassing all these constraints and denoted by $\mathcal{D}$ is defined in [118] to replace the previous distance $d$ and satisfies the Eikonal-type equation:

$$|\nabla \mathcal{D}| = f(x, y) = \varepsilon + \beta |\nabla I|^2 + \nu P_d + d_{AM},$$

where $\varepsilon$ is a small parameter, $I$ is a denoised version of $z$—which can be the smoothed image $G*z$ or the image resulting from a few iterations of a denoising process applied to $z$, $P_d$ is the Euclidean distance as used before and $d_{AM}$ is an anti-marker distance exhibiting the following properties: it penalises pixels close to the set $\mathcal{AM}$ and it ensures rapid decay of the penalty away from the set $\mathcal{AM}$. The design of $d_{AM}$ may be based on a geodesic distance $\bar{d}_{AM}$ satisfying $|\nabla \bar{d}| = \varepsilon + \beta |\nabla I|^2 + \nu P_d(\mathcal{AM})$. Then in [118], we suggest to take $d_{AM} = (\exp(\alpha(1 - \bar{d}) - 1)/(\exp(\alpha) - 1)$ which can highlight the contribution of $\mathcal{AM}$ while reducing its influence on $\mathcal{M}$.

Note that the Eikonal equation is equipped with Dirichlet boundary conditions (i.e., $\bar{d} = 0$ at $\mathcal{M}$) thus falling within the framework of boundary value problems, and can be solved efficiently by a $O(N)$ implementation of the fast marching algorithm (see [152]). It yields the following model [118]

$$\min_{u, c_1, c_2} \int_{\Omega} g(|\nabla z|)|\nabla u| d\Omega + \int_{\Omega} (\lambda_1 u|z - c_1|^2 + \lambda_2(1 - u)|z - c_2|^2) d\Omega \\ + \mu \int_{\Omega} \mathcal{D} u d\Omega + \alpha \int_{\Omega} v(u) d\Omega$$

where $v(u)$ enforces $0 \le u \le 1$ as done in [32]. The shifting of $\mathcal{D}$ from the first component to a separate term was motivated by [90].

While Model 6 is perhaps the most robust up to now, it still assumes that the underlying given image is approximately of piece-wise constant intensities and without textures, just as in the Chan-Vese model [33]. To allow more generality, there are scopes and needs to design new models.
## 2.8 Other possible models

There exist many interesting ideas that remain to be fully explored.

(i). Assuming that a user has provided 2 sets of input as before, markers $\mathcal{M}$ within the object and anti-markers $\mathcal{AM}$ within the background, [45] defines a distance-like label function

$$L(x, y) = \begin{cases} +1 & \text{if } (x, y) \in \mathcal{M}, \\ -1 & \text{if } (x, y) \in \mathcal{AM}, \\ 0 & \text{elsewhere,} \end{cases}$$

which may be used to replace or enhance our distance function. The suggestion in [45] is to add a regularisation term to influence optimization for the level set function (object $\phi > 0$)

$$E_{user}(\phi) = - \int_{\Omega} L(x, y)\mathrm{sign}(\phi(x, y))d\Omega.$$

This idea is directly related to our works as demonstrated.

(ii). To ensure that an indicator function based on user input can re-adjust any classified (segmented) pixels, [15] suggested the new definition

$$L(x, y) = H(\phi) + [1 - 2H(\phi)] \int_{z \in \mathcal{N}(x, y)} M(z)dz$$

where

$$M(z) = \sum_{i=1}^{m} \delta(z - (x_i, y_i)) \quad \text{with} \quad (x_i, y_i) \in \mathcal{M}$$

and $\mathcal{N}(x, y)$ denotes a infinitesimal neighborhood of $(x, y)$. Here, $L(x, y) = 0$ for pixels in set $\mathcal{M}$ and 1 for pixels in set $\mathcal{AM}$. However $L(x, y) = H(\phi(x, y))$ at other pixels away from markers. The new $H$ that can reflect the feedback of a user is found by

$$E_{user}(\phi) = \int_{\Omega} \int_{\Omega} (L(x', y') - H(\phi(x, y)))^2 K(x, x', y, y')d\Omega d\Omega'$$

where $K$ is a Gaussian kernel defined by

$$K(x, x', y, y') = \frac{1}{2\pi\sqrt{|\Lambda|}} \exp\left\{-\frac{1}{2}(x - x' \ y - y')\Lambda^{-1}\begin{pmatrix} x - x' \\ y - y' \end{pmatrix}\right\}$$

and $\Lambda$ is a $2 \times 2$ covariance matrix. In this method, only the set $\mathcal{AM}$ is required because all pixels in this set have to be fixed, i.e., if such pixels from the set are in the foreground, they will be assigned to background and vice versa through matching $H(\phi)$ to $L$. This idea, mainly directed to post-processing steps, is a bit different from other formulations.

(iii). The current selection models, as discussed so far, assume that a given image exhibits a piecewise constant distribution of intensities (due to their use of Chan-Vese-like fitting terms). There exist newer and convex models [1, 27] derived from the Mumford-Shah [102] model, well-suited for more general images. Hence the
models depicted in this section and dedicated to geometric constraint enforcement
may be extended to such works.

(iv). Geometric constraints may be phrased in other forms apart from a set $M$. An interesting (and challenging) problem is to define a shape constraint, often useful when one intends to overcome problems related to missing data due to occlusions (e.g. in cells imaging, medical imaging or vehicles recognition). See works in [138, 61, 80]. Such methods may involve ideas borrowed from image registration involving small deformations or large rigid deformations.

(v) Another interesting class of research directions falls within the scope of boundary convexity constraint. This helps segmentation in case of very noisy images or missing data. See the recent works of [91, 94, 129].

Finally it should be noted that the above discussed models can be extended in a
simple manner to deal with 3D images; in fact many were implemented and tested
in 3D. As also remarked above, it remains to develop models based on frameworks
beyond piecewise constant intensities that can process texture images effectively.
This will be a future direction.

3 Topological Prior Knowledge

The second class of methods we depict hereafter intends to partition a given image
into semantically significant constituents while fulfilling some topological require-
ments. As stressed in [126], integrating such constraints is a difficult task for two
reasons. First, due to the dual nature of topology which is both a global property and a
local one: small and localized changes on a geometrical shape may modify its global
connectivity. Second, since topology is a continuous concept whose properties are
difficult to transpose in the discrete setting. Two different kinds of prior knowledge
are investigated :

(a) prescribed topology enforcement in the sense that the segmented target should be homeomorphic to the original shape supplied by the user —two objects being homeomorphic provided they can be deformed into each other by a continuous, invertible mapping —or should exhibit a prescribed number of connected components/holes. Topology enforcement in segmentation is particularly important when a user's requirement is not in visual agreement with the data, i.e. in the case where, without including those topological constraints, most segmentation models would fail: Figure 3 would show two objects, while a single cell would be outlined in Figure 4.

Figures 3 and 4 taken from [85] provide two illustrations of prescribed topology enforcement. In Figure 3, we aim at segmenting the two discs while maintaining the same topology throughout the process, meaning that we expect to get a simply connected shape. Figure 4 illustrates the case where the initial condition is made of two disjoint closed curves. We expect to have both curves evolving without merging. If visually the blood cells look glued, individual cell segmentation is
required. Here, the final segmentation shows that the two cells are disconnected, in compliance with the user's requirement.

<figure><img src="image_6.png" /><figcaption>Fig. 3: Segmentation steps of a synthetic image with two disks when topological constraints are applied.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 4: Segmentation steps of two blood cells close to each other when topological constraints are applied.</figcaption></figure>

(b)regularity enforcement on the edge set of the segmentation, thus influencing the topology of the segmenting curves/shapes, with an emphasis on variational
models, due to their ability to include multiple criteria. This kind of approach
does not fall exactly within the scope of topological prior-based methods —since
it does not intend to prescribe the topology of the targeted object—but influences
nevertheless the regularity and so in some way the topology of the final shape
by removing undesirable small patterns. In this regard, Figure 5 taken from [3]
(courtesy of Luis Alvarez, Universidad de Las Palmas de Gran Canaria, Spain)
illustrates how a geometrical partial differential equation can be used for level set
regularisation, i.e. to remove small scale features and spurious oscillations. By
choosing adequately a forcing term appearing in the partial differential equation
(PDE) that dictates the front evolution, one can keep more or less detail in the
final segmenting curve.

<figure><img src="image_8.png" /><figcaption>Fig. 5: On the left, the initial condition $\Gamma_0$ is obtained by thresholding the intensity values of the image $I$, i.e. $\Gamma_0 = \partial \{x \mid c_1 \le I(x) \le c_2\}$, $c_1$ and $c_2$ being two fixed values. On the right, obtained regularised segmentation.</figcaption></figure>

**3.1 Topology prescription**

The necessity of designing topology-preserving processes arises in many applica-
tions, e.g. in medical imaging to preserve the contextual relations between organs,
or when reconstructing for instance the human brain ([34]). Despite its highly folded
nature ([60, 126]), the intrinsic unfolded structure of the cortex is the one of a
2D sheet, and assuming that the midline hemispheric connections are artificially
closed, each cortical hemisphere is homeomorphic to the sphere, implying that this
topological feature must be fulfilled by the segmentation. In the context of curve
evolution-based models and more specifically level set-based approaches ([109]),
two main channels of thought have been investigated: methods resting on digital
topology ([21, 82, 82]), in particular on the concepts of simple point, multisimple
point, well-composedness, and methods relying on specifically designed criteria in
the objective function (purely continuous ones), penalizing curvature and affecting
thus the shape of the curve.

**3.1.1 Digital Topology**

It is generally agreed that the implicit framework of the level set setting displays
several advantages over parametric methods when tracking a front that propagates:
the evolving contour is embedded in a higher dimensional level set function avoiding
thus parameterization issues, the model is intrinsic, i.e. invariant to a reparameteri-
zation of the curve, able to handle topological changes (merging and splitting —note
nevertheless that some works aim to reconcile parametric implementations with han-
dling of topological changes as in [112] for instance, where the authors address the
issue of moving object segmentation using interpolating B-spline curves in a spatial
multi-resolution approach and with a penalty over the length of the contour. Topolog-
ical changes are managed by making the most of the variation diminishing property
exhibited by B-splines —), geometrical properties of the front such as curvature or
outwards unit normal vectors are easily derived from the level set function, and its
evolution is straightforwardly phrased in an Eulerian framework. Nevertheless, this
topological flexibility proves to be undesirable in many applications (e.g. mitotic
cell tracking [63]). That is to say, parametric methods seem naturally more suited
to deal with these topological constraints. Nevertheless, the level set formulation
remains widely used in topology-preserving segmentation models, partly due to its
popularity and the fine properties it exhibits. The topology enforcement takes the
form of constraints applied to the implicit contour to fulfill the user's requirements.

In the spirit of front propagation approaches combined with digital topology
requirements (reconciling then the intrinsic discrete nature of digital images with the
continuous representation with which they are identified in variational frameworks),
Han et al. ([72, 73]) propose a model preserving the topology of the implicit contour
while the embedding level set function is evolving. The key idea of the model
lies in the concept of simple points ([16, 17]) identified as those points whose
addition or removal leave the topology unchanged. Their characterisation relies on
the computation of two topological numbers: the equivalence between the fact for a
point of being simple, and the specific values of these two topological characteristics
is proved in [16]. It is complemented by the basic definitions necessary to effectively
calculate these two numbers. The algorithm reads as follows, the topology of the
zero level set being assumed equivalent to the topology of the boundary of the digital
object it defines : at each iteration, one monitors the changes of sign of the level set
function and prevents it from changing sign at grid points that are not simple. The
derived algorithm is thus pixel-based, applicable on lattice structures only, not on
arbitrary data sets, and the result produced is highly dependent on the order in which
the points are treated in the narrow band.
Still in a level set evolution framework, Ségonne ([125]) investigates the concept of multisimple points identified as those points whose addition or removal do not create or delete handles in the image. The formal mathematical characterisation of these points —derived from the one of simple points —can be found in [125, Sub-section 3.1]. Unlike [72, 73], the resulting algorithm allows connected components to merge, split or vanish, ensuring at the same time that the genus (topological invariant describing the number of handles) of the initial active contour is preserved, and relaxes then the constraints on the initial condition.

Contrasting with these concepts of simple points and multisimple points, Tustison et al. ([139]) propose a topological variant of well-composedness for maintaining the topology of the evolving digital front —in *n* dimensions, a set being said well-composed if and only if the boundary of its continuous counterpart is a (*n* − 1)-manifold ([21]), obviating thus the requirement of specifying one of the classical connectivity relations since in 2D it reduces to the connectivity (4,4), ((6,6) in 3D) —. This characterisation implies that a correspondence between a *n*-D digital set and the boundary surface of its continuous analog is required. This correspondence is made explicit in [83, Section 1]. This connectivity condition alone is not sufficient to constrain the topology of the evolving digital front, requiring thus the extension of the concept of well-composedness to its topological variant by incorporating critical configurations. In this regard, Tustison et al. ([139]) identify points that preserve both the well-composedness property and the image topology, sole points that can be added or removed while the front progresses. Once the evolution of the initial configuration subject to strict topology preservation via topological well-composedness is achieved, and since the interface that separates the well-composed genus zero components satisfies the digital Jordan separation theorem —discrete counterpart of the Jordan separation theorem that states that a surface homeomorphic to the sphere might be separated into two objects by a simple, continuous, closed curve (Jordan curve) along the surface ([104]) —, a procedure to glue together the neighboring components by adding the part of the Jordan surface separating them is initiated, yielding a single well-composed genus zero object.

In the context of cellular morphology analysis, Yu et al. ([155]) address the critical issue of individual cell segmentation. This task is challenging since a cell may stick to other ones, sharing common boundaries, or even overlap others with no clear outline. The authors propose a novel algorithm (that constitutes again a compromise between discrete and continuous formulations) : once the nuclei of the cells are identified and segmented, a front propagation is performed using generalized Voronoï diagrams to avoid overlaps.

In a purely discrete setting, Chen et al. ([35]) propose incorporating topological prior knowledge into random field image segmentation to encode more global topological properties such as connectedness of a labeled region. The labeling functional is classically designed as the sum of unary potentials and pairwise potentials, and is subject to topological constraints prescribing the number of connected components and holes. Waggoner et al. ([144]) develop a new Markov Random Field based multi-labeling technique to enforce topology in multi-label image segmentation. They apply both specific adjacency relations between each pair of segments, and a
connectedness property on each region. More precisely, the algorithm consists in
defining a segmentation template exhibiting the desired topology and in spreading
it towards a target image

Among a posteriori (also termed as retrospective) topology correction methods,
one can cite the work by Ségonne et al. ([126]) dedicated to geometrically accurate
topology correction of cortical surfaces. Denoting by *C* the cortical surface —typ-
ically instantiated as a polygonal tessellation with vertices, edges and faces —, by
*S* a sphere and by *N* : *S* → *C* a mapping from *S* to *C*, their work is based on the
observation that a necessary and sufficient condition for *C* to carry the desired spher-
ical topology is that the mapping *N* is a homeomorphism. Worded in these terms,
the problem is difficult since large deformations are required to retrieve the less
smooth highly convoluted surface. The authors propose instead to find a mapping
*M* : *C* → *S*. If the cortical surface carries the desired topology, i.e. χ(*C*) = 2, χ(*C*)
denoting the Euler characteristic which is a topological invariant, nothing must be
done. If not (which is most often the case in practice since *C* is likely to exhibit topo-
logical defects), the proposed algorithm (also inspired by [60]) operates as follows.
The authors first seek a mapping *M* : *C* → *S* which is a quasi-homeomorphism,
i.e. a homeomorphism on as much of the surface as possible. This step is achieved
by unfolding and smoothing the folded cortical surface by spherical inflation so that
the obtained surface resembles the one of a sphere with origin the centroid of the
initial surface. The position of a given vertex is evolved according to a geometric
flow involving movement towards the centroid of its neighboring vertices, while
projecting out the average inwards movement it creates over the whole surface, with
in addition a radial term driving each vertex to the surface of a sphere with pre-
scribed radius. Once this spherical inflation step is achieved, a quasi-homeomorphic
mapping *M* is generated by minimising a function that penalizes regions in which
the determinant becomes negative or zero. Topological defects are then detected as
locally non invertible regions (where *M*⁻¹ is multivalued), i.e. as parts of the sphere
displaying overlapping triangles, and the correction procedure is applied, taking into
account geometrical accuracy (expected local curvature, local intensity distribution)
and topological consistency. More precisely, correcting a topological defect amounts
to identifying the number of handles (or equivalently holes) it contains, a handle be-
ing characterised by the existence of non contractible curves (also termed as non
separating loops), simple closed curves that cannot be continuously deformed on the
manifold into a single point. —This concept is of importance since a non separating
loop associated to a hole gives two strategies to remove this hole: either one fills the
area enclosed by the non separating loop to patch the hole, or one empties the area
inside to open the hole.—Motivated by this characterisation, the algorithm reads as
follows. Given a handle, several non separating loops are randomly generated. For
each produced curve, the faces that form the loop are removed from the topological
defect mesh and the resulting open mesh is sealed (either by filling the hole or by
cutting the handle). The accuracy of the resulting candidate solution is optimized,
based on active contour patches. In the end, the candidate configuration that opti-
mizes a Bayesian energy functional is selected, both maximizing goodness of fit of
the produced surface with respect to the available image information, and complying
with topological consistency.

In the same vein as in [126] and motivated by the fact that surface reconstruction
methods go beyond the simple use of volumetric images alone in the context of struc-
tural and functional brain data analysis, Yotter et al. ([154]) propose to retrospectively
repair topological and geometrical defects (that impair the true nature of the cortical
anatomy) on the brain surface mesh, using spherical harmonics, this theory allowing
to quantify structural differences between shapes. Spherical harmonics are functions
defined on the surface of a sphere. They form a complete set of orthogonal functions
on the sphere and might be used to represent functions defined on the surface of a
sphere in the same spirit as Fourier series.

The processing chain reads as follows: (i) the uncorrected surface mesh is mapped
onto a sphere (this mapping is not a homeomorphism) meshed with triangles; (ii) a
regularly sampled grid is then overlaid on the previously obtained sphere surface;
(iii) for each regularly sampled spherical point, one identifies the intersecting triangle
of the sphere mesh (by finding the closest triangle of the tessellated sphere surface,
a specific strategy being developed to favor some of the triangles related to topo-
logical defects). Then using barycentric coordinates in this triangle, the coordinates
of the fiber vertex lying on the original mesh surface are determined, yielding three
functions defined on the sphere: each regularly sampled spherical point is associated
with the three coordinates giving its location on the original cortical surface. (iv)
every function is expanded in the spherical harmonic basis where the coefficients
are defined as the L²-inner product of the function and the basis functions. Using
the computed coefficients, two surfaces are then reconstructed: the first one is a
high-frequency surface employing all coefficients, while the second one is a smooth
surface reconstructed from filtered coefficients using a low-pass filter. Vertices from
the low-pass filtered reconstruction are patched into the high-frequency one in re-
gions that previously contained defects (and that are likely to display pikes after the
spherical harmonic-based reconstruction). At last, a post-processing step is applied
to correct self-intersections.

Despite the fact that deep learning-based methods are beyond the scope of this
contribution, we would like to end this part by highlighting some methods that in-
tertwine the soundness of these approaches (that yield remarkable results whenever
sufficient labeled data can be collected) with variational models and specially ac-
tive contour-based models, capable of encoding high-level shape features such as
topology. In [137], Thierbach et al. propose to combine convolutional neural net-
works with topology-preserving geometric deformable models ([19]) in the context
of neural cell bodies segmentation from light-sheet microscopy, while limiting man-
ual annotations. The training step is achieved with simple cell centroid annotations
and the final segmentation provides accurate results complying with the topological
requirements (no cell splitting/merging).
### 3.1.2 Purely Continuous Methods

If the above methods make a trade-off between the intrinsic discrete nature of digital images and the continuous formulation of front propagation, some take the side of focusing only on continuous aspects. In [133], the authors incorporate a novel nonlocal geometric flow into image-based evolutions of active contours in order to preserve topology. The relevant term that is minimised with respect to the curve C is inspired by the knot energy and is defined by $E(C) = \frac{1}{2} \iint_{C \times C} \frac{dp \, dp'}{\|C(p) - C(p')\|^\gamma}$ ($\| \cdot \|$ denoting the Euclidean norm and $\gamma$ being a tuning parameter), penalizing thus spatial proximity of the curve points and subsequently its curvature.

The work [2] introduces a curve evolution method based on level sets for shape optimization models arising in material science, and is directed towards the class of problems involving constraints on the number of connected components. The algorithm is designed in order for the narrow band of the evolving contour to avoid overlaps. More precisely, their model minimises $F_\mu(\Phi) = F(\Phi) + \mu R(\Phi)$ with $\mu \ll 1$, $\Phi$ being the evolving level set function assumed to be a signed-distance function, $F$ being a general shape optimization functional, while $R(\Phi)$ is the topological constraint. As previously sketched, this constraint that includes shape topology (i.e. prescribed number of connected components and holes), component size, and distance between components, ensures that the evolving topology is equivalent to the initial one —the prior on the initial topology is thus strong —and is incorporated into the optimization problem via a logarithmic barrier technique as

$$R(\Phi) = - \int_{\partial D} \log [\Phi(x + d \nabla \Phi(x))] ds - \int_{\partial D} \log [-\Phi(x - l \nabla \Phi(x))] ds,$$

with $D = \{x | \Phi(x) > 0\}$ and $d > 0, l > 0$ given parameters. Parameters $d$ and $l$ influence on the distance between distinct connected components and on the size of the connected components themselves. Although devoted to a different application, the work [120] uses a similar idea to the one developed in [133] to prevent pieces of the same curve from colliding, merging or breaking. The goal is to track thin long objects that evolve, with applications to the automatic extraction of road networks in remote sensing images. The authors propose interesting nonlocal regularisations on the curve C parameterized by $p \in [0, 1]$ phrased as $E(C) = - \int_0^1 \int_0^1 t(p) \cdot t(p') \Psi(\|C(p) - C(p')\|) dp dp'$, with $t(p)$ denoting the tangent vector to the curve at point $C(p)$ and $\|C(p) - C(p')\|$ being the Euclidean distance between the curve points $C(p)$ and $C(p')$. The function $\Psi$ is chosen to be $\Psi(l) = \sinh^{-1}(1/l) + l - \sqrt{1 + l^2}$, thus decreasing on $[0, +\infty]$. Other nonlocal forms are considered as well, and geometric motions of thin long objects are obtained. The implicit representation by level sets is used for the implementation. In [119], the authors carry on their ideas, but this time, in a phase-field approach.

Still in the prospect of a local treatment of topology preservation, [29, Section 4] is dedicated to the tracking of interfaces with fixed topology. The model relies on a coupled system of PDEs involving the level set function $\varphi$ embedding the
propagating front, and the arclength function $\Psi - \bar{\Psi}$ being conjugate to $\varphi$ in the sense that the two form an orthogonal coordinate system on the zero level set of $\varphi$, and on an accurate estimate of the Jacobian $J$ of the interface function and its conjugate ($J$ tends to 0 at merge points, while it tends to $\infty$ at pinch points). Motivated by geometric considerations, Le Guyader and Vese ([85]) complement the classical geodesic active contour model by a nonlocal component interpreted as a repelling force. More precisely, the level set function $\Phi$ being assumed to be a signed-distance function to the evolving contour $C$ and $l > 0$ denoting a tuning parameter, the following functional is incorporated into the classical geodesic active contour model phrased in the level set framework:

$$E(\Phi) = - \int_{\Omega} \int_{\Omega} G(\|x - y\|^2) \langle \nabla \Phi(x), \nabla \Phi(y) \rangle H(\Phi(x) + l) H(l - \Phi(x)) \\ H(\Phi(y) + l) H(l - \Phi(y)) \, dx \, dy,$$

the potential function $G$ measuring the closeness of the two points $x$ and $y$, $H$ denoting the 1D Heaviside function. Again, the goal is to penalize spatial proximity of curve points belonging to a narrow band around the zero level set and subsequently the curvature of the level lines. This idea is then revisited in [123] in the context of region-based active contours.

An energy involving a fixed-width band around the evolving curve as in [85] is also introduced in [97] to achieve a proper trade-off between local features of gradient-like terms and global region characteristics, and to weaken the strong assumption of uniformity of intensity over regions classical region-based models rely on.

More recently, still in an effort to ensure orientation preservation and to address the issue of stability with respect to noise —for instance when the shapes exhibit multiple disjoint objects that should be viewed as a whole—, a new framework based on quasiconformal mappings has been introduced in [31]. Given an image containing an object to be segmented together with the desired prescribed topology (that can be viewed as a shape prior), a simple template image is deformed so that it matches the boundary of the target object. In this regard, the model may be seen as a joint segmentation/registration one. The deformation undergone by the moving shape is dictated by the Beltrami equation and relies on the fine properties of quasiconformal mappings. Quasiconformal mappings can be defined as follows ([88]) (*we restrict ourselves to quasiconformal mappings that are homeomorphisms between plane domains*):

A sense-preserving homeomorphism $f$ of the domain $G$ is called quasiconformal if its maximal dilation $K(G)$ is finite. If $K(G) \le K < \infty$, then $f$ will be called $K$-quasiconformal.

As from the one hand, the maximal dilatation of a non-conformal sense-preserving homeomorphism is always greater than 1, and from the other hand, if $f$ is conformal, $K(G) = 1$, $K(G)$ can be viewed as a measure of deviation from conformality. The following result gives a necessary and sufficient condition ensuring that a given homeomorphism $f: \Omega \to \Omega'$ in $W_{loc}^{1,2}(\Omega)$ is $K$-quasiconformal.
**Theorem 1** [7, Theorem 2.5.4] Suppose $f: \Omega \to \Omega'$ is a homeomorphic $W_{\text{loc}}^{1,2}$-mapping. Then $f$ is K-quasiconformal if and only if

$$ \frac{\partial f}{\partial \bar{z}}(z) = \mu(z) \frac{\partial f}{\partial z}(z) \text{ for almost everywhere } z \in \Omega, $$

where $\mu$, called the Beltrami coefficient of $f$ is a bounded measurable function satisfying

$$ \|\mu\|_{\infty} \le \frac{K-1}{K+1} < 1. $$

With $\frac{\partial f}{\partial z} = \frac{1}{2}(\frac{\partial f}{\partial x} - i \frac{\partial f}{\partial y})$ and $\frac{\partial f}{\partial \bar{z}} = \frac{1}{2}(\frac{\partial f}{\partial x} + i \frac{\partial f}{\partial y})$, setting $f(z) = f(x + iy) = u(x, y) + iv(x, y)$ (so in the context of registration, the sought deformation is $\varphi = (u, v)^T$), the Jacobian is defined by $J_f(z) = \left|\frac{\partial f}{\partial z}(z)\right|^2 - \left|\frac{\partial f}{\partial \bar{z}}(z)\right|^2 = \frac{\partial u}{\partial x}(x, y) \frac{\partial v}{\partial y}(x, y) - \frac{\partial u}{\partial y}(x, y) \frac{\partial v}{\partial x}(x, y) = \det \nabla \varphi(x, y)$. Consequently, if $f, W_{\text{loc}}^{1,2}$-homeomorphism, is K-quasiconformal,

$$ J_f(z) = \left|\frac{\partial f}{\partial z}(z)\right|^2 - \left|\frac{\partial f}{\partial \bar{z}}(z)\right|^2 = \left|\frac{\partial f}{\partial z}(z)\right|^2 \left(1 - |\mu(z)|^2\right), $$

entailing that $J_f(z) = \det \nabla \varphi(x, y) > 0$ a.e., since $\|\mu\|_{\infty} < 1$.

Ensuring boundedness of the Beltrami coefficient ($\|\mu\|_{\infty} < 1$) thus implies positivity of the related deformation Jacobian determinant.

Equipped with this material, the authors state the main theoretical result that says that for any object in $\mathbb{C}$ with arbitrary topology, there exists a homeomorphism between the object and the simple circular domain with the same topology, which motivates the introduction of the concept of Beltrami representation of shapes. Given a natural integer $g \in \mathbb{N}$, a $g$-holed circular domain, $\tilde{D}_g \subset \Omega \subset C$, and a shape $D$ with the same topology as $\tilde{D}_g$, as there exists a quasiconformal mapping $f^\mu: \mathbb{C} \to \mathbb{C}$ associated with Beltrami coefficient $\mu$ such that $f^\mu(\tilde{D}_g) = D$ and $f^\mu = \text{Id}$ in $C \setminus \Omega$, the shape $D$ can thus be represented by $\mu$, which is called the Beltrami representation of $D$. It yields the following minimisation problem relying on the Beltrami representation of shapes. Note that the Beltrami representation is defined by the deformation from $\tilde{D}_g$ to $D$. Denoting by $\Omega \subset \mathbb{C}$ an image domain, by $I: \Omega \to \mathbb{R}$ an image including an object $D \subset \Omega$, $D$ being supposed of genus $g$, by $J: \Omega \to \mathbb{R}$ an image of $\tilde{D}_g$, called topological prior and modelled as a two-phase partition defined by:

$$ J(z) = \begin{cases} c_1 & \text{if } x \in \tilde{D}_g, \\ c_2 & \text{if } x \in \Omega \setminus \tilde{D}_g, \end{cases} $$

the authors propose minimising the following energy with respect to $\mu$:
$$E(\mu) = \int_{\Omega} |\mu|^2 + \eta \int_{\Omega} (I \circ f^{\mu} - J)^2 + \lambda \int_{\Omega} |\nabla \mu|^2,$$

subject to Dirichlet boundary condition $\mu = 0$ on $\partial\Omega$ and inequality constraint $\|\mu\|_{\infty} < 1$ on $\Omega$. The model was extended in [157] to impose $\|\mu\|_{\infty} < 1$ and without having to computing $\mu$ directly while allowing the use of a converging Gauss-Newton method.

Thus the optimisation problem encompasses three components: while the pairing of the intensities between $I$ and $J$ is ensured by the second term of the functional, the last one guarantees some smoothness on the deformation mapping $f^{\mu}$ whereas the constraint on $\|\mu\|_{\infty}$ insures admissibility of the Beltrami representation of the object in $D$. The connection to the Chan-Vese data fidelity term can be made explicit by making a simple change of variable in the second component. Also the regularisation of the Beltrami coefficient is now a substitute for the classical shape perimeter minimisation. High accuracy and stability of the proposed model with different segmentation tasks are then exemplified as demonstrated in Figure 6 in which we try to preserve the 0-genus property of the shape —note nevertheless that the algorithm can deal with shapes of higher genuses; for instance, if we go back to the example of Figure 3, it would suffice to build a proper topological prior image $J$ made of two separate shapes —. Classical segmentation models would fail to segment the desired shape as a whole because of the visible occluded regions: the bear ears would be disconnected from the body.

<figure><img src="image_9.png" /><figcaption>Fig. 6: On the left, prescribed topology superimposed on the input image. On the right, obtained segmentation.</figcaption></figure>

## 3.2 Regularisation enforcement on the evolving front

Although not directly related to topological prior-based methods, models relying on regularity enforcement prove to be efficient to remove undesirable small patterns and oscillations. In that, they influence both the global and local topology of the recovered shape and are connected in some way to the models depicted in Subsection 3.1.
### 3.2.1 Regularisation by geometric flows

Controlling the asymptotic states of geometric equations with respect to a prescribed forcing term is the subject of [3], in the context of level set regularisation. With suitable forcing terms, stabilization in a finite time of radial solutions can be demonstrated, making this modelling relevant for nonlinear image filtering or segmentation, in order to remove for instance spurious oscillations or small scale objects, to control the size of an object or on the contrary to enforce merging. Beyond the theoretical study of particular geometrical partial differential equations it includes, this work intends to provide some insight on how to build PDEs in order to solve image-related problems, whether it be segmentation or filtering. More precisely, the considered PDEs are geometrical ones —these equations defining an hypersurface evolution —, among which parabolic perturbed mean curvature-based equations of the form :

$$ \left\{ \begin{array}{l} \displaystyle \frac{\partial u}{\partial t} = F(\nabla u, \nabla^2 u) + k(x) |\nabla u| \quad \text{in } (0, T) \times \mathbb{R}^n, \\ u(0, x) = u_0(x) \quad \text{on } \mathbb{R}^n, \end{array} \right. \qquad (1) $$

with $F(\nabla u, \nabla^2 u) = \text{div} \left( \frac{\nabla u}{|\nabla u|} \right) |\nabla u|$, this latter quantity being the mean curvature (—ensuring thus smoothness of the evolving curve—), while $k$ denotes a forcing term, $k$ being assumed bounded Hölder continuous (so with rather weak required regularity —only Hölder continuity in comparison to the classical Lipschitz regularity —). This function $k$ that may depend on the image contents is the parameter that can be adjusted by the user according to his needs. Note also that this class of PDEs falls within a broader framework studied in [64] for instance. The geometric feature of these PDEs is conveyed by the fact that $F: \mathbb{R}^n \setminus \{0\} \times S^n \to \mathbb{R}$ satisfies $F(\lambda p, \lambda X + \sigma p \otimes p) = \lambda F(p, X), \forall \lambda > 0, \forall \sigma \in \mathbb{R}, \forall p \in \mathbb{R}^n, \otimes$ denoting the tensor product in $\mathbb{R}^n$ and $S^n$ denoting the space of $n \times n$ real symmetric matrices. It expresses that the zero level set of function $u$ only depends on the zero level set of the initial condition and not on the initial condition itself, and the composition of any solution with a nondecreasing function remains a solution of the equation. Such PDEs fall within the framework of the viscosity solution theory ([44]). Theoretical issues/qualitative properties of the hypersurface evolution are investigated like comparison principle or existence/uniqueness of the solution, with fewer requirements on function $k$ (Hölder continuity only). Special care is taken to the qualitative properties of the hypersurface evolution. This analysis is fully meaningful once we have studied the shape of radial solutions, making it possible to derive some qualitative properties (asymptotic behaviour) of the (unique) solution of the problem associated with an unspecified (but smooth enough) initial condition.

Considering as initial hypersurface $\Gamma_0$ of $\mathbb{R}^n$ the boundary of a bounded open set $U_0$, and denoting by $u_0: \mathbb{R}^n \to \mathbb{R}$ any bounded uniformly continuous function (Lipschitz continuous if $k$ is merely Hölder continuous) that satisfies :
$$u_0(x) = \begin{cases} u_0(x) < 0 & \text{if } x \in U_0 \\ u_0(x) > 0 & \text{if } x \in (\mathbb{R}^n - \bar{U}_0) \\ 0 & \text{if } x \in \Gamma_0 = \partial U_0, \end{cases}$$

we define the sets $\{U_t\}_{t \ge 0}$ and $\{\Gamma_t\}_{t>0}$ as

$$U_t = \{x \in \mathbb{R}^n, u(t, x) < 0\}$$

and

$$\Gamma_t = \partial U_t,$$

with $u := u(t, x)$ the unique viscosity solution of (1) for the initial datum $u_0(x)$. The authors first prove that $U_t$ and $\Gamma_t$ are independent of the choice of $u_0$ and only depend on $U_0$. Then they provide a comparison principle saying that if $U_0$ and $\bar{U}_0$ are bounded open sets satisfying the inclusion relation $U_0 \subseteq \bar{U}_0$, then for $t \ge 0$, this inclusion relation still holds, i.e. $U_t = \{x \in \mathbb{R}^n, u(t, x) < 0\} \subseteq \bar{U}_t = \{x \in \mathbb{R}^n, \hat{u}(t, x) < 0\}$, $u$ and $\hat{u}$ being the unique solutions of the PDE associated respectively with initial condition $u_0$ and $\bar{u}_0$. This later result will enable us to infer the asymptotic behaviour of an evolving shape given an initial condition. In order to get a clear view of the asymptotic states of the solution of (1) according to the choice of the forcing term $k$, the authors focus on the radial solution shape analysis. In that purpose, let $U_0 = B_{r_0}(x_0)$ be the ball centered at $x_0$ with radius $r_0$, and let us choose as initial datum $u_0$, the signed distance function defined by:

$$u_0(x) = d_{B_{r_0}(x_0)}(x) = \begin{cases} |x - x_0| - r_0 & \text{if } |x - x_0| < r_0 + S \\ S & \text{otherwise,} \end{cases}$$

for some $S > 0$. Standard computations lead to the following equation in radial coordinates:

$$\frac{\partial u}{\partial t}(t, r) = \left( \frac{n-1}{r} \operatorname{sgn}(u_r(t, r)) + k(r) \right) |u_r(t, r)|, \quad (2)$$

with sgn being the sign function. In this case, $U_t$ is given by the ball $B_{r(t)}(x_0)$ where $r(t)$ satisfies $u(t, r(t)) = 0, \forall t > 0$. By deriving this relation with respect to $t$ and by substituting $\frac{\partial u}{\partial t}$ with the right-hand side of (2), the problem amounts to solving an ordinary differential equation in $r$ for which an explicit expression of the solution can be provided. The shape of $r(t)$ is then investigated for several choices of the forcing term $k$ as well as its asymptotic behaviour, which suggests that by choosing the forcing term $k$ properly, some stabilization properties of the propagating front (the radius may stabilize in a finite time) can be expected as well as some particular responses such as shrinkage of the shape or on the contrary expansion of the radius with time. These observations combined with the monotonicity principle mentioned above (preservation of the inclusion property) motivate the application of this model as level set regularisation (refer to Figure 5 for an application).
As an alternative to the design of segmentation models including explicit regularisations, some recent advances have been made in the field of numerical schemes and more precisely, on the development of higher-order schemes for level set-based segmentation methods.

### 3.2.2 Higher-order schemes for level set-based segmentation models

In [56], Falcone and collaborators focus on a level-based segmentation model including a modified velocity inspired by prior related works by Malladi, Sethian and Vemuri ([95]) and more precisely, on a hybrid numerical scheme designed to avoid spurious oscillations around discontinuities of the solutions and/or jumps in the gradient. The propagating front is assumed to evolve in its normal direction with a velocity $v$ depending only on time and space variables, and supposed to be of constant sign in time. Its evolution is dictated by the following first order Hamilton-Jacobi nonlinear equation of Eikonal type:

$$ \left\{ \begin{array}{l} v_t + c(t, x, y) |\nabla v| = 0, \quad (t, x, y) \in (0, T) \times \mathbb{R}^2, \\ v(0, x, y) = v_0(x, y), \quad (x, y) \in \mathbb{R}^2, \end{array} \right. \qquad (3) $$

with $v_0$ a proper representation of the initial front $\Gamma_0$, satisfying

$$ \left\{ \begin{array}{ll} v_0(x, y) < 0, & (x, y) \in \Omega_0, \\ v_0(x, y) = 0, & (x, y) \in \Gamma_0, \\ v_0(x, y) > 0, & (x, y) \in \mathbb{R}^2 \setminus \overline{\Omega_0} \end{array} \right. $$

High-order schemes have been proposed to solve (3), most of them being based on non oscillatory local interpolation technique, for which nevertheless general convergence theorems are lacking. These limitations motivate the introduction of a new class of high-order schemes for time-dependent Hamilton-Jacobi equations grounded on filtered schemes. The design of these filtered schemes relies on a simple coupling of a monotone scheme and a high-order scheme, which allows inheriting both the property of convergence to the weak viscosity solution of the monotone scheme —known however to be at most first order accurate —and the higher accuracy of high-order schemes —that prove to be in general unstable —by properly connecting these two schemes, guaranteeing then global convergence. This is the main focal point of [56] after proposing a way to compute a modified (in the sense, extended) velocity $c$ in (3) ensuring regularity of the front evolution. As the front represents the boundary of an evolving shape, and since segmentation aims to extract object shapes from a given image, the front should stop moving in the vicinity of the desired object boundaries. The question of designing a suitable image-related speed function naturally emerges. From the modelling, this speed has only meaning on the zero level set function over the entire domain. In order for the evolution equation to have consistent meaning for all the level sets, an extension of the image-related speed is introduced, this extension being governed by the following property extracted from [95]:
- the image-related speed function must be devised so that level sets moving under this speed function cannot collide. A natural way of designing it is to let the speed at a point P lying on a level set {$v = c$} be the value of the speed at a point Q such that Q is the closest point to P lying on the level set {$v = 0$}. Q is uniquely determined whenever the normal direction in P is well-defined.

The novelty of [56] relies then on the method of construction of this extended velocity: it is based on the central premise that if the layout of the level sets is known at initial stage and if all the points in the normal direction to the zero level set evolve according to the same law, it sounds reasonable to expect that all such points will keep their relative distance unchanged as time flows. This observation leads to the following definition of the extended velocity $\tilde{c}$:

$$ \tilde{c}(x, y, v, v_x, v_y) = c(x - d(v) \frac{v_x}{|\nabla v|}, y - d(v) \frac{v_y}{|\nabla v|}), $$

$d$ being a distance function. The adaptive filtered scheme is then introduced, composed of two intermediate schemes —the monotone one and the high-order one —related by a filter function $F$ that switches between the two schemes according to smoothness indicators.

# 4 Joint Segmentation and Registration Models

Segmentation and registration are cornerstone steps in many image processing chains, which combined, can significantly improve the accuracy of both processes.

## 4.1 Motivations

Like segmentation, registration can be achieved with a large variety of methodologies. In [130], Sotiras et al. provide an extensive overview of existing registration techniques in a systematic manner, by identifying the main components they consider to be part of a registration algorithm, namely the deformation model —or how the deformation is viewed —, the cost function designed to enforce the shape matching, and the optimisation technique adopted to complete the minimisation. The deformation $\varphi$ to be searched is viewed as a minimiser (uniqueness defaults in general) of a specifically designed cost function, the problem being mathematically hard to solve, due to its under-constrained, nonlinear and non convex nature, and its strong dependency on the considered application. For instance, when the images are of different modalities, the quality of registration is no longer assessed by intensity distribution alignment, but by the measurement of shape and geometric feature matching, requiring to design specific metrics. According to Sotiras and collaborators, an image registration algorithm consists of three main components:
(i) a deformation model describing the setting in which the objects to be matched are interpreted and viewed, and allowing to favor certain properties of the deformation: physical models, purely geometric models, models including a priori knowledge, etc.;

(ii) a cost function which generally comprises two terms: a first one quantifying the misalignment between the deformed template and the reference, and the second one regularizing the deformation, regularisation prescribing the nature of the deformation;

(iii) an optimization method.

The deformation model, which is thus the first ingredient, actually motivates the way the deformation $\varphi$ is built in order to apply to a specific task:

(i) by analogy with physical models: for instance, elastic models ([24]) in which the shapes to be matched are considered as the observations of the same body before and after being subject to constraints, fluid models ([38]) in which the shapes to be matched are viewed as fluids evolving in accordance with Navier-Stokes equations, diffusion models ([58]), curvature models ([59]), flows of diffeomorphisms ([14]) and nonlinear models ([26], [52], [53], [86], [122], [114], [111]) to allow for large deformations;

(ii) by interpolation or approximation driven models: it means that the deformation is described in a parameterizable set. The displacements are considered to be known on a restricted set and are then extrapolated or approximated on the whole domain. The family of interpolation strategies includes: radial basis functions ([156]), elastic body splines ([49]), free-form deformations ([124]), basis functions from signal processing ([6]), and piecewise affine models. These models are rich enough to describe the transformations, while having low degrees of freedom;

(iii) by including a priori knowledge (through conditioning statistically image matching or biomechanical/biophysical models - for instance, tumor growth model or biomechanical model of breast tissue ([41]) -) or shape a priori in order to penalize configurations that diverge too much from it.

Additional constraints can be applied in order for the deformation to exhibit suitable properties such as topology or orientation preservation (one-to-one property of the deformation) ([78], [38], [103], [105]), symmetry, inverse consistency (which means that interchanging the Template and the Reference should not impact on the produced result) ([151]), volume preservation ([70]), lower and upper bounds on the Jacobian determinant ([71]), etc.

The second component of an image registration method is the objective function or the matching criterion, that is, how the available data are exploited to drive the registration process. Ideally, it should be devised in order to comply with the nature of the observations to be registered and should put the emphasis on salient features. There exist numerous types of matching criterion which can be regrouped into three categories as follows:

(i) iconic methods: these concern intensity-based methods, attribute-based methods and information - theoretic approaches;
(ii) geometric methods: they aim to establish correspondences between landmarks (reliable anatomical locations for instance);

(iii) hybrid methods that summarize both types of approaches.

Finally, the last component is the optimization method, consisting of the following
types:

(i) continuous methods in which the variables are assumed to take real values and the objective function to be differentiable: gradient descent ([14]), conjugate gradient ([98]), Newton-type methods, Levenberg-Marquardt, and stochastic gradient descent methods ([146]);

(ii) discrete methods (contrary to continuous methods, they perform a global search and exhibit better convergence rates than the continuous methods): graph-based ([135]), belief propagation and linear programming methods;

(iii) miscellaneous methods: greedy approaches and evolutionary algorithms.

For images including several objects, registration cannot just track the changes of a
particular one. Yet, in some applications, we are only interested in tracing only one
of these objects, resulting in a linear process of the two tasks: segmentation should
be achieved first, and then registration, meaning that segmentation and registration
are processed sequentially, one task after another, without correlating them, which
in practice may propagate errors from step to step. Still, as structure/salient compo-
nent/shape/geometrical feature matching and intensity distribution comparison rule
registration, combining the segmentation and registration tasks into a single frame-
work sounds relevant. Beyond the fact it may reduce propagation of uncertainty,
jointly performing these tasks yields positive mutual influence and benefit on the
obtained results as exemplified in Figure 7. Accurate segmented structures allow to
drive the registration process correctly, providing then a reliable deformation be-
tween the encoded structures, not only based on intensity distribution comparison
(local criterion) but also on geometrical and topological features (nonlocal feature)
and edge transfer —thus diminishing the influence of noise—. Besides, registration
can be viewed as the incorporation of prior information to guide the segmentation
process, in particular for the questions of topology preservation (the unknown de-
formation is substituted for the classical evolving contour and the related Jacobian
determinant is subject to positivity constraints) and geometric priors (since the reg-
istration allows to overcome the issue of weak boundaries). An overview of prior
related works dedicated to joint segmentation and registration is produced in the next
section.

**4.2 Overview of existing methods**

Several scientific works suggest to combine segmentation and registration to take
advantage of both processes. In [153], Yezzi et al., propose performing jointly
segmentation and registration. Denoting by $R: \Omega \subset \mathbb{R}^2 \to \mathbb{R}$ and $T = \bar{R}: \bar{\Omega} \subset \mathbb{R}^2 \to \mathbb{R}$ the two images containing a common object to be registered and segmented,
their goal is to find a closed curve $C \subset \Omega$ which captures the boundary of an object in image $R$ and another closed curve $\tilde{C} \subset \tilde{\Omega}$ which captures the boundary of the corresponding object in image $\tilde{R}$, these closed curves being related through the mapping $g : \mathbb{R}^2 \to \mathbb{R}^2$, $g \in G$ (finite dimensional group) by $\tilde{C} = g(C)$. While the fidelity criterion is defined in terms of a region-based energy, the regularizer is based on the mean curvature flow in order to ensure that the evolving contour remains smooth. A generalization of this model is introduced in [140].

In [141], Vemuri et al. propose a coupled PDE model to perform both segmentation and registration. In the first PDE, the level sets of the source image (i.e. the Template) are evolved along their normals with a speed defined as the difference between the target (i.e. the Reference) and the evolving source image. In [86], Le Guyader and Vese introduce a segmentation model based on the active contour without edges model that is now solved using registration techniques. The shapes to be matched are viewed as Ciarlet-Geymonat materials and are implicitly modelled by level set functions. Denoting by $\Omega$ a connected bounded open subset of $\mathbb{R}^3$ with Lipschitz boundary $\partial\Omega$, by $R$ the Reference image, and modelling the shape of the Template image $T$ via a Lipschitz continuous function $\Phi_0$ whose zero level line corresponds to this shape boundary, the problem consists in searching for a deformation $\bar{\varphi} \in W$ with

$$ W = \{ \Psi \in W^{1,4}(\Omega, \mathbb{R}^3), \text{Cof} \nabla \Psi \in L^2(\Omega, M_3(\mathbb{R})), \\ \det \nabla \Psi \in L^2(\Omega), \Psi = \Psi_0 \text{ on } \partial\Omega \text{ and } \det \nabla \Psi > 0 \text{ a.e.} \} $$

realising the minimum of

$$ I(\varphi) = v_1 \int_{\Omega} (R - c_1(\varphi))^2 H(\Phi_0(\varphi)) dx + v_2 \int_{\Omega} (R - c_2(\varphi))^2 (1 - H(\Phi_0(\varphi))) dx \\ + \int_{\Omega} W_{CG}(\nabla \varphi) dx, $$

with

$$ c_1(\varphi) = \frac{\int_{\Omega} R H(\Phi_0(\varphi)) dx}{\int_{\Omega} H(\Phi_0(\varphi)) dx}, \quad c_2(\varphi) = \frac{\int_{\Omega} R (1 - H(\Phi_0(\varphi))) dx}{\int_{\Omega} (1 - H(\Phi_0(\varphi))) dx}, $$

and $W_{CG}$ standing for the Ciarlet-Geymonat stored energy function. It is defined by $W_{CG}: F \in M_3^+(\mathbb{R}) = \{A \in M_3(\mathbb{R}) | \det A > 0\} \mapsto W_{CG}(F) = a_1 \|F\|^2 + a_2 \|F^T F\|^2 + b \|\text{Cof } F\|^2 + \Gamma(\det F) + e$, $M_3(\mathbb{R})$ being the set of real square matrices of order 3, $\text{Cof } F$ denoting the cofactor matrix of $F$, $\| \cdot \|$ the Frobenius norm, and $\Gamma$ being a convex function which penalises expansions and contractions of the deformation that are too large. Following [86], the model by [76] proposed a simple extension by adopting a high order regulariser for the deformation field that offers advantages for certain classes of problems.
In [93], Lord et al. present a unified model that simultaneously treats segmentation and registration based on metric structure comparisons (see also the more recent work [65] based on Expectation Maximization algorithm that incorporates a glioma growth model for atlas seeding). In [5], the authors propose a new variational PDE-based level set method for a simultaneous image segmentation and non-rigid registration (the expected transformation is not parametric, i.e., is not expanded in some basis functions) using prior shape and intensity information. While the segmentation is obtained by determining a non-rigid deformation of the prior shape, the non-rigid registration consists of both a global rigid transformation (transformations are restricted to rotations and translations) and a local non-rigid deformation. A joint segmentation and registration algorithm for infant brain images with the goal to accurately characterise structure changes is presented in [149]. The emphasis is put on the interest of combining both tasks owing to dynamic appearance change with rapid brain development. In [66], relevant accurate segmented structures allow to drive correctly the registration process. The proposed model integrates both the active contour framework and the dense deformation fields of optical flow framework. In [54], Droske et al. aim to match the edges and the normals of the two images by applying a Mumford-Shah type free discontinuity problem. More recently, Ozeré et al. ([110]) have introduced a variational joint segmentation/registration model combining a measure of dissimilarity based on weighted total variation and a regularizer based on the stored energy function of a Saint Venant-Kirchhoff material. In the same spirit, Debroux et al. ([51]) provide a nonlocal topology-preserving segmentation guided registration model, theoretically well-motivated and capable of handling large and smooth deformations. The shapes to be matched are viewed as hyperelastic materials and more precisely as Saint Venant-Kirchhoff ones, and are implicitly modelled by level set functions. These are driven to minimise a functional containing both a nonlinear-elasticity-based regularizer prescribing the nature of the deformation, and a criterion that forces the evolving shape to match intermediate topology-preserving segmentation results. In [20], a joint segmentation/optimal transport model is analyzed to determine the velocity of blood flow in vascular structures. A convex variational method is used and primal-dual proximal splitting algorithms are implemented. At last, in [148], the author wonders about the behavior of phase field approximations of the Mumford-Shah model when used for joint segmentation and registration.

We conclude this section with a special focus on a recent combined segmentation/registration framework.

## 4.3 A mixed segmentation/registration model based on a nonlocal characterisation of weighted total variation

In 2019, Debroux and Le Guyader [50] propose a unified variational model in a hyperelasticity setting. The dissimilarity measure relates local and global (region-based) information, since relying on the weighted total variation and nonlocal shape
descriptors inspired by the piecewise constant Mumford-Shah model. Including
the weighted total variation enables one to consider a larger class of images (not
necessarily of the same modality) and to compare shapes (alignment of the level
curves) rather than intensities. Also, in practice, the obtained deformed Templates
are more consistent with the complex topologies/thin structures involved. In addition
to theoretical results (existence of minimisers, connection to the segmentation step,
etc.), a nonlocal characterisation of weighted semi-norms is provided as well as
asymptotic results and Γ-convergence properties. We now show some details of this
model. Assume Ω ⊂ R² is of class C¹. Denote by R : Ω → R the Reference image
assumed to be sufficiently smooth and by T : Ω → R the Template image. We
assume that T is compactly supported on Ω to ensure that T ∘ ϕ is always defined
and we assume that T is Lipschitz continuous. It can thus be considered as an
element of the Sobolev space W¹,∞(R²). Let ϕ : Ω → R² be the sought deformation
supposed to be a smooth orientation-preserving mapping. The deformation gradient
is ∇ϕ : Ω → M₂(R), the set M₂(R) being the set of real square matrices of order 2.
The deformation to be searched ϕ is seen as the minimal argument of a specifically
designed objective function including a regularisation on ϕ prescribing the nature
of the deformation and which is modelled by the component ∫Ω QW(∇ϕ) dx in the
functional to be minimised (—its design is more precisely motivated hereafter —),
and a term measuring alignment or how the available data are exploited to drive the
registration process. This later one is itself decomposed into three components: a
first one ensuring alignment of the edges of both the deformed Template T ∘ ϕ and the
Reference and expressed in terms of the weighted total variation var_g T ∘ ϕ, a second
one which is merely the L²-fidelity term ||R - T ∘ ϕ||² L²(Ω) insuring intensity pairing,
and a last one, ∫Ω [(R - c₁)² - (R - c₂)²] T ∘ ϕ dx, inspired by the work of Bresson
et al. ([23]) which guarantees region matching. Again, the design of these terms is
justified below. To allow large deformations, the shapes to be matched are viewed
as hyperelastic materials, and more precisely as Saint Venant-Kirchhoff ones ([40]).
This outlook rules the design of the regularisation on ϕ which is thus based on the
stored energy function of a Saint Venant-Kirchhoff material. We recall that the right
Cauchy-Green strain tensor (viewed as a quantifier of the square of local change in
distances due to deformation) is defined by C = ∇ϕT ∇ϕ = FTF. The Green-Saint
Venant strain tensor is defined by E = ½ (C − I). Associated with a given deformation
ϕ, it is a measure of the deviation between ϕ and a rigid deformation. We also need
the following notations: A : B = trA²B, the matrix inner product and ||A|| = √A : A,
the related matrix norm (Frobenius norm). The stored energy function of a Saint
Venant-Kirchhoff material is defined by W<sub>SVK</sub>(F) = ̂W(E) = λ/2 (tr E)² + μ tr E², λ
and μ being the Lamé coefficients. To ensure that the distribution of the deformation
Jacobian determinants does not exhibit contractions or expansions that are too large
and to avoid singularity as much as possible, we complement the stored energy
function W<sub>SVK</sub> by the term μ (det F − 1)² controlling that the Jacobian determinant
remains close to 1. The weighting of the determinant component by parameter μ
allows to recover a property of convexity for the function Ψ introduced later. (Note
that the stored energy function W<sub>SVK</sub> alone lacks a term penalizing the determinant:
it does not preclude deformations with negative Jacobian. The expression of its
quasiconvex envelope is more complex since involving explicitly the singular values of $F$. Also, when they are all lower than 1, the quasiconvex envelope equals 0, which shows bad behavior under compression). Therefore, the regularisation can be written, after intermediate computations, as $W(F) = \beta(\|F\|^2 - \alpha)^2 - \frac{\mu}{2}(\det F)^2 + \mu(\det F - 1)^2 + \frac{\mu(\lambda+\mu)}{2(\lambda+2\mu)}$, where $\alpha = 2\frac{\lambda+\mu}{\lambda+2\mu}$ and $\beta = \frac{\lambda+2\mu}{8}$. Although meaningful, function $W$ takes on a drawback since it is not quasiconvex (see [48, Chapter 9] for a complete review of this notion), which raises an issue of a theoretical nature since we cannot obtain the weak lower semi-continuity property. The idea is thus to replace

$$W \text{ by its quasiconvex envelope defined by } QW(\xi) = \begin{cases} W(\xi) & \text{if } \|\xi\|^2 \ge 2\frac{\lambda+\mu}{\lambda+2\mu}, \\ \Psi(\det \xi) & \text{if } \|\xi\|^2 < 2\frac{\lambda+\mu}{\lambda+2\mu}, \end{cases}$$

and $\Psi$, the convex mapping such that $\Psi : t \mapsto -\frac{\mu}{2}t^2 + \mu(t-1)^2 + \frac{\mu(\lambda+\mu)}{2(\lambda+2\mu)}$ (see [110] for the derivation), for which the minimal argument is $t=2$. The regularizer is now complemented by a dissimilarity measure inspired by the unified model of image segmentation and denoising introduced by Bresson et al. ([23]), designed to overcome the limitation of local minima and to deal with global minimum.

In that purpose, let $g: \mathbb{R}^+ \to \mathbb{R}^+$ be an edge detector function satisfying $g(0) = 1$, $g$ strictly decreasing and $\lim_{r \to +\infty} g(r) = 0$. From now on, we set $g := g(|\nabla R|)$ and for theoretical purposes, we assume that $\exists c > 0$ such that $0 < c \le g \le 1$ and that $g$ is Lipschitz continuous. We then use the generalization of the notion of function of bounded variation to the setting of BV-spaces associated with a Muckenhoupf's weight function depicted in [12]. We follow Baldi's arguments and notations to define the weighted BV-space related to weight $g$.

For a general weight $w$, some hypotheses are required (fulfilled here by $g$). More precisely, $\Omega_0$ being a neighborhood of $\bar{\Omega}$, the positive weight $w \in L_{\text{loc}}^1(\Omega_0)$ is assumed to belong to the global Muckenhoupf's $A_1 = A_1(\Omega)$ class of weight functions, i.e., $w$ satisfies the condition:

$$C w(x) \geq \frac{1}{|B(x,r)|} \int_{B(x,r)} w(y) dy \quad \text{a.e.} \quad (4)$$

in any ball $B(x, r) \subset \Omega_0$. Now, denoting by $A_1^*$ the class of weights $w \in A_1$, $w$ lower semicontinuous (Isc) and that satisfy condition (4) pointwise, the definition of the weighted BV-space related to weight $w$ is given by:

**Definition 1 ([12, Definition 2])** Let $w$ be a weight function in the class $A_1^*$. We denote by $BV(\Omega, w)$ the set of functions $u \in L^1(\Omega, w)$ (set of functions that are integrable with respect to the measure $w(x) dx$) such that:

$$\sup \left\{ \int_{\Omega} u \operatorname{div}(\varphi) dx : |\varphi| \le w \text{ everywhere, } \varphi \in \operatorname{Lip}_0(\Omega, \mathbb{R}^2) \right\} < \infty, \quad (5)$$

with $\operatorname{Lip}_0(\Omega, \mathbb{R}^2)$ the space of Lipschitz continuous functions with compact support. We denote by $\operatorname{var}_w u$ the quantity (5).
Remark 1 In [12], Baldi defines the BV-space taking as test functions elements of Lip$_0$(Ω, R$^2$). Classically in the literature, the test functions are chosen in C$^1_c(Ω, R^2)$. It can be proved that these two definitions coincide thanks to mollifications and density results.

To explain (5), we give the following result [12, Remark 10]:

Remark 2 Given a weight w sufficiently smooth, if E is a regular bounded open set in R$^2$, with boundary of class C$^2$, then |∂E|(Ω, w) = var$_w$ χ$_E = ∫Ω∩∂E w dH$^1$, which can be interpreted in the case where w = g as a new definition of the curve length with a metric that depends on the Reference image content.

Equipped with this material (—and due to the properties of function g: it is obviously L$^1$, continuous and it suffices to take C = 1c to satisfy (4) pointwise—), we propose introducing as dissimilarity measure the following functional:

$$
\begin{equation}
\begin{split}
W_{fid}(\varphi) = \operatorname{var}_g T \circ \varphi &+ \frac{\nu}{2} \int_{\Omega} (T \circ \varphi(x) - R(x))^2 dx \\
&+ a \int_{\Omega} \left[ (c_1 - R(x))^2 - (c_2 - R(x))^2 \right] T \circ \varphi(x) dx,
\end{split}
\tag{6}
\end{equation}
$$

with $c_1 = \frac{\int_{\Omega} R(x) H_{\varepsilon}(T \circ \varphi(x)-\rho) dx}{\int_{\Omega} H_{\varepsilon}(T \circ \varphi(x)-\rho) dx}$ and $c_2 = \frac{\int_{\Omega} R(x)(1-H_{\varepsilon}(T \circ \varphi(x)-\rho)) dx}{\int_{\Omega} (1-H_{\varepsilon}(T \circ \varphi(x)-\rho)) dx}$ —we dropped the dependency on $\varphi$ to lighten the expressions —, $H_{\varepsilon}$ denoting a regularisation of the Heaviside function and $\rho \in [0,1]$ being a fixed parameter allowing to partition $T \circ \varphi$ into two phases and yielding a binary version of the Reference. $\rho$ can be estimated by analyzing the Reference histogram to discriminate two relevant regions or phases. For instance, through histogram shape-based methods, clustering-based methods, entropy-based methods, object attribute-based methods, spatial methods, or local methods ([127]). This proposed functional emphasizes the link between the geodesic active contour model ([28]) and the piecewise-constant Mumford-Shah model ([102]): if $\tilde{T}$ is the characteristic function of the set $\Omega_C$, bounded subset of $\Omega$ with regular boundary $C$, $\text{var}_g \tilde{T}$ is a new definition of the length of $C$ with a metric depending on the Reference content (so minimising this quantity is equivalent to locating the curve on the boundary of the shape contained in the Reference), while $\int_{\Omega} [(c_1 - R(x))^2 - (c_2 - R(x))^2] \tilde{T}(x) dx$ approximates $R$ in the $L^2$ sense by two regions $\Omega_C$ and $\Omega \setminus \Omega_C$ with two values $c_1$ and $c_2$. Indeed, $\text{var}_g \tilde{T} = \int_{\Omega \cap C} g dH^1$, and if $c_1$ and $c_2$ are fixed (which is in practice the case in the alternating algorithm), $\int_{\Omega} [(c_1 - R(x))^2 - (c_2 - R(x))^2] 1_{\Omega_C} dx$ is equivalent to minimising $\int_{\Omega} (c_1 - R(x))^2 1_{\Omega_C} dx + \int_{\Omega} (c_2 - R(x))^2 1_{\Omega \setminus \Omega_C} dx$.

In the end, the global minimisation problem denoted by (QP) —that stands for Quasiconvex Problem— is stated by:

$$
\inf_{\varphi \in W = \text{Id} + W_0^{1,4}(\Omega, \mathbb{R}^2)} \bar{I}(\varphi) = W_{\text{fid}}(\varphi) + \int_{\Omega} QW(\nabla \varphi) \, dx \quad (\text{QP})
$$

which is a relaxed problem from the following formulation
$$ \inf_{\varphi \in W = \text{Id} + W_0^{1,4}(\Omega, \mathbb{R}^2)} \bar{I}(\varphi) = W_{\text{fid}}(\varphi) + \int_{\Omega} W(\nabla \varphi) \, dx. \quad (P) $$

Here $\varphi \in \text{Id} + W_0^{1,4}(\Omega, \mathbb{R}^2)$ means that $\varphi = \text{Id}$ on $\partial\Omega$ and $\varphi \in W^{1,4}(\Omega, \mathbb{R}^2)$. $W^{1,4}(\Omega, \mathbb{R}^2)$ denotes the Sobolev space of functions $\varphi \in L^4(\Omega, \mathbb{R}^2)$ with distributional derivatives up to order 1 which also belong to $L^4(\Omega)$. $W$ is a suitable space due, in particular, to the $\|F\|^4$ component in $W(F)$. Note that from generalized Hölder's inequality, if $\varphi \in W^{1,4}(\Omega, \mathbb{R}^2)$, then $\det \nabla \varphi \in L^2(\Omega)$. Now we justify that $\text{var}_g T \circ \varphi$ is well-defined. In [4], Ambrosio and Dal Maso prove a general chain rule for the distribution derivatives of the composite function $v(x) = f(u(x))$, where $u: \mathbb{R}^n \to \mathbb{R}^m$ has bounded variation and $f: \mathbb{R}^m \to \mathbb{R}^k$ is Lipschitz continuous. A simpler result is given when $u \in W^{1,p}(\Omega, \mathbb{R}^m)$ for some $p, 1 \le p \le +\infty$, resulting in our case in $T \circ \varphi \in W^{1,4}(\Omega) := W^{1,4}(\Omega, \mathbb{R}) \subset BV(\Omega) \subset BV(\Omega, g)$, since $g \le 1$. Some theoretical results can be found in [50] among which, existence of minimisers for problem (QP), connection between the minimum of (QP) and the infimum of (P), and derivation of a related nonlocal problem, motivated by the strength and robustness of nonlocal methods exemplified in many image processing tasks such as image denoising, color image deblurring in the presence of Gaussian or impulse noise, color image inpainting, color image super-resolution, or color filter array demosaicing ([77]). The next part is dedicated to the derivation of a nonlocal counterpart of problem (QP). In practice, in terms of quantitative and qualitative accuracy, this nonlocal model gives better results than those obtained with the local one, with higher Dice coefficients, in particular when the shapes to be matched exhibit fine details or complex topologies.

The statement of the nonlocal problem relies on the following nonlocal approximation of the weighted total variation (or nonlocal weighted BV semi-norm) by a sequence of integral operators involving a differential quotient and a radial mollifier sequence. It is inspired by prior works by Dávila and Ponce dedicated to the design of nonlocal counterparts of Sobolev and BV semi-norms. Let $(\rho_n)_{n \in \mathbb{N}}$ be a sequence of radial mollifiers satisfying: $\forall n \in \mathbb{N}, \forall x \in \mathbb{R}^2, \rho_n(x) = \rho_n(|x|); \forall n \in \mathbb{N}, \rho_n \ge 0; \forall n \in \mathbb{N}, \int_{\mathbb{R}^2} \rho_n(x) dx = 1; \forall \delta > 0, \lim_{n \to +\infty} \int_{\delta}^{+\infty} \rho_n(r) r dr = 0$. Then the following theorem holds:

**Theorem 2** Let $\Omega \subset \mathbb{R}^2$ be an open bounded set with Lipschitz boundary and let $f \in BV(\Omega, g) \subset BV(\Omega)$ as $0 < c \le g \le 1$ everywhere. Consider $(\rho_n)_{n \in \mathbb{N}}$ defined previously. Then

$$ \lim_{n \to +\infty} \int_{\Omega} g(x) \left[ \int_{\Omega} \frac{|f(x) - f(y)|}{|x - y|} \rho_n(x - y) dy \right] dx \\ = \left[ \frac{1}{|S^1|} \int_0^{2\pi} \left| e \cdot \left( \begin{array}{c} \cos(\theta) \\ \sin(\theta) \end{array} \right) \right| d\theta \right] \text{var}_g f = K_{1,2} \text{var}_g f, $$

with $e$ being any unit vector of $\mathbb{R}^2$ and $S^1$ being the unit sphere in $\mathbb{R}^2$.

Motivated by the asymptotic properties of the above nonlocal quantity, we propose using this characterisation as a substitute for the weighted total variation of $T \circ \varphi$,
yielding the following NonLocal Problem (NLP) :

$$
\begin{equation}
\begin{aligned}
& \inf_{\varphi \in \mathrm{Id} + W_0^{1,4}(\Omega, \mathbb{R}^2)} \left\{ E_n(\varphi) = \frac{1}{K_{1,2}} \int_{\Omega} g(x) \left[ \int_{\Omega} \frac{|T \circ \varphi(y) - T \circ \varphi(x)|}{|x-y|} \rho_n(x-y) \, dy \right] dx \right. \\
& \qquad \left. + a \int_{\Omega} [(c_1 - R)^2 - (c_2 - R)^2] T \circ \varphi \, dx \right. \\
& \qquad \left. + \frac{\nu}{2} \|T \circ \varphi - R\|_{L^2(\Omega)}^2 + \int_{\Omega} QW(\nabla \varphi) \, dx \right\}.
\end{aligned}
\tag{NLP}
\end{equation}
$$

In addition to an existence result for problem (NLP), the authors provide a Γ-convergence theorem relating the approximated problem to the original one when n tends to +∞ as well as a numerical approximation based on the introduction of two auxiliary (i.e. splitting) variables, T̃, simulating T ◦ ϕ and V, approximating ∇ϕ, the underlying idea being to transfer the nonlinearity on V. An asymptotic result is stated ([50, Theorem 3.4]), together with a result ([50, Theorem 3.6]) relating segmentation and registration. The method has then been applied on MRI images of a patient cardiac cycle (courtesy of Caroline Petitjean, LITIS, University of Rouen, France) : the Reference corresponds to end diastole (ED), i.e. when the heart is the most dilated, while the Template corresponds to end systole (ES), i.e. when the heart is the most contracted. In Figure 7, we provide the Reference R, the Template T, the binary Reference obtained thanks to c₁ and c₂ which is rescaled to 0-1 from the nonlocal numerical method, the deformed template, the deformation grid which does not exhibit any overlap (yielding thus the physical well-definedness of the deformation), T̃ the simplified version of the deformed Template, the segmentation of the Reference obtained thanks to T̃, and the oscillatory part resulting from R − T̃.

<figure><img src="image_10.png" /><figcaption>Fig. 7: Mapping of cardiac MRI images (ED-ES) (size : 150 × 150).</figcaption></figure>
# 5 Other Related Models

The topics related to segmentation and registration are huge. The area is active and fast growing. We only briefly mention a few directions.

## 5.1 Optimal flow frameworks

When a sequence of images $z_1, z_2, ...$ are given e.g. from functional MRI or from frames of a video image, segmented objects in $z_1$ are hoped to be registered to the evolved features in subsequent images. This may be realised by optimal flow registration methods or by joint segmentation and registration models [50, 25, 42].

## 5.2 Shape priors

Given a shape prior $\psi_0$ intended for an image $z$, there exist several models in the literature trying to segment an object $\psi$ in $z$ that is some transformed version of $\psi_0$. This seemingly simple and useful task is highly non-trivial to realise, unless $\psi$ is a parametric (e.g. affine) transform of $\psi_0$; see [46, 69] and many references therein. One fundamental challenge is that registration models such as (QP) are highly capable to transform one shape to another [51] and hence if not constrained, registration would attempt to find a match of objects by essentially ignoring the given shape prior.

## 5.3 Deep learning models

Deep learning models have been extremely popular for solving models in segmentation, registration or joint segmentation and registration [55, 150]. In fact, supervised learning for image segmentation and unsupervised learning for image registration are widely used for various image applications. Current and emerging works show novelties in terms of new network designs for segmentation while of new energy (loss) functions.

## 5.4 Multi-modal problems

Segmentation of multi-modal images as a sole task seems to pose no particular or additional challenges, although segmentation of an arbitrary image is an unfinished business due to various inherent difficulties such as low contrast, strong noise,
possibly non-periodic textures and missing data. Among many competing models, unsurprisingly, deep learning methods [134] are increasingly used. Moreover, one can make use of deep learning ideas for multi-modal images to obtain a more accurate segmentation through fusion [159].

The related task of image registration for multi-modal images is particularly challenging if one wishes to have robustness. A particularly reliable method suitable for registering multi-modal images is the use of quasi-conformal maps [87] if landmark points could be identified. If it is not possible to find reliable landmark points, the design of suitable dissimilarity measures to discriminate objects across modalities is a key. Refer to [37, 136].

Once a dissimilar measure is defined, development of a registration model and also a joint segmentation and registration model can follow from works for single modality images [51].

## 6 Conclusion

By covering a broad spectrum of constraint types, whether it be geometrical constraints to identify a single object among several ones or topological conditions to ensure that the segmented shape is homeomorphic to the initial one, we have intended in this survey, to show the utility of including such additional a priori information to achieve more accurate results, in compliance with the physics of the problem or the anatomical reality. Still, however abundant the literature is on this topic, scientific obstacles remain in particular in the way to reconcile the intrinsic global nature of topology with its more local one, which gives tremendous prospects for the future.

## References

1. Alberti, G., Bouchitté, G., Dal Maso, G.: The calibration method for the Mumford-Shah functional and free-discontinuity problems. Calculus of Variations and Partial Differential Equations **16**(3), 299–333 (2003)

2. Alexandrov, O., Santosa, F.: A topology-preserving level set method for shape optimization. Journal of Computational Physics **204**(1), 121–130 (2005)

3. Alvarez, L., Cuenca, C., Díaz, J.I., González, E.: Level Set Regularization Using Geometric Flows. SIAM Journal on Imaging Sciences **11**(2), 1493–1523 (2018)

4. Ambrosio, L., Dal Maso, G.: A general chain rule for distributional derivatives. Proc. Amer. Math. Soc. **108**(3), 691–702 (1990)

5. An, J.H., Chen, Y., Huang, F., Wilson, D., Geiser, E.: A variational PDE based level set method for a simultaneous segmentation and non-rigid registration. In: J.S. Duncan, G. Gerig (eds.) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2005: 8th International Conference, Palm Springs, CA, USA, October 26-29, 2005, Proceedings, Part I, pp. 286–293. Springer Berlin Heidelberg (2005)

6. Ashburner, J., Friston, K.J.: Nonlinear spatial normalization using basis functions. Human brain mapping **7**(4), 254–266 (1999)
7. Astala, K., Iwaniec, T., Martin, G.: Elliptic Partial Differential Equations and Quasiconformal Mappings in the Plane. Princeton University Press (2009)

8. Aubert, G., Kornprobst, P.: Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations. Applied Mathematical Sciences. Springer-Verlag (2001)

9. Ayed, I.B., Li, S., Islam, A., Garvin, G., Chhem, R.: Area prior constrained level set evolution for medical image segmentation. In: J.M. Reinhardt, J.P.W. Pluim (eds.) Medical Imaging 2008: Image Processing, vol. 6914, pp. 27–32. SPIE (2008)

10. Badshah, N., Chen, K.: Image selective segmentation under geometrical constraints using an active contour approach. Communications in Computational Physics **7**, 759–778 (2010)

11. Bai, X., Sapiro, G.: A Geodesic Framework for Fast Interactive Image and Video Segmentation and Matting. In: 2007 IEEE 11th International Conference on Computer Vision, pp. 1–8 (2007)

12. Baldi, A.: Weighted BV functions. Houston J. Math. **27**(3), 683–705 (2001)

13. Barrett, W., Mortensen, E.N.: Interactive live-wire boundary extraction. Medical Image Analysis **1**(4), 331–341 (1997)

14. Beg, M., Miller, M., Trouvé, A., Younes, L.: Computing Large Deformation Metric Mappings via Geodesic Flows of Diffeomorphisms. International Journal of Computer Vision **61**(2), 139–157 (2005)

15. Ben-Zadok, N., Riklin-Raviv, T., Kiryati, N.: Interactive level set segmentation for image-guided therapy. In: IEEE International Symposium on Biomedical Imaging (ISBI), pp. 1079–1082 (2009)

16. Bertrand, G.: Simple points, topological numbers and geodesic neighborhoods in cubic grids. Pattern Recognition Letters **15**(10), 1003–1011 (1994)

17. Bertrand, G.: A Boolean characterization of three-dimensional simple points. Pattern Recognition Letters **17**(2), 115–124 (1996)

18. Blake, A., Zisserman, A.: Visual Reconstruction. MIT Press, Cambridge (1989)

19. Bogovic, J.A., Prince, J.L., Bazin, P.L.: A multiple object geometric deformable model for image segmentation. Computer Vision and Image Understanding **117**(2), 145–157 (2013)

20. Boink, Y.: Combined modelling of optimal transport and segmentation revealing vascular properties (2016)

21. Boutry, N., Géraud, T., Najman, L.: A tutorial on Well-Composedness. Journal of Mathematical Imaging and Vision **60**(3), 443–478 (2018)

22. Boykov, Y.Y., Jolly, M.: Interactive graph cuts for optimal boundary map; region segmentation of objects in N-D images. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, vol. 1, pp. 105–112 1 (2001)

23. Bresson, X., Esedoğlu, S., Vandergheynst, P., Thiran, J.P., Osher, S.: Fast global minimization of the active contour/snake model. J. Math. Imaging Vis. **28**(2), 151–167 (2007)

24. Broit, C.: Optimal registration of Deformed Images. Ph.D. thesis, Computer and Information Science, University of Pennsylvania (1981)

25. Brox, T., Malik, J.: Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence **33**(3), 500–513 (2010)

26. Burger, M., Modersitzki, J., Ruthotto, L.: A hyperelastic regularization energy for image registration. SIAM Journal on Scientific Computing **35**(1), B132–B148 (2013)

27. Cai, X., Chan, R., Zeng, T.: A two-stage image segmentation method using a convex variant of the Mumford–Shah model and thresholding. SIAM Journal on Imaging Sciences **6**(1), 368–390 (2013)

28. Caselles, V., Kimmel, R., Sapiro, G.: Geodesic Active Contours. International Journal of Computer Vision **22**(1), 61–79 (1997)

29. Cecil, T.: Numerical methods for partial differential equations involving discontinuities. Ph.D. thesis, Dept. Math., Univ. California, Los Angeles (2003)

30. Chambolle, A., Cremers, D., Pock, T.: A Convex Approach to Minimal Partitions. SIAM Journal on Imaging Sciences **5**(4), 1113–1158 (2012)
31. Chan, H.L., Yan, S., Lui, L.M., Tai, X.C.: Topology-Preserving Image Segmentation by Beltrami Representation of Shapes. Journal of Mathematical Imaging and Vision **60**(3), 401–421 (2018)

32. Chan, T.F., Esedoglu, S., Nikolova, M.: Algorithms for finding global minimizers of image segmentation and denoising models. SIAM Journal on Applied Mathematics **66**(5), 1632–1648 (2006)

33. Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Transactions on Image Processing **10**(2), 266–277 (2001)

34. Chen, C., Freedman, D.: Topology Noise Removal for Curve and Surface Evolution. In: B. Menze, G. Langs, Z. Tu, A. Criminisi (eds.) Medical Computer Vision. Recognition Techniques and Applications in Medical Imaging, pp. 31–42. Springer Berlin Heidelberg, Berlin, Heidelberg (2011)

35. Chen, C., Freedman, D., Lampert, C.H.: Enforcing topological constraints in random field image segmentation. In: CVPR 2011, pp. 2089–2096 (2011)

36. Chen, D., Zhang, J., Cohen, L.D.: Minimal paths for tubular structure segmentation with coherence penalty and adaptive anisotropy. IEEE Transactions on Image Processing **28**(3), 1271–1284 (2019)

37. Chen, K., Lui, L.M., Modersitzki, J.: Image and surface registration. In: Elsevier Handbook of Numerical Analysis. Processing, Analyzing and Learning of Images, Shapes, and Forms: Part 2, chap. 15, pp. 579–611. North Holland (2019)

38. Christensen, G., Rabbitt, R., Miller, M.: Deformable Templates Using Large Deformation Kinematics. IEEE Trans. Image Process. **5**(10), 1435–1447 (1996)

39. Chuang, K.S., Tzeng, H.L., Chen, S., Wu, J., Chen, T.J.: Fuzzy C-means clustering with spatial information for image segmentation. Computerized Medical Imaging and Graphics **30**(1), 9–15 (2006)

40. Ciarlet, P.: Elasticité Tridimensionnelle. Masson (1985)

41. Clatz, O., Sermesant, M., Bondiau, P.Y., Delingette, H., Warfield, S.K., Malandain, G., Ayache, N.: Realistic simulation of the 3-D growth of brain tumors in MR images coupling diffusion with biomechanical deformation. IEEE Transactions on Medical Imaging **24**(10), 1334–1346 (2005)

42. Cohen, I.: Nonlinear variational method for optical flow computation. In: Proceedings of the 8th Scandinavian Conference on Image Analysis (SCIA), pp. 523–530. Springer (1993)

43. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence **24**(5), 603–619 (2002)

44. Crandall, M., Ishii, H., P.-L., L.: User's guide to viscosity solutions of second order partial differential equations. Bulletin of the American Mathematical Society **27**(1), 1–67 (1992)

45. Cremers, D., Fluck, O., Roussou, M., Aharon, S.: A probabilistic level set formulation for interactive organ segmentation. In: Medical Imaging 2007: Image Processing, vol. 6512, pp. 304–312. SPIE (2007)

46. Cremers, D., Tischhäuser, F., Weickert, J., Schnörr, C.: Diffusion snakes: Introducing statistical shape knowledge into the Mumford-Shah functional. International Journal of Computer Vision **50**(3), 295–313 (2002)

47. Criminisi, A., Sharp, T., Blake, A.: GeoS: Geodesic Image Segmentation. In: D. Forsyth, P. Torr, A. Zisserman (eds.) Computer Vision – ECCV 2008, pp. 99–112. Springer Berlin Heidelberg, Berlin, Heidelberg (2008)

48. Dacorogna, B.: Direct Methods in the Calculus of Variations, Second Edition. Springer (2008)

49. Davis, M.H., Khotanzad, A., Flamig, D.P., Harms, S.E.: A physics-based coordinate transformation for 3-D image matching. IEEE Transactions on Medical Imaging **16**(3), 317–328 (1997)

50. Debroux, N., Le Guyader, C.: A joint segmentation/registration model based on a nonlocal characterization of weighted total variation and nonlocal shape descriptors. SIAM Journal on Imaging Sciences **11**(2), 957–990 (2018)

51. Debroux, N., Ozeré, S., Le Guyader, C.: A non-local topology-preserving segmentation guided registration model. J. Math. Imaging Vision pp. 1–24 (2017)
52. Derfoul, R., Le Guyader, C.: A relaxed problem of registration based on the Saint Venant-Kirchhoff material stored energy for the mapping of mouse brain gene expression data to a neuroanatomical mouse atlas. SIAM Journal on Imaging Sciences **7**(4), 2175–2195 (2014)

53. Droske, M., Rumpf, M.: A Variational Approach to Non-Rigid Morphological Registration. SIAM J. Appl. Math. **64**(2), 668–687 (2004)

54. Droske, M., Rumpf, M.: Multiscale Joint Segmentation and Registration of Image Morphology. IEEE Transactions on Pattern Analysis and Machine Intelligence **29**(12), 2181–2194 (2007)

55. Estienne, T., Vakalopoulou, M., Christodoulidis, S., Battistela, E., Lerousseau, M., Carre, A., Klausner, G., Sun, R., Robert, C., Mougiakakou, S., Paragios, N., Deutsch, E.: U-ReSNet: Ultimate Coupling of Registration and Segmentation with Deep Nets. In: D. Shen, T. Liu, T.M. Peters, L.H. Staib, C. Essert, S. Zhou, P.T. Yap, A. Khan (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2019, pp. 310–319. Springer International Publishing (2019)

56. Falcone, M., Paolucci, G., Tozza, S.: A High-Order Scheme for Image Segmentation via a modified Level-Set method. arXiv e-prints (2018). URL https://arxiv.org/abs/1812.03026

57. Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient Graph-Based Image Segmentation. International Journal of Computer Vision **59**(2), 167–181 (2004)

58. Fischer, B., Modersitzki, J.: Fast diffusion registration. AMS Contemporary Mathematics, Inverse Problems, Image Analysis, and Medical Imaging **313**, 11–129 (2002)

59. Fischer, B., Modersitzki, J.: Curvature based image registration. J. Math. Imaging Vis. **18**(1), 81–85 (2003)

60. Fischl, B., Liu, A., Dale, A.M.: Automated manifold surgery: constructing geometrically accurate and topologically correct models of the human cerebral cortex. IEEE Transactions on Medical Imaging **20**(1), 70–80 (2001)

61. Fuzhen, H., Xuhong, Y.: Image segmentation under occlusion using selective shape priors. In: A. Campilho, M. Kamel (eds.) Image Analysis and Recognition, pp. 89–95. Springer Berlin Heidelberg, Berlin, Heidelberg (2010)

62. Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., Martinez-Gonzalez, P., Garcia-Rodriguez, J.: A survey on deep learning techniques for image and video semantic segmentation. Applied Soft Computing **70**, 41 – 65 (2018)

63. Geiping, J.A.: Comparison of topology-preserving segmentation methods and application to mitotic cell tracking. Westfälische Wilhelms-Universität Münster (2014)

64. Giga, Y., Goto, S., Ishii, H., Sato, M.H.: Comparison principle and convexity preserving properties for singular degenerate parabolic equations on unbounded domains. Indiana University Mathematics Journal **40**(2), 443–470 (1991)

65. Gooya, A., Pohl, K., Bilello, M., Cirillo, L., Biros, G., Melhem, E., Davatzikos, C.: GLISTR: Glioma Image Segmentation and Registration. IEEE Transactions on Medical Imaging **31**(10), 1941–1954 (2012)

66. Gorthi, S., Duay, V., Bresson, X., Cuadra, M.B., Castro, F.J.S., Pollo, C., Allal, A.S., Thiran, J.P.: Active deformation fields: Dense deformation field estimation for atlas-based segmentation using the active contour framework. Medical Image Analysis **15**(6), 787–800 (2011)

67. Gout, C., Le Guyader, C., Vese, L.A.: Segmentation under geometrical conditions with geodesic active contour and interpolation using level set methods. Numerical Algorithms **39**(1), 155–173 (2005)

68. Grady, L.: Random Walks for Image Segmentation. IEEE Trans. Pattern Anal. Mach. Intell. **28**(11), 1768–1783 (2006)

69. Gu, Y., Rice, M., Xiong, W., Li, L.: A new approach for image segmentation with shape priors based on the potts model. In: Proceedings of APSIPA Annual Summit and Conference 2017, pp. 12–15. IEEE (2017)

70. Haber, E., Modersitzki, J.: Numerical methods for volume preserving image registration. Inverse Probl. **20**(5), 1621–1638 (2004)

71. Haber, E., Modersitzki, J.: Image registration method with guaranteed displacement regularity. Int. J. Comput. Vision **71**(3), 361–372 (2007)
72. Han, X., Xu, C., Braga-Neto, U., Prince, J.L.: Topology correction in brain cortex segmentation using a multiscale, graph-based algorithm. IEEE Transactions on Medical Imaging 21(2), 109–121 (2002)

73. Han, X., Xu, C., Prince, J.L.: A topology preserving level set method for geometric deformable models. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(6), 755–768 (2003)

74. He, J., Kim, C.S., Kuo, C.C.J.: Interactive Segmentation Techniques: Algorithms and Performance Evaluation. Springer Publishing Company, Incorporated (2013)

75. Hosni, A., Rhemann, C., Bleyer, M., Rother, C., Gelautz, M.: Fast Cost-Volume Filtering for Visual Correspondence and Beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence 35(2), 504–511 (2013)

76. Ibrahim, M., Chen, K., Rada, L.: An improved model for joint segmentation and registration based on linear curvature smoother. Journal of Algorithms & Computational Technology 10(4), 314–324 (2016)

77. Jung, M., Bresson, X., Chan, T.F., Vese, L.A.: Nonlocal Mumford-Shah regularizers for color image restoration. IEEE Trans. Image Process. 20(6), 1583–1598 (2011)

78. Karaçali, B., Davatzikos, C.: Estimating topology preserving and smooth displacement fields. IEEE Trans. Med. Imag. 23(7), 868–880 (2004)

79. Kass, M., Witkin, A., Terzopoulos, D.: Snakes: Active contour models. International Journal of Computer Vision 1(4), 321–331 (1988)

80. Kihara, Y., Soloviev, M., Chen, T.: In the shadows, shape priors shine: Using occlusion to improve multi-region segmentation. pp. 392–401. IEEE (2016)

81. Klodt, M., Cremers, D.: A convex framework for image segmentation with moment constraints. In: 13th IEEE International Conference on Computer Vision (ICCV), pp. 2236–2243 (2011)

82. Kong, T., Rosenfeld, A.: Digital topology: Introduction and survey. Computer Vision, Graphics, and Image Processing 48(3), 357–393 (1989)

83. Latecki, L.J.: D Well-Composed Pictures. Graphical Models and Image Processing 59

84. Le Guyader, C., Gout, C.: Geodesic active contour under geometrical conditions: theory and 3D applications. Numerical Algorithms 48(1), 105–133 (2008)

85. Le Guyader, C., Vese, L.A.: Self-Repelling Snakes for Topology-Preserving Segmentation Models. IEEE Transactions on Image Processing 17(5), 767–779 (2008)

86. Le Guyader, C., Vese, L.A.: A combined segmentation and registration framework with a nonlinear elasticity smoother. Computer Vision and Image Understanding 115(12), 1689–1709 (2011)

87. Lee, Y.T., Lam, K.C., Lui, L.M.: Landmark-Matching Transformation with Large Deformation Via n-dimensional Quasi-conformal Maps. Journal of Scientific Computing 67(3), 926–954 (2016)

88. Lehto, O., Virtanen, K.: Quasiconformal Mappings in the Plane. Springer-Verlag (1973)

89. Li, C., Kao, C., Gore, J.C., Ding, Z.: Implicit Active Contours Driven by Local Binary Fitting Energy. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–7 (2007)

90. Liu, C., Ng, M.K.P., Zeng, T.: Weighted variational model for selective image segmentation with application to medical images. Pattern Recognition 76, 367–379 (2018)

91. Liu, J., Tai, X.C., Luo, S.: Convex Shape Prior for Deep Neural Convolution Network based Eye Fundus Images Segmentation (2020). URL https://arxiv.org/abs/2005.07476

92. Liu, Y., Yu, Y.: Interactive Image Segmentation Based on Level Sets of Probabilities. IEEE Transactions on Visualization and Computer Graphics 18(2), 202–213 (2012)

93. Lord, N., Ho, J., Vemuri, B., Eisenschenk, S.: Simultaneous Registration and Parcellation of Bilateral Hippocampal Surface Pairs for Local Asymmetry Quantification. IEEE Trans. Med. Imaging 26(4), 471–478 (2007)

94. Luo, S., Tai, X.C., Huo, L., Wang, Y., Glowinski, R.: Convex Shape Prior for Multi-Object Segmentation Using a Single Level Set Function. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 613–621 (2019)
95. Malladi, R., Sethian, J.A., Vemuri, B.C.: Shape modeling with front propagation: a level set approach. IEEE Transactions on Pattern Analysis and Machine Intelligence 17(2), 158–175 (1995)

96. McGuinness, K., O'Connor, N.E.: A comparative evaluation of interactive segmentation algorithms. Pattern Recognition 43(2), 434–444 (2010). Interactive Imaging and Vision

97. Mille, J.: Narrow band region-based active contours and surfaces for 2D and 3D segmentation. Computer Vision and Image Understanding 113(9), 946–965 (2009)

98. Miller, M., Trouvé, A., Younes, L.: On the Metrics and Euler-Lagrange Equations of Com- putational Anatomy. Annu. Rev. B. Eng. 4(1), 375–405 (2002)

99. Modersitzki, J.: Numerical Methods for Image Registration. Oxford University Press (2004)

100. Mortensen, E., Morse, B., Barrett, W., Udupa, J.: Adaptive boundary detection using 'live- wire' two-dimensional dynamic programming. In: Proceedings Computers in Cardiology, pp. 635–638 (1992)

101. Mory, B., Ardon, R.: Fuzzy Region Competition: A Convex Two-Phase Segmentation Frame- work. In: F. Sgallari, A. Murli, N. Paragios (eds.) Scale Space and Variational Methods in Computer Vision, pp. 214–226. Springer Berlin Heidelberg, Berlin, Heidelberg (2007)

102. Mumford, D., Shah, J.: Optimal approximation by piecewise smooth functions and associated variational problems. Commun. Pure Applied Mathematics 42(5), 577–685 (1989)

103. Musse, O., Heitz, F., Armspach, J.P.: Topology preserving deformable image matching using constrained hierarchical parametric models. IEEE Trans. Image Process. 10(7), 1081–1093 (2001)

104. Nakahara, M.: Geometry, Topology and Physics. Taylor & Francis (2003)

105. Noblet, V., Heinrich, C., Heitz, F., Armspach, J.P.: 3-D deformable image registration: a topology preservation scheme based on hierarchical deformation models and interval analysis optimization. IEEE Trans. Image Process. 14(5), 553–566 (2005)

106. Nosrati, M.S., Hamarneh, G.: Incorporating prior knowledge in medical image segmentation: a survey. arXiv e-prints (2016). URL https://arxiv.org/abs/1607.01092

107. Ohlander, R., Price, K., Reddy, D.R.: Picture segmentation using a recursive region splitting method. Computer Graphics and Image Processing 8(3), 313–333 (1978)

108. Oliveira, F.P., ao Manuel R.S. Tavares, J.: Medical image registration: a review. Computer Methods in Biomechanics and Biomedical Engineering 17(2), 73–93 (2014)

109. Osher, S., Sethian, J.A.: Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations. Journal of Computational Physics 79(1), 12–49 (1988)

110. Ozeré, S., Gout, C., Le Guyader, C.: Joint Segmentation/Registration Model by Shape Align- ment via Weighted Total Variation Minimization and Nonlinear Elasticity. SIAM Journal on Imaging Sciences 8(3), 1981–2020 (2015)

111. Pennec, X., Stefanescu, R., Arsigny, V., Fillard, P., Ayache, N.: Riemannian Elasticity: A Statistical Regularization Framework for Non-Linear Registration. In: J.S. Duncan, G. Gerig (eds.) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2005: 8th International Conference, Palm Springs, CA, USA, October 26-29, 2005, Proceedings, Part II, pp. 943–950. Springer Berlin Heidelberg (2005)

112. Precioso, F., Barlaud, M.: B-spline Active Contour with Handling of Topology Changes for Fast Video Segmentation. EURASIP Journal on Advances in Signal Processing 2002(6), 555–560 (2002)

113. Price, B.L., Morse, B., Cohen, S.: Geodesic graph cut for interactive image segmentation. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3161–3168 (2010)

114. Rabbitt, R., Weiss, J., Christensen, G., Miller, M.: Mapping of Hyperelastic Deformable Templates Using the Finite Element Method. In: Proceedings SPIE, vol. 2573, pp. 252–265. SPIE (1995)

115. Rada, L., Chen, K.: A new variational model with dual level set functions for selective segmentation. Communications in Computational Physics 12(1), 261–283 (2012)

116. Rada, L., Chen, K.: Improved selective segmentation model using one level-set. Journal of Algorithms and Computational Technology 7(4), 509–540 (2013)
117. Rao, S.R., Mobahi, H., Yang, A.Y., Sastry, S.S., Ma, Y.: Natural Image Segmentation with Adaptive Texture and Boundary Encoding. In: H. Zha, R.i. Taniguchi, S. Maybank (eds.) Computer Vision – ACCV 2009, pp. 135–146. Springer Berlin Heidelberg, Berlin, Heidelberg (2010)

118. Roberts, M., Chen, K., Irion, K.: A Convex Geodesic Selective Model for Image Segmentation. Journal of Mathematical Imaging and Vision **61**(5), 482–503 (2019)

119. Rochery, M., Jermyn, I., Zerubia, J.: Phase field models and higher-order active contours. In: Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1, vol. 2, pp. 970–976 Vol. 2 (2005)

120. Rochery, M., Jermyn, I.H., Zerubia, J.: Higher Order Active Contours. International Journal of Computer Vision **69**(1), 27–42 (2006)

121. Rother, C., Minka, T., Blake, A., Kolmogorov, V.: Cosegmentation of Image Pairs by Histogram Matching - Incorporating a Global Constraint into MRFs. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 1, pp. 993–1000 (2006)

122. Rumpf, M., Wirth, B.: A Nonlinear Elastic Shape Averaging Approach. SIAM Journal on Imaging Sciences **2**(3), 800–833 (2009)

123. Schaeffer, H., Duggan, N., Le Guyader, C., Vese, L.: Topology preserving active contours. Communications in Mathematical Sciences **12**(7), 1329–1342 (2014)

124. Sederberg, T., Parry, S.: Free-form Deformation of Solid Geometric Models. SIGGRAPH Comput. Graph. **20**(4), 151–160 (1986)

125. Ségonne, F.: Active Contours Under Topology Control—Genus Preserving Level Sets. International Journal of Computer Vision **79**(2), 107–117 (2008)

126. Ségonne, F., Pacheco, J., Fischl, B.: Geometrically Accurate Topology-Correction of Cortical Surfaces Using Nonseparating loops. IEEE Transactions on Medical Imaging **26**(4), 518–529 (2007)

127. Sezgin, M., Sankur, B.: Survey over image thresholding techniques and quantitative performance evaluation. J. Electron. Imaging **13**(1), 146–168 (2004)

128. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence **22**(8), 888–905 (2000)

129. Siu, C.Y., Chan, H.L., Lui, L.M.: Image segmentation with partial convexity prior using discrete conformality structures. SIAM J. Image Sci to appear (2020)

130. Sotiras, A., Davatzikos, C., Paragios, N.: Deformable medical image registration: A survey. IEEE Transactions on Medical Imaging **32**(7), 1153–1190 (2013)

131. Spencer, J., Chen, K.: A convex and selective variational model for image segmentation. Communications in Mathematical Sciences **13**(6), 1453–1452 (2015)

132. Storath, M., Weinmann, A.: Fast Partitioning of Vector-Valued Images. SIAM Journal on Imaging Sciences **7**(3), 1826–1852 (2014)

133. Sundaramoorthi, G., Yezzi, A.: More-than-topology-preserving flows for active contours and polygons. In: Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1, vol. 2, pp. 1276–1283 (2005)

134. Taghanaki, S.A., Abhishek, K., Cohen, J.P., Cohen-Adad, J., Hamarneh, G.: Deep semantic segmentation of natural and medical images: a review. arXiv e-prints (2019). URL https://arxiv.org/abs/1910.07655

135. Tang, T., Chung, A.: Non-rigid Image Registration Using Graph-cuts. In: Medical image computing and computer-assisted intervention : MICCAI International Conference on Medical Image Computing and Computer-Assisted Intervention, vol. 10, pp. 916–924 (2007)

136. Theljani, A., Chen, K.: An augmented Lagrangian method for solving a new variational model based on gradients similarity measures and high order regularization for multimodality registration. Inverse Problems and Imaging **13**(2), 309–335 (2019)

137. Thierbach, K., Bazin, P.L., Gavriilidis, F., Kirilina, E., Jäger, C., Morawski, M., Geyer, S., Weiskopf, N., Scherf, N.: Deep Learning meets Topology-preserving Active Contours: towards scalable quantitative histology of cortical cytoarchitecture. bioRxiv (2018)

138. Thiruvenkadam, S.R., Chan, T.F., Hong, B.W.: Segmentation under occlusions using selective shape prior. SIAM J. IMAGING SCIENCES **1**, 115–132 (2009)
139. Tustison, N.J., Avants, B.B., Siqueira, M., Gee, J.C.: Topological Well-Composedness and Glamorous Glue: A Digital Gluing Algorithm for Topologically Constrained Front Propagation. IEEE Transactions on Image Processing **20**(6), 1756–1761 (2011)

140. Unal, G., Slabaugh, G.: Coupled PDEs for non-rigid registration and segmentation. In: Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, vol. 1, pp. 168–175 (2005)

141. Vemuri, B., Ye, J., Chen, Y., Leonard, C.: Image Registration via level-set motion: Applications to atlas-based segmentation. Medical Image Analysis **7**(1), 1–20 (2003)

142. Vese, L.A., Chan, T.F.: A Multiphase Level Set Framework for Image Segmentation Using the Mumford and Shah Model. International Journal of Computer Vision **50**(3), 271–293 (2002)

143. Vese, L.A., Le Guyader, C.: Variational Methods in Image Processing. Mathematical and Computational Imaging Sciences Series. Chapman & Hall/CRC, Taylor & Francis (2015)

144. Waggoner, J., Zhou, Y., Simmons, J., Graef, M.D., Wang, S.: Topology-Preserving Multi-label Image Segmentation. In: 2015 IEEE Winter Conference on Applications of Computer Vision, pp. 1084–1091 (2015)

145. Wang, L., Li, C., Sun, Q., Xia, D., Kao, C.Y.: Active contours driven by local and global intensity fitting energy with application to brain mr image segmentation. Computerized Medical Imaging and Graphics **33**(7), 520–531 (2009)

146. Wells, W.M., Viola, P., Atsumi, H., Nakajima, S., Kikinis, R.: Multi-modal volume registration by maximization of mutual information. Medical image analysis **1**(1), 35–51 (1996)

147. Werlberger, M., Pock, T., Unger, M., Bischof, H.: A variational model for interactive shape prior segmentation and real-time tracking. In: X.C. Tai, K. Mørken, M. Lysaker, K.A. Lie (eds.) Scale Space and Variational Methods in Computer Vision, pp. 200–211. Springer Berlin Heidelberg, Berlin, Heidelberg (2009)

148. Wirth, B.: On the Gamma-limit of joint image segmentation and registration functionals based on phase fields. Interfaces Free Bound. **18**(4), 441–477 (2016)

149. Wu, G., Wang, L., Gilmore, J., Lin, W., Shen, D.: Joint segmentation and registration for infant brain images. In: B. Menze, G. Langs, A. Montillo, M. Kelm, H. Müller, S. Zhang, T.W. Cai, D. Metaxas (eds.) Medical Computer Vision: Algorithms for Big Data: International Workshop, MCV 2014, Held in Conjunction with MICCAI 2014, Cambridge, MA, USA, September 18, 2014, Revised Selected Papers, pp. 13–21. Springer International Publishing (2014)

150. Xu, Z., Niethammer, M.: DeepAtlas: Joint Semi-Supervised Learning of Image Registration and Segmentation. arXiv e-prints (2019). URL https://arxiv.org/abs/1904.08465

151. Yanovsky, I., Thompson, P.M., Osher, S., Leow, A.D.: Topology preserving log-unbiased nonlinear image registration: Theory and implementation. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 1–8 (2007)

152. Yatziv, L., Bartesaghi, A., Sapiro, G.: *O*(N) implementation of the fast marching algorithm. Journal of Computational Physics **212**(2), 393–399 (2006)

153. Yezzi, A., Zollei, L., Kapur, T.: A variational framework for joint segmentation and registration. In: Mathematical Methods in Biomedical Image Analysis, pp. 44–51. IEEE-MMBIA (2001)

154. Yotter, R.A., Dahnke, R., Thompson, P.M., Gaser, C.: Topological correction of brain surface meshes using spherical harmonics. Human Brain Mapping **32**(7), 1109–1124 (2011)

155. Yu, W., Lee, H.K., Hariharan, S., Bu, W., Ahmed, S.: Evolving Generalized Voronoï Diagrams for Accurate Cellular Image Segmentation. Cytometry. Part A: the journal of the International Society for Analytical Cytology **77A**(4), 379–386 (2010)

156. Zagorchev, L., Goshtasby, A.: A comparative study of transformation functions for nonrigid image registration. IEEE Transactions on Image Processing **15**(3), 529–538 (2006)

157. Zhang, D., Chen, K.: A novel diffeomorphic model for image registration and its algorithm". Journal of Mathematical Imaging and Vision **60**(8), 1261–1283 (2018)

158. Zhang, J., Chen, K., Yu, B., Gould, D.: A local information based variational model for selective image segmentation. Inverse Problems and Imaging **8**(1), 293–320 (2014)
159. Zhou, T., Ruan, S., Canu, S.: A review: Deep learning for medical image segmentation using multi-modality fusion. Array **3-4**, 100004 (2019)

160. Zhu, H., Meng, F., Cai, J., Lu, S.: Beyond pixels: A comprehensive survey from bottom-up to semantic image segmentation and cosegmentation. Journal of Visual Communication and Image Representation **34**, 12–27 (2016)