A Differentiable Topological Notion of Local Maxima for Keypoint Detection

Giovanni Barbarani 1 Francesco Vaccarino 1 Gabriele Trivigno 2 Marco Guerra 3 Gabriele Berton 2
Carlo Masone 2

Abstract

details (on a fine-grained scale). Therefore, an ideal scalespace operator should be coherent across different scales bydetecting, as local extrema, the same keypoints at a greater
scale s2 and also at the smaller scale s1 < s2 . For an
appropriate relaxation of these requirements, under certain
hypotheses, the work in [14] indicated that the convolution
with a Gaussian of a variance equal to the scale parameteris the best solution among linear operators.

In computer vision, keypoint detection is a fundamental task, with applications spanning fromrobotics to image retrieval; however, existinglearning-based methods suffer from scale dependency and lack flexibility. This paper introduces
a novel approach that leverages Morse theory
and persistent homology, powerful tools rooted
in algebraic topology. We propose a novel loss
function based on the recent introduction of a notion of subgradient in persistent homology which
achieves competitive performance in keypoint repeatability and introduces a principled and theoretically robust approach to the problem.

Many classical handcrafted keypoints detectors exploit thescale-space theoretical framework [15], [16], the most popular of which is SIFT [17]. In particular, the latter one
operates by building a feature maps pyramid from the image
by repeatedly applying Gaussian convolution and downsample operations. Ultimately, keypoints are detected as local
extrema of the features maps.
Recently, several learning-based detectors have been introduced, which, in the spirit of deep learning, propose
to forego the formal definition of keypoints and rely ona data-driven approach to teach a neural network how to
select salient points [18]–[25]. Inspired by scale-space theory, at inference time, these approaches model keypoints as
local maxima of a scalar map that is the output of a respective learnable convolutional neural network. However, attraining time, several differentiable relaxations have been
applied. For example, D2Net [22] makes use of a pixel
score calculated in a fixed-size patch centered on the pixel.
Similarly, R2D2 [21] considers the local maxima within afixed-size N × N sliding window. DISK [23] provides aprobabilistic formulation where the probability of being a
keypoint still depends on a softmax logit calculated only on
a neighboring patch.

1. Introduction
The ability to extract points from an image in consistent
way across different views (keypoint detection) is a fundamental task of computer vision that found applications as
a basic step of many complex applications of visual localization [1]–[3], SLAM [4]–[6], Structure-from-Motion and
3D reconstruction [7]–[9], as well as retrieval and place
recognition [10], [11].
A theoretical framework for the problem is provided byscale-space theory [12], [13]. In this context, the keypointsof an image I ∈ RH×W are modeled as the set of local
extrema (maxima and minima) of a suitable one-parameter
image operator F (I, s) ∈ RH×W , where the parameters represents the scale at which we are looking for. The
principle used to design the aforementioned scale-space
operator is the non-creation property: when processing theimage at multiple scales, what is noticeable from a distance(on a coarse scale), should have already been visible in the

Despite these recent innovations based on deep learning,classical handcrafted solutions still remain competitive andoften outperform their learnable counterparts. We hypothesize that one of the main reasons for this is that the current
formulation of keypoints adopted in the deep learning literature is based on a fixed-size patch-wise differentiable
relaxation of the concept of local maxima. Indeed, this approach incentivizes models to detect keypoints at a given
frequency, introducing a scale dependency that is in directcontrast with the non-creation property that earlier literature has identified as a key requirement. Therefore, a new

1

Department of Mathematical Sciences ”Giuseppe Luigi Lagrange”, Politecnico di Torino, Italy 2 Department of Control
and Computer Engineering, Politecnico di Torino, Italy 3 Institut
Fourier, Université Grenoble Alpes, France. Correspondence to:
Giovanni Barbarani <giovanni.barbarani@gmail.com>.
Published at the 2 nd Differentiable Almost Everything Workshop
at the 41 st International Conference on Machine Learning, Vienna, Austria. July 2024. Copyright 2024 by the author(s).

1A Differentiable Topological Notion of Local Maxima for Keypoint Detection

approach, based on a scale-agnostic and differentiable formulation of local maxima, is needed to develop unbiased
learnable methods for keypoint detection.

path. An example of this case can be seen in fig. 1.
2.2. Discrete Morse Theory

To this end, we propose a novel scale-independent formulation of keypoints based on Morse theory [26] and persistenthomology [27], [28] from algebraic topology. This formulation leverages the connection between local maxima
and differentiable topological invariants [29], [30], offering a rigorous and differentiable solution without requiringhardcoded hyperparameters that determine the density or
frequency of keypoints. Furthermore, we demonstrate thevalidity of our approach by achieving promising results onbenchmarks for keypoints repeatability. Our implementation and trained models have been publicly released1 .

In the context of digital images, we rely on a discretizedversion of the former theoretical framework, namely discrete
Morse theory [31], and the concept of cubical complex.
A cubical complex is a finite family K of objects Q ⊂Rd , such that: for all Q ∈ K, there is a set of integers
IQ := {l1 , . . . , ld } such that Q = I1 × · · · × Id , with
Ij = [lj , lj + 1] or Ij = [lj , lj ]; and, if P, Q ∈ K theneither P ∩ Q = ∅ or P ∩ Q ∈ K. When I = [l, l], itis called degenerate, and the number of non-degenerateintervals in Q is its dimension, while d is usually called its
embedding number. The 0−dimensional cubes are points,
1−dimensional cubes are edges, 2−dimensional cubes are
squares, and so on. Let P ⊂ Q ∈ K, then P is called a faceof Q, and, if the inclusion is proper, dim P ≤ dim Q − 1.
A cubical complex K is a partially ordered set (poset) via
inclusion and, if (P, ≤P ) is another poset with f : K → P
a monotonically not decreasing map, that is P ⊂ Q impliesf (P ) ≤P f (Q), then it is possible to create a sublevel
filtration of K as follows: ∅ ⊂ K1 ⊂ K2 · · · ⊂ Kt = K
where Ks := {Q ∈ K : f (Q) ≤P ps ∈ P}. Topologicalinvariants of these sublevel sets and their behavior along
the filtration are some of the main topics in topological data
analysis. In particular, they have been studied in Morsetheory, discrete Morse theory, and persistent homology.

2. Background2.1. Morse Theory
The relationship between critical points of a function (extrema and saddle points) and the evolution of a topology
can be intuitively explained using the following analogy:
picture the graph of a 2D scalar function as a landscape.
When we flood this landscape, we witness a series of transformations: lakes emerge from the lowest valley regions;lakes surround mountains, leaving only their peaks above
water, and, ultimately, the lakes blend when they submerge
the peaks.
Morse theory [26] is the mathematical framework that precisely captures the relationship between critical points andchanges in topology. Formally, a smooth scalar function
h defined on a smooth manifold is a Morse function if it
has only non-degenerate critical points, i.e., having nonzero Hessian determinants only. This condition is not restrictive: indeed, up to an infinitesimal perturbation, every
differentiable function on a compact is Morse. Given a
2D compact surface X and the choice of a Morse function h, we can study the evolution of the sublevel sets
Xt = {x ∈ X : h(x) ≤ t} for an increasing t. These
sets can be considered the union of the bottom of the lakes
obtained by pouring water onto our landscape up to level t.
When t reaches the value corresponding to a minimum ofh, the sublevel changes by adding a new point: a new connected component (lake) is born. When t reaches a saddle
point s = (p, t) with t = h(p), two things could happen: (i)the saddle s merges two lakes into one, or (ii) the saddle s
creates a single span bridge over a lake, thus producing a
new closed path (loop) in the component. Therefore, a saddle either reduces connected components or creates a loop.
Finally, when t reaches a maximum value, it corresponds tocompletely submerging the terrain and its closed paths, andthis can be seen as filling the hole surrounded by a closed
1

In this setting a 2D−greyscale digital image is represented
as cubical complexes as follows: 0−cubes are the image
pixels laying on the vertices of an integral rectangular latticein R2 ; 1−dimensional cubes are the edges connecting pixelthat differ by 1 in precisely one coordinate; 2−dimensional
cubes, i.e. squares, are the obvious ones. Let I : {0 −cubes} → [0, 1] be the function assigning to each pixel its
values. Then, we can associate I with a function f : KI →[0, 1] by f (Q) = maxP ∈K0 : P ⊆Q I(P ). The complex KI ,with fI and the corresponding filtration will be called the
cubical complex associated to I. In layman’s terms, we canthink of the filtration of a cubical complex as the sublevelsets of a step function, where vertices, edges, and faces
gradually appear. As shown in fig. 2, this process resultsin loops forming and disappearing at certain critical times,that are in correspondence with the respective saddle edgeor maximum face.
2.3. Persistent Homology
Every topological feature e, that appears in the evolutionof the sublevel sets, is associated with a pair of values(b(e), d(e)), b(e) < d(e), the birth time and the death timeof e. If e is a connected component, b(e) corresponds toa minimum of h, and d(e) must be a saddle. On the other
hand, if e is a loop, then b(e) corresponds to a saddle pointhttps://github.com/gbarbarani/MorseDet

2

A Differentiable Topological Notion of Local Maxima for Keypoint Detection

z2

z1

Figure 1: The evolution of the sublevel sets of a surface filtered by height, i.e. value on the z axis. As the height crosses z1 ,a new loop is born in correspondence with a saddle (green point), then the loop changes smoothly until z hits z2 , the valueof a corresponding maximum (blue point), and the loop disappears. z1 and z2 are respectively the birth time and the deathtime of the topological feature.

3. Topological Detector Loss

(a)

(c)

We aim to learn a feature extractor Fθ that, given an input image I ∈ RH×W ×C , outputs a set of discrete pixel
locations {ki } ∈ R2 . We propose a novel keypoint detector, which we call MorseDet, based on a CNN backbone,that outputs a scalar feature map which we call height map,in an analogy with the terminology used in Morse theory.
Thus, forwarding an image through the adopted network,
we obtain a response map Fθ (I) = H ∈ RH×W .

(b)

At inference time, the keypoints are simply obtained by
performing a fast non-maximum suppression algorithm that
selects the locations corresponding to a local maximum of
the height map with a value above a given threshold γ.

(d)As for the previous approaches, a training instance is composed of two images I1 , I2 and a ground-truth correspondences map between them U ∈ RH×W ×2 , more explicitly
U [i, j] = (i′ , j ′ ) if and only if the pixel (i′ , j ′ ) of the second image corresponds to the pixel (i, j) in the first image;notice that U is defined only on covisible regions.
(e)

(f)

Figure 2: (a) A scalar image can be associated with (b) a
cubical complex made of vertices, edges, and squares. In
both examples, the color represents the pixel value and thefiltration value of the cube, i.e., the maximum value amongits vertices. Adding the cubes according to the filtration
time, we observe the following sequence: (c) still no loop
exists, (d) a loop is born as a saddle edge is added, (e) the
loop persists, and (f) the loop gets closed by a maximum
square. The pixels responsible for the birth and death of theloop are marked in red.

In contrast with previous heuristical methods, during thetraining process, we model keypoints bijectively with thelocal maxima of the feature map. We refer to a local maximum via the associated topological feature, i.e. the loopthat spawns around the critical point and that gets closed
at its peak. Let G(H) be the set of such topological features of the cubical complex associated with H. Every element e ∈ G(H) can be associated with the coordinates ofa (creator) saddle s(e) ∈ R2 and a (destructor) maximumm(e) ∈ R2 . The birth time of e is the value attained by
H at its creator saddle, i.e., b(e) = H[s(e)], in the sameway, the death time of e is the value of a local maximumd(e) = H[m(e)].

and d(e) to a maximum. The persistence is defined as
Pers(e) = d(e) − b(e) and it represents the life span of a
topological feature [27], [28].

We make use of the persistence of e, as defined in sec. 2.3,
to measure the magnitude of a topological feature. Notice3

A Differentiable Topological Notion of Local Maxima for Keypoint Detection

gradient by extending their values continuously along thearbitrary direction.

that this quantity does not depend on the shape or extensionof the region filled by the maximum, i.e. the scale, but only
on how prominent the peak is.

For many functions of the persistence terms, such as theirsum, stronger results hold [29], [30]. These functions are
locally Lipschitz, and the process described above provides
a subgradient along the trajectory V . In these cases, wehave a guarantee for the convergence to a local minimum of
the gradient methods. In contrast, our loss function is notcontinuous at the boundaries of the regions S and its value
strictly depends on the choice of the arbitrary perturbation,
but we still obtain a directional derivative.

For convenience, we define the error matrix between twoheight maps, H1 and H2 , based on the correspondences map
U:
E[i, j] = H1 [i, j] − H2 [U [i, j]](1)

if U is defined on (i, j), otherwise E[i, j] = 0. At thispoint, we introduce a new term that takes into account how
the maps H1 and H2 differ at the topologically relevant(correspondent through U ) positions, namely the boundary
similarity:
Sim(e) = E[s(e)]2 + E[m(e)]2

4. Experiments
Our experiments assessed the ability of MorseDet to predict repeatable keypoints that are robust to changes in scale,viewpoint, or illumination. We adopted the HPatches benchmark and repeatability metric to compare our model against
the best current deep learning detectors, namely D2-Net,
R2D2, SuperPoint, DISK, and ALIKED, as well as the classical handcrafted solution SIFT. Due to space constraints,
the details of the implementation, experimental settings, andresults are covered in app. A.

(2)

Given a positive constant α, our detector loss is finallydefined as
X

Ldet (H1 , H2 ) = −Pers(e) [Pers(e) − αSim(e)]e∈G(H1 )To summarize our findings, we observed that MorseDet performs better than the other learned detectors against scale
shifts, ranking second on average in this scenario after SIFT.
Moreover, we found that, in viewpoint and illumination
tests, our model achieves the most consistent performance.
Notably, while SIFT demonstrates strong scale invarianceproperties, it fails to generalize as effectively in noisy settings, such as changes in illumination.

(3)
In practice, minimizing the loss function aims to increase
the number and prominence of local maxima in the height
map, as long as they are reproducible across similar images.
3.1. Differentiability
Consider a vectorized scalar image H ∈ RHW where all
entries are distinct, and let d be the minimum distance
between these values. When moving within a neighborhood defined by the open ball B(H, d), the order in which
the cubes spawn along the associated filtration remains
unchanged, as do the critical times locations. Therefore,
within a region where the entries follow a strict orderS = {x ∈ RHW | xσ(1) < · · · < xσ(HW ) }, the function PS that selects the relevant entries for the m criticaltime pairs, PS (H) = (b1 , d1 , . . . , bm , dm ), acts as a fixedlinear projection. The persistence and boundary similarity
terms of our loss can thus be expressed as compositions of
PS with the height maps H1 and H2 . Consequently, the loss
function is differentiable almost everywhere.

5. Conclusion and Limitations
In this paper, we introduce an algebraic topology techniqueto address a ubiquitous problem in the image matching
literature: the scale dependency of keypoints due to patchwise differentiable relaxation of local maxima. By utilizing the concept of sublevel set filtration and its connection with Morse theory, we model local maxima in a scaleindependent manner that is, thanks to persistence homology,
suitable for gradient methods. Experimental results demonstrate the validity of our approach, showing strong resultsin terms of keypoints repeatability against changes in scale,viewpoint, and illumination.

A problem arises when multiple entries in H have the samevalue. To address this, we can introduce an arbitrary perturbation that ensures all values are distinct while preserving
any other order relations. For instance, Hϵ = H+ϵV , whereV [i, j] = i+Ij
2IJ and ϵ is smaller than d, the minimum positive difference between distinct entries. For ϵ ∈ (0, d), allHϵ belong to the same region S corresponding to a fixedprojection PS . This allows us to compute the loss and its

Nevertheless, challenges remain in terms of differentiability (see sec. 3.1). The regularization term we introduced
meets only minimal requirements and lacks a guarantee of
convergence for gradient methods due to the discontinuous
nature of the loss function. Therefore, we believe there issignificant potential for further improvements by extending
our framework to achieve complete differentiability.

4

A Differentiable Topological Notion of Local Maxima for Keypoint Detection

References
[1]

[2]

[3]
[4]

[5]

[6]

[7]
[8]

[9]

[10]

[11]

[12]

[13]
[14]
[15]

[16]
[17]

[18]

[19]

P.-E. Sarlin, C. Cadena, R. Siegwart, and M. Dymczyk,“From coarse to fine: Robust hierarchical localization atlarge scale,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019,
pp. 12 716–12 725.
T. Sattler, W. Maddern, C. Toft, et al., “Benchmarking
6DOF outdoor visual localization in changing conditions,”in CVPR, 2018.
C. Toft, W. Maddern, A. Torii, et al., “Long-Term Visual
Localization Revisited,” TPAMI, pp. 1–1, 2020.
R. Mur-Artal, J. M. M. Montiel, and J. D. Tardós, “Orbslam: A versatile and accurate monocular slam system,”IEEE Transactions on Robotics, vol. 31, no. 5, pp. 1147–
1163, 2015.
H. Durrant-Whyte and T. Bailey, “Simultaneous localization and mapping: Part i,” IEEE robotics & automation
magazine, vol. 13, no. 2, pp. 99–110, 2006.
T. Bailey and H. Durrant-Whyte, “Simultaneous localization and mapping (slam): Part ii,” IEEE robotics & automation magazine, vol. 13, no. 3, pp. 108–117, 2006.
J. L. Schönberger and J.-M. Frahm, “Structure-frommotion revisited,” in CVPR, 2016.
J. Heinly, J. L. Schonberger, E. Dunn, and J.-M. Frahm,“Reconstructing the world* in six days *(as captured by
the yahoo 100 million image dataset),” in Proceedingsof the IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), Jun. 2015.
J. L. Schönberger, E. Zheng, M. Pollefeys, and J.-M.
Frahm, “Pixelwise view selection for unstructured multiview stereo,” in ECCV, 2016.
G. Barbarani, M. Mostafa, H. Bayramov, et al., “Are local
features all you need for cross-domain visual place recognition?” In Proceedings of the IEEE/CVF Conference on
Computer Vision and Pattern Recognition, 2023, pp. 6154–
6164.
H. Noh, A. Araujo, J. Sim, T. Weyand, and B. Han, “Largescale image retrieval with attentive deep local features,” in
CVPR, 2017.
T. Lindeberg, “Scale-space theory: A basic tool for analyzing structures at different scales,” Journal of applied
statistics, vol. 21, no. 1-2, pp. 225–270, 1994.
T. Lindeberg, “Scale invariant feature transform,” 2012.
J. J. Koenderink, “The structure of images,” Biologicalcybernetics, vol. 50, no. 5, pp. 363–370, 1984.
K. Mikolajczyk and C. Schmid, “Scale & affine invariant
interest point detectors,” International Journal of Computer
Vision, vol. 60, pp. 63–86, Oct. 2004.
H. Bay, T. Tuytelaars, and L. Van Gool, “SURF: Speeded
up robust features,” in ECCV, 2006.
D. G. Lowe, “Distinctive image features from scaleinvariant keypoints,” International journal of computer
vision, vol. 60, pp. 91–110, 2004.
K. M. Yi, E. Trulls, V. Lepetit, and P. Fua, “Lift: Learned
invariant feature transform,” in Computer Vision–ECCV
2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VI 14,
Springer, 2016, pp. 467–483.

[20]

[21]

[22]

[23]

[24]

[25]

[26]
[27]

[28]

[29]

[30]

[31]

[32]

[33]

5

N. Savinov, A. Seki, L. Ladicky, T. Sattler, and M. Pollefeys, “Quad-networks: Unsupervised learning to rank forinterest point detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017,
pp. 1822–1830.
Y. Tian, B. Fan, and F. Wu, “L2-net: Deep learning of
discriminative patch descriptor in euclidean space,” 2017
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6128–6136, 2017.
J. Revaud, C. De Souza, M. Humenberger, and P. Weinzaepfel, “R2d2: Reliable and repeatable detector and descriptor,” in Advances in Neural Information Processing
Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F.
d’Alché-Buc, E. Fox, and R. Garnett, Eds., vol. 32, Curran
Associates, Inc., 2019.
M. Dusmanu, I. Rocco, T. Pajdla, et al., “D2-Net: A Trainable CNN for Joint Detection and Description of Local
Features,” in Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
M. Tyszkiewicz, P. Fua, and E. Trulls, “Disk: Learning
local features with policy gradient,” Advances in Neural Information Processing Systems, vol. 33, pp. 14 254–14 265,
2020.
X. Zhao, X. Wu, J. Miao, W. Chen, P. C. Y. Chen, and Z. Li,“Alike: Accurate and lightweight keypoint detection anddescriptor extraction,” IEEE Transactions on Multimedia,
vol. 25, pp. 3101–3112, 2023.
X. Zhao, X. Wu, W. Chen, P. C. Chen, Q. Xu, and Z. Li,“Aliked: A lighter keypoint and descriptor extraction network via deformable transformation,” IEEE Transactions
on Instrumentation and Measurement, 2023.
J. W. Milnor, Morse theory. Princeton university press,
1963.
Edelsbrunner, Letscher, and Zomorodian, “Topologicalpersistence and simplification,” Discrete & Computational
Geometry, vol. 28, pp. 511–533, 2002.
A. Zomorodian and G. Carlsson, “Computing persistenthomology,” in Proceedings of the twentieth annual symposium on Computational geometry, 2004, pp. 347–356.
M. Carriere, F. Chazal, M. Glisse, Y. Ike, H. Kannan, and Y.
Umeda, “Optimizing persistent homology based functions,”in International conference on machine learning, PMLR,
2021, pp. 1294–1303.
J. Leygonie, S. Oudot, and U. Tillmann, “A framework for
differential calculus on persistence barcodes,” Foundationsof Computational Mathematics, pp. 1–63, 2021.
V. Robins, P. J. Wood, and A. P. Sheppard, “Theory andalgorithms for constructing discrete morse complexes fromgrayscale digital images,” IEEE Transactions on patternanalysis and machine intelligence, vol. 33, no. 8, pp. 1646–
1658, 2011.
V. Balntas, K. Lenc, A. Vedaldi, and K. Mikolajczyk,“Hpatches: A benchmark and evaluation of handcrafted
and learned local descriptors,” in Proceedings of the IEEE
conference on computer vision and pattern recognition,
2017, pp. 5173–5182.
K. Lenc and A. Vedaldi, “Large scale evaluation of local
image feature detectors on homography datasets,” BMVC,2018.

A Differentiable Topological Notion of Local Maxima for Keypoint Detection
[34]

[35]

[36]

[37]

I. Rey-Otero, M. Delbracio, and J.-M. Morel, “Comparing
feature detectors: A bias in the repeatability criteria,” in2015 IEEE International Conference on Image Processing(ICIP), IEEE, 2015, pp. 3024–3028.
D. DeTone, T. Malisiewicz, and A. Rabinovich, “Superpoint: Self-supervised interest point detection and description,” in Proceedings of the IEEE conference on computer
vision and pattern recognition workshops, 2018, pp. 224–
236.
I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in International Conference on Learning Representations, 2019.
Z. Li and N. Snavely, “Megadepth: Learning single-view
depth prediction from internet photos,” in Proceedings ofthe IEEE conference on computer vision and pattern recognition, 2018, pp. 2041–2050.

6

A Differentiable Topological Notion of Local Maxima for Keypoint Detection

A. Experiments Details
A.1. Dataset and Metrics
We assessed the capability of our method to predict repeatable keypoints using the well-established HPatches benchmark[32]. This dataset comprises 116 scenes, split into 696 images, with the first 57 scenes emphasizing variations in illuminationand the subsequent 59 containing changes in viewpoint. Each sequence in the dataset comprises image pairs of increasing
difficulty. We focus on this dataset, given that it represents a classical, longstanding benchmark for the task of keypoint
detection, to assess the validity of our framework.
Regarding evaluation, our main concern is comparing the quality of the extracted keypoints for different detectors. Thus,
as a metric, we use the formulation of repeatability proposed in [33], which evaluates the consistency of keypoints acrossdifferent images while overcoming the issues of previous definitions of repeatability, which can bias toward detecting clustersof keypoints [34]. This adaptation aims to assess the unique association of keypoints by preventing a single keypoint frommatching multiple counterparts, in detail, it quantifies the proportion of keypoints that are each other’s nearest neighbors in
the corresponding images and are closer than a predefined distance threshold, considering both the coordinate system ofboth the images.
A.2. Baselines
In our study, we benchmark our model against a range of established detectors and state-of-the-art models to ensure acomprehensive evaluation:• SIFT [17]: a handcrafted detector designed to be robust against scale changes.
• D2-Net [22]: employs a multi-scale inference approach, detecting local maxima across multiple output maps.
• R2D2 [21]: an unsupervised detector that uses multi-scale inference.
• SuperPoint [35]: a semi-supervised detector trained to generalize to real images from labeled synthetic shapes.
• DISK [23]: utilizes a probabilistic formulation to jointly model detection and matching.
• ALIKED [25]: features deformable convolution, adapting receptor fields to the supports of keypoints.
A.3. Implementation Details
We adopted L2Net [20] as backbone with the last convolutional layer modified as in R2D2. For training, we employed
AdamW [36] as optimizer, with batches of 8 pairs of images with resolution 208 × 208. To generate image pairs, we follow
the protocol from R2D2, using the same training datasets (WASF). Hyperparameter search and early stopping are performedon the validation split of MegaDepth [37] used in [35]. The final hyperparameters configuration included α = 10, weightdecay equal to 0.005, and repeatability threshold γ = 0.7 for inference. The training process on a single TITAN X GPU
with 12GB of VRAM concluded in approximately 10 hours until convergence.

Method
D2-NetR2D2
SIFTSuperPoint
DISK
ALIKED
MorseDet (ours)

250
21.1
27.3
34.9
42.4
42.2
14.8
44.3

Illumination
500 1000 2000
22.0 23.6 26.4
28.6 29.8 30.5
37.2 38.8 40.4
47.7 49.8 49.5
45.9 49.8 54.2
24.4 37.3 47.0
47.3 50.3 53.4

4000
28.7
30.7
41.2
49.4
57.4
51.9
55.2

250
12.1
24.3
37.8
27.5
30.6
6.5
40.6

500
13.6
25.5
38.9
36.0
35.0
10.6
42.8

Viewpoint
1000 2000
19.5 18.6
26.5 27.6
39.9 40.7
43.6 46.8
39.3 44.0
18.1 29.7
44.6 46.1

4000
22.1
28.3
40.4
46.4
47.6
43.1
47.2

Table 1: Repeatability for illumination and viewpoint splits of HPatches, computed using various values for the maximum
number of keypoints allowed. The best and second-best results are indicated in each column.

7

A Differentiable Topological Notion of Local Maxima for Keypoint Detection

Method
D2-NetR2D2
SIFTSuperPoint
DISK
ALIKED
MorseDet (ours)

Avg
24.6
48.5
63.6
60.6
56.0
18.7
62.2

75%
31.9
55.7
75.9
73.3
71.8
24.2
82.2

50%
19.2
56.2
64.8
63.0
57.4
16.5
63.0

25%
22.8
33.7
50.2
45.6
38.8
15.4
41.3

Table 2: Repeatability of the detector on resized HPatches images as the scale factor progressively reduces. The best andsecond-best results are indicated in each column.

A.4. HPatches Benchmarks
Detector Repeatability
In this experiment, we evaluate the detector repeatability across changes in point of view and illumination on the common
benchmark HPatches. Following [21], we provide results across different values for the maximum number of detected
keypoints allowed. The results are shown in the tab. 1, where the metrics are averaged across all thresholds up to 5px.
We can see that MorseDet’s keypoints achieve consistently good performances, regardless of the number of keypoints or
settings (i.e. illumination and viewpoint changes), being either best or second best across the table. Some other methods
perform competitively with MorseDet under specific settings, although none is competitive in all cases. Notably, DISK has
strong results with a high number of keypoints, and SIFT is second best with fewer keypoints under viewpoint changes but
performs poorly under illumination changes. On average, SuperPoint is second-best.
Scale Repeatability
We posit that models employing a fixed-size window approach for keypoint modeling during training learn to predict
keypoints at a specific frequency. Building on this premise, such models may struggle to consistently replicate keypoints
under rescaling transformations. To study this idea in isolation, we designed the following experiment using the imagesof HPatches. We evaluated for every method the repeatability metric between every image resized to 1000×1000, andthe image resized to smaller sizes to have approximately 75%, 50%, and 25% the pixel area of the original image. As thenumber of keypoints deeply influences repeatability, we limit keypoints to 500, to ensure that every method uses the same
number of keypoints at every scale for fair comparisons, thus also measuring how the methods can prioritize their mostrobust keypoints. The metrics are summarized in the tab. 2 by their average above all the thresholds till 5px.
The results show that MorseDet obtains second-best results on average after SIFT. In particular, MorseDet shines with
75% image resize (i.e. to images of 750×750), outperforming the second best method, SIFT, by 6.3 points. For extreme
scale changes (i.e., 25% of the original resolution), the best model is SIFT, which is a handcrafted detector built to bescale-invariant, followed by SuperPoint and MorseDet. Overall, the only learnable model competitive with MorseDet isSuperPoint, which benefits from a human-informed prior on keypoints. Notably, despite SIFT being proposed nearly two
decades ago, it still outperforms modern detectors in this setup; MorseDet performs significantly better than every otherlearnable method in this task. This is a direct consequence of the fact that previous learnable methods lack a principledframework for modeling local maxima, which is our method’s core contribution.
A.5. Qualitative Results
Fig. 3 shows an example of our model’s height map and detected keypoints. It demonstrates how the model adapts thefrequency of keypoints to the image content, effectively detecting both large-scale corners and fine-grained details without
8

A Differentiable Topological Notion of Local Maxima for Keypoint Detection

(a)

(b)

Figure 3: An example of MorseDet’s (a) height map and (b) detected keypoints on a HPatches image. Our model adapts thefrequency of its keypoints to the scale of the image content.
creating artifacts in low-textured regions.

9