The magnitude vector of images
Authors: Michael F Adamer
Leslie O'Bray
Edward De Brouwer
Bastian Rieck
Karsten Borgwardt
Venue: ICLR 2022 Submitted (2022)
Keywords: magnitude
magnitude vector
edge detection
adversarial robustness
metric space
algebraic topology

Abstract: The magnitude of a finite metric space is a recently-introduced invariant quantity. Despite beneficial theoretical and practical properties, such as a general utility for outlier detection, and a close connection to Laplace radial basis kernels, magnitude has received little attention by the machine learning community so far. In this work, we investigate the properties of magnitude on individual images, with each image forming its own metric space. We show that the known properties of outlier detection translate to edge detection in images and we give supporting theoretical justifications. In addition, we provide a proof of concept of its utility by using a novel magnitude layer to defend against adversarial attacks. Since naive magnitude calculations may be computationally prohibitive, we introduce an algorithm that leverages the regular structure of images to dramatically reduce the computational cost.

Under review as a conference paper at ICLR 2022
THE MAGNITUDE VECTOR OF IMAGES
Anonymous authors
Paper under double-blind review
ABSTRACT
The magnitude of finite metric space is recently-introdueed invariant quantity. Despite beneficial theoretical and practical properties, such as a general utility for outlier detection, and close connection to Laplace radial basis kernels, magnitude has received little attention by the machine learning community so far. In this work, we investigate the properties of magnitude on individual images, with each image forming its own metric space. We show that the known properties of outlier detection translate to edge detection in images and we give supporting theoretical justifications. In addition, we provide a proof of concept of its utility by using a novel magnitude layer to defend against adversarial attacks. Since naive magnitude calculations may be computationally prohibitive, introduce an algorithm that leverages the regular structure of images to dramatically reduce the computational cost.
INTRODUCTION
The topology community has recently invested much effort in studying a newly introduced quantity called magnitude (Leinster 2010). While it originates from category theory, where it can be seen as a generalization of Euler characteristics to metric spaces, the magnitude of a metric space is most intuitively understood descriptive scalar, this quantity extends the set of other well known descriptors such as the rank, diameter or dimension. However, unlike those descriptors, the properties of magnitude are not yet well understood.
attempt measure the effective size of metric space. As
Even though the metric space structure of dataset of the utmost importance understand e.g. the regularisation behaviour of classifiers, magnitude has received little attention by the machine learning community so far. However, it turns out that one can instead investigate the magnitude of each data point separately, considering each data instance as its own metric space. Following this line of thought, magnitude vectors were introduced as a way to characterise the contribution of each data sample to the overall magnitude, such that the sum of the elements of the magnitude vector amounts to the magnitude. As shown in previous works, the magnitude vectors can detect boundaries of a metric space, with boundary points having larger contribution to magnitude (Bunch et al., 2021).
In this work, we seek to advance the research about magnitude in machine learning by addressing these challenges. Since the metric space structure of an entire dataset may not be the most useful application in machine learning, we instead therefore consider the magnitude of each individual data point by endowing each of them with a metric space structure and explore its properties. In partieular, because of their ubiquity and the large dimensionality, we focus our analysis on image datasets, where each individual image seen metric space. We then extend previous results by investigating the theoretical properties associated with magnitude vector in such a context. In addition, we study the properties of the magnitude vector for improving the adversarial robustness of existing neural network architectures for image classification. We propose novel, fully differentiable, magnitude layer, which can serve as an input layer for any deep learning architecture. We show that it results in more robustness for several types of adversarial attacks, with an acceptable reduction in classification performance, paving the way for a new exciting direction in the creation of robust neural architectures. Moreover, since naive magnitude calculations may be computationally prohibitive for large images, we introduce a new algorithm that dramatically speeds up the computation of the magnitude vector. Leveraging the regular structure of images, this allows to conveniently approximate magnitude vectors for large images with minimal error. Intractable computational runtime often stymies the applicability of magnitude in machine learning and therefore hinders further the research of it; our algorithm opens

---

Under review as a conference paper at ICLR 2022
the door to using magnitude efficiently in machine learning research. Equipped with the speed up algorithm, we showcase possible use cases of magnitude vectors in machine learning in the realm of in edge detection and adversarial robustness.
Our contributions are summarized as follows:
We formalize the notion of magnitude vectors for images and investigate the impact of the choice of different metrics.
We introduce an algorithm that dramatically speeds up the computation of magnitude with little loss, which removes a critical roadblock in using magnitude vectors in machine learning applications.
We provide theoretical framework to understand the edge detection capability of magnitude vectors and report empirical supporting evidence.
We demonstrate the capabilities novel, fully differentiable, magnitude layer for improv- ing adversarial robustness of computer vision architectures.
2 THEORY
In this section, we introduce essential notions of the theory of magnitude and magnitude vectors and develop the theoretical background that will help interpreting the empirical results.
2.1 MATHEMATICAL BACKGROUND
We start by formally introducing the notion of a finite metric space.
Definition 1. metric space is an ordered pair (B,d), where B is a finite set and d is metric on B. We denote the cardinality of B by [B].
In our application the set B is set of vectors B R" and the metric considered will be the E, norm. In order to define the magnitude of such space we first define the similarity matrix.
Definition Given a finite metric space M (B, d). its similarity matrix is SM with entries (i,j) e- BB) for Bi, B3 B.
The similarity matrix can be seen as the weights arising from an exponential kernel. We are now in position to define the magnitude vector and the magnitude of finite vector space.
Definition 3. Given finite metric space M = (B, d) with [ and similarity matrix Ças with inverse CM. the magnitude vector of element B; is given by w, = Σラ= magnitude of M, mag Eij=1 (i.j) Σm= W/.
1 SM (i.j). Moreover, the
Not every finite metric space has magnitude. In particular, the magnitude is defined when the similarity matrix is not invertible; the magnitude therefore characterizes the structure of metric space some extent. It should also noted that the definition of the magnitude vector is reminiscent of optimising support vector machine. This connection has been pointed out for the Fuclidean norm by Bunch et al. (202T).
2.2
THEORETICAL RESULTS
2.2. THE MAGNITUDE OF AN IMAGE
This work focuses the analysis of magnitude on images, by considering each individual image as its own metric space. We then first define how we build such a metric space from an image and then prove the existence of magnitude on images.
We refer to the metric spaces on images as image metric spaces and define them as follows.
Definition Let E R be an image with e channels, height n and width m. Let {V Vigi E I...., m; j E I....,n} / be the set of c-dimensional vectors of pixel values in the image. Then the image metric space M(B,d d) is given by a set of vectors C Re+2 of the form B = (1)
(i,j, ...., "..,,,))) (0)T: T. Vay V) together with metric d TR++2.
2

---

Under review as a conference paper at ICLR 2022
Informally, we put all vectors corresponding pixel values grid, concatenating the position vector with the pixel value vector and use the resulting vectors as the ground set B for our finite metric space. Let note that [B] m X n and therefore the number of points in the metric space can be quite large even for moderately-sized images, a potential limitation that we address in Section 2.3
We now turn to investigating when an image metric space has magnitude and, by extension, a magnitude vector. This is priori not clear since, as we saw, the existence of magnitude depends on the properties of the metric space.
From the definition of magnitude (Definition 3). it is readily seen that image metric space M has magnitude if and only if its similarity matrix Çar is invertible. As generic x matrices are invertible (i.e. subjecting any non-invertible matrix result in an invertible matrix), we conclude that generic image metric spaces have magnitude. In fact, since the vectors b E B of the image metric space by a factor > can be rescaled, we can define scaled metrics d(.,-) + td(- -). This rescaled metric space has magnitude except for finitely many > 0 (Leinster et al. 2017 Proposition 2.8). In other words, we can always find scaling which gives rise to a magnitude vector. The univariate function defined by this scaling is called the magnitude function.
random perturbation will almost certainly
In general, we are interested in computing the magnitude of all images in whole dataset, and not only of a single image, such that we can compare their magnitude (vectors). Therefore, we would like theoretical guarantees that there exists scaling such that every image in a dataset has magnitude vector.
Let (BB d) be an image metric space. If d is an Ey norm with 0 < p < 2, then every image metric space M has magnitude.
Proposition 5.
Proof. This a special case of (Meckes) 2013 Theorem 3.6).
The above proposition theoretically only applies to lp norm with < p < 2. However, in practice, we find that on the various datasets we considered in this work, l, norm with p 4, 10 and the Hamming distance also lead to invertible similarity matrices, and thus to the existence of magnitude.
The image metric space exhibits substantial structure; in particular, there is underlying regular subspace (the grid). To quantify this further, we use the notion of a product space.
Lemma 6. Let Mi (B di) and M2 B2, d2) be two finite metric spaces with di d2 being lp norms. Its product metric space M = M1 x M2 is a metric space with metric Pp (B) x B2) x (Bi x B2) R such that
Pz(( x vn), (w2 x w2)) = v/ (di (2), + d2(yn, U2)P, 1 1Sps0.
Proof. This is a special case of (Dovgoshey & Martio 2009).
Note that the product space formulation gives lot of freedom, since every positively weighted combination of di and d2 is also a metric on the product space.
Magnitude vector images based harmonic analysis We now present our main result of this subsection which is an interpretation of the magnitude vector for images based on harmonic analysis. For this consider grey scale images, however, our reasoning generalises colour images. First, notice that a grey scale image can be seen as discretisation of surface in R i.e. == f(x, u). where x, y are the pixel positions and is the brightness. In general, it not clear what an "outlier" on a continuous surface or curve is. In this paper we define an outlier as a point on the surface where the gradient large, i.e. neighbouring points are further away w.r.t. some distance measure. In the image case, outliers are points where the brightness value is substantially different between neighbouring pixels. We can use this reasoning to define a filtration J on the vectors of B, bij (i, 0 ())"" namely 0 C B()) C B(2) .... C B(K) B such that 1 8(k) for 1,..., K, where the g(k) are different brightness thresholds. Due to symmetry in the problem we also define F with criterion u/j 5(k), Further, define the projection onto RR2 p R IR? p(b.)) + (i.j). By
3

---

Under review as a conference paper at ICLR 2022
7
7
(a)
(b)
(c)
(d)
A schematic example of the filtration. In (a) we introduce three thresholds, 0, 0.5, 1. In FF all points below the grey planes are projected to the unit square and in F' all points above are projected. (b) is an illustration of the 0.5 level of the filtration F and (c) is is the same for F', Point size and colour indicate the magnitude vector values. In (d) we reconstruct the magnitude vector.
Figure 1:
considering the projections of each subset of F (F') break the problem down into successive boundary detection of compact subsets of R². To extend this reasoning to colour images we can either consider each channel grey scale image or define multi-dimensional filtrations. A visual deseription of our reasoning found in Figure
To investigate the effect of different metries boundary detection, consider the behaviour of the weighting vector on the 2-dimensional grid. We closely follow the argument of Bunch et al. (2021) extending their results the product space metric in the case when d, are lp norms with p 1, 2 and
p((2 x 21), (x2g 22)) oi (ar,, 22) + (=2dg(11.2)) 22)
for some positive weights a1 a2. For image we can consider the as two-dimensional vectors indicating the position on the unit square and 2; as the corresponding brightness value. Consider regular 2d grid and the equation defining weighting on the grid points Cyr (1....,1)7.. Using
continuous analogue, this can be written as a convolution (Bunch et al. 2021)
f f(x ((== y)v(r)
/ e aadda..) vly) = (0)))()))
8 v())
Jg²a
where 1jo is the indicator function and d is any translation invariant metric. Using the Fourier transform,
F(f)(E) = c-12m= (
(3)
its well-known properties Folland 1999) and the convolution theorem, we can derive an intuitive understanding of the magnitude vector and the effects of specific metrics. In the case of d(-,-) being the Euclidean (Ez) norm, it has been shown in (Bunch et al. 2021 that
3 (a2
Σ
02) 40 = U,
(4)
where we generalised the result of Bunch al. (021 to the unit square. We interpret equation as the weighting u being constant in the interior of the unit square (0f is zero in interior) and different this constant on the boundary. In fact, the weighting will also be constant the edges and corners.
In the Euclidean case it has been shown (Bunch er al., (022)) that the intuitive understanding translates rigorously to compact subsets of R with n odd using distribution theory. Guided by this, we continue our argument using the Fourier transform.
If d(:,-) is the e norm, i.c. the Manhattan or Cityblock distance, we obtain similar result to equation
2 I
(a2
0?)
Io.11 = c
(5)
4

---

Under review as a conference paper at ICLR 2022
in other words, the & norm admits the same interpretation as Eg, however, this time the differential operators acts multiplicatively on each dimension.
For the product space metric pl.,-) ajdı (:.-) + azd2(;,-) and d, are lp with p 1, 2 it follows that the Fourier transform is a product of the single-metric results. Equipped with this insight, we can now explain the edge detection capabilities of the magnitude vector.
Consider filtration F (and F') on grey scale image and choose a subset of vectors ( from the filtration, Apply the projection map p to every vector in this subset. The transformed set is grid with potentially "missing" grid points the domain [0, 12] X m.]. From the results on the unit square and the discretising of the operator, we expect a constant weighting vector except on the boundaries of the grid, i.e. points adjacent to the missing grid points and on the boundaries of the domain. This procedure can be performed on every B(s) E F and the final magnitude vector can be seen as a combination of the weightings of each step (see Figure DT. We find that our expectations from the theoretical result agree well with empirical results (see Appendix D). Moreover, empirically find that Hamming distance and bp norms for # 1, have similar properties.
2.3 COMPUTATIONAL CONSIDERATIONS AND SPEED UP
As can be seen from Definition 3 the computation of the magnitude vector is achieved via matrix inversion. This is fundamentally expensive computation since for an m x n image, we need to invert an N x N square matrix with N mn. In other words, even for moderately sized images there is considerable computational challenge to overcome.
The authors of Bunch et al.) (2021) propose an iterative algorithm based on the a small subspace of the finite metric space M' C M. Suppose the magnitude vector of M' is known, then, when adding a single extra point to M', the computation of the updated magnitude vector reduces to a simple matrix-vector multiplication. Empirically, we find that even for MNIST images this computation is considerably slower than the naive implementation due to the large number of steps in the algorithm. Furthermore, the full distance matrix of the image needs to be stored, which can quickly exceed memory.
When considering the metric space of images, we also have one fundamental advantage over methods, namely grid structure. While every pixel in an image has a weighting, every weighting of pixel also be associated to position computation of weighting vectors should also implicitly consider this additional structure. Therefore, we propose divide-anddconques procedure to be able handle large images efficiently.
wwo-dimensional grid. Any algorithm which speeds up the
According to the theoretical considerations of Subsection 2.2 the magnitude vector is constant in the interior of the unit square and also constant along its edges and corners. From this analysis we gain one key insight, namely creating patches from the image would result in different magnitude vector values along the edges, but all other magnitude vector elements would remain the same. To counter the resulting "edge effects" of the patches, we can choose overlapping patches, calculate the magnitude vector of the patches, and crop the transformed sub-images. Finally, piecing the patches together, we obtain an approximation of the full magnitude vector. As shown in Subsection 3.1 this results in considerable speed up with manageable error. Furthermore, it allows us to even calculate the magnitude vector of high-resolution images.
even faster inference of the magnitude vector is required, we can view the calculation as a special type of image-to- image-to-image translation problem and use current techniques available.
EXPERIMENTS
Given a theoretically-basee algorithm to efficiently calculate the magnitude vector on large images, we can investigate its effect empirically. We turn now to our experiments which begin first with an analysis of the runtime using the speed up algorithm. Armed with a fast calculation, we are then free explore potential applications of the magnitude vector in machine learning applications. We investigate the utility of magnitude vectors in two domains which magnitude was not originally designed for, namely in edge detection and adversarial robustness, and find that it can prove useful.
5

---

Under review as a conference paper at ICLR 2022
0.30
0.07
1500
1250
0.20
0.04
750
0.15
0.03
300
2.05
0
0.00
0.00
16 32
64
128
256
16
32
64
128
256
16
32
64 128 256
Number of patches per dimansion
Number of patches per dimansion
Number of patches per dimension
(b)
Figure 2: (a) The computation time in seconds of the magnitude vector calculation for each of the ten images. Even a small number of patches the computational time deceases drastically. (b) and (c) show box plots of the approximation errors. Although the maximum error can be quite large compared to the maximum difference of 1, the Fröbenius error remains small, indicating a good overall approximation.
3.1 FAST MAGNITUDE CALCULATION AND HIGH-RESOLUTION IMAGES
To illustrate the benefits of the patching method, we consider the first images of the NIH Chest X-ray dataset (Wang et al. 2017). The dataset consists of 100,000 de-identified X-ray (grey scale) images with a resolution 1024 x 1024. Hence, full-scale magnitude calculation would involve 10 matrix which is far beyond current computational power.
inverting roughly 106
To estimate the error induced by the patching algorithm, we rescale the images resolution and calculate the magnitude via matrix inversion. Then, we divide the rescaled image into patches and repeat the calculation. We used the é norm and weights of for both the grid points and intensity values. To ensure comparability we min-max scaled the magnitude vectors. The error is measured in maximum absolute deviation on a pixel level and also in terms Fröbenius distance between the two magnitude vectors
256 256
(magniude vector exact
magnitude vector patches )z
error
(6)
Σ, (magnitude vector
The results can be found in Figure 2
We observe significant reduction (several order of magnitudes)
when using our fast magnitude calculation method, with limited reconstruction error.
To illustrate the effect of the magnitude vector full-scale images we used the patch-algorithm on the first image of the dataset (see Supplementary Figure We also investigated effect of increasing the weight ag of the pixel values in the product space metric and the results can be found in the Supplementary Material.
3.2 MAGNITUDE VECTORS AND EDGE DETECTION
Now that we have computationally efficient method calculate magnitude, turn towards the first of two novel applications of magnitude: developing proof-of-concept of the capabilities of magnitude for edge detection. Indeed, shown in Section 2] the magnitude vector expected to be higher on edges present in the images. This leads naturally to an edge detection algorithm where pixels whose magnitude vector values are larger than a threshold are classified as edges.
To assess the performance of magnitude-basee approach on this task, we compare it with the famous Canny detector (Canny) 1986). which has been widely used for edge detection in images. Because the definition of an edge is subjective concept, we use the edges given by the Canny detector as ground truth and tune the hyper-parametent of our magnitude-basee edge detector such as to match the Canny edges as close possible. We use the Sørensen- Dice coefficient, D, to assess the similarity between both types of edges. Both algorithms can be seen as classifier on a pixel level operating on two classes (I edge, no-edge)) 1), thereby outputting mask with same dimensions of the original image. The Sørensen- -icc coefficient between one predicted edge mask (Edgenat) and the reference edge mask (Edgeret) then writes:

---

Under review as a conference paper at ICLR 2022
Table 1: Sørensen-Dice Dince coefficient on held-out test of the magnitude-based method with respect to the Canny method.
FASHIONMNIST
CIFAR
X-ray
0.59+0.11
0.61+0.00
0.33+00.07
2TP
D ((dgehhat Edgoret)
2TP + FP +
where TP, FP and FN stands for the number True Positive, False Positive and False Negative pixels predictions respectively.
We then use Bayesian optimization of the Sprensen-Dies coefficient (Sorensen 1948 Dice I 1945) on training set find the best hyperparameters of the magnitude- based edge detector. In particular, we tune the threshold at which magnitude vectors predict for an edge and the 02 from Equation I We then report the metric on hold-out test set, and the results shown in Table [ We this experiment on three different datasets: FashionMNIST, CIFAR and an X-ray image dataset. For the X-ray dataset, we use the patched version as described Section 2.3 On Figure 3 show examples from the held-out test set on the Fashion MNIST dataset. In Appendix A we show examples from the two other datasets. visual inspection shows that edges can be recovered quite accurately. However, the edges masks obtained through magnitude appear more noisy.
Figure 3:
Examples of edge detection on the FashionMNIST dataset. The left-most column show the initial images, the center shows the edges mask obtained with the Canny detector, the right-most column shows the edges mask obtained with the magnitude- based approach.
3.3 MAGNITUDE VECTORS AND ADVERSARIAL ROBUSTNESS
As saw in Subsection 3.2 and Supplementary Figure the magnitude calculation transforms original image into a representation its edges. We now want to explore another facet of its utility in machine learning by investigating its potential use for improving adversarial robustness. In the following, the datasets we use are MNIST, KMNIST and FashionMNIST.
The magnitude layer is zero-parameter feature transformation consisting, sequentially, in

---

Under review as a conference paper at ICLR 2022
1. the computation of the distance matrix,
2. the exponentiation of the negative distance matrix element-wise,
3. the inversion the resulting matrix,
4. the summing of the resulting matrix over its rows.
Crucially, this layer be used as an input layer to any image-classification network. While the transformation is still, in principle, differentiable, we are presented with numerous opportunities to introduce step function into the calculation. One possibility is inspired by feature squeezing (Xu et al. 2017), which quantizes the pixel values. In the magnitude calculation we can differentiably quantize the values of the inverse similarity matrix just before the summation in step 4. This step function ensures that the gradients with respect to the input data are zero and, therefore, any white-box attacks which rely on this gradient (such FGSM, PGD, Carlini-Wagner) Wagner) must fail.
While the introduction of the step function guarantees trivial "robustness", we also investigate the efficacy of transferred adversarial examples. To this end, we train two LeNet models (LeCun et al.) 1989) for 30 epochs each on our datasets, one with magnitude input layer, one without (the base model). We then generate adversarial examples w.r.t. the base model and test them on the model with magnitude. Our results for an FGSM attack and various quantization levels are summarised in Table 2 evasion, emphasise that these results
We note that, although our model shows the best performance in terms of adversarial
directly comparable. While simple FGSM attacks fail altogether on the squeezed magnitude layer due to the step function, we calculated robustness via transfer of adversarials from a conventional LeNet model. The feature squeezing setup similar to the original setup from Xu et al. (2)17). however, we used our own quantization layer which divides the features into equally spaced levels and the models were trained on the squeezed images. The base model is a simple undefended LeNet. We also investigate the robustness properties of a non-quantized magnitude input layer which E norm, which shows substantially increased robustness over the baselines of the undefended model and for large feature squeezing.
The number of levels in the feature squeezing was chosen to be small due to the fact that there are large brightness in the datasets we have mainly white objects on black background and, therefore, a large compression would be expected to give the largest robustness. The levels of the magnitude vector were chosen to be larger as the models performed worse with smaller levels.
Levels
Base
Squoeze Levels
Standalone 2.
30
100
1000
3
0
97.79
97.63
99:01
95.68
588.3
0.05
71.90
72.88
68.23
94.78
100.0
100.0
100.0
90.78
0.1
34,50
65
62.02
81.57
100.0
190.0
100.0
81.36
0.2
54.95
60.65
100.0
54.84
21.76
32.20
0.4
46.14
61.36
54.57
14.37
100.0
3.47
2.53
39.77
0.5
28.00
43.87
39.34
11.95
18:43
3.47
2.53
33.65
0.6
6.26
0.7
23.56
12
35.03
12.95
6.26
2.24
21.36
0
$1.25
$3.15
80.26
$8.06
$7.00
$7.34
$8.25
K3:36
0.05
53.91
64.27
53.25
78.34
100.0
100.0
100.0
70.77
32.52
190.0
0.2
27.05
34.93
25.14
18.69
100.0
100.0
19.88
41.64
KMINITT
0.3
27.94
30.94
28.20
6.56
100.0
3.89
19.88
35.44
24.60
31.7
24.77
3.97
100.0
3.89
0.6
16.32
21.74
16.51
3.85
5:31
3.89
2:31
25:26
0.7
16.24
20.94
16.39
4,03
5.31
3.89
1,50
20.55
0
K2.82
82.85
82.82
0.05
77.42
74.67
24.79
100.0
100.0
100.0
$5.43
0.1
56.85
61. 48
66.17
10.70
100.0
100.0
100.0
43.39
0.2
48.65 18.86
$3.89
60.34
5.10
100.0
100.0
23.68
35.24
FashienMNIST
38.1
51.77
100.0
4.76
0.5
27.
33.13
43.29
1.42
23.26
3.37
4.76
20.45
0.6
78.33
31. 78
42.00
1.55
4.47
3.37
4.76
22
4.47
Table The accuracies of the attacked networks. From left to right: quantized magnitude layer with different quantization levels, the undefended model, the model with feature squeezing and various levels, and the magnitude layer as an input layer. The zero epsilon values are the base accuries. For the non-zero epsilon, the accuracy is the ratio of aon-adversarial examples to base accuracies.
8

---

Under review as a conference paper at ICLR 2022
4 RELATED WORK
An early version of magnitude has been introduced by Soooo & Polasky (994) where it was defined as an "effective number of species", however with little mathematical development. The subject has been picked up formally by Leinster (010)) where connection to theory of enriched categories has been made. From there, the magnitude of many mathematical objects has been studied, such as spheres [Willerton] 2014), odd balls (Willerton) 2018; 2017), compact sets (Meckes) 2013 Leinster
Willerton 2013). convex bodies (Meckes (220)) and graphs (Leinster 2019).
As stated in the introduction, magnitude and magnitude vectors have not been studied from machine learning perspective except for the work of [Bunch et al.] (202T) that analysed the capability of magnitude vectors for boundary detection. In contrast, our work extends the results from this study to images and provide the necessary theoretical grounds. To the best of our knowledge, this represents the first work exploring magnitude vectors for images. In particular, using magnitude vectors for adversarial robustness has never been explored before.
5 DISCUSSION
This paper investigated the magnitude vector of images. In the first part, we stated several theoretical properties of the magnitude vector on images and showed how the outlier detection property translates to edge detection in images. Although the theoretical part is based on analogies, rather than rigorous mathematical proofs, we can still distil the essence of the behaviour of the magnitude vector. In particular, these theoretical insights are confirmed by practical experiments. We also propose an algorithm for approximate calculation of the magnitude vector leading significant speed up of computations. While this works well in practice, as shown in Subsection 3.1 the exact computation of magnitude vector on a metric space with large number of points still remains an open problem.
Our algorithm to speed up the computation allows magnitude to be used more generally as the computational bottleneck of its naive computation can be bypassed. Investigating the capabilities of magnitude vectors in machine learning applications, we studied two application areas in more depth, namely edge detection and adversarial robustness, showing promising and overall favourable results in both cases.
As for edge detection, we compared a magnitude- approached with the well known Canny edge detector. We show that the overlap between the edges found by the Camny detector and the magnitude vector noteworthy, though the magnitude vector can appear a bit noisier. In particular, for low resolution colour images (e.g. CIFAR- 10), the edges are very noisy and thorough investigation for the magnitude vector on colour images should be conducted. This is nonetheless impressive, since the magnitude vector is general-purpose purpose image transformation, unlike the Canny detector, and therefore shows an exciting potential for improving exisiting edge detection tools.
We also considered a potential application of the magnitude vector a simple mean of defence against adversarial attacks. We first noted that a step function such as a quantization function can be introduced into the magnitude calculation, providing trivial but effective defence against any white box attack. We went further by investigating the property of transferred adversarial examples. Our model shows reasonable robustness and performs better than the baseline method of feature squeezing. With these encouraging results we aim to investigate the use of magnitude in adversarial evasion and detection in future work.
REPRODUCIBILITY STATEMENT
We provide all code to create figures and results this paper in the supplementary material. All datasets are publicly available and downloadable via the code provided.
REFERENCES
Eric Bunch, Jeffery Kline, Daniel Dickinson, Suhaas Bhat, and Glenn Fung. Weighting vectors for machine learning: numerical harmonic analysis applied to boundary detection. arXiv preprint arXiv:2/06. /06.027. 2021.

---

Under review as a conference paper at ICLR 2022
John Canny, A computational approach edge detection, IEEE Transactions pattern analysis and machine intelligence, (6):679-65) 698, 1986.
Lee Dice. Measures of the amount of ecologic association between species. Ecology. 26(3): 297-302, 1945.
Oleksiy Dovgoshey and Olli Martio. Products of metric spaces, covering numbers, packing numbers and characterizations of ultrametric spaces. arXiv preprint arXiv:0903. /,26, 2009.
Gerald B Folland. Real analysis: modern techniques and their applications, volume 40. John Wiley & Sons, 1999.
Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard Howard, Wayne
Hubbard, and Lawrence D Jackel. Backpropagation applied handwritten zip code recognition. Neural computation, 1(4):541-551, 1989.
Tom Leinster. The magnitude of metric spaces. arXiv preprint arXiv: 1012.8557 2010.
Tom Leinster. The magnitude a graph. In Mathematical Proceedings of the Cambridge Philosoph- ical Society, volume 166, pp. 464. Cambridge University Press, 2019.
Tom Leinster and Simon Willerton. On the asymptotic magnitude of subsets of euclidean space. Geometriae Dedicata, 164(1):287-310, 2013.
Tom Leinster, Mark W Meckes, and Nicola Gigli. The magnitude of metric space: from category theory to geometric measure theory. pp. 156-193, 2017.
Mark W Meckes. Positive definite metric spaces. Positivity, 17(3):333-77, 2013.
Mark W Meckes. Magnitude, diversity, capacities, and dimensions of metric spaces. Potential
Analysis, 42(2):549-577 -72,, 2015.
Mark W Meckes. On the magnitude and intrinsic volumes of convex body in euclidean space. Mathematika, 66(2):333 -3544435355-13 2020.
Andrew Solow and Stephen Polasky. Measuring biological diversity. Environmental and Ecological Statistics, 1(2):95-103, 1994.
Th A Sorensen. A method of establishing groups of equal amplitude in plant sociology based on similarity of species content and its application to analyses of the vegetation on danish commons. Biol. Skar. 5:1-34, 1948.
Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M Summers. Chestx-ray8; Hospital-scaa scall chest x-ray database and benchmarks on weakly-supervisee classi- fication and localization of common thorax diseases. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2097-2106, 2017.
Simon Willerton. On the magnitude of spheres, surfaces and other homogeneous spaces. Geometriae Dedicata, 168(1):291-310, 2014.
Simon Willerton. The magnitude of odd balls via hankel determinants of reverse bessel polynomials. arXiv preprint arXiv: mttp::320..0.27... 2017.
Simon Willerton. On the magnitude of odd balls via potential functions. arXiv preprint 10X00:1804.02174.. 2018.
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. arXiv preprint 1rXiv:1704.01755. 2017.
A SUPPLEMENTARY ILLUSTRATIONS OF THE EDGE DETECTION CAPABILITIES OF THE MAGNITUDE VECTORS.
On Figures 4 lndd] 5] we show examples of edge detection from the test sets of the CIFAR and X-ray datasets respectively.
10

---

Under review as a conference paper at ICLR 2022
Orchs
Grry Tesos
Figure 4: Three randomly selected examples of edge detection on the CIFAR dataset. Left left-most column show the initial images, the center shows the edges mask obtained with the Canny detector, the right-most column shows the edges mask obtained with the magnitude- based approach.
Dngra
DRAOK
Figure 5: Two randomly selected examples of edge detection on the X-ray dataset. Left left-most column show the initial images, the center shows the edges mask obtained with the Canny detector, the right-most column shows the edges mask obtained with the magnitude-basee based approach.
B MAGNITUDE VECTORS WITH DIFFERENT METRICS
11

---

Under review as a conference paper at ICLR 2022
VR
322
Original
Hamming
p=1 2= Hamming,
p 1, 1p= 1
D/
Figure 6: A comparison of the effect of different metrics in the magnitude calculation on KMNIST.
The first column is the original image and the following three columns are the magnitude vectors with different Ep metrics. The last three columns use a product space metric with a₁ = a = 2 = 1. grid metric bp and brightness metric Ep'
C THE MAGNITUDE OF FULL SCALE IMAGES
Original
a2
02
100
Q2
200
022 300
Figure 7: An illustration of the effect of ag on the magnitude calculation of full-scale images.
D COMPARISON OF THE FILTRATION AND THE FULL MAGNITUDE
12

---

Under review as a conference paper at ICLR 2022
7
7
7
Original
Full magnitude
Thresholds: {0, 1} Thresholds: 0.5, 1}
Thresholds: {0, 0.1,.... 1}
Figure 8: An illustration of the magnitude based on filtrations with metric 41
13


Reviews:
[{"review_id": "qPcjEyVlJmR", "reviewer": "Reviewer_f2ou", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- The magnitude of a metric space seems to have various interesting properties encoding other quantities of interest.\n- The fast magnitude computation method is interesting and seems to be efficient although the scale of the significance of approximation error needs more discussion.\n\nWeaknesses:\n- There is a lack of motivation for considering the magnitude as a quantity for any purpose. My understanding from prior work is that the magnitude is a rather theoretical curiosity that “turns out to encode many invariants … including, volume, capacity, dimension and intrinsic volume” (Leinster 2017). But after reading this paper as well as prior work, I still do not have an intuition as to what it is expected to encode about an image. I will expand on this issue below under the experimental limitations.\n- Theoretical contributions of the work are not clear. A few definitions, propositions and lemma are restated from prior work. Together they seem to provide expressions for computing the magnitude of a single image. However, all of these come from prior work.\n- My understanding is that the contribution in the Section 2.2 is the particular definition of metric space for an image (Def. 4). The rest of the section applies prior tools to this metric space. There is a lack of justification for this definition. For example, why should one treat individual pixels as a point and why should we use spatial coordinates in the vector representation?\n- The notation is confusing. For example, Def. 3 and Def. 1 use n to denote both the cardinality of B and the dimensionality of elements in B. Eq. 4 and 5 are also using notations with proper definition that comes from Bunch et al 2021.\n- The significance of Section 3.2 is not clear. Are authors claiming “The magnitude of an image is as good as the Canny Edge detector.”? If so, can authors clarify why this is a significant observation? Given Figure 3, I’m not sure I would agree with this claim either, as the magnitude-based approach has mostly failed on the image of the pants.\n- The adversarial robustness results in Section 3.3 need improvement considering major works on attacking discretization layers that achieve spurious robustness through gradient masking / gradient obfuscation. As tested by authors, black-box attacks are one approach, however, FGSM is not a particularly strong attack. Various other guidelines are suggested in [1,2].\n\n[1] Athalye, Anish, Nicholas Carlini, and David Wagner. \"Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples.\" International conference on machine learning. PMLR, 2018.\n[2] Florian Tramèr, Nicholas Carlini, Wieland Brendel, Aleksander Madry:\nOn Adaptive Attacks to Adversarial Example Defenses. NeurIPS 2020", "summary_of_the_paper": "This paper explores the application of the quantity “magnitude of a metric space” for images. Based on prior work, magnitude is a quantity originating from category theory extended to metric spaces with surprising geometric properties (Leinster 2017). This paper defines the magnitude as a vector valued function of a metric space (Def. 3). Through defining an image as a metric space, the magnitude of a single image is defined (Def. 4). Authors then state conditions under which magnitude of an image is known to exist (Prop 5). Then the paper defines a filtration operation that computes the magnitude vector on multiple thresholded projections of the input. Section 2.3 introduces an approximate computation method for the magnitude and Section 3.1 evaluates the efficiency and error of this approximation. Section 3.2 applies the method for edge detection and compares it to Canny detector. Section 3.3 uses the magnitude for adversarial robustness and claims gains.", "summary_of_the_review": "The paper requires major improvements to motivation, empirical results, and rigour of theoretical contributions.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "hPYWOj9ZOe3", "reviewer": "Reviewer_qYUC", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Page 2\n\n#### Definition 1\nThis should read \"A finite metric space is an ordered pair...\"\n\n#### Definition 4\n- The notation to define $V$ is nonstandard and unclear.\n- The notation $M(B,d)$ is nonstandard: It should either be $M$ or $(B,d)$ unless $M$ some kind of mapping. I.e. $M=(B,d)=M(B,d) $ is not a consistent choice of notation.\n- In the definition of $B$ (as well as $V$) it would be wise to avoid an unrestricted comprehension.\n\n## Page 3\n\n> As generic $n \\times n$ matrices are invertible (i.e. subjecting any non-invertible matrix to a random perturbation will almost certainly result in an invertible matrix), we conclude that generic image metric spaces have magnitude.\n\n- This is not a conclusion, it is an assumption.\n\n> In fact, since the vectors b ∈ B of the image metric space by a factor t > 0 can be rescaled, we can define scaled metrics $d(·, ·) \\mapsto td(·, ·)$.\n\n- This only makes sense if $d$ is induced by a norm. The sentence, as written, does not make sense.\n\n#### Lemma 6\n- This is a standard metric space construction, and applies even more generally than its statement here. Its inclusion is inappropriate.\n\n> We can use this reasoning to define a filtration $F$ on the vectors of B...\n- Typo, should be $v_{ij}^{(k)}$ not $v_{ij}^{(1)}$. \n- observe the se are just the sublevel sets of the function $f$, calling this a \"filtration\" is confusing.\n\n##  Page 4\nCan you please explain the purpose of the derivations and what the results are? It is unclear what here is new or novel and what is a result from Bunch et al. (2021). \n\n\n## Pages 5 and 6\nI would an explanation here of what the new algorithm is. Additionally if you are going to claim that your algorithm is significantly faster you need to show a baseline to compare the speed with others, e.g. Bunch et al. (2021). Furthermore why have you chosen an X-ray dataset as the only dataset to present benchmarks with?\n\n\n\n\n\n\n", "summary_of_the_paper": "The authors consider a relatively new finite metric space quantity known as the \"magnitude\" and its applications within computer vision. \n\nThe paper proceeds with introducing and defining the magnitude quantity before exploring some key properties.  The contributes the authors outline are a study of \n* relationship between 'outlier detection' and 'edge detection'\n* introduction of 'magnitude layer' as an adversarial defence \n* new algorithm to speed up the computation of the magnitude vect", "summary_of_the_review": "The reason that I have recommended this paper are several fold. \n\nChiefly it is unclear what the novel contribution of the authors are in this paper. Taking the contributions one by one: \n - It is unclear what the benefit or novelty there is from the study of outliers and edge detection\n- The improved algorithm is not explained, presented, or studied empirically with reference to any baseline.\n- The new layer the authors present involves a computationally prohibitive matrix inversion that will limit its applicability. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "3d-vQ94t9mc", "reviewer": "Reviewer_WNxv", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\nThe paper discuss a concept that is relatively new in the machine learning community. The magnitude of a metric space has been studied in topology, and this paper demonstrates that it can potentially be used in image-related ML problems.\n\nWeakness:\nGiven the results in the current paper, I am not fully convinced that the concept of magnitude vector is really important for learning problems. To better demonstrate its importance, the authors should provide more comprehensive experimental results.\n\nFor edge detection: the authors only provided some visual examples for the Fashion MNIST dataset. From the three images that the authors provided in the paper, the results of magnitude-based edge detection is visually worse than Canny detector. The authors did not provide any quantitative results on the quality of edge detection. I think a more rigorous study and evaluation is needed to show its benefits.\n\nFor adversarial robustness, the authors only provided results on three relatively simple datasets: MNIST, KMNIST, Fashion MNIST, with a relatively weak attack (FGSM). To demonstrate the method of the proposed approach, the authors should provide results on more datasets and more baseline algorithms.\n\n====\nAfter response: the authors did not post specific response so I decided to keep my score.", "summary_of_the_paper": "This paper discuss a concept of magnitude vector of images. The authors provided a fast algorithm for computing the magnitude vector, and discuss the potential applications in edge detection and adversarial robustness.", "summary_of_the_review": "The experimental results are not strong enough to demonstrate the benefits of the concept of magnitude vector of images.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "1pCoiEdslWz", "reviewer": "Reviewer_cxFw", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper does not provide any theoretically-backed reason why magnitude vectors of images should be considered, and in fact the given definition of a vector space associated with an image appears quite dubious: the existence of the magnitude of an image depends on individual brighness values, which are irrelevant to the actual overall meaning of the image; in addition, the metric mixes the position of the pixels with their brightness, which looks very unnatural.\n\nThe algorithm to speed up the computation of magnitude vectors is also not theoretically backed in any way, and only supported by empirical experiments.\n\nFinally, the two given application of magnitude vectors to computer vision are very weak. In edge detection, it is shown that with appropriate hyperparameter tuning, magnitude vectors can recover a noisy version of the output of the Canny detector. The authors never observe that edge detection is a local problem, whereas the magnitude vector is a global quantity; this appears to me as an intrinsic theoretical limitation to the application of magnitude vectors to edge detection. As for adversarial robustness, it seems quite obvious that generated adversarial examples for a base LeNet are classified more correctly by a modified LeNet (regardless of how the network is actually modified).\n\nHere are some additional minor comments:\n- After Definition 1: a *finite* set of vectors\n- Lemma 6: this is obvious, there is no need to cite a 2009 preprint\n", "summary_of_the_paper": "This paper applies the machinery of magnitude vectors of metric spaces to extract information from individual images (which can be themselves be seen as finite metric spaces having one point for each pixel). After some theoretical considerations, the paper proposes a divide-and-conquer algorithm to approximate the magnitude vector of an image which is significantly faster than the naive inversion of a $N\\times N$ matrix (where $N$ is the number of pixels in the image). Finally, the magnitude vector is shown to correlate with the result of traditional edge detectors (after hyper-parameter tuning) and an application to adversarial robustness is described.", "summary_of_the_review": "This paper presents an application of magnitude vectors to image, which is not convincing both from a theoretical perspective (why should the given definition be useful? why should the given speed-up algorithm work well?) and from an empirical perspective (the results on edge detection and adversarial robustness are weak).", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper discusses a relatively new concept called "magnitude" for finite metric spaces and investigates its potential applications in machine learning, in particular for computer vision.

Reviewers generally agree that this is an interesting concept and appreciate the algorithm for reducing its computational cost. 
However, there are concerns (1) that the concept is not well-motivated theoretically for machine learning problems 
(2) the experimental results, for edge detection and adversarial robustness, are not convincing. More rigorous empirical work should be carried out.
Decision: Reject