Scalable Neural Network Geometric Robustness Validation via Hölder Optimisation
Authors: Yanghao Zhang
Panagiotis Kouvaros
Alessio Lomuscio
Venue: NeurIPS 2025 poster ()
Keywords: Robustness Validation
Hölder Optimisation
Geometric robustness
Hilbert curve dimensionality-reduction
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: NN robustness validation against geometric perturbations using Hölder Optimisation

Abstract: Neural Network (NN) verification methods provide local robustness
guarantees for a NN in the dense perturbation space of an input.
In this paper we introduce H$^2$V, a method for the validation of
local robustness of NNs against geometric perturbations. H$^2$V
uniquely employs a Hilbert space-filling construction to recast
multi-dimensional problems into single-dimensional ones and Hölder
optimisation, iteratively refining the estimation of the Hölder constant 
for constructing the lower bound.
In common with current methods, Hölder optimisation might theoretically
converge to a local minimum, thereby resulting in a robustness result
being incorrect. However, we here identify conditions for H$^2$V
 to be provably sound, and show experimentally that even
outside the soundness conditions, the risk of incorrect results can be
minimised by introducing appropriate heuristics in the global
optimisation procedure. Indeed, we found no incorrect 
results validated by H$^2$V on a large set of benchmarks from
SoundnessBench and VNN-COMP.
To assess the scalability of the approach, we report the results
obtained on large NNs ranging from Resnet34 to Resnet152 and vision
transformers.  These point to state-of-the-art scalability of the approach when
validating the local robustness of large NNs against geometric
perturbations on the ImageNet dataset.  Beyond image tasks, we show
that the method's scalability enables for the first time the
robustness validation of large-scale 3D-NNs in video classification
tasks against geometric perturbations for long-sequence input frames
on Kinetics/UCF101 datasets.

Scalable Neural Network Geometric Robustness Validation via Hölder Optimisation
Yanghao Zhangl:
Panagiotis Kouvaros'
Alessio Lomuscio 1.2
1 Safe Intelligence, UK
2 Department of Computing, Imperial College London, UK
{yanghao, panagiotis, a
Abstract
Neural Network (NN) verification methods provide local robustness guarantees for NN in the dense perturbation space an input. In this paper we introduce H² method for the validation of local robustness of NNs against geometric perturbations. HDV uniquely employs a Hilbert space-filling construction to recast multi-dimensional problems into dinenssonaa ones and Hölder optimisa- tion, iteratively refining the estimation of the Hölder constant for constructing the lower bound. In common with current methods, Hölder optimisation might theoretically converge to a local minimum, thereby resulting in a robustness result being incorrect. However, we here identify conditions for H2V to be provably sound, and show experimentally that even outside the soundness conditions, the risk of incorrect results can be minimised by introducing appropriate heuristics in the global optimisation procedure. Indeed, we found no incorrect results validated by HºV large set of benchmarks from SoundnessBench and VNN-COMP. To assess the scalability of the approach, we report the results obtained on large NNs ranging from Resnet34 to Resnet152 and vision transformers. These point state-of- the-ar art scalability of the approach when validating the local robustness of large NNs against geometric perturbations on the ImageNet dataset. Beyond image tasks, we show that the method's 'sss scalability enables for the first time the robustness validation of large-scale 3D-NNs in video classification tasks against geometric perturbations for long-sequence input frames on Kinetics/UCF10) datasets.
Introduction
As well known, Neural Networks (NNs) are inherently vulnerable to adversarial perturbations [I]. i.e., their output is susceptible to fragilities, or attacks, the neighbourhood of correctly processed inputs. In the context of machine vision models, input perturbations generating such fragilities can take various forms including noise, geometric changes, and illumination variations. Evaluating the robustness of model, i.e., its resistance to such small input changes, is particularly important in safety-critical applications.
The area of robustness verification 2 consists of methods providing formal guarantees that a model locally robust in regions of the input space defined by a test point and a particular perturbation. With the exceptions discussed in the Related Work (Section 5). methods for the verification of local robustness are often theoretically sound (if method reports that the model is either locally robust or not robust in region, then that is guaranteed to be the case), and often complete (given unlimited time and resources a method can always resolve the local robustness query).
well-known difficulty sound and complete verification methods their scalability: the robustness verification problem is theoretically NP-hard [3 and present State-of- the-Art (SoA) methods often fail to scale to large models, large inputs, or large perturbations [4) thereby hindering the application
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

of the methods in many noteworthy applications. Consequently, incomplete methods like adversarial testing are routinely used in applications evaluate the robustness of large models [5] 6 Yet. adversarial testing is known to fail to identify fragilities in a large number of cases, potentially instilling false sense of robustness the developer.
Building on these considerations, we introduce H²V, a method leveraging Hilbert curve mapping and Hölder optimisation for robustness Validation. H is formally sound when executed under certain configurations. However, when run outside these settings, false positives are theoretically possible but unlikely. We argue that this limitation has negligible impact on practical robustness evaluation for two key reasons. Firstly, assessing model's 's robustness in real-world applications relies on aggregating results from numerous inputs and perturbations; it is the cumulative evidence, rather than any single local robustness result, that drives conclusions. Secondly, the theoretical soundness of SoA verification tools is often compromised in practice by system-leve floating point errors rendering the distinction between theoretical and practical guarantees less significant.
H²V is based Hölder optimisation and follows advanced developments in the area of global optimisation 19 As such, in line with many global optimisation methods [12) for NN analysis, its convergence the global minimum can be assured theoretically only if appropriate optimisation parameters are chosen. Given this, we refer to H²V as a robusiness validation method, rather than one for robustness verification, because in general the method may be unsound. However, in what follows we identify circumstances where the method is theoretically sound, thereby falling into the category of traditional verification methods. In cases where this assumption cannot be established, we demonstrate that in practice the optimisation procedure at the heart of H2V results in no incorrect robustness results all the validation benchmarks that we studied, including SoundnessBench (}. indeed outperforming in terms of soundness all current SoA and theoretically sound methods.
major feature of V lies in its scalability. As we demonstrate below, H²V enables the validation of models with hundreds of millions of tunable parameters, thereby enabling the robustness analysis of models in many present applications.
In summary, our contributions are as follows:
We propose H²V, a global optimisation method for the validation of NNs based on space- filling dimensionality reduction and Hölder optimisation. We provide theoretical conditions for theoretical convergence, hence soundness. We illustrate that when these theoretical convergence conditions are not met, the potential of a robustness error in a single query is well contained. Indeed, no errors were found in the extensive evaluation reported.
We V validate the local robustness of models of up to 300M tunable parame- ters, including ResNet152 and Vision Transformers for image classification tasks, against geometric properties (rotation, scaling, and translation) the large-scale ImageNet dataset.
We use H2V validate the geometric robustness of 3D ResNet models in video classification tasks for streams of 32 x 3 x 256 256 inputs.
The rest of the paper is organised follows. In Section we present key notions of use throughout. We present H²V in Section where we give the technical details of the validation approach and present soundness conditions. In Section[4] we evaluate H2V on large NNs for image classification and video classification; we also evaluate the correctness of the implementation empirically on SoundnessBench and additional benchmarks from VNN-COMP 4 Section 5 discusses related work. We conclude in Section 6
2 Preliminaries
This section outlines the background concepts and notation that facilitate the exposition of the validation method presented in the next section.
Hölder/Lipschite constant. A function f: R + R is said to be Hölder continuous with exponent
E (0, 1] if there exists a smallest constant H 2 0. called the Hölder constant, such that for all ar (a, b), the following inequality holds: [f(x) - -f(')) < H|2 I'10. Lipschitz continuity [33 a special case of Hölder continuity when a = 1, in which case H becomes the Lipschitz constant
L. These constants represent the highest rate at which the function can change in the interval.

---

Hilbert space-filling curve. A space- filling curve [14 is a function h R + RN that maps the unit interval x E [0, 1] onto a multidimennional hypercube D {0= E [a. b] C E.::
{h(z) 0 s 1] (0 R S 0, < b, b
(I)
The function is surjective; for every point in the hypereube, there exists at least one point in the interval which maps onto it.
The first examples of space-filling curves date back to Peano 14] the one we adopt here is due to Hilbert [15]. Their definitions are given in the limit of infinitely many refinements of recursive constructions. Each recursive step discretises the space at fixed resolution determined by parameter m, thereby producing an m-approximation of the space. Specifically, the hypercube is subdivided into 2 xm smaller hypercubes with 2nh subdivisions along each dimension. The Hilbert curve for an m-approximation, denoted m traverses these unit hypercubes in a continuous manner, thus preserving spatial locality. As m - oc, the approximation converges the true space-filling Hilbert curve, which fully covers the entire hypercube in the limit. An example of Hilbert curves hN mN=33,==3()) can be seen on the left of Figure
A property of Hilbert curves is that the multi-dimensional minimisation problem of a Lipschitz continuous function f R. + R (N, N) can be accurately reduced to the one-dimensional problem along the m-approximation the Hilbert curve hN.m [16]:
min f(0) min f(h(x)) Rs min mm = min f(x), BERN
=E(0.1]
=E[0,1]
where for brevity f(x) denotes ((bN.(x))) Further, f(x) is Hölder continuous with exponent 1/N:
Vx, x' E [0. 1f(x) f(x')| S H( ((1==x11)+
where 2L N +3 is the Hölder constant and the Lipschitz constant of the original f.
Neural Networks with Lipschitz continuity. We consider Lipschitz continuous neural networks (NNs) g: RRN R NNs comprising convolutional, fully connected, and contrast-normalisatioo layers with ReLU activation functions are Lipschitz continuous [17]. Further, softmax layers, as well as sigmoid and hyperbolic tangent activation functions, also satisfy Lipschitz continuity (18). We here focus on classification tasks where each input æ E R is assigned to the class ÿ among a set of classes c) determined by the largest NN output, i.e., 1 arg max g(x);- j=l.....
Local robustness verification. Given NN g R + RC, an input æ to g. and perturbation space (xx)) of æ. the robustness verification problem establishes whether the class prediction of the network is consistent within the perturbation space. In other words, the problem is to determine whether:
Ya' E Q(x): arg max g(x);; arg max g(xx)((()) )-
By taking (19.z.z) = g(x'), max g(x')(())))))))){)))) where = arg max g(x); this is equivalent to i
establishing whether:
Val E ((x): [[g xt, x²) > 0.
A NN g is said to be certifiably robust on input with respect the perturbation space N(x:) if Eq. (5) holds. Any violation of this property, ie., 3x' E M(x): [gg x, x') < O, indicates the presence of counterexample (or attack, or fragility). A common perturbation space is the one generated by Ep norms around x, defined as l(x:) = ( || - x' æ'Ilp S }} for a perturbation budget E IR.
Local geometric robustness. A perturbation space that is of particular interest in computer vision is defined in terms of geometric perturbations on the input, such as rotation, translation, isotropic scaling combinations thereof [19] A geometric perturbation is 2D affine transformation Ag that provides a mapping between source coordinates (x* (x") of the input and target coordinates (x*, y²) of the transformed input:
2x
[X y Xssn y
- Xisnng X cos y
thor tver
Ao
y 1
where 0 = by, A, thor ever the transformation parameters, with y representing the rotation angle, A denoting the scaling factor, and thor yeer controlling the horizontal and vertical translation. Each
3

---

Aool
f(0)
4000
Figure From left to right: Hilbert curve mapping; k-th iteration H²V's optimisation; The subsequent iterations of H² optimisation. The orange lines represent the constructed lower bound, and the blue lines are auxiliary lines used to compute the splitting point efficiently.
pixel value Va in the transformed image can be computed by calculating the pre- pre-image of the pixel under Ag and interpolating the (possibly non-integer) resulting coordinates using any interpolation scheme. We here adopt Spatial Transformation Networks (20) with bilinear interpolation to determine
these values: Va'.y' Σn Σm Unm max(0, 1 - |ø ml) max(0, 1 ly" n|), where Uhw is the value of the pixel with coordinates (n, m).
Given interval constraints θ C R the geometric perturbation space s(x=) (V 0) 0 E )) for an input x is the set of all transformed inputs for each 0 E O where each transformed input (ax, 0) is obtained by determining Ve., for each pixel (I, y) of the input. Establishing the local robustness of NNs with respect to this space can be used to assess their robustness to geometric distortion effects, such as tilted camera orientation (rotation), positional shifts (translation), and zoom variations (isotropic scaling) [2T].
Two key properties enable the derivation and efficiency of the validation procedure introduced in the next section. Firstly, prepending geometric transformation modules to Lipschitz continuous NNs preserves Lipschitz continuity [22]. Secondly, the perturbation dimensionality, i.e., the number of parameters, of these modules is very low when compared to the input dimensionality, iee, the number of pixels, of norm-based perturbation modules. We will exploit this to provide an effective reduction to one dimension for a global optimisation method for more than one input dimensions.
In the following we consider the local geometric robustness problem vO E θ
f(g', x, (ax 0)) > 0, Since g' and are fixed,
where g' denotes NN prepended with geometric module on input 19 we briefly denote the problem by vo E O f(0) > 0.
3 Hölder-based Global Optimisation for Neural Network Validation
In this section we present H2 Hölder-based global optimisation method for addressing the local geometric robustness problem defined the previous section.
The method aims to find a solution to the optimisation problem min f(0) s.t. 0 E O, where @ an N d-dimensional hyperrectangle encoding the perturbation space around an input. If the solution to the problem is greater than zero, then the robustness problem is answered positively by H2V. To enable the utilisation scalable methods, H²V first transforms the multivariate function of the optimisation problem into univariate equivalent using the Hilbert space filling-curve. Some optimisation methods for the resulting univariate problem require knowledge of the Hölder constant [23] while some others do not [II]. Following the intractability of the accurate estimate of the constant [9, we here adapt a method from the latter category that relies on adaptive estimations of the constant throughout the optimisation process 9 10 [8] As we discuss in more detail below, while the method does not theoretically guarantee the identification of the global minimisers of the optimisation objective,
demonstrate below, the potential for any potential unsound result is well-contained view of widely employed settings in the resulting global optimisation problem.
In the following, we provide technical exposition of HºV. To ease its presentation, and without loss of generality, we assume that the input domain θ the hypercube (0, 1) . We begin with short. technical overview.

---

Overview. Figure | illustrates two consecutive iterations of H²V. Having transformed the multi- variate objective function into a univariate one, the algorithm iteratively operates on increasingly tighter intervals of the one-dimensional input range. For each interval, it computes a lower-bounding piecewise function (the orange dashed lines in the figure) using an estimation of the Hölder constant for that interval. Based on this lower-bounding function, it then computes lower bound for the interval (the minimum between žleft and Fright in the figure). At each iteration, the algorithm chooses the interval with the lowest bound to split into tighter intervals at the point where the estimated lower bound is observed. Lower-bounding facilitate the next iteration. The algorithm terminates when an optimisation budget is reached that reflects the minimum length of the selected interval. Below, we discuss this procedure in detail.
functions for the new sub-intervals are then computed to
Initialisation. V is initialised by: (i) transforming the multi-dimensee dimensional optimisation problem min f(0) s.t. 0 E [0, 1J to the one-dimensional problem min f(x) s.t. E [0. 1] using the Hilbert space-filling curve, (ii) setting I = {[0, 1]} to the set of initial intervals, and (iii) letting = ( to be the set of already considered intervals. Then, for each iteration k 2 1, H2V executes the following steps.
Step 1: Adaptive estimation of the Hölder constant. For each interval E I, with = [a, b). HaV f(o]](()] computes a local Hölder constant H; = |e-b/178 and a global Hölder constant h2 = max H₃ | i E I). Based on these constants, it derives its (adaptive) estimation of the Hölder constant for interval as H. r max {A. 17: {). where:
k max Hy 1 E [í ne, + nx]} } is the local component of the estimation, which represents the maximum value among the local constants of interval and its neighbouring intervals (i.e, all intervals within a given number of hops n. from i, including i itself);
" hi X Xmax
la-hl
is the global component of the estimation, where Xmax
max {(b aa' [a' [y] E I is the widest interval;
{ a small value that prevents H, from becoming O, thereby accounting for f(x) varying
over [0, 1]: r > the reliability parameter of the algorithm.
Intuitively, the adaptive estimation H, is dominated by the global component whenever an interval is large (and thus the local estimates are not reliable), and by the local component whenever interval small (and thus the local estimates are more accurate). A practical enhancement is introduced in more details below to mitigate potential underestimations of the constant.
Step 2: Estimation of the lower bounds of the intervals. For each interval E I, with i [a, the algorithm computes the point
b+a f(b) f(a) 8; 2
2H; a) -
This point the intersection of the lines Tlett (x) and Fright (I) (see the blue solid lines in Figure which are defined as
a) -² _0= a + fla). Teett (x) = - (b a) -22 + H, (b-
(8)
(x) = (b -a) للاا - 1b b b+ f(b).
These lines simplify the piccewise lower bounding functions of f within the interval (the orange dashed lines in the figure); we refer to [9] for a formal description of the functions using the estimated Hölder constant. The lower bound of the interval is then estimated as 4 min( (Zeeft, Fright), where
Zleft = f(a) H, - I/N Bright = j(b) Å, (b 1/)1/N.
The bound 4 corresponds to the minimum value of the lower bounding functions evaluated at the 8;. Step 3: Convergence and refinement. H2V selects the interval I with the minimum lower bound estimate 4. Then,
If the length of the interval i = [a, b) is smaller than the optimisation budget, i.e., 1b- - al S e, it executes Step 4 and terminates;
5

---

Otherwise, it splits the selected interval with respect to E [a, bl, and updates I t I {i] U {[a,s,), s,l, [s, 6]}, O t OU {i}. It then repeats from Step 1.
Step 4: Calibration and output. Hav computes an estimation of the minimum of the function as fm min (()))(() [a, bj E I]. and an estimation of the lower bound of the function as mm = min (4 j IUO). The bound 4m is then calibrated as 4m <- Im 7, where = 6-2-(m+))A + (e/2) L and H/ are the present estimates of the global Lipschitz and Hölder constants, and m is the resolution of the Hilbert approximation. The calibration, which is theoretically analysed below, accounts for (i) approximation errors in the dimensionality reduction along the Hilbert curve, and (ii) the constrained nature of the optimisation budget within which the algorithm operates. Following the calibration, H²V produces its output as follows:
If mm > O, then it returns robust, i.e., a positive answer to the robustness of the underlying network.
If fm < 0, then it returns non-robust, along with a counterexample hymm (x) corresponding to the value for which f(x) fm-
We now proceed to analyse the algorithm's soundness and examine practical methods for sustaining high reliability and computational efficiency. We begin by showing that calibrated (as per Step 4) lower bound for the reduced one-dimensional space translates to a lower bound for the original N N-diiensional space.
Theorem 1. Let 17 be a lower bound of the one-dimensional problem min f(x) s.t. [0, 1] over the Hilbert space-filling curve. Then, we have that
1 2-2 (10 VN H (E/2) =..
(10)
where 1* is the optimal solution of the multi- problem min f(0) s.t. 0 E 10,
Proof. The proof is included in the Appendix.
Note that the first calibration term results from the approximation of the Hilbert curve reduction, while the second is consequence of the limited optimisation budget. When the resolution of the Hilbert approximation is high enough, egg., m = 50 in our experiments, the magnitude of the former term is negligible. Differently, the magnitude of the latter term grows with the number of dimensions, thus hindering the efficacy of H²V to high-dimensional input domains.
Next, show that given sufficiently large enough value for the reliability parameter H²V implements a sound verification procedure.
Theorem 2. There exists r" s.t. for all r", H2V outputs robust iff vO 10, f(0) >
Proof. The result follows immediately from Theorem D and Theorem 3.8 in 9
Note that Theorem 2 does not provide a constructive way of determining r". Consequently, in practice, the Hölder constant can be underestimated at any iteration and interval, which may impact the localisation of the global minimisers and the convergence speed. Consequently, H²V may output robust when the local geometric robustness problem not robust. This reflects all existing global op- timisation-basee verification methods that do not necessitate knowledge of the true Lipschitz/Hölder constant [18 24 Note however that if 22V reports non-robust, meaning the model is fragile in the specified neighbourhood, the conclusion is definitive. Note also that if in Step an overestimation of the Hölder constant is used, then H2V is theoretically guaranteed to return sound results as the corollary below formalises.
Corollary 1. Let H* be the true Hölder constant. for every iteration of Step 1, A, 2 H", then HºV outputs robust iff v0 E 10, [(0) > 0.
Proof. The corollary is a direct consequence of Theorems 3.6 and 3.8 in 9
Tight bounds for the Hölder constant are in general intractable to compute (9) while fast and loose
bounds lead to major performance degradation of the verification procedure, empirically analysed

---

in the Appendix. In the light of this, H²V relies on estimations of the constant as detailed in Step 1, but implements the following operational enhancements that remedy the potential underestimation of the constant.
Practical enhancements. To ensure high reliability, following convergence (i.e, when the opti- misation budget reached), H²V iteratively increases the value of the global Hölder constant li, and the number of neighbourhood intervals n,e used in the estimation of the Hölder constant until either (i) a different interval is selected at Step 3. or (ii) a time limit (given as a parameter) is reached. Intuitively, if the algorithm converges to local minimum following underestimation of the Hölder constant, the iterative adjustment of the constant will eventually trigger an escape from said minimum. To further enable high practical efficacy, HºV implements two global optimisation strategies (9] First, it employs a heuristic whereby dynamically adapts the Hölder constant based on both local and global information as detailed in Step 1. Second, for every iteration, following the selection of an interval i = [a. b] and division thereof as per split point s; at Step 3. it re-estimates the lower bound of an interval j the next iteration only if one of the following conditions hold: (i) j is adjacent or contained in i; (ii) the length of is equal to maxi (iii) the local Hölder constant for the sub-intervals of is greater than the global Hölder constant hg. These express the necessary conditions for triggering a change in the estimation of the lower bound lj of each interval j (as per the definition of the adaptive estimation of the Hölder constant H, in Step 1). Taken together, these enhancements contribute towards achieving high efficiency and a very high degree of correctly answered verification problems.
4 Experimental Evaluation
Experimental Setup. Our experiments were conducted on a workstation equipped with a 16-core AMD Ryzen 9 9950X CPU, 192 GB of RAM, running Linux kernel 6. 114.0-2--generie and an NVIDIA RTX 5090 GPU with 32 GB of graphics memory. The implementation is in Python; the Hilbert space-filling curve mapping is implemented by using the hilberteurve library (25). The experimental evaluation is aimed to evaluate the practical applicability of the approach. We establish this by assessing the scalability of the approach on very large NNs and its reliability in practice. As we discuss below, our findings suggest that: the method scales to models such as vision transformers and video models that to our knowledge could not be previously verified and the implementation achieves the highest level of correctly answered verification queries.
In terms of geometric perturbations, we denote the rotation operation as R(7), where the angle varies within the range +y,, and the scaling operation as S()), where the scaling factor ranges between 1 1 X. Let T(t) represent the translation operation, shifting an input by up =t proportionally in both the horizontal and vertical directions. Here we consider the combination of these three types of geometric transformations to evaluate the model's robustness in terms of its robust accuracy, i.e., the percentage of samples reported robust in the geometric neighbourhood considered. We report only the highlights in the rest of this section but base our conclusions on the comprehensive benchmarking for the method also reported in the Appendix.
Large NNs for Image Classification. To evaluate the performance of H image classification for large NNs, we benchmarked the robust accuracy obtained by the tool on 9 models from timn of different sizes, ranging from 19M (Gmlp) to 300M (Large ViT ViT16x16) tunable parameters. These include several ResNet models up to ResNet152, trained to a good level of accuracy. The dataset used ImageNet, with input sizes of either 3 224 x 224 or 3 299 x 299, depending on the model con- figuration. The verification queries consisted of any combination of input transformation consisting of rotation, translation and isotropic scaling with parameters 20 10%, and 10%, respectively. We set the timeout budget for each verification query to 1200s (20 minutes) and report the average runtime of H in seconds. The runtimes are computed with respect the robust and non-robust cases and do not include the timeouts. To the best of our knowledge, GeoRobust [22] is the only available tool that can handle such queries on high dimensional inputs for such large NNs. In particular, none of the tools in VNN-COMP 4 21] can resolve such queries. We provide further benchmarking for completeness in the Appendix. Table Ireporrss the results obtained for 500 ImageNet samples. We observe that in most cases H²V signifcantly outperforms GeoRobust in terms of finding more
bbtpss//uuggingfie..oooosss
7

---

Table I: Evaluation results on 500 images from ImageNet against the perturbation combination (4 dimensions) of rotation (20%). translation (10%) and isotropic scaling (10%). The baseline performances are adopted from 22 except for ResNet-34, for which we rerun the experiment to obtain updated results due to changes in the timm (PyTorch Image Models) library.
Timeruts (%)
Robust Accuracy (8)
Average Runtime
Model
Input Size
Acc (%)
(M)
GooRobust
H
GeoRobust
H
HPV
Inception V3
3 299 x 299
73.4
24
4.0
0.6
24.2
23.0
150.51
ResNer34
299 x 299
72.0
4.2
1.0
27.8
26.0
101.55
ResNet50
3 290 x 290
78.4
26
22.9
1.8
31.1
40.8
253.97
ResNet101
299 299
2.2
48.2
47.0
430.27
R5sNet152
3 290 x 290
79.6
7.2
2.0
46.2
46.8
477.01
Mixer
x 224 x 221
72.2
60
3,8
4.0
23,4
20.2
206.01
Gmip
3 224 x 221
6.2
36.8
30.6
327.11
Swin
x 224 x 224
80.2
88
21.4
9.6
13.2
8.2
199.29
Large ViTunsau
x 224 221
10.6
40.2
29.0
496.16
GeoRobusi's lerminnaiin criterion the query count: therefore, here present the pencentage cases that remaii undecided.
counterexamples whilst exhibiting smaller percentage of timeouts. GeoRobust is shown have superior performance on non-ResNet models in terms of robust accuracy. However, further analysis of these results indicate that GeoRobust often incorrectly reported model robust. Indeed, Håv identified several counterexamples (i.e., 8 for Inception V3, 9 for ResNet34. 2 for ResNet50, 5 for ResNet101, 12 for ResNet152, 9 for Mixer, 15 for Gmlp, 21 for Swin, 30 for ViT) verification queries that were reported robust by GeoRobust. We suspect this is because GeoRobust's underlying global optimisation procedure can often use underestimations of the Lipschitz constant.
The results suggest that the VIT model considered is less robust than some ResNet models. This raises the question to why the patch-based attention mechanisms do not translate into improved robustness [26]. The results here only refer to geometric robustness and require further analysis.
Large NNs for Video Classification. To further evaluate the performance of HºV, we now report experimental results obtained when assessing the robustness large models used for video classification. For this considered end-to-end RGB 3D-NNs without flow information trained on the Kinetics-400 dataset (27). Specifically, we evaluated 5 pre-trained NNs from the open- source library PyTorch Video 28 with network parameters ranging from 3. 3.79M to 32.45M, and up to 32 x 3 x 256 x 256 input dimensions: Slow-R50 29 R(2+1)D-R50
30 X3D_M [1],
13D-R50 32 and C2D-R50
33
Table 2: Benchmarking static geometric robustness of video classification models against geometric transforms (R(20") S(10%) + T(10%))
Frame
Frame Rate
Average Runtime (s)
Model
No. Params (M)
Clean Ace (%)
Timeouts (%)
Robust Acc (5)
Length
X3D_M
16
3.79
73.0
3.0
37.0
1210.70
C2D-R50
8
8
24.33
73.0
3.0
36.0
768.88
T3D-R50
8
8
28.04
74.0
1.0
45.0
11 18.28
R(2+1)D-R50
16
28.11
76.0
2.0
45.0
1874.70
Slow-R50
32.45
2.0
1238.82
For the evaluation, we randomly selected 100 videos from the dataset and evaluated the robustness of the models against perturbations applied entire video (see the Appendix for technical details). The perturbations consisted of combinations of rotation, scaling and translation, using the same perturbation intensity used for the images above. We the timeout budget for each verification query to 3600s (60 minutes) and report the average runtime of H2V in seconds. The runtimes are computed with respect to the robust and non-robust cases and do not include the timeouts. In terms of baselines, to the best of our knowledge, the only two verification methods applicable to video tasks are (34) and (35]. However, the former analyses the robustness of the extracted optical flow, rather than perturbing the RGB frames directly, thereby limiting its real-world applicability and

---

Table Soundness validation on SoundnessBench.
Beachmark
Input Dincusionality
No. Params
Tool
No. Robust
No. Non-Robust
No. Unknown
No. Unsound
Average Runtime (s)
B-CCRWW
19
57
6.74
CNNI
25-75
353K
PYRAT
12
64
3.50
HPV
27
66.06
OJ-CCRWN
25-75
354K
PYRAT
5
15.32
It'v
16
58
0
63.31
ad-CROWN
8
67
4.35
CNN3
25-75
606K
PYRAT
9.63
IF
03-CROWN
10
2.12
CNN AvgPool
25.75
353K
PyRAT
6
54
18.
H"V
12
48
65.47
ad-CROWN
CNN Tanh
25-75
353K
PYRAT
0.00
HºY
19
63.34
oU--CROWN
2
29
0.50
CNN Sigmoid
25-75
353K
PyRAT
30
0.36
H²V
OU-CCROWN
20
0.79
3.13M
PYRAT
20
51
7.23
H-V
30
39
54.32
Table 4: Evaluation on TLL Verify Bench, a VNN-COMP benchmark.
Benchmark
Dinensionality
No. Params
Tool
No. Robust
No. Non-Robust
No. Unknown
No. Unsomnd
Avenge Runtine (6)
TL.. Verify
G-CCRWW
15
D
0
37.93
17k- 67M
PYRAT H'V
4
0
44,29
Bench
0
0
27.86
comparability with task. The latter only scale to small NNs with small input sizes, hence it is not comparable with H2V.
The results are reported in Table It can be observed that H²V was able to resolve large proportion of the verification queries with minimum rate timeouts. To our knowledge, this is the first time that large vidco classifiers evaluated for local robustness. In our results Slow-R50 and R(2+1)D R50 achieved the highest robust accuracy (47.0% and 45. 55.0%, respectively). These findings indicate that architectural refinements and expanding model capacity could potentially benefit robustness against geometric transformations.
Soundness Validation. We discussed in Sections and that since H² is based on global optimisation, may return unsound results. This theoretical possibility can be mitigated, as is routinely done in optimisation, by carefully choosing the optimisation parameters. The ablation studies of several relevant hyper-parameters, including the reliability parameter, the resolution parameter of the Hilbert curve, and the optimisation budget, are provided in the Appendix.
In what follows we evaluate the empirical soundness of H²v. We do this in two ways. Firstly, we evaluate the results obtained by H"V on SoundnessBench [8]. This is a recently released neural network verification benchmark, specifically designed for the validation of the correctness of verifiers by including the ground truth of the verification queries. Secondly, HºV on www-demiiiooaatty perturbations from VNN-COMP [4. In this case the ground truth not known, and thus, similarly to VNN-COMP, we compare the results against those produced by the SoA tools. total, we evaluated the soundness of H2V on 460 robustness queries. We found that all the results produced by H²V on BoundnessBench were correct (i.c., matching the ground truth provided), and all the results produced by H²V on the VNN-COMP benchmarks were in line with those reported by a.--CRWNN [36].
report the results obtained by

---

We report the results obtained on SoundnessBench in Table using a timeout of 100 seconds. The benchmark comprises 24 models (primarily CNNs and MLPs) and includes 240 verification queries that ought to be resolved as robust and 186 that ought to be resolved as non-robust. Note that the latter include carefully hidden adversarial examples that are challenging for verifiers to discover. It has recently reported that several mainstream NN verifiers, including a.9-CROWN [66,, NeuralSAT [7], and Marabou (38], answer some of the instances incorrectly [8]. In contrast, H² returned the correct result for all the queries, and achieved the highest number of verified queries. We refer to the
Appendix for an exposition of the detailed performance of each method.
Lastly, reports the results obtained on the TLL Verify Bench benchmark from VNN-COMP [4]. using& default timeout of 600 seconds. The benchmark was selected because of the low-input dimensionality (N = 2) of the perturbations it includes, which makes it amenable to analysis via H2V. We observe that H achieves the same verification results as a.3-CROWN (batch size 1) and PYRAT (3), while being more efficient and exhibiting no unsound cases.
5 Related work
An extensive body of literature exists on the verification of NN robustness against lg ep-bounded
and other local perturbations; we refer to 2 40] for surveys on the area. A variety of methods
are used ranging from Mixed-Integer Linear Programming 41, to SMT [37 38 abstract
interpretation [39 45 49,, and wranchaadd-ouue with symbolic interval propagation [6 50 56
All of these differ from H2V in that they are theoretically sound. This guarantee does not always translate into sound implementations since unavoidable floating point approximations may impact the correctness of the bounds generated by symbolic interval propagation methods [7 []] In contrast, outside the soundness envelope discussed in Section 3 H²V may in principle return "robust" for a model that admits attacks. We noted that this eventually is remote and can be mitigated by an appropriate choice of optimisation parameters as our experiments demonstrate.
In terms of scalability, the approaches cited above, notably symbolic interval propagation, outperform V2V for large dimensionality problems. However, shown in the previous section, H2 V considerably outperforms all SoA on geometric perturbations, including the approaches targeting geometric (see also discussion in the Appendix).
robustness directly 22 22 59
Much closer to H are existing methods based on global optimisation difference between HºV and these methods is that the former couples Hölder optimisation with dimensionality reduction technique, thereby scaling to larger models and to higher dimensions, as empirically demonstrated. We note that existing optimisation methods provably converge only if particular parameters can be chosen. However, this choice is closely related to establishing an upper bound of the Lipschitz constant. This is normally intractable for large models and only estimations can be used in practice, thereby potentially resulting in unsound results, as we empirically observed.
[8
24 60]
The key
Conclusions
We presented H²V, a novel method for the robustness validation of NNs against geometric perturba- tions. We demonstrated that H²v outperforms SoA robustness verification methods against geometric perturbations and scales to vision classification models of hundreds of millions of tunable parameters and large inputs, enabling, for the first time to our knowledge, the validation of large video classifiers. These results enable the rigorous validation of present vision systems including transformer- baased architectures. We noted that, differently from several present verification methods, the theoretical soundness of H2V cannot be guaranteed in all cases, but we presented the reasons why we do not regard this as a limitation in practice.
Acknowledgements
Alessio Lomuscio acknowledges partial support from the Royal Academy of Engineering via a Chair of Emerging Technologies. We would like to thank Jean-Guillaume Durand and Christopher Brix for their suggestions to improve the paper. All authors are grateful to the anonymous referees for valuable comments that led to considerable improvements to this article.
10

---

References
[1] lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. arXiv preprint arXiv:/412.65722 2014.
[2] Xiaowei Huang, Daniel Kroening, Wenjie Ruan, James Sharp, Youcheng Sun, Emese Thamo, Min Wu, and Xinping Yi. A survey of safety and trustworthiness of deep neural networks: Verification, testing. adversarial attack and defence, and interpretability. Computer Science Review, (700070))) 100270, 2020.
[3] Guy Katz, Clark Barrett, David L Dill, Kyle Julian, and Mykel J Kochenderfer. Reluplex: An efficient SMT solver for verifying deep neural networks. In Proceedings of the 29th International Conference on Computer Aided Verification (CAVI7), Lecture notes in computer science, pages 97-117. Springer, 2017.
[4] Christopher Brix, Stanley Bak, Taylor T Johnson, and Haoze Wu. The fifth international verification of neural networks competition (VNN-COMP 2024): Summary and results. arXiv preprint 2412.19985, 2024.
[5] Naveed Akhtar, Ajmal Mian, Navid Kardan, and Mubarak Shah. Advances in adversarial attacks and defenses in computer vision: A survey. IEEE Access, 9:155161-15519, 2021.
[6] Alex Serban, Erik Poll, and Joost Visser. Adversarial examples on object recognition. ACM Computing Surveys, 53(3):1 1-38, 2021.
[7] Kai Jia and Martin Rinard. Exploiting verified neural networks via floating point numerical error. arXiv preprint arXiv:2003. 03001,, 2020.
[8] Xingjian Zhou, Keyi Shen, Andy Xu, Hongji Xu, Cho-Jui Hsieh, Huan Zhang, and Zhouxing Shi. SoundnessBench: soundness benchmark for neural network verifiers. arXiv preprint arXiv:2412.03154, 2025.
[9] Daniela Lera and Yaroslav D Sergeyev. Global minimization algorithms for holder functions. BIT Numerical Mathematics, 42: 19-133, 2002.
[10] Daniela Lera and Yaroslav D Sergeyev. Lipschitz and hölder global optimization using space- filling curves. Applied Numerical Mathematics, 60(1-2):111-122) 2010.
[11] Yaroslav D Sergeyev, Roman G Strongin, and Daniela Lera. Introduction to global optimization exploiting space-filling curves. Springer, 2013.
[12) Vladimir Grishagin, Ruslan Israfilov, and Yaroslav Sergeyev. Convergence conditions and numerical comparison of global optimization methods based on dimensionality reduction schemes. Applied Mathematics and Computation, 318:270-280, 2018.
[13] Houshang Sohrab. Basic Real Analysis. Springer, 2014.
[14] Giuseppe Peano. Sur une courde qui remplit touteune aire plane. Mathematische Annalen, 36(1):157-160, 1890.
[15] David Hilbert. Ueber die stetige abbildung einer linie auf ein flächenstück. Mathematische Annalen, 38(3):459-465 1891.
[16] Roman Strongin and Yaroslav D Sergeyev. Global optimization with non-convex constraints. Springer, 2013.
[17] Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint 1:1.v:1312.6199. 2013.
[18] Wenjie Ruan, Xiaowei Huang, and Marta Kwiatkowska. Reachability analysis of deep neural networks with provable guarantees. Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI18), page 265 1-269,, ijcai.org, 2018.

---

[19] Panagiotis Kouvaros and Alessio Lomuscio. Formal verification of CNN-based perception systems. arXiv preprint arX1v:1811.11333, 2018.
[20] Max Jaderberg, Karen Simonyan, Andrew Zisserman, and Koray Kavukcuoglu. Spatial trans- former networks. In Proceedings of the 29th Annual Conference on Neural Information Processing Systems (NeurIPSI55, pages 2017-2025. Curran Associates Inc., 2015.
[21] Ben Batten, Yang Zheng, Alessandro De Palma, Panagiotis Kouvaros, and Alessio Lomuscio. Verification of geometric robustness of neural networks via piecewise linear approximation and lipschitz optimisation. Proceedings of the 27th European Conference on Artificial Intelligence (ECAI24), pages 2362-2369. IOS Press, 2024.
[22] Fu Wang, Peipei Xu, Wenjie Ruan, and Xiaowei Huang. Towards verifying the geometric robustness of large-scale neural networks. In Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAA/23), pages 15197-15205. AAAI Press, 2023.
[23] Eric Gourdin, Brigitte Jaumard, and Rachid Ellaia. Global optimization of hölder functions. Journal of Global Optimization, 8(4):323-34 348, 1996.
[24] Chi Zhang, Zhen Chen, Peipei Xu, Geyong Min, and Wenjie Ruan. Verification on out-of- distribution detectors under natural perturbations. Machine Learning, 114(3), 2025.
[25] Gabriel Altay.
hilbertcurve (python package).
pttps://pypi.org/prroo
hilbertcurve/
[26] Daquan Zhou, Zhiding Yu, Enze Xie, Chaowei Xiao, Animashree Anandkumar, Jiashi Feng, and Jose M Alvarez. Understanding the robustness in vision transformers. In Proceedings of the 39th International Conference on Machine Learning (ICML22), pages 17377-27339 PMLR, 2022.
[27] Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijaya- narasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, Mustafa Suleyman, and Andrew Zisserman. The kinetics human action video dataset. arXiv preprint
2017.
[28] Haoqi Fan, Tullie Murrell, Heng Wang, Kalyan Vasudev Alwala, Yanghao Li, Yilei Li, Bo Xiong, Nikhila Ravi, Meng Li, Haichuan Yang, et al. Pytorchvideo: A deep learning library for video understanding. Proceedings of the 29th ACM International Conference on Multimedia (ACMMM21), pages 3783-3786. 2021.
[29] Kensho Hara, Hirokatsu Kataoka, and Yutaka Satoh. Learning spatio-temporal features with 3d residual networks for action recognition. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCVI7) Workshops, pages 3154 3154-3160, 2017.
[30] Du Tran, Heng Wang, Lorenzo Torresani, Jamie Ray, Yann LeCun, and Manohar Paluri. A closer look at spatiotemporal convolutions for action recognition. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPRI8), pages 6450-6459, 2018.
[31] Christoph Feichtenhofer. X3d: ceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR20), pages 203-213, 2020.
Expanding architectures for efficient video recognition. In Pro-
[32] Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? new model and the kinetics dataset. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR17), pages 6299-6308, 2017.
[33] Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. Non-local neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR18), pages 7793,7803, 2018
[34] Min Wu and Marta Kwiatkowska. Robustness guarantees for deep neural networks on videos. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition (CVPR20), pages 311-320, 2020.
12

---

[35] Samuel Sasaki, Diego Manzanas Lopez, Preston K Robinette, and Taylor T Johnson. Robustness verification of video classification neural networks. In Proceedings of the 13th IEEE/ACM International Conference on Formal Methods in Software Engineering (FormaliSH255), pages 22-33. IEEE, 2025.
[36] Huan Zhang, Shiqi Wang, Kaidi Xu, Linyi Li, Bo Li, Suman Jana, Cho-Jui Hsieh, and J Zico Kolter. General cutting planes for bound- wwwttddooooptttaassseeeeee neural network verification. In Pro- ceedings of the 36th Annual Conference on Neural Information Processing Systems (NeurIPS22),
pages 1656-1670. Curran Associates Inc., 2022.
[37] Hai Duong, Thanhvu Nguyen, and Matthew Dwyer. A DPLL(T) framework for verifying deep neural networks. arXiv preprint arXiv:2307. 10266, 2023.
[38] Haoze Wu, Omri Isac, Aleksandar Zeljić. Teruhiro Tagomori, Matthew Daggitt, Wen Kokke, Idan Refaeli, Guy Amir, Kyle Julian, Shahaf Bassan, et al. Marabou 2.0: a versatile formal analyzer of neural networks. In Proceedings of the 36th International Conference on Computer Aided Verification (CAV24), pages 249-264. Springer, 2024.
[39] Augustin Lemesle, Julien Lehmann, and Tristan Le Gall. Neural network verification with PyRAT. arXiv preprint arXiv:2410.23903, 2025.
[40] Changliu Liu, Tomer Arnon, Christopher Lazarus, Clark Barrett, and Mykel J Kochenderfer. Algorithms for verifying deep neural networks. arXiv preprini arXiv: 1XXiv:903330673: 2019.
[41] Vincent Tjeng, Kai Xiao, and Russ Tedrake. Evaluating robustness of neural networks with mixed integer programming. arXiv preprint arXiv:1711.07336, 2017. 2017.
[42] Alessio Lomuscio and Lalit Maganti. An approach to reachability analysis for feed-forward ReLU neural networks. arXiv preprint 1706.07351, 2017.
[43] Elena Botoeva, Panagiotis Kouvaros, Jan Kronqvist, Alessio Lomuscio, and Ruth Misener. Efficient verification of relu-based neural networks via dependency analysis. In Proceedings of the 34th AAAI Conference Artificial Intelligence (AAAI20), pages 3291-3299, AAAI Press, 2020.
[44] Panagiotis Kouvaros and Alessio Lomuscio. Towards scalable complete verification of relu neural networks via dependency-basee based branching. In Proceedings of the 30th International Joint Conference on Artificial Intelligence (IJCA12I), pages 2643-220 26540.650. ijcai.org, 2021.
[45] Timon Gehr, Matthew Mirman, Dana Drachsler-Cohee Petar Tsankov. Swarat Chaudhuri, and Martin Vechev. AI:: Safety and robustness certification of neural networks with abstract interpretation. In JEEE Symposium on Security and Privacy (SP18), pages 3-18. IEEE, 2018.
[46] Gagandeep Singh, Timon Gehr, Matthew Mirman, Markus Püschel, and Martin Vechev. Fast and effective robustness certification. In Proceedings of the 32nd Annual Conference on Neural Information Processing Systems (NeurIPSI8) page 10825-10836. Curran Associates Inc., 2018.
[47] Gagandeep Singh, Timon Gehr, Markus Püschel, and Martin Vechev. An abstract domain for certifying neural networks. Proceedings of the ACM on Programming Languages, 3(POPL):11 30, 2019.
[48] Niklas Kochdumper, Christian Schilling, Matthias Althoff, and Stanley Bak. Open-and closed- loop neural network verification using polynomial zonotopes. In Proceedings of the 15th NASA Formal Methods Symposium (NFM23), pages 16-36. Springer, 2023.
[49] Diego Manzanas Lopez, Sung Woo Choi. Hoang-Duun Tran, and Taylor T Johnson. Nnv 2.0: The neural network verification tool. In Proceedings of the 35th International Conference on Computer Aided Verification (CAV23), pages 397-412. Springer, 2023.
[50] Kaidi Xu, Huan Zhang, Shiqi Wang, Yihan Wang, Suman Jana, Xue Lin, and Cho-Jui Hsieh. Fast and complete: Enabling complete neural network verification with rapid and massively parallel incomplete verifiers. In Proceedings of the 9th International Conference on Learning Representations (ICLR2I). OpenReview.net, 2021.
13

---

[51] Rudy Bunel, Jingyue Lu, Ilker Turkaslan, Philip HS Torr, Pushmeet Kohli, and M Pawan Kumar. Branch and bound for piecewise linear neural network verification. Journal of Machine Learning Research, 21(42):1-39 2020.
[52] Huan Zhang. Tsui-Wei Weng. Pin-Y Yu Chen, Cho-Jui Hsieh, and Luca Daniel. Efficient neural network robustness certification with general activation functions. In Proceedings of the 32nd Annual Conference on Neural Information Processing Systems (NeurIPS18), page 4944-4953.
Curran Associates Inc., 2018.
[53] Shiqi Wang, Huan Zhang. Kaidi Xu, Xue Lin, Suman Jana, Cho-Jui Hsich, and J Zico Kolter. Beta-crown: Efficient bound propagation with per-neuron split constraints for neural network robustness verification. In Proceedings of the 35th Annual Conference on Neural Information Processing Systems (NewrIPS2I), pages
Curran Associates Inc., 2021.
[54] Patrick Henriksen and Alessio Lomuscio. DEEPSPLIT: an efficient splitting method for neural network verification via indirect effect analysis. In Proceedings of the 30th International Joint Conference on Artificial Intelligence (IJCA121), pages 2549-2555. ijcai.org, 2021.
[55] Panagiotis Kouvaros, Benedikt Brückner, Patrick Henriksen, and Alessio Lomuscio. Dynamic back-substitution in bound- wwwddddoooationbaassseee neural network verification. In Proceedings of the 39th AAAI Conference on Artificial Intelligence (AAA125), pages 27383-27391, AAAI Press, 2025.
[56] Benedikt Brücckner and Alessio Lomuscio. Verification of neural networks against convolutional perturbations via parameterised kernels. In Proceedings of the 39th AAAI Conference on Artificial Intelligence (AAAI25), pages 27215-27223. AAAI Press, 2025.
[57] Linyi Li, Maurice Weber, Xiaojun Xu, Luka Rimanic, Bhavya Kailkhura, Tao Xie, Ce Zhang, and Bo Li. TSS: Transformation-speeiii smoothing for robustness certification. In Proceedings of the 202/ ACM SIGSAC Conference on Computer and Communications Security (CCS21), pages
ACM, 2021.
[58] Zhongkai Hao, Chengyang Ying, Yinpeng Dong, Hang Su, Jian Song, and Jun Zhu, GSmooth: Certified robustness against semantic transformations via generalized randomized smoothing. In Proceedings of the 39th International Conference on Machine Learning (ICML22), pages 1465-8483 PMLR, 2022.
[59] Mislav Balunovic, Maximilian Baader, Gagandeep Singh, Timon Gehr, and Martin Vechev. Certifying geometric robustness neural networks. In Proceedings of the 33rd Annual Con- ference on Neural Information Processing Systems (NeurIPS19), pages 15313 1532. Curran Associates Inc., 2019.
[60] Chi Zhang, Wenjie Ruan, and Peipei Xu. Reachability analysis of neural network control systems. arXiv preprint arXiv:230f./2/000 2023.
[61] Rem Yang, Jacob Laurel, Sasa Misailovie, and Gagandeep Singh. Provable defense against geometric transformations. In Proceedings of the 1 11th International Conference on Learning Representations (ICLR23). OpenReview.net, 2023.
[62] Kaidi Xu, Zhouxing Shi, Huan Zhang, Yihan Wang, Kai-Wei Chang, Minlie Huang, Bhavya Kailkhura, Xue Lin, and Cho-Jui Hsich. Automatic perturbation analysis for scalable certified robustness and beyond. In Proceedings of the 34th Annual Conference on Neural Information Processing Systems (NeurIPS2O), pages 1129-1141. Curran Associates Inc., 2020.
[63] Hirokatsu Kataoka, Tenga Wakamiya, Kensho Hara, and Yutaka Satoh. Would mega- mega-scale datasets further enhance spatiotemporal 3D CNNs? arXiv preprint arXiv:2004. 2020.
[64] Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. UCF101: A dataset of 101 human actions classes from videos the wild. arXiv preprint 12t.iv::112.04442 2012.
[65] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint 2017.
[66] Donald R Jones and Joaquim R R A Martins. The DIRECT algorithm: 25 years later. Journal of Global Optimization, 79(3):521-566, 2021.
14

---

A Proof of Theorem 1
Lemma 1. (11 Theorem 2.13) Let li be the solution of the lower bound along the Hilbert curve m (x) for f(0) satisfying the Lipschitz condition with constant L:
1 S min (m min f(z), I [0.1].
(11)
Considering the infinite optimisation budget, i.e,, c = O, then the true lower bound for the multi- dimensional problem min f() in the original space the entire region [0, 1) 1|N:
1/ L.2 (m++1) SN..
(12)
Proof can be found in 10 Thm. 2.13.
Theorem 1, Let 17 be a lower bound of the one-dimensional problem min f(x) s.t. x E [0, over the Hilbert space-filling curve. Then we have that
2. - H (c/2) St"
(13)
where (* the optimal solution of the multi-dimensional problem min f(0) s.t. 0 E [0, 1Ja
Proof. The first calibration term directly comes from Lemma. 1 Let f [0, 1] satisfy the
Hölder condition with exponent I/N and constant H > O:
1 f(v) s H |x 2/N, Vx, [0.
Assume that during V optimisation process, Theorem in the main manuscript has been satisfied, i.e., r* such that A. 2 H, for each interval, the algorithm terminates when the selected interval [2y-1, .T.] has length 0, = Ir; 21-1l S
Let
x = arg min f(x), f = arg fins sampled endpoints x;}, 2E[0.1]
and denote closest to the true minimizer x². Then, upon termination (9, the true minimizer x" will only fall into the chosen interval. By definition of as the nearer endpoint, we have
f(t). Let & represent the endpoint of the interval [2;=1, that is
gl -nin(l=-=---1l11 12" x1
1x.
34
1x"
xz
5,
(14)
2
2
2
From the Hölder condition for the function f:
1/N
(f(E) - f(x*)) 11 x² 1IN < H1 (5)
(15)
Since j ming f(xx) s f(E).
- f(x f( < 1 ( 1/N
(16)
Therefore, H 1/N is incorporated into the lower bound in Eq. 13 the second calibration
term, thereby completing the proof.
Summary: The first calibration term results from the approximation of the Hilbert curve reduction, while the second a consequence of the limited optimisation budget. When the resolution of Hilbert approximation is high enough, e.g., = 50 in our experiments, the magnitude of the former term is negligible. Differently, the magnitude of the latter term grows with the number of dimensions, thus hindering the efficacy of H2V to high-dimensional input domains.
15

---

B
Ablation Study
In this section report several ablation studies for some ryper-parametert used in method. Specifically, we conducted ablation studies on the reliability parameter y on two benchmarks with ground-truth: TLL Verify Bench from VNN-COMP (2 dimensions) and MLP5 benchmark from BeundnessBench (10 dimensions). The timeout is set to 100 seconds. We report the counts of Robust, Non-Robust, and Unknown cases, respectively, along with the average runtime in seconds. The runtimes are computed with respect to the robust and non-robust cases and do not include the timeouts. note an
The results show that H²V outputs unsound results for any value of 1. However, increased difficulty in answering verification queries with bigger values of r. Empirical results from the optimisation literature [TT] also suggest that the use of small value close 1, e.g., 1.3, yields close to optimal solutions. This is aligned with the observations in [
In addition, the computational complexity of modern libraries providing the Hilbert space mapping grows linearly in m, so any value of m > 30, which already provides small calibration errors (see mathematical expression (Eq. 10 in the paper), would be adequate to use. We report ablation study on m in Table which confirms this intuition: any value of m > 10 resolved all verification queries with stable overheads.
Also note that the optimisation budget corresponds to the typical optimisation gap present in all convergent algorithms. As shown in Table smaller value generally enables the method to solve more verification queries, as it allows exploration finer- graingd search space. value is thus contingent to available computation and temporal resources.
Table 5: Ablation study on the reliability parameter r.
Benchmark
Perturbation Dimensionality
No. Params
No. Robust (Sound)
No. Non Robust
No. Unknown
Average
Runtime (s)
1.1
(5(15)
26.45
1.3
15 (15)
27.81
1.5
15 (15)
26.06
TLL Verify Bench
17k-67M
(12)
6
6 (6)
17
17.60
(
1.1
(10)
47.08
10
SoundnexsBBncc ML.55 (epsilon 0.2)
1.5
(
39.27
3.13M
6
(
13
68.20
10
(:00)
Table 6: Ablation study the Hilbert curve resolution parameter m.
Benchmark
Perturhation Dimensionality
No. Params
m
No. Robust (Sound)
No. Non-Robust
No. Unknowa
Average Runtime
(
30
(15)
17
25.22
TLLL Verify Bench
2
17k 67M
15 (15)
26.06
70
15 (15)
17
26.57
(15)
27.39
10
10 (10)
44.85
(10)
$6.35
SoundnessBench MLP5 (epsilon 0.2)
10
3.13M
50
10 (10)
2
50.81
70
(10)
52.29
90
(9)
2
47.92
C Additional Experimental Evaluation
We here provide additional experimental results for image and video tasks.
16

---

Table Ablation study on the optimisation budget parameter
Benehmark
Perturbation Dimensionality
No. Params
No. Robust (6ound)
No. Non-Robust
No. Unknown
Runtime
Average
6
15 (15)
17
0
25.91
TLL Verily Bench
2
17k-67M
155)
26.60
(15)
Ie 15
15 (15)
17
0
26.06
0 (0)
14
5 (5)
57.16
SoundnessRench MLP5 (epsilon (.2)
10
3.13M
Ic
(10)
2
45.10
Ie 13
(00(10)
46.23
le (10)
50.81
C.1 Verification on Image Classification
We return to the simple case where we consider geometric perturbation in one dimension only. Table reports the results obtained on the MNIST and CIFAR-I0 datasets. H2V outperforms other methods, including TSS [57] GeoRobust [2], DeepG [59], and GSmooth [58]
Table 8: Comparing with baseline methods on MNIST/CIFAR-IO against rotation and scaling. Baselines performance adopted from 22 57 58).
Dunset
Model
Claaa Ace
Pert
Advenarial Accuracy (%)
Robust Accuracy
GridSeachh
HPV
DeepG
MINST
Small
CNN
90.4
S(309L)
R:50")
99.2
99.2
08.2
99.2
98.2
$5.0
85.8
95.9
95.7
97.2
99.2
55 35
83:45
84.0
R(10")
74.8
74.8
74.8
52.5
55.6
74.6
216.35
81.2
R(30")
69.4
66.6
66.4
66.4
10.6
63.6
66.2
66.4
169.37
80.8
S(305))
67.0
63.4
63.4
6344
00.0
54.3
58.8
02.8
03.4
170.23
Additionally, conducted experiments on Tiny ImageNet evaluate the WideResNet (with 7.94M parameters), adopted from Certified Geometric Training (CGT) [61] which uses auto LiRPA [62 to perform geometric robustness verification. Experimental results in TallstSillsstaaee all images and returns any counterexample found.
illustrate that H2V verifies
Table 9: Comparing with baseline method CGT on the Tmay-lm against rotation and scaling perturbations.
Timeouts (%)
Robust Accuracy (%)
Average Runtime (s)
Perturbation
Clean Accuracy (%)
CGT
H2V
CGT
H²y
HºV
S(2%)
33.10
4.06
0.00%
21.44
25,50
48,99
R(5%)
32.01
3.98
0.00%
17.49
21.47
45.58
C.2 Verification on Video Classification
We reported the results on several large 3D-NNs on the Kinetics dataset in the main manuscript. Here report results geometric robustness validation the UCF101 dataset.
Long-length video classification often requires segmentation into shorter clips, achieved by consider- ing [Petp]- Two metrics are often used for assessing model performance: clip-level accuracy and video-level accuracy. Clip-level accuracy records the proportion of correctly predicted clips across all videos. Video-level accuracy is computed by aggregating the predicted probabilities of individual clips each video and using the accumulated probabilities to compute the final prediction accuracy at the video level. Let Vαx, 0) represent the perturbed video/clip. we then analyse the video classifier's robustness by simulating the effects of geometric distortions, including tilted camera orientation (rotation), positional shifts (translation), and zoom variations (scaling). We focus
17

---

geometric robustness against frame-agnostic perturbations at the video level. In this setting. the same transformation is applied consistently across all frames, and we evaluate the classifier's ability to handle such uniform geometric distortions. This setup captures the overall impact of static transformations, such as those caused by incorrect camera positioning. The experiments conducted on the Kinetics dataset presented in the main manuscript also belong to this category.
Analogously, the local geometric robustness problems for video classification can be formulated as checking whether v0 E - f(0) > O. In our setting this is solved by evaluating Vor E [0, 1] f(x)) > 0 in the corresponding optimisation problem. We here focus on a 3ID-ResNet50 model, adapted from [33] to explore its robustness on the UCF101 dataset [64]. The model processes entire videos using frame length of 16 and frame rate of 1, with 46.4M network parameters. Since the frame-agnostic perturbations are applied uniformly across all frames in a video, the number of perturbation dimension ranges from to 4.
We select the first two videos of each class in the test set, which consists of 202 videos in total. Since we are not aware of any previous work supporting this setup. to provide a baseline comparison, we adapted another global optimisation based verification method, DeepGO [18] with a pre-defined Lipschitz constant L = 8 (follow their settings), PGD [65 and as two strong baselines for comparison. We controlled the termination of all methods by setting maximum number of queries K 100, 000 and timeout as 2400 seconds (40 minutes). The 31>RRsNNet50 model we are adopting achieves 87.13% clean accuracy across the 202 videos. Detailed results against various consistent geometric perturbations at video level are summarised in Table 10 As we can see, H&V obtains the best adversarial accuracy and robust accuracy and these values coincide. This means that the method could solve all queries for this model and inputs with no unknowns. We are also able to establish that for the parameters analysed, rotation has the highest impact on the fragility the model. As discuss in the main part of the paper, global optimisation- based verification methods without guarantees on the soundness of the upper bound the Lipschitz constant may converge to local minimum. As consequence, they may potentially derive unsound results, indicated in Table I0]
Table 10: Evaluation of static geometric robustness for a 3D-ResNet50 model with 87.13% clean video accuracy on UCF101 dataset. Below R, S, T refer to R(20)), ((10%), T(10%), respectively.
Perturbation Dimensionality
Adversarial Accuracy (%)
Robust Accuracy (%)
Average Runtime (8)
Perturbation
PGD
H²V
DeepGO
H
H²V
51.98
43.07
45.05
43.07
458.10
2
76.73
66.83
70.30
66.83
634.15
1
75.74
74.75
75.74
74.75
929.69
R + T
3
48.02
34.65
37.13
34.65
358.03
R + S
2 50.10
35.64
36.14
35.64
342.10
T
3
72.77
58.91
61.39
58.91
613.48
S
4
40.10
22.77
25.25
22.77
286.33
Figure 2 illustrates the impact of rotation (R), translation (T), scaling (S), and their combination on the static geometric robustness in video classification under different perturbation magnitudes. Note that H²V able to solve all the robustness verification among the following perturbation settings. A clear trend emerges, indicating that the perturbation magnitude increases, robust accuracy consistently declines. While individual transformations already contribute to performance degradation, their combinations exacerbate the effect, leading to even more significant accuracy drops. Among these three geometric factors, scaling (S) appears to be the most stable than shifting and angular transformations, as its impact on accuracy degradation is relatively lower compared to rotation and translation.
C.3 Detailed Experimental Results on SoundnessBench and VNN-COMP
Table 11 reports detailed verification results on BoundnessBench [ for a, C.R-CRON PYRAT and H respectively, with default timeout setting 100 seconds for each instance. It can be seen that,
hwwe////eiiccceelliieeee//ppppaa https://git.framaee https//ggifaaaaecc.oo//////////p cooppbbbyyyt/
18

---

Aecuncy
RM5,, 5128) 11293
R110") 5(5%) TI5%)
R120%). 5(10%) 1110%
Cortited 0.6
0.0
Porturbation increasing Magnitude
Figure 2: Robust accuracy decreases as perturbation magnitude increases.
differently from previously reported, in our tests, all three tested tools now pass the soundness tests. Previously [8] identified unsound results in a,8-CROWN [36). It can be observed that H²V provides more positive answers for the verification queries, and finds counterexamples for queries that cannot be solved by other tools. This highlights its effectiveness in NN verification.
For additional benchmarking from VNN-COMP 4 we consider TLL Verify Bench, which has the low-input dimensionality of 2 of the perturbation. The TLL Verify Bench consists of Two- Level Lattice (TLL) NNs; it has been proposed as a means of comparing TLL-specific verification algorithms with general-purposs purpose NN verification algorithms. We used the default timeout setting, 600 seconds, described in VNN-COMP 2023 [4. H2V achieved the same SOA performance on these two benchmarks a CROWN.
Table 11: Soundness validation on SoundnessBench. Numbers in bold denote that counterexamples were found.
Peturbation Settings
No. (Clean Lives))
Resnlts for Clean Enstinees (No. UNSAT Unknown)
Resolts for Unverifablle Instances (No. UNSAT | SAT Unknown)
Avenge Rumtime (5)
Madel Name
Topun Size
+3-CRRWW
PyRAT
HPV
w/1-CRONN
PyRAT
HPV
e.8CROWN
PyRAT
HPV
CNN 1 Cons
0.2
1x5555
10
10
10
10 10
0 0
0
0 0
1165
1.02
59.92
CNN 1 Cons
0.2
3xSus
10
911
0
10
10 0
0 0/019
010
01 0
13.51
15.87
73.68
7
0/0
010
Cone
0.5
3x5vd
0/0|8
CNN 2 Com-
0.2
1x5x5
10
9
5
0
0
2.11
15.32
58.74
CNN 2 Com
0.2
8x5x55
10
3
0 10
6
0 0/017
010
0 0
15.38
70.92
CNN 2 Com
0.5
10 10
10
01
0 0101
010
010
CNN Con-
0.2
1xdx5
10
110
9.63
64.14
CNN 3 Cons
0.2
10
10
10
19
0 0
0
69.45
CNN 3 Coms
0.5
1x5s5
10
10
10
21 8
0 0/019
01019
0 019
$3.89
0
AvgPool
1x5x5
01010
01010
CNN ArgPool
02
3xdx5
10
01010
86.15
CNN AvgPood
0.5
1x5x5
10:9
0 10
01 10
1199
01010
01019
011
70.52
CNN AugPool
0.5
5x55
10
01 10
10
0.
01812
010
0
0.74
10
0
010
0
58.04
CNN Tanh
02
3xd5xd
110
69.23
CNN Sigmoid
02
1x5x5
10
2
01011
010
0.50
0.36
56.45
CNN Sigmoid
0.2
3xdxx5
10 10
10
0 10
10110
0
010
0 0 10
70.43
MLP Hidden
02
10
10 9
10
10
10
0/217
01019
019
0,30
2.10
49.05
u.5
119
3
0/019
0
019
MLP5 Halda
02
MLP5 Hidden
0.5
10
10
119
1
713
0/019
01019
0 019
1.31
16.83
53.66
19

---

D Comparison with the State-of-the-Art Methods
In this section, we compare H²V with aB--CRWN, the SoA tool from VNN-COMP 4.. The verification queries from our main experiments cannot be used for this comparison. Firstly, the tools do not support the affine -anee and grid_sampl required for the geometric perturbations via the Spatial Transformer Networks. Secondly, the tools do not scale to the sizes of the models considered in the main experiments. To illustrate this, we used the Reslet188ccife the a33-RROS repository, and considered different model sizes by adjusting the width of the layers through thje in_planes parameter of the benchmark. For the verification of the models, we constructed one-pixel perturbations (corresponding to three dimensions), serving comparable low-dimensional proxy with H²V capabilities. We then performed verification queries for e 0.1 and 0.3.
model from
Table 12 reports the results obtained on this simplified setting. We observe that a; 4.8-CROWN verifies the smallest case at c = 0.1 but either timeouts or runs out of memory (OOM) as increases or the model grows. In particular, a.3-CROWN runs out of memory (OOM) on a relatively small model with 175,002 parameters. In contrast, H consistently returns robust with lower and more stable memory consumption. conclusion, SoA methods cannot scale to models of tens/hundreds of millions of parameters as we do here, or to ImageNet-scale inputs with geometric perturbations.
Table 12: Verification results against one-pixel perturbation for CIFARI0 models.
Model
In planes
No. Params
Perturbution Magnitude
Perturbation Dimensionality
Tool
Result
Peak Memory (MB)
Runtime (6)
PGD
Unknown
656
12.75
2
11,270
0.1
3
aR-CROWN
Robust
644
0.73
HºV
Robust
68.33
PGD
Unknown
656
13.32
ResNett8_coo
11,270
af-CROWN
Unknown
4298
Timeout
H"V
Robust
636
61.02
PGD
Unknown
678
14.16
175,802
0, CROWN
Unknown
Timeout
HFV
Robust
56.54
E Performance Analysis of H2V With Soundness Guarantees
In Section 3 we noted that if Step of H uses an overestimation of the Hölder constant, then the method is theoretically guaranteed to produce sound results. Here, we use H2V" denote H2V with such overestimations. We then compare the scalability of H²v and H²V* allowing us to empirically identify the practical limitations of H²
For this, we use the CIFARIO and RESNET models from VNNCOMP and 1-pixel perturbations. 13blc reportss the results. These show that 2 is able to verify the models associated with small Lipschitz constants but timeouts when the constant exceeds a certain value. In contrast, H2V validates all models and scales to models with significantly larger Lipschitz constants (e.g., around 8c+50 for ResNet34) and much larger models (over 300M parameters). as reported in the paper. We stress that, unlike much of the existing verification literature, model size itself is not the primary bottleneck in our setting. This instead the value of the Lipschitz constant, as these experiments demonstrate.
F Further implementation details of H²V
In the main manuscript, we have described the optimisation process of our method in detail. The pseudocode of H2V is also outlined in Algorithm 1 In the algorithm, we use K to control the maximum number of queries considered. The algorithm initiates the reliability parameter with r 1.3, as recommended by [II. When the algorithm converges (i.e., when it reaches the optimisation budget), the size the neighbourhood is increased nt. t- n1 + 1, and the value of the global Hölder constant iteratively loosened (i.e., hig <- h4 1.3), until different interval is selected in Step 16 of the algorithm. This process repeated until the the optimisation budget reached for the same interval for 25 times, at which point the estimated lower bound is used to answer the geometric
20

---

Table 13: Verification results on different models on the CIFARIO examples.
Model
Upper bound of Lipschitz Constant
No. Params
Perturbation Magnitude
Perturbation Dimensionality
Method
Result
Runtime (8)
HPV
Robust
marabou--ciaarllkffffrr 10_ [mmmal)
L2288c+02
2,456
0.1
Robust
2,68
HPV
Robust
53.37
marahou caaeahoouuillalOiiiiiii cilar 10__0mall
2,0159e+002
9,008
Robust
HPV
Robust
65.02
marabou IilffrociaaaaOOO [mmaall
4,2458e+02
H'V
Robust
161.73
#VV
Robust
56.98
cifar10 reebeee//tt//eseee22
2.2218e+08
112,006
H"V*
Unknown
Timeout
HFV
c ressne/rreeee
7. 138005c+13
123,734
0.1
3
HF
Unknown
Timeout
HPV:; IPV with un upper hound of Lipschitz constant, hence am upper hound of Hölder constant as well.
robustness query. As shown the experimental results, this solution provide a good approximation for the unknown Hölder constant.
G Additional Related Work
Besides what previously referenced, further approaches have been put forward for the verification of local robustness against geometric perturbations. We here compare these with H
By constructing the set allowed perturbation values using an l bound for each pixel and replacing it with convex relaxation, [9 47 performance bounds.
provide an over-approximmttio for the solution but result in loose
TSS 57 and GSmooth 58 demonstrate the potential of randomised smoothing to certify robust- ness against individual geometric transformations. However, these approaches provide statistical guarantees which are of different nature from other approaches. Also, as presented, they do not support combinations of transformations. Methods such as DeepG (59) and PWL [21 cater for combined transformations but rely computationally expensive techniques like DeepPoly 47 or VENUS making them infeasible for large NNs due to scalability issues.
investigates the worst-case combinations of transformations affecting a network's output for image inputs, by leveraging direct optimisation [66]. However, as shown in our main experiments, some unsound results have been revealed on the ImageNet dataset, which requires further analysis in terms of possible underestimations of the Lipschitz constant. Similar issues apply to existing verification methods based on global optimisation like DeepGO [18] as also shown in Table 10
GeoRobust 22
21

---

Algorithm 1 Geometric Robustness Validation of Neural Networks via H
Input:
Original input a and its corresponding label y; N-dimensional perturbation variables of the geometric transformation
[y, Phoiionntaly Evertical: A| with bounds 1, b R.,, Hilbert curve
mn (-), threat NN model; the property function f. the optimisation budget e, and the maximum number of queries K.
Output: {UNSAT |SA Unknown]
I: zo 0, - 1, k = 2
2: 2 f(x,) ((h(r;)) = 0,
h(()) is the m-approximation the Hilbert curve > Initialisation for set of initial intervals
3: O O.I - 1])
4: fm = min {f(a). f(b) [a I
5: if fm < 0 then
6: return SAT
> Misclassification
7: end if
8: while k < K do
9: for [a, b] E I do
10:
Compute the estimation the Hölder constant H;
11:
bta 8, 2
f(a)-f(a)
Compute the intersection point for the interval
28, (b-o)
12:
Zleft = M(a) H1(8: - a) 1/N & Compute the estimated lower bound from the left side
13:
Bright = f(b) H. 8,) L/N & Compute the estimated lower bound from the right side
14:
min(2left: Zright)
15:
end for
Locate the interval from I. with [a, b] who has minimum lower bound estimate
17:
if (b al S then
Compute the estimation of the lower bound of the function In min {4 IjezuO]. Compute the calibration of the lower bound wn + Im if (m 0 then
19:
21:
return UNSAT else
The property is validated to hold
23:
return Unknown end if
25:
end if
26:
Compute k
Accept 8; as the new trial point b Split the interval [a, bj according to 8;
27:
I e- I' (i) (1a 8.). 11.
Compute the current minimum of the function as fm = min f(a), f(b) [a, b] Z}.
28:
29:
if I then
30:
return SAT
b- Found a counterexample
end ir
32:
end while
return Unknown
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction have clearly stated the main contributions, scope and potential limitation.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discuss the limitation and potentially unsound results may be obtained via our algorithm, but this seldom happens.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
23

---

Answer:
[Yes]
Justification: The complete proof' are provided in the appendix. Theorems are properly referenced in this paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We describe the proposed algorithm in detail and include its pscudocode in the Appendix. An experienced researcher should be able to reproduce the method within reasonable time. The code will be open- sourced.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.

---

reproducible, they 5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [No]
Justification: The code will be open-sourced, and the datasets we evaluate are publicly available; no private benchmarks are used.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/nips. wwtpcc/pccppcpcceeeooooooop htte:/gguuddes/ooeeuuubbiiiii wwwpanssssonppoo ssinnolicyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Detailed settings for our experiments are provided in the experiments section and in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification: Our method is deterministic, producing identical results when executed on the same machine.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
25

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: Detailed settings, environment configurations, and software package informa-
tion for experiments are provided in the experiments section and in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http://////eerppp..cppppp//////e
Answer: [Yes]
Justification:
We have read and acknowledged the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Societal impacts have been discussed explicitly the introduction. As paper concerns a technical approach to Al Safety. Specifically, it is intended to contribute towards the assessment Al systems before they are deployed. We regard this as strongly positive social impact.
26

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We have properly cited the original paper that produced the code package or dataset.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
27

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the datasets.paperssiii paperswi thcode. www.com//aaaa datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
28

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMS as any important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy hatt:p//eearrrppccc/ooooeereeee/ /c/////oeeeeeeccc//22////.. for what should or should not be described.
29


Reviews:
[{"review_id": "Qpd5mDlnTu", "reviewer": "Reviewer_zTf7", "summary": "The work describes a verification approach, HOVer, that estimates the local robustness of neural networks to low-dimensional geometric attacks rotation, scaling, and translation. HOVer initially maps the N-dimensional perturbation space to one dimension along a Hilbert space-filling curve before applying an adaptive Hölder global-optimisation process, along with a set of heuristics to ensure that a reliable bound results. Empirical results show that HOVer scales to reasonably large models (ResNet-152, ViT-Large) and 3-D CNNs on video classification, and outperform prior work (GeoRobust) while remaining reliable (no incorrect results) on 500 ground-truth queries of SoundnessBench and VNN-COMP.", "questions": "How many queries saw their outcome change with growing r, and what was the accompanying runtime overhead?\n\nHow would the method scale if more geometric parameters (e.g. shear, aspect-ratio) are taken into account?", "limitations": "Yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\nThe paper describes a new combination of Hilbert curve dimensionality reduction and adaptive Hölder optimization to allow 1-D search over 4-parameter geometric spaces.\n\nThe evaluation shows that this approach scales to the largest models (to my knowledge) for verified video tasks.\n\nAlthough the certified accuracy of HOVer is comparable to GeoRobust, and the adversarial accuracy is lower on most models, it appears that HOVer is likely the better solution for certified classification as GeoRobust is unsound.\n\n**Weaknesses**\n\nAlthough most prior geometric certification techniques could not come close to scaling to models of the size used in this eval, it would still be helpful to see how the performance compares to those methods, albeit on smaller models.\n\nThe evaluation covers accuracy and soundness, but does not mention anything about the cost and how it scales with model and data size. Without this, the audience cannot tell how practical the method really is.\n\nWhile the approach seems to improve on prior work in terms of model scale, the significance of doing so for certified geometric robustness is somewhat limited. There is quite a bit of prior work on this problem, but its lack of impact on practical applications does not stem from the cost of those methods, but rather the impact that they have on accuracy and the fact that the threat model is not well aligned with practical concerns.", "was_revised": false, "final_justification": "", "rebuttal": "**Question 1.**  \nEmpirical results from the optimisation literature suggest that using a small value close to 1, e.g., 1.3, yields close\nto optimal solutions [1]. We report an ablation study on r in Table 1, which confirms these empirical observations. Specifically, we conducted three ablation studies on the reliability parameter r on two benchmarks where the ground-truth for the verification queries is known: TLL Verify Bench from VNN-COMP (2 dimensions) and MLP5 from SoundnessBench (10 dimensions).\nWe report the counts of SAFE, UNSAFE, and Unknown cases, along with the average runtime in seconds (SAFE+UNSAFE, excluding Timeout). The results show that HOVer outputs no unsound results for **any value of r>1**, but it becomes increasingly challenging to solve verification queries with bigger values of r. This is in line with literature findings [1] which suggest that big values for r should not be used.\n\n\n\n**Table 1: Ablation study on the reliability parameter r on Benchmark TLL and MLP5**\n| **r** |TLL|SAFE (sound)|UNSAFE|Unknown|Runtime (s)| MLP5|SAFE (sound)|UNSAFE|Unknown|Runtime (s)|\n|-----------|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------|\n| **1.1**   ||15(15)|17|0|1.00||  16(16)|3|14|20.92|\n| **1.3**   ||15(15)|17|0|1.04|| 14(14)|1|18|10.58|\n| **1.5**   ||13(13)|17|2|1.16|| 10(10)|2|21|10.58|\n| **3**   ||1(1)|17|14|0.12||0|2|31|11.96|\n| **6**   || 0|17|15|0.03||0|2|31|59.56|\n| **10**  | |0|17|15|0.03||0|2|31|32.79|\n\n\n**Question 2.** \n\nSince any composition of scaling, shearing, rotation, and translation can be represented as an affine\ntransformation of only 7 parameters, HOVer scales to the models reported in the paper for any composition of transformations.\n\nSpecifically, although our initial experiments considered only 4 parameters, our approach extends seamlessly to the 7‑parameter affine group by replacing the transformation matrix with: \n\n$$\n \\left[\n\\begin{matrix}\n    \\lambda_x (\\cos \\gamma + s_x \\sin \\gamma) &  \\lambda_y (-\\sin \\gamma + s_x \\cos \\gamma) & t_x \\\\\\\\\n    \\lambda_x (\\sin \\gamma + s_y \\cos \\gamma) & \\lambda_y (\\cos \\gamma - s_y \\sin \\gamma) & t_y \\\\\n\\end{matrix} \n\\right]\n$$\n\nwhere\n$\\lambda_x$ and $\\lambda_y$ are independent scaling factors along the horizontal and vertical axes;\n$\\gamma$ is the rotation angle;\n$s_x$ and $s_y$ are the horizontal and vertical shear coefficients;\n$t_x$ and $t_y$ represent translations along the x-axis and y-axis.\n\n\nWe evaluated 100 randomly sampled ImageNet images using ResNet‑50 with our method (HOVer) under both 4‑parameter and 7‑parameter perturbations. The termination threshold was set to 500 seconds or 50,000 queries per sample. As shown in Table 2, which reports the SAFE/UNSAFE/Unknown cases and the average runtime for the SAFE + UNSAFE cases, there is only a modest performance drop when moving from isotropic scaling to anisotropic scaling plus shearing (due to the increased number of geometric parameters and the resulting higher number of unsolved cases). Nevertheless, HOVer remains able to verify higher‑dimensional geometric transformation spaces.\n\n\n**Table 2: Robust performance on 100 ImageNet images (ResNet‑50), with clean Acc 70%**\n\n|  Geometric Perturbation| SAFE                 | UNSAFE| Unknown | Runtime (s)|\n|----|------------------------|--------------------|--------------|--------------|                                                           \n| Rotation + Translation + Isotropic Scaling                         | 33  |      60   |      7      | 59.49 |            \n| Rotation + Translation + Anisotropic Scaling + Shearing                        | 20  |      66   |      14      | 32.36|   \n\n**[Scaling]**\n\nIn Appendix B, we have also reported the performance comparisons on smaller models for the MNIST, CIFAR‑10, and TinyImageNet datasets, where we evaluate against prior geometric certification techniques, including DeepG, GSmooth, TSS, and Auto_LiRPA with CGT. \n\nThe timing tables below report the average running time (in seconds) for SAFE + UNSAFE cases, excluding timeouts. Measurements are based on 100 ImageNet images and 50 Kinetics videos, evaluated across neural networks ranging from 22M to 300M parameters and data sizes from image to video. We will include all the runtimes in all tables in our revised version.\n\n**Table 3: Runtime comparison on the ImageNet dataset**\n | Model| NN  params      |    Clean accuracy      | SAFE|UNSAFE| Unknown | Runtime (s)|\n|----|------------------------|----|-|--------------------|--------------|--------------|                                                           \n| ResNet34     |22M|         64%         | 20  |      77   |      3|     29.67    |            \n| ResNet50         |26M|  70%      | 30  |      60   |      7      | 59.49 |  \n| ResNet101               |45M|  69%      | 37 |53 |10     |       97.77  |     \n| ResNet152      |60M|     74%           | 39 |51| 10    |      104.30   | \n| ViT                |300M|    79%| 13 |67| 20      |    65.80     |     \n\n**Table 4: Runtime comparison on the Kinetics dataset**\n | Model| NN  params      |    Clean accuracy      | SAFE|UNSAFE| Unknown | Runtime (s)|\n|----|------------------------|----|-|--------------------|--------------|--------------|                                                           \n| x3d_m     |3.79M|         76%         | 16  |      30   |    4  |     194.98    |            \n| c2d_r50         |24.33M|  78%      | 12  |      31   |      7      | 145.84 |  \n\n\n**[Geometric transformations]** \n\nGeometric transformations are important in many practical applications including object detection, since geometric adversarial vulnerability is evidenced in practice [2][3]. We emphasise that our method is specifically designed for robustness analysis with respect to geometric perturbations. Within this scope, we do not consider said perturbation as a limitation, as all combinations of geometric transformations can be effectively encoded using our approach. However, we acknowledge that if the goal were to develop a general-purpose method for neural network analysis, this focus would indeed constitute a limitation.\n\n**[Comparison with the SoA]**\n\nThe Appendix includes a comparison on small models against the previous SoA that can solve geometric certification queries.\n\n-----\n[1] Introduction to global optimization exploiting space-filling curves, 2013\n\n[2] Exploring the Landscape of Spatial Robustness, 2017\n\n[3] Adversarial examples based on object detection tasks, 2023"}, {"review_id": "zpdtVqQg4a", "reviewer": "Reviewer_vUgZ", "summary": "This paper introduces HOVER, a novel global optimization framework for verifying the local robustness of neural networks against geometric perturbations (e.g., rotation, translation, scaling). Unlike traditional methods constrained by input dimensionality or heavy reliance on theoretically sound (but often impractical) optimization guarantees, HOVER leverages a dimensionality reduction strategy via Hilbert space-filling curves and adapts a Hölder continuous optimization method that is capable of verifying robustness for very large models (e.g., ResNet152, ViTs, and 3D CNNs).\n\nThe approach formulates the robustness verification problem as a univariate optimization task, reducing the original high-dimensional geometric perturbation space to a single dimension. HOVER provides empirical reliability via adaptive estimation of Hölder constants, interval refinement heuristics, and dynamic calibration, even though theoretical soundness is not always assured. Extensive experiments show that HOVER: 1. Outperforms previous methods on benchmarks like ImageNet, Kinetics, and UCF101; 2. Is applicable to both image and video classification; 3. Performs competitively on SoundnessBench and VNN-COMP, often with higher empirical correctness than “sound” solvers like αβ-CROWN.", "questions": "1. The practical reliability of HOVER depends heavily on the heuristic tuning of parameters like the reliability constant r. Could you include an ablation study showing how variations in r affect false positives/negatives or convergence time across benchmarks?\n\n2. The current method is explicitly limited to low-dimensional transformations. Do you see a path toward generalizing the Hilbert-based Hölder optimization to moderately high-dimensional perturbation spaces (e.g., 20–50 dimensions)? What are the theoretical or practical barriers?\n\n3. Are there examples where HOVER returns an incorrect robustness label? What mitigation strategies are in place for safety-critical applications where even one unsound result could be unacceptable?\n\n4. Can you provide wall-clock runtime comparisons between HOVER and GeoRobust, PGD, or αβ-CROWN on the same tasks?\n\n5. You fix m = 50 throughout the experiments. Could you show the effect of varying m on both verification success rate and runtime? This would clarify its role in the trade-off between accuracy and speed.", "limitations": "Partially addressed. The authors acknowledge the limitation of HOVER's applicability to low-dimensional geometric transformations and mention that noise perturbations are better handled by symbolic interval methods. However, the discussion of societal or safety risks is absent. Given the focus on robustness and safety-critical use cases, it would be important to address: 1. The risks of false confidence in verification outcomes; 2. The challenges in applying HOVER under distribution shifts; 3. The potential for misuse or overtrust in industrial settings.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n\nHOVER can verify large-scale models, including 3D CNNs for video, which is unprecedented in the robustness verification literature for geometric perturbations.\n\nThe use of Hilbert curves for dimensionality reduction enables the transformation of multivariate geometric robustness problems into tractable univariate optimization tasks. This is both elegant and effective.\n\nDespite theoretical limitations, HOVER shows very strong empirical soundness across 500+ verification queries, outperforming several state-of-the-art verifiers.\n\nThe authors present results on diverse tasks (image and video classification), architectures (CNNs, ResNets, ViTs, 3D CNNs), and datasets (ImageNet, Kinetics, UCF101), along with comparisons against certified and adversarial baselines.\n\nThe appendices thoroughly cover theoretical proofs, ablation experiments, extended results, and implementation details, improving the paper's transparency and credibility.\n\nWeaknesses\n\nWhile empirical correctness is high, the algorithm lacks a constructive guarantee of convergence to a global minimum unless the unknown optimal Hölder constant is accurately estimated. This may be problematic in safety-critical applications.\n\nThe core contributions (space-filling curves + Hölder optimization + heuristics) are well-engineered but do not introduce fundamentally new theory. The work is closer to a strong engineering improvement than a conceptual leap.\n\nThe method is only applicable to low-dimensional perturbation spaces (e.g., geometric transformations). It does not generalize to pixel-level or noise-based robustness verification tasks, limiting its scope compared to symbolic interval propagation or MILP-based solvers.\n\nOn certain architectures like Swin or ViT-Large, HOVER underperforms in certified robustness compared to GeoRobust. Some claims about “no unsound results” would benefit from further third-party validation.\n\nThe choice of reliability parameter r, budget ε, and re-estimation strategies are empirical and somewhat arbitrary. No ablation study shows how sensitive results are to these choices.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the comments. We address the specific\nquestions as follows.\n\n**Question 1.**\n\n**[Reliability parameter r]** \n\nEmpirical results from the optimisation literature suggest the usage of a small value close to 1, e.g., 1.3, yields close to optimal solutions [1]. We report an ablation study on r in Table 1, which confirms the observations in [1]. Specifically, we conducted three ablation studies on the reliability parameter r on two benchmarks with ground-truth: TLL Verify Bench from VNN-COMP (2 dimensions) and MLP5 from SoundnessBench (10 dimensions).\nWe report the counts of SAFE, UNSAFE, and Unknown cases, along with the average runtime in seconds (SAFE+UNSAFE, excluding Timeout). The results show that HOVer outputs no unsound results for **any value of r>1**, but it becomes hard to answer verification queries with bigger values of r, which, given the above, should not be used. In summary: we have never witnessed a false positives in any of the experiments. Any counterexample is provably correct, so we cannot have any false negatives, and indeed never encountered one.\n\n**Table 1: Ablation study on the reliability parameter r**\n| **r** |TLL|SAFE (sound)|UNSAFE|Unknown|runtime| MLP5|SAFE (sound)|UNSAFE|Unknown|runtime (s)|\n|-----------|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------|\n| **1.1**   ||15(15)|17|0|1.00||  16(16)|3|14|20.92|\n| **1.3**   ||15(15)|17|0|1.04|| 14(14)|1|18|10.58|\n| **1.5**   ||13(13)|17|2|1.16|| 10(10)|2|21|10.58|\n| **3**   ||1(1)|17|14|0.12||0|2|31|11.96|\n| **6**   || 0|17|15|0.03||0|2|31|59.56|\n| **10**  | |0|17|15|0.03||0|2|31|32.79|\n\n**[Hilbert curve resolution parameter m]** \n\nThe computational complexity of\nmodern libraries providing the Hilbert space mapping grows linearly in\nm, so any value of m > ~30, which already provides small calibration\nerrors (see mathematical expression (10) in the paper), would be\nadequate to use. We report an ablation study on m in Table 2 which confirms this intuition - any value of m > 10\nresolves all verification queries with stable overheads.\n\n**Table 2: Ablation study on the Hilbert curve resolution parameter m**\n| **m** |TLL|SAFE (sound)|UNSAFE|Unknown|Runtime| MLP5|SAFE (sound)|UNSAFE|Unknown|Runtime (s)|\n|-----------|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------|\n| **10**   || 0 |17|15|0.18||14(14)|2|17|13.02|\n| **30**   || 15(15)|17|0|0.33|| 13(14)|2|18|7.59|\n| **50**   || 15(15)|17|0|0.33||14|1|18|10.58|\n| **70**   ||15(15)|17|0|0.33||14|2|17|16.13|\n| **90**   ||15(15)|17|0|0.34||14|2|17|11.37\n\n**[Optimisation budget parameter ϵ]**\n\nThe optimisation budget\ncorresponds to the typical optimisation gap present in all convergent\nalgorithms. As the ablation study in Table 3\nshows, the smaller its value the more verification queries can be\nsolved, albeit with longer runtime. Its value is thus contingent to\navailable computation and temporal resources.\n\n**Table 3: Ablation study on the optimisation budget parameter ϵ**\n| **ϵ** |TLL|SAFE (sound)|UNSAFE|Unknown|Runtime (s)| MLP5|SAFE (sound)|UNSAFE|Unknown|Runtime (s)|\n|-----------|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------|\n| **1e-3**   |  12|17|3|0.62|||0|0|33|-\n| **1e-6**   | 15|17|0|0.68|||4|0|29|2.33\n| **1e-9**   | 15|17|0|0.78|||9|0|24|2.52|\n| **1e-13**   |15|17|0|0.88|||13|1|19|5.03|\n| **1e-15**   |15|17|0|1.04|||14|1|18|10.58|\n\n**Question 2.**\n\nHOVer is designed for geometric verification, and therefore inherently\nrestricted to low‑dimensional perturbation spaces.  As shown in\nTheorem 1, the convergence bound includes an irreducible residual term\n$H \\cdot (\\epsilon/2)^{1/N}$. Because $(\\epsilon/2)^{1/N}\\rightarrow 1$ as $N$ grows, the\nresulting approximation gap, governed by the Hölder constant, also\ngrows with N, thereby deteriorating convergence and giving overly\nloose certificates in higher dimensions. Consequently, pushing the method \nto high dimensionality transformations such as noise remains an\nopen challenge. However, note that extending Hover to other smaller dimensionality perturbations such as contrast,\nluminosity, bias field, blur, etc, is entirely feasible.\n\n**Question 3.**\n\nIn our extensive evaluations, we have observed no cases where HOVer\nproduces an incorrect robustness result. In principle, by further\nincreasing r beyond what we used in the experiments, we reduce even\nfurther the possibility of any unsound results. Currently however,\nHOVer provides no formal mitigation for potential unsoundness. We\nbelieve this to be an acceptable trade-off in the context of a robustness analysis\nacross many test points for the very large models here that can now be analysed.\n\n**Question 4.** \n\n**[GeoRobust and PGD]**\n\nWe ran 100 randomly sampled ImageNet images through ResNet‑50 considered in\nthe paper and compared the performance of our method (HOVer) against PGD and GeoRobust under the 4‑parameter perturbations. The termination threshold was set to 500 seconds or 50,000 queries per sample.\nTable 4 reports the average runtime observed. As expected, while fast PGD cannot in principle establish any safe cases. In comparison to GeoRobust, HOVer is markedly more performant.\n\n**Table 4: Comparison with GeoRobust and PGD**\n|  Rotation + Translation + Isotropic Scaling | SAFE  | UNSAFE| Unknown |Runtime (s)|\n|----|------------------------|--------------------|--------------|--------------|                                                           \n| HOVer                        | 33  |      60   |      7      | 59.49|                                                            \n| GeoRobust           | 33   |      57   |      10      | 286.01| \n|PGD|-|34|64| 4.18|\n\n**[$\\alpha\\beta$-CROWN]**    \n\nWe have attempted to run αβ‑CROWN on the ImageNet‑scale networks (with more than 24 million NN parameters) that we analyse here (e.g. ResNet‑50 on a 3 × 224 × 224 input and Inception‑v3 on 2 × 299 × 299), but we were not able to. One major obstacle is operator coverage: although the Spatial Transformer Network (STN) can be represented in the computation graph, to our knowledge the current αβ‑CROWN implementation (nor any other IBP/SIP tools) does not provide the bounded operations for geometric perturbations, specifically, `affine_grid` and `grid_sample`.\n\nA further obstacle is scalability: αβ‑CROWN’s memory footprint grows roughly linearly with both the number of neurons and the full input resolution, which makes branching/splitting prohibitively expensive. To clarify this, we constructed a one‑pixel perturbation (corresponding to 3 dimensions) as a comparable low‑dimensional proxy, as our geometric experiments involve low‑dimensional perturbations. In this simplified setting and in contrast to HOVer, αβ‑CROWN often times out or runs out of memory (OOM). We refer to  **`the response to Question 2 of Reviewer rxtg`** for the experimental results.\n\n\n**Question 5.** \n\nPlease see the answer for Question 1.\n\n**[Comment on Low-dimensional perturbation spaces]** \n\nGeometric transformations are\nimportant in many practical applications, including object detection\nsince geometric adversarial vulnerability is evidenced in practice\n[2].  HOVer is specifically designed for robustness\nanalysis with respect to geometric perturbations. Within this scope,\nwe do not consider said perturbation as a limitation, as all\ncombinations of geometric transformations can be encoded using\nHOVer. We noted elsewhere that the same approach will be effective on\nother low-dimensional perturbations, such as bias-field, contrast,\nluminosity and blur.  However, we acknowledge that if the goal were to\ndevelop a general-purpose method for neural network analysis, this\nfocus would indeed constitute a limitation.\n\n**[Comment on Soundness]** \n\nWe agree it would be ideal to have a theoretical guarantee on no false\npositives. As argued in the paper, we believe this is an acceptable\ntrade-off in the context of robustness analysis for very large models\nthat could not be analysed before. We remark that we observed no false\npositives in all the experiments run.\n\n**[Comment on Underperformance wrt GeoRobust]** \n\nWe observed in the paper\nthat GeoRobust reported incorrect results in some cases. We believe\nthis is due to an underestimation of the Lipschitz constant in their\napproach; this leads to faster resolution of some cases.\n\n**Limitations (possible overconfidence)**. \n\nWe have found no cases where the answer of the tool was incorrect. But we would like to stress that no safety critical systems, such as aviation systems, are validated purely on the basis of formal verification. An assurance case in the industry is much more comprehensive and verification is only a part of the process. Even verification results on sound tools are evaluated in the context of tool qualification and confidence and the possibility of errors creeping up in various ways. This is also likely to be the case with ML systems also in view of the difficulty of coverage quantification of the Operational Design Domain. Therefore we do not believe this risk to be high. Still, we do not believe this removes value from the method, as it significantly pushes forward  robustness validation for large models, something that was not possible before.  \n \n----\n[1] Introduction to global optimization exploiting space-filling curves, 2013\n\n[2] Adversarial examples based on object detection tasks, 2023"}, {"review_id": "KyHE5LgfpP", "reviewer": "Reviewer_rxtg", "summary": "The paper proposes an optimization-based approach for the robustness analysis of neural networks w.r.t. geometric perturbations.\nTo this end, the paper phrases the problem as a low-dimensional optimization problem (where the few dimensions correspond to the configuration of the perturbation,\ne.g. rotation or scaling) and uses a space filling curve to reduce the problem into a one-dimensional verification problem.\nSubequently, a Hölder-constant based optimization approach is deployed.", "questions": "- **(Q1)** Would the authors be willing to change the title of their paper to make it more transparent, that the approach does not come with a provable soundness guarantee  \n  (e.g. \"approximate verification\", \"robustness analysis\", \"approximate certification\" or something similar)\n- **(Q2)** Have the authors performed any experiments that demonstrate that their tool indeed outperforms alpha-beta-CROWN or other VNNCOMP participants?\n  Are there fundamental reasons why the benchmarks used for evaluation (e.g. ImageNet) cannot be evaluated using alpha-beta-CROWN via an encoding as a computational graph?\n\nI want to emphasize that I am very willing to increase my score if these questions are addressed.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Strengths:**\n- **(S10)**\tThe innovative approach is conceptually interesting in its usage of space filling curves to reduce the dimensionality of the optimization problem.\n- **(S20)**\tThe results are very promising as the approach is capable of scaling its analysis technique to NNs much larger than traditionally analyzed\n\t\tin the field of NN verification, e.g. ImageNet networks or Video Classification Networks.\n- **(S30)**\tWhile some of the details of the optimization approach are quite technical, the paper provides a good overview of the approach.\n\n**Opportunities for Improvement:**\n- **(O10)**\tA significant portion of the paper is spent on discussing the theoretical unsoundness of the approach. I appreciate that the authors were up front about this.\n\t\tTo this end, my understanding of the paper's argument is that (1) empirically the obtained results so far have always been correct,\n\t\tand (2) that other State of the Art verifiers are unsound, too. I find this argument unconvincing: Yes, some verification tools have\n\t\tbugs that lead to unsound results, but for any of the methods underlying alpha-beta-CROWN, NeuralSAT, Marabou or other NN verifiers we know that it is,\n\t\tin principle, possible to implement them in a sound manner and these bugs should be fixable (even if this may require some assumptions about floating point operation order).\n\t\tThe purpose of NN verification is precisely to go *beyond* what can be achieved via empirical methods and hence an empirical soundness result is insufficient in my view.\n\t\tTo this end, it is also important to emphasize that the field of NN verification is broader than \"just\" robustness verification. Hence, even if robustness analysis\n\t\tmight get away without rigorous soundness guarantees, other application scenarios do not and require that something labeled \"NN verification\" actually comes with a rigorous soundness guarantee.\n\t\tThis might be nitpicking (with my luck I probably end up being reviewer 2), but I believe it is important to be careful with terms like \"verification\": Without a soundness\n\t\tguarantee in the classical sense it is my stance that this approach should not be called verification.\n\t\tMy suggestion would be to relabel it as, e.g., \"approximate verification\" (see, e.g., approximate model counting), \"robustness analysis\" or \"approximate certification\".\n- **(O20)**\tThe paper mentions that tools from VNNCOMP are not capable of resolving queries of the kind evaluated (line 288ff):\n\t\tTo this end, I know that, e.g., alpha-beta-CROWN is in principle able to process computational graphs which, in my understanding, should be able to capture the geometric transformations\n\t\tperformed in the paper. This makes me wonder whether the authors actually tried to apply alpha-beta-CROWN and it failed or whether they performed a comparison on some other benchmarks\n\t\twhere they demonstrated that alpha-beta-CROWN is indeed less performant than their proposed approach.\n\t\tEspecially given the low input dimensionality I expect that some NN verification tools might scale further than is evaluated in classical VNNCOMP benchmarks.\n\n**Minor Notes:**\n- Line 48: \"Hölder\" (o->ö)\n- Line 188: Where is $h_k$ defined?\n- Line 230: \"fi*r*st\": missing r\n- Line 338: Unnecessary \"the\"", "was_revised": true, "final_justification": "Overall, I find the author's rebuttal convincing.\nI have hence moved my score to \"Accept\" under the assumption that the authors will rephrase the contribution from \"Verification\" to something akin to robustness analysis (or similar).\nThe comparison to alpha beta CROWN nicely showcases that their approach indeed outperforms other techniques in the considered setting. The authors' rebuttal lifts my concerns outlined in (O10) and (O20).", "rebuttal": "**Question 1.**\n\nWe fully agree with the reviewer’s remarks. Our goal was\nto be as transparent and scientifically precise as possible, and we\nwelcome the opportunity to further improve this. We are happy to refer\nto our approach as a method for \"robustness analysis\" or \"robustness\nvalidation\" and change the title accordingly and all referenced text within\naccordingly. Indeed, we may prefer these over \"approximate verification\"\nas they more clearly emphasise the contrast to \"verification\", in the\nspirit of the reviewer's comment, which we endorse. That said, we\nbelieve that the method here presented strictly goes beyond, and is\nconceptually different from, the kind of empirical robustness analysis\ntypically performed using PGD and derivatives. We welcome further comments\non this.\n\nIf the paper is accepted, we will revise the paper to acknowledge\nthat, in principle, a sound method can indeed be implemented\ncorrectly. We had absolutely no intention of being dismissive to\nexisting work, which we believe has great value, and are grateful for\nthe opportunity to clarify our position.\n\n**Question 2.**\n \nAs discussed in the paper tools from VNNCOMP cannot solve queries on geometric perturbations on large models that we evaluate here; this is not only due to the nature of the perturbations but also the large input size and large models. \n\nWe have attempted to run αβ‑CROWN on the ImageNet‑scale networks (with more than 24 million NN parameters) that we analyse here (e.g. ResNet‑50 on a 3 × 224 × 224 input and Inception‑v3 on 2 × 299 × 299), but we were not able to.\nOne major obstacle is operator coverage: although the Spatial Transformer Network (STN) can be represented in the computation graph, to our knowledge the current αβ‑CROWN implementation does not provide the bounded operations for geometric perturbations, specifically, `affine_grid` and `grid_sample`. \n\nA further obstacle is scalability: αβ‑CROWN’s memory footprint grows roughly linearly with both the number of neurons and the full input resolution, which makes branching/splitting prohibitively expensive. To illustrate this, we constructed a one‑pixel perturbation (corresponding to 3 dimensions) as a comparable low‑dimensional proxy, as our geometric experiments involve low‑dimensional perturbations. \n\nWe used the ResNet18_cifar model provided in αβ‑CROWN’s repository and used its wider variant to increase the inplanes from 2 to 8, yielding a larger model. We then performed verification queries at ε = 0.1 and ε = 0.3 on these two models over a random pixel, reasonably assuming that they are expected to be robust (i.e., safe) within these perturbation radii. \n\nTable 1 reports the results for this simplified setting, where αβ‑CROWN verifies the smallest case at ε=0.1 but reports TIMEOUT  or runs out of memory (OOM) as ε  increases or the model grows; in contrast, HOVer consistently returns safe with lower and more stable memory as it only requires the forward pass. In our experiments αβ‑CROWN runs out of memory (OOM) on a relatively small model with 175,802 parameters. In conclusion we see no way to scale it to models with tens/hundreds of millions of parameters as we do here, or to ImageNet‑scale inputs.\n\n**Table 1: Verification results on one‑pixel perturbation for CIFAR10 models**\n| Model (#params) & ε            | Method         | Result / Peak Mem (MB) / Runime (s) |\n|-----------------------------------|----------------|-------------------------------------|\n| ResNet18_cifar (#11,270), ε=0.1  | PGD (10k steps × 5 restarts)    | unknown / 656 / 12.75                |\n|                                    | αβ‑CROWN        | safe / 644 / 4.68                     |\n|                                    | HOVer           | safe / 636 / 30.28                    |\n| ResNet18_cifar (#11,270), ε=0.3  | PGD (10k steps × 5 restarts)    | unknown / 656 / 13.32                 |\n|                                    | αβ‑CROWN        | unknown / 4298 / Timeout               |\n|                                    | HOVer           | safe / 636 / 30.20                     |\n| ResNet18_cifar model 8 inplanes  (#175,802), ε=0.1   | PGD (10k steps × 5 restarts)    | unknown / 678 / 14.16                 |\n|                                    | αβ‑CROWN        | out of memory (OOM)                                   |\n|                                    | HOVer           | safe / 638 / 24.88                     |\n\n\n**Minor Notes.**\n\nWe thank the reviewer for pointing out these typos which will be correct in the final version if the paper is accepted. \n$h_k$ is defined in lines 182-183, which is the global Hölder constant so far."}, {"review_id": "AVPcYKILJk", "reviewer": "Reviewer_BsfB", "summary": "This paper introduces HOVER, a novel method for verifying the robustness of neural networks against low-dimensional geometric perturbations like rotation, scaling, and translation. The core technical contribution is a clever use of Hilbert space-filling curves to reduce the multi-dimensional verification task to a one-dimensional problem. This problem is then solved using Hölder optimisation with an adaptive estimation of the Hölder constant. The authors provide an extensive evaluation showing that HOVER scales to very large models, including Vision Transformers and, for the first time, 3D-NNs for video classification. While the method is not theoretically guaranteed to be sound, the authors provide empirical evidence from SoundnessBench and VNN-COMP showing it produces no incorrect results in their experiments.", "questions": "I think generally it is a good paper, even though with a potential unsoundness concern, I will increase my score if my following questions are well answered:\n1. Your work makes a clear trade-off, sacrificing theoretical soundness for practical scalability. Could you provide a quantitative assessment of this trade-off? For instance, what is the performance and runtime cost if you modify HOVER to use more conservative, sound bounding techniques (like interval arithmetic for the lower bounds) instead of the current optimisation approach? This would clarify whether an unsound approach is necessary or merely convenient.\n2. For HOVER to be adopted, users need guidance on setting critical parameters. Can you provide practical heuristics or an empirical study showing how the choice of the reliability parameter r, the Hilbert curve resolution m, and the optimisation budget ϵ affects the trade-off between runtime, memory usage, and the tightness of the verification bound?\n3. Your evaluation against SoA verifiers is limited. Why were general-purpose open-source verifiers like $\\alpha,\\beta$-CROWN not benchmarked on the main proposed geometric robustness tasks? A direct comparison would be necessary to establish if HOVER's specialized approach is truly superior to state-of-the-art general methods on this specific problem class. Furthermore, can you clarify the missing baseline results for GeoRobust in Table 1?", "limitations": "Yes, the authors acknowledge their main limitations", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper's main strength is its achievement in scalability. By successfully verifying models with hundreds of millions of parameters (e.g., ResNet152, ViT) and enabling the first-ever robustness analysis of large video classifiers, the work pushes the practical boundary of what is possible in verification. This is driven by a novel technical approach that creatively combines dimensionality reduction with global optimisation. Furthermore, the extensive experimental validation is a major plus; the perfect record on SoundnessBench is an empirical counterargument to the method's primary weakness.\n\nHowever, the paper's central weakness is its lack of theoretical soundness. For a method presented in the context of \"verification,\" whose very purpose is to provide formal guarantees, this is a fundamental limitation. The optimization can theoretically converge to a local minimum, leading to an incorrect certificate. The authors' argument that the \"aggregation of these results, not a single query, that enables the analysis of a model's robustness\" is a pragmatic but concerning stance from a safety-critical perspective.\n\nOther notable weaknesses include:\n1. The method is explicitly designed for and limited to low-dimensional perturbation spaces. It is not a general-purpose verifier and cannot handle the high-dimensional lp​-norm perturbations that are the focus of most verification literature.\n2. The method's reliability hinges on parameters like the reliability parameter r and the optimisation budget ϵ. The paper offers no constructive procedure for selecting them, leaving their tuning to user expertise and heuristics.", "was_revised": true, "final_justification": "Your response to **Question 2** is excellent. The comprehensive ablation studies for parameters, along with concrete recommendations based on optimization literature, significantly strengthen the paper's practical usability.\n\nYour response to **Question 3** is also convincing. The technical explanation of $\\alpha,\\beta$-CROWN's scalability limitations (memory footprint growing with neurons and input resolution, lack of STN operator support) provides compelling evidence for why existing general-purpose verifiers cannot handle your problem scale.\n\nHowever, your response to **Question 1** - the most fundamental issue I raised - is insufficient. You state it's \"unclear\" how HOVER could be combined with sound methods and dismiss IBP after testing on only one small model. This sidesteps my core request: a quantitative assessment of the soundness vs. performance trade-off.\nThe theoretical soundness issue I identified as your paper's \"central weakness\" remains unresolved. For a method presented as \"verification,\" the lack of formal guarantees is problematic, especially for safety-critical applications. Your empirical evidence (500 queries with no false results in Soundbench) is valuable but doesn't substitute for theoretical analysis.", "rebuttal": "We thank the reviewer for the comments. We address the specific\nquestions and provide clarifying points to he identified weaknesses.\n\n**Question 1.**\n\nWe have run HOVer on 500 verification queries from the SoundBench benchmark, for which the ground truth result is\nknown. We have observed no unsound results. HOver is based on global optimisation and, unlike other\nmethods like SIP, it does not compute bounds for the nodes of the model. So it is unclear to us how it could be combined with SIP or IBP. For a comparison we did use IBP (auto_LiRPA) to compute bounds for a small ResNet model\n(WideResNet trained on TinyImageNet) - the bounds derived were too loose to establish a good robustness result (see Appendix B1).\n\n**Question 2.**\n\n**[Guidance on parameter r]** \n\nConsolidated results from the optimisation literature suggest the that a value slightly higher than 1, e.g., 1.3,\nyields close to optimal solutions [1]. We report an ablation study on r in Table 1 on two benchmarks with ground-truth: TLL Verify Bench from VNN-COMP (2 dimensions) and MLP5 from SoundnessBench (10 dimensions).\nWe report the counts of SAFE, UNSAFE, and Unknown cases, along with the average runtime in seconds (SAFE+UNSAFE, excluding Timeout), where the result also confirms [1] . Hence, we would just recommend that the consolidated results from the optimisation literature are appropriate to be used here.\n\n**Table 1: Ablation study on the reliability parameter r**\n\n| **r** |TLL|SAFE (sound)|UNSAFE|Unknown|Runtime (s)| MLP5|SAFE (sound)|UNSAFE|Unknown|Runtime (s)|\n|-----------|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------|\n| **1.1**   ||15(15)|17|0|1.00||  16(16)|3|14|20.92|\n| **1.3**   ||15(15)|17|0|1.04|| 14(14)|1|18|10.58|\n| **1.5**   ||13(13)|17|2|1.16|| 10(10)|2|21|10.58|\n| **3**   ||1(1)|17|14|0.12||0|2|31|11.96|\n| **6**   || 0|17|15|0.03||0|2|31|59.56|\n| **10**  | |0|17|15|0.03||0|2|31|32.79|\n\n**[Guidance on the Hilbert curve resolution parameter m]**\n\n The computational complexity of the Hilbert space mapping grows linearly\n in m.  Any value of m > 30, which already provides small calibration\n errors (see mathematical expression (10) in the paper), would be\n adequate to use. We report an ablation study on m in Table 2, which confirms this intuition - any value of m > 10\n resolves all verification queries with stable overheads.\n\n**Table 2: Ablation study on the Hilbert curve resolution parameter m**\n| **m** |TLL|SAFE (sound)|UNSAFE|Unknown|Runtime (s)| MLP5|SAFE (sound)|UNSAFE|Unknown|Runtime (s)|\n|-----------|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------|\n| **10**   || 0 |17|15|0.18||14(14)|2|17|13.02|\n| **30**   || 15(15)|17|0|0.33|| 13(13)|2|18|7.59|\n| **50**   || 15(15)|17|0|0.33||14(14)|1|18|10.58|\n| **70**   ||15(15)|17|0|0.33||14(14)|2|17|16.13|\n| **90**   ||15(15)|17|0|0.34||14(14)|2|17|11.37\n\n**[Guidance on the Optimisation budget parameter ϵ]** \n\nThe optimisation budget corresponds to the typical optimisation gap\npresent in all convergent algorithms. As the ablation study in Table\n3 shows, the smaller its value the more\nverification queries can be resolved, albeit with longer runtime. Its\nvalue is thus contingent to the complexity on the queries in question\nand resources. Our recommendation is to tune ϵ in line with the\ncomprehensive ablation study provided.\n\n**Table 3. Ablation study on the optimisation budget parameter ϵ**\n| **ϵ** |TLL|SAFE (sound)|UNSAFE|Unknown|Runtime (s)| MLP5|SAFE (sound)|UNSAFE|Unknown|Runtime (s)|\n|-----------|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------|\n| **1e-3**   ||  12(12)|17|3|0.62||0|0|33|-\n| **1e-6**   || 15(15)|17|0|0.68||4(4)|0|29|2.33\n| **1e-9**   || 15(15)|17|0|0.78||9(9)|0|24|2.52|\n| **1e-13**   ||15(15)|17|0|0.88||13(13)|1|19|5.03|\n| **1e-15**   ||15(15)|17|0|1.04||14(14)|1|18|10.58|\n\n**Question 3.**\n\nWe have attempted to run αβ‑CROWN on ImageNet‑scale networks (with more than 24 million NN parameters) that we analyse here (e.g. ResNet‑50 on a 3 × 224 × 224 input and Inception‑v3 on 2 × 299 × 299), but we were not able to.\nOne major obstacle is operator coverage: although the Spatial Transformer Network (STN) can be represented in the computation graph, in our understanding the current αβ‑CROWN implementation does not provide the bounded operations for geometric perturbations, specifically, `affine_grid` and `grid_sample`. \n\nA further obstacle is scalability: αβ‑CROWN’s memory footprint grows roughly linearly with both the number of neurons and the full input resolution which makes branching/splitting prohibitively expensive. To illustrate this, we constructed a one‑pixel perturbation (corresponding to 3 dimensions) as a comparable low‑dimensional proxy, as our geometric experiments involve low‑dimensional perturbations. \n\nWe used the ResNet18_cifar model provided in αβ‑CROWN’s repository and used its wider variant to increase the inplanes from 2 to 8, yielding a larger model. We then performed verification queries at ε = 0.1 and ε = 0.3 on these two models over a random pixel, reasonably assuming that they are expected to be robust (i.e., safe) within these perturbation radii. \n\nTable 4 reports the results for this simplified setting, where αβ‑CROWN verifies the smallest case at ε=0.1 but TIMEOUT or runs out of memory (OOM) as ε  increases or the model grows; in contrast, HOVer consistently returns safe with lower and more stable memory as it only requires the forward pass. In our experiments αβ‑CROWN runs out of memory (OOM) on a relatively small model with 175,802 parameters. In conclusion we see no way to scale it to models with tens/hundreds of millions of parameters as we do here, or to ImageNet‑scale inputs with geometric perturbations.\n\n**Table 4: Verification results on one‑pixel perturbation for CIFAR10 models**\n| Model (**#params**) & ε            | Method         | Result / Peak Mem (MB) / Time (s) |\n|-----------------------------------|----------------|-------------------------------------|\n| ResNet18_cifar (11,270), ε=0.1  | PGD (10k steps × 5 restarts)    | unknown / 656 / 12.75                |\n|                                    | αβ‑CROWN        | safe / 644 / 4.68                     |\n|                                    | HOVer           | safe / 636 / 30.28                    |\n| ResNet18_cifar (11,270), ε=0.3  | PGD (10k steps × 5 restarts)    | unknown / 656 / 13.32                 |\n|                                    | αβ‑CROWN        | unknown / 4298 / Timeout               |\n|                                    | HOVer           | safe / 636 / 30.20                     |\n| ResNet18_cifar model 8 inplanes  (175,802), ε=0.1   | PGD (10k steps × 5 restarts)    | unknown / 678 / 14.16                 |\n|                                    | αβ‑CROWN        | out of memory (OOM)                                   |\n|                                    | HOVer           | safe / 638 / 24.88                     |\n\n\n\n**[Comment on Low-dimensional perturbation spaces]**\n\nGeometric transformations are important in many practical applications including\nobject detection since geometric adversarial vulnerability is\nevidenced in practice [2].  HOVer is specifically\ndesigned for robustness analysis with respect to geometric\nperturbations. Within this scope, we do not consider said perturbation\nas a limitation, as all combinations of geometric transformations can\nbe encoded using HOVer. We noted elsewhere that the same approach will\nbe effective on other low-dimensional perturbations, such as\nbias-field, contrast, luminosity and blur.  However, we acknowledge\nthat if the goal were to develop a general-purpose method for neural\nnetwork analysis, this focus would indeed constitute a limitation.\n\n**Clarification**\n\nThe missing baseline result for GeoRobust in Table 1 is due to an update in the timm (PyTorch Image Models) library, which previously caused the ResNet model to achieve only ∼9% robust accuracy (abnormally low). The updated model now achieves a more reasonable ∼25% robust accuracy. We will rerun GeoRobust using the updated model and include the result in the revised version.\n\n-----\n[1] Introduction to global optimization exploiting space-filling curves, 2013\n\n[2] Adversarial examples based on object detection tasks, 2023"}]

Author rebuttal:

Meta-review: Over the course of a very active discussion period, the authors assuaged the concerns of all reviewers except one, who did not respond to the authors or myself after multiple prompts. The actual concerns of that reviewer appear to me as minor ones that the reviewers addressed in their responses. Overall, this paper describes an interesting method that is notable in its capability for scaling to the types of architectures in use today for state-of-the-art tasks. That makes it highly relevant for NeurIPS and worthy of acceptance.
Decision: Accept (poster)