Achieving Domain-Independent Certified Robustness via Knowledge Continuity
Authors: Alan Sun
Chiyu Ma
Kenneth Ge
Soroush Vosoughi
Venue: NeurIPS 2024 poster (2024)
Keywords: Lipschitz continuity
robustness
certified robustness
adversarial robustness
Primary area: safety_in_machine_learning
TL;DR: We present knowledge continuity, a novel definition which aims to certify robustness of neural networks across continuous/discrete domains.

Abstract: We present *knowledge continuity*, a novel definition inspired by Lipschitz continuity which aims to certify the robustness of neural networks across input domains (such as continuous and discrete domains in vision and language, respectively). Most existing approaches that seek to certify robustness, especially Lipschitz continuity, lie within the continuous domain with norm and distribution-dependent guarantees. In contrast, our proposed definition yields certification guarantees that depend only on the loss function and the intermediate learned metric spaces of the neural network. These bounds are independent of domain modality, norms, and distribution. We further demonstrate that the expressiveness of a model class is not at odds with its knowledge continuity. This implies that achieving robustness by maximizing knowledge continuity should not theoretically hinder inferential performance. Finally, to complement our theoretical results, we present several applications of knowledge continuity such as regularization, a certification algorithm, and show that knowledge continuity can be used to localize vulnerable components of a neural network.

Achieving Domain-Independent Certified Robustness via Knowledge Continuity
Alan 12 Chiyu Ma², Kenneth Ge', Soroush Vosoughi²
CCarnegie Mellon University, Dartmouth College falansun, (kge))aandre cmu edu, {chiyu.ma, gr. soroush. rosoughi)@darttooooooo edu
Abstract
We present knowledge continuity, novel definition inspired by Lipschitz continuity which aims to certify the robustness of neural networks across input domains (such continuous and discrete domains in vision and language, respectively). Most existing approaches that seek to certify robustness, especially Lipschitz continuity, guarantees.
lie within the continuous domain with norm and cettribution-dependde In contrast, our proposed definition yields certification guarantees that depend only on the loss function and the intermediate learned metric spaces of the neural net- work. These bounds are independent of domain modality, norms, and distribution. We further demonstrate that the expressiveness of model class not odds with its knowledge continuity. This implies that achieving robustness by maximizing knowledge continuity should not theoretically hinder inferential performance. Fi- nally, complement our theoretical results, we present several applications of knowledge continuity such as regularization, a certification algorithm, and show that knowledge continuity can be used to localize vulnerable components of neural network!.
1 Introduction
Deep neural networks (DNNs) have demonstrated remarkable generalization capabilities. Their robust- ness, however, has been considerably more difficult achieve. Robustness refers to the preservation of model performance under natural or adversarial alterations of the input [18]. DNNs' lack of robustness, highlighted by seminal works such as [24, 66] and recently [7. 5], poses significant challenges their adoption in critical applications, underscoring concerns for Al safety and trustworthiness [20, 30, 9, 7].
Though issues of robustness emerged from computer vision applications, they have since spanned multiple domains 35, 72, 75, 7]. This research trajectory has not only prompted significant advance- ments in robustness improvements through architectural, training, and dataset augmentations, but also unveiled the sophistication of adversarial atlacks-the process through which counterexamples robustness are generated [1, 35, 72, 75, 7]. Along the progress made in these parallel directions, great deal of work has gone into certified robustness which seeks to provide theoretical robustness guarantees. Certification is desirable
generally transcends any particular task, dataset, or model.
As a result, Lipschitz continuity has emerged, promising certified robustness by essentially bounding the derivative of neural network's output with respect to its input. In this way, Lipschitz continuity directly captures the volatility of a model's 's performance, getting at the heart robustness. Such an approach has proven its merit in computer vision, facilitating robustness under norm and distributional assumptions [29. 59, 78, 76). Its inherent ease and interpretability has lead to widespread adoption as means to measure and regulate robustness among practitioners well [71, 12, 21, 68, 54].
CCodehase for experiments can be found https://githin com/ aal.aaaannn00////eeaee
38th Conference on Neural Information Processing Systems (NeurTPS 2024).

---

Despite these successes in computer vision. there are fundamental obstacles when one tries to apply Lipschitz continuity into discrete or non -otteetrizable domains such natural language. Firstly, characterizing distance in this input-output space is highly nontrivial, as language does not have naturally-endower distance metric. Additionally, suppose we impose some distance metric on the input-output space [49, 16]. For such a metric to meaningfully characterize adversarial perturbations, it cannot be universally task-invariant. Consider the two sentences (a) "I am happy." (b) "I am sad." The ground-truth label of (a) is invariant to the perturbation (a) - (b), if the task is sentence-Structure identification, but it would not be preserved for a task like sentiment classification. Lastly, key architectures such as the Transformer [70] are provably not Lipschitz continuous [36]. Most of these challenges are not unique to language, and they represent a strong divide of our understanding of and continuous domains [22, 46].
robustness in bascrete/non-metriaaee
To address these issues, we propose a new conceptual framework which we call knowledge continuity. At its core, we adopt the following axiom:
Robustness is the stability of a model's performance with respect to its perceived knowledge of input-outpin relations.
Concretely, framework is grounded the premise that robustness is better achieved by focusing on the variability of a model's loss with respect to its hidden representations, rather than forcing arbitrary metrics its inputs and outputs. Our approach results in certification guarantees independent of domain modality, norms, and distribution. We demonstrate that the expressiveness of a model class is not at odds with its knowledge continuity. In other words, achieving robustness by improving knowledge continuity should not theoretically hinder inferential performance. We show that in continuous settings (i.e. computer vision) knowledge continuity generalizes Lipschitz continuity and inherits its tight robustness bounds. Finally, we present an array of practical applications using knowledge continuity both an indicator to predict and characterize robustness as well as an additional term in the loss function to train robust classifiers. In sum, our contributions are threefold:
Introduction of knowledge continuity, a concept that frames robustness variability of model's loss with respect to its hidden representations.
We theoretically show that knowledge continuity results in certified robustness guarantees that generalize across modalities (continuous, discrete, and non-metrizable) Moreover, this robustness does not come at the expense of inferential performance.
We present several practical applications of knowledge continuity such using it train more robust models, in both language processing and vision, identify problematic hidden layers, and using its theoretical guarantees to formulate a novel certification algorithm.
Although our results apply to all fiscretenoonmmeriiaatee and continuous spaces, throughout the paper we invoke examples from natural language as it culminates the aforementioned challenges. Further, the ubiquity of large language models make their robustness a timely focus.
2 Related Works
There have been extensive studies on developing robust neural networks with theoretical guarantees. With respect to our contributions, they can be organized into the following categories.
Certified robustness with Lipschitz continuity. The exploration of Lipschitz continuity as a cornerstone for improving model robustness has yielded significant insights, particularly in the domain of computer vision. This principle, which ensures bounded derivatives of the model's output with respect to its input, facilitates smoother model behavior and inherently encourages robustness against adversarial perturbations. This methodology, initially suggested by [24], has since been rigorously analyzed and expanded upon. Most theoretical results in this area focus on certifying robustness with respect to the :z-norm [11, 86, 25, 2, 38, 29, 4]. A recent push, fueled by new architectural developments, has also expanded these results into Coo -norm perturbations [89, 88, 90]. Further, Lipschitz
algorithms also serve practitioners as a computationally effective way to train more robust. models [68, 78, 69, 13J. This stands in contrast to (virtual) adversarial training methods which brute-foree the set of adversarial examples, then veratively retrain on them [50, 63. 80]. Though Lipschitz continuity has seen much success continuous domains, it does not apply to non- metrizable domains such language. Further, architectural limitations of prevalent models such as

---

the Transformer [70, 36] exacerbate this problem. These challenges highlight critical need for new approach that can accommodate the specificities of discrete and non-metrizable domains while providing robustness guarantees.
Achieving robustness in pecerrte/nonmmetrzzall spaces. Non-metrizable spaces, where it is non- trivial to construct a distance metric on the input/output domains, pose a unique challenge to certified robustness. Instead of focusing on point-wise perturbations, many studies have opted to examine how the output probability distribution of a model changes with respect to input distribution shifts by leveraging information bottleneck methods [67, 73, 53] (see also ditto---iitributionn generaliza- tion: [42, 83, 60].. Most of these bounds lack granularity and cannot be expressed in closed-form. In contrast to these theoretical approaches, recent efforts have refocused on directly adapting the principles underlying Lipschitz continuity to language. Virtual adversarial training methods such as [43, 85] mimic the measurement of Lipschitz continuity by comparing changes in the textual embeddings with the Kl.-divergence of the output logits. Along these lines, techniques akin to those used in adversarial training in vision have also been translated to language, reflecting shift towards robustness centered around the learned representation space [40, 23, 35]. Though these approaches have seen empirical success, they lack theoretical guarantees. As a result, their implementations and success rate is heavily lask-dependent [43, 85). There have also been attempts to mitigate the non- Lin-hiischitzeess of Transformers [87, 82] by modifying its architecture. These changes. however, add significant computational overhead.
Other robustness approaches. In parallel, other certified robustness approaches such as randomized smoothing [12, 39, 37] give Mfte-of-theeard certification for 2--based perturbations. Notable works such as [34, 74] have sought to generalize these techniques into language, but their guarantees strongly depend on the type of perturbation being performed. On the other hand, analytic approaches through convex relaxation inductively bound the output of neurons in a ReLU network across layers [79, 81, 77]. These works, however, are difficult to scale and also do not transfer easily to discrete/non-metriaablee domains.
Our approach, inspired by Lipschitz continuity, distills the empirical intuitions from the works of [43, 85] and provides theoretical certification guarantees independent of perturbation-type [34, 74] and domain modality. We demonstrate that knowledge continuity yields many practical applications analogous to Lipschitz. continuity which are easy to implement and are computationally competitive.
3 Preliminaries
Notations. Let R20 = [0. co). For any function f & y, we denote graph(f) := [(x, X x y : f(x) = y). For E N, let [n] denote the set .n). (x. Px). (2, Py.
are probability spaces and (X x y. Fx @ Px x yy) denotes the product measurable space of the probability spaces X', Y. Since our contribution focuses on the supervised learning regime, we colloquially refer Y. as the input and labels, respectively. We call any probability measure Pxxy absolutely continuous to Px X y (i.e. (Px x P))(E) = 0 = (X) = 0) data distribution and denote it as D Dxy. If dz) is a metric space with metric dz and c Z. then for any z E Z, dz(z, A) inf aEA dz z). We say that metric space, (Z.dz), is bounded by B. Denote by Idz Z 2 the identity function on Z. Let y x - R220 be loss function where L(y,y) = 0 and only if y= y'. For any f x X' + y and (x, y), (x'. , E X' x y, we denote sC( (x', )²) : |C((()))) - C((())))))))))))))))) y'), essentially the absolute difference in loss between (x, y) and (x', y²). Unless otherwise specified, it will be assumed that f is a measurable function from X to y with metric decomposition (see Def. 1).
some B E R20, if sup=,=ez d(z,z'
Lipschitz continuity. Given two metric spaces (X, dx), dy)dy) a function f x - Vis K-Lipschitz continuous if there exists K e R2 such that for all x, x' E Ex, dy(s(x), f(x))))(()) (dx(x,x))
4 Knowledge Continuity
In this section, we provide the formal definition of knowlege continuity and explore its theoretical properties.
We start by defining model's perceived knowledge through a rigorous treatment of its hidden representation spaces. By considering the distance between inputs in some representation space in conjunction with changes in loss, result in mcasure of volatility analogous to Lipschitz continuity.
3

---

Bounding this volatility in expectation then directly leads to our notion of knowledge continuity. With these tools, we demonstrate host of theoretical properties of knowledge continuity including its certification of robustness, guarantees of expressiveness, and connections to Lipschitz continuity in continuous settings. We summarize our theoretical contributions as follows:
We define the perceived knowledge of a model as well as volatility and knowledge continuity within model's representation space (see Def. 2, 3, 4, respectively).
We prove that knowledge continuity implies probabilistic certified robustness under perturbations in the representation space and constraining knowledge continuity should not hinder the expressiveness of the class of neural networks (see Thm. 4.1 and Prop. 4.3, 4.4, respectively).
We prove that in some cases knowledge continuity is equivalent (in expectation) to Lipschitz continuity. This shows that our axiomization of robustness aligns with existing results when perturbation with respect to the input is well-defined (see Prop. 4.6, 4.8).
4.1 Defining Perceived Knowledge
Knowledge is generally understood as a relational concept: it arises from the connections we make between ideas, experiences, and stimuli [26]. Herein, we capture the perceived knowledge of a model by focusing on the relations it assigns to input-input pairs. Specifically, these relations are exposed by decomposing a function f : X y into projections to intermediate metric spaces. Formally, Definition 1 (Metric Decomposition). We say that admits metric decomposition if there exists metric spaces (Z..d)..... (Z d,)) with metrics d, for k E [n] such that
1. (Z dg) endowed with its Borel a-algebra.
2. There exists measurable mappings hy h hy...., h2 where hn : 2' - 2, h4 Ze - 2k+1
k E [n [n-11 1], and h Zn
3. ʃ = h, n oh,
oli-
Remark 1. If is metric space with metric dy and Px is its Borel o-algebra, then for any measurable
mapping f X - y there exists the trivial metric decomposition
(4.1) Therefore, in computer vision applications where dx) ) = (R".6,) ) for some n E Z*, we can apply this trivial decomposition to yield bounds which mirror the certification guarantees of Lipschitz continuity. This is discussed in detail in Section 4.5.
f= foldx-
To the best of our knowledge, all deep learning architectures admit metric decompositions, since their activations are generally real-valued. So, for all subsequent functions from X' to y, unless otherwise specified, we assume they are measurable and possess metric decomposition. Further, we denote f h2 - o ... oh, ohg and adopt the convention of calling hg the h hidden layer. In Appendix A, we present several metric decompositions for variety of architectures.
For any metrii-decomposible function, an immediate consequence of our definition is that its metric decomposition may not be unique. However, in the context of neural networks, this is a desirable property. Seminal works from an array of deep learning subfields such as semi-supervised learn- ing [57], manifold learning [5]], and interpretability [10, 47] place great emphasis on the quality of learned representation spaces by examining the induced-topor topology of their metrics. This often does not affect the typical performance of the estimator, but has strong robustness implications [33]. Our results, which are dependent on particular metric decompositions, capture this trend. In Section 4.4, we discuss in detail the effects various metric decompositions on our theoretical results.
4.2 Defining Knowledge Continuity
We first introduce what means for a model's 's performance to be volatile at a data point relative to its metric decomposition. Then, we contrast knowledge continuity with Lipschitz continuity, pointing out key differences that will allow us to prove more general bounds.
Definition 2 (k-V V-Voaaiilityy). Let f : X y and L be any loss function. The k-volatility of a point (x. E X x y which we denote as of (x. is given by
sc(xM(x'.)))
(x. ELESDDDDD SC0S(())
(4.2)
fx),

---

X
o.... hh h (Zk, dk)
y
Nn o...hee2 Nu+1
Sparse Knowledge Continuity
Knowledge Continuity
Knowledge Discontinuity
((f(z), v)
Figure 1: Examples of knowledge (dis)continuities. f X y a measurable map. and (2 d one of its hidden representations. The color of the points indicates loss. denotes knowledge continuity induced by sparsity: an isolated concept with no knowledge relations close to So, any perturbation moves far away with high probability. Smooth changes in loss around * implies knowledge continuity. Finally, is not knowledge continuous due to drastic changes in loss nearby. Notice that the classification of points is independent of input/output clustering behavior since X may not be endowed with a metric.
where dg is the distance metric associated with f t hidden layer.
By performing some algebra on the definition, we see that it decomposes nicely into two distinct terms: sparsity of the representation and variation in loss.
[(((x),) - (((())))
(xx Ex
(x),
-((()))))) 2(f(),))
=[((()).)) (x)
(4.3)
sparsity
variation loss
Our notion of volatility essentially measures the change in performance with respect to perturbations model's perceived knowledge. In particular, Eq. 4.3 reveals that there are interactions in play which we illustrate in Fig. 1. Informally, we say that (x, y) is highly volatile if there is large discrepancy in performance between it and points that are perceived be conceptually similar. Therefore, highly volatile points capture inaccurate input-input knowledge relations. Additionally, (x. y) experiences low volatility the space around is sparse with respect Dx In other words, any set of perturbations applied in Z, would push (x. y) far away, with high probability. This makes (x. y) isolated concept with little knowledge relationships associated with
Similar to Lipschitz continuity, the boundedness the k-volatility f across the data distribution is crucial and we denote this class of functions knowledge continuous.
Definition 3 (Pointwise e- nowneeee Continuity). We say that f is e-knowledge continuous at (x, y) y with respect a function loss function E, and hidden layer if o² (x.,)) <.
Conversely, we say that (x, y) is e-knowledge discontinuous if the previous inequality does not hold. Further, (x, y) is simply knowledge discontinuous if a ox y) is unbounded. Now, we extend this definition globally by considering the k-volatility between all pairs of points.
5

---

Definition 4 (Expected c- e-Knowledge Continuity), ). We say that f is e-knowledge continuous with respect loss function and hidden layer k if
E(ry>-lo (x. y)] <c
(4.4)
Though the functional forms of Lipschitz continuity and knowledge continuity are similar, there are important differences that allow us to prove more general results. Firstly, unlike Lipschitz continuity which an analytical property of the model f, knowledge continuity is a statistical one. In this way, non-typical data points, even if they are volatile, are ignored, whereas Lipschitz continuity treats all points equally. This is necessary in many discrete applications, as projecting a countable input space onto a non-countable metric space inevitably results in a lack of correspondence thereof. Moreover, ground-truth relations from X' - y may not be well-defined on all X:: consider sentiment classification of dipha-numeric UUID string dog-cat classification of Gaussian noise. Secondly, the knowledge continuity of an estimator is measured with respect to the loss function rather than its output. This property allows us achieve the expressiveness guarantees in Section 4.4, since it places no restrictions on the function class of estimators. Lastly, knowledge continuity measures the distance between inputs with the endowed metric in its hidden layers. This flexibility allows to define knowledge continuity even when the input domain is not metric space.
4.3 Certification of Robustness
Our first main result demonstrates that e-knowledge continuity implies probabilistic certified robust- ness in the hidden representation space. In Theorem 4.1, given some reference set A C X x V, we bound the probability that a 6-sized perturbation in the representation space away from will result in an expected change in loss. In other words, knowledge continuity is able to characterize the robustness of any subset of data points with positive measure.
Theorem 4.1. Let A C & X y such that P [A]
> 0 and 6. O.
Let A'
I : 2 y is e-knowledge continuous with respect to the hidden layer indexed by and (Z ) is bounded by B > 0, then
(K,))EXX E(XYyDDx (X,yEA
P PO..D-D (x,00(())), 5
Eō
(4.5)
1 c -22
log P[AÍ
Proof sketch. We apply the definition of conditional probability P)A|B) P(A B() and bound P), P(B), separately. The numerator, P[A' and (x( < 6). is upper-bounded through an application of Markov's Inequality. On the other hand, we apply known concentration inequalities to lower bound P[d, (xx,,, f f*(A)) < 5], combining these results in the theorem. We present the proof in its entirety in Appendix B.
This demonstrates that knowledge continuity results in certification of robustness, independent of distance metric and domain modality. The assumption of boundedness and requirement to know P[A] can be lost by taking limits of Eq. 4.5 with respect to B and P[A]. This yields the following corollary. Corollary 4.2. If (Z d()) is unbounded. then
P P(N IA' I (,("*(x) < 60 ed
(4.6)
n11 P[AA)
!fP[A] = 0, then
co
P PaNc>-D [A' 1A'-D (x((*()) < <
(4.7)
Proof. These results follow from directly taking the limit as B - co and applying some of the bounds acquired in the proof of Thm. 4.1. This yields Eq. 4.6. Next, jointly taking the limit as P[A] - and B - co results in Eq. 4.7.
In both Thm. 4.1 and Cor. 4.7, we yield probabilistic guarantees like [12], rather than deterministic ones. Though deterministic bounds are desirable, the stochasticity of our framework is necessary

---

for its generalization across different domains. For most continuous, metrizable applications (like computer vision), models learn hidden representation space where most minute changes in this space correspond to tangible inputs. The same cannot be said for many discrete or non-metrizable applications. In natural language processing, the correspondence between the learned representation space and the input is sparse, resulting in lots of "dead space": portions of the hidden representation space that do not correspond to any input [3, 19J. And so, by incorporating the data distribution into our bounds, we implicitly adjust for this: assigning zero-measure to the aforementioned "dead space."
4.4 Expressiveness
Our second main result demonstrates that e-knowledge continuity can be achieved without theoretically compromising the accuracy of the model. In other words, universal function approximation is an invariant property with respect to e-knowledge continuity. Universal approximation results have seen a great deal of theoretical work, they put limits what neural networks represent (1,431, As discussed in Section 2, Lipschitz continuous functions do not achieve universal function approximation with respect the set of all functions, in particular, non-continuous ones. However, we show that under strong conditions this is achievable with knowledge continuity.
First, let us formally define a universal function approximator.
Definition 5 (Universal Function Approximator). Suppose that L is Lebesque-integrable in both coordinates. Let P c yd be set of measurable functions from X - y such that for any f E, there exists y & Dxy such that y (ggraph(f) = 1. Then, V C F is a universal function approximator P for every f E and every > 0, there exists f E V such that
functiion
[ ((f(x)))) <.
(4.8)
We now show any universal function approximator can be made robust through the trivial metric decomposition.
Proposition 4.3. Let U c yd be a universal function approximator of y with respect to some loss function L. Then, for any f E y and sequence E E1, ...... such that E, - 0 there are a sequence of e, kn-knowledge continuous functions in U such that f [f(((x)), y) y < E for n E N.
Proof. Choose Sn E U such that f /,,((()))))))))))))) duy < Eep Consider the l-layer metric decomposi- tion of f, h1 X² 2, where Zi X equipped with the trivial metric (d, (x, y) = 1 if x # y and 0 otherwise). Then, fn = h. So, it follows that
sC (x
/
Eo (x.y)
dx(h,(x),, (x))) duy.
Jn
- ACtx d fn
c,
and by the construction of fa, the proof is completed.
In other words, if our estimator was given "infinite representational capacity." robustness can be trivially achieved by isolating every point as its own concept (as discussed in Section 4.2). More generally, if we instead considered a generalized discrete metric (fix e E [0, oo), d(x, y) e if and only if = y and d(xx, y) O, otherwise), then as co, k-volatility converges pointwise to 0 almost everywhere assuming that the loss is finite almost everywhere. In practice, we find these degenerate decompositions to be unreasonable robustness is not well-defined as any perturbation would lead to a point that is perceived to be infinitely far away. In this sense, our framework accounts for different notions of robustness, strong and weak. The next result builds on Prop. 4.3 and demonstrates how stronger notion of robustness will affect expressiveness. These added constraints make it so that trivial metric decompositions are no longer possible unless the metric in X' is also trivial. We state this formally below, note the highlighted differences between this and Prop. 4.3.
they also trivialize robustness. For example, if c = co, then
Proposition 4.4. Suppose (X, Nx,xx) (V (Y.x). dy) := (X, dx) are compact metric spaces, F C y is the set of all continuous functions from X' to y such that dx'(x.'))' < and L be Lipschitz continuous both coordinates. Then, there exists a universal function approximator V of F that knowledge continuous (i.e. E Eo², (x. y) < co for some k).
7

---

Proof sketch. We show an outline of the proof here and defer the full proof to Appendix C. By the Stone-Weierseee Weierstrass Theorem, the set of Lipschitz continuous functions dense in the set of all continuous functions from X' to V. Since C is Lipschitz continuous in both coordinates, through some algebra, Ex y) < co, where h = Idx and we yield the statement of the theorem.
The additional constraint / dx(xx ddy, requires data points to be sparsely layed out in the representation space. As discussed previously, this assumption is generally reasonable for discrete applications. In conjunction with Prop. 4.3, we have shown that the class of knowledge continuous functions is strictly larger than the class of Lipschitz continuous ones. Though we show that universal approximation by knowledge continuous networks is achievable, it is unclear whether these results still hold if the "tightness" of the metric decompositions bounded. Specifically, the construction in Prop. 4.3 results in metric decomposition with infinite Hausdorff dimension. Is it possible to achieve Prop. 4.3 in its most general form if we only consider the set of all knowledge continuous functions with metric decompositions with finite Hausdorff dimension? Based the theoretical and empirical results of [62. 33], respectively, we conjecture in the negative and leave its resolution open. Conjecture 4.5. U C y² is a universal function approximator with respect to some Lebesgue- integrable loss function L. Then, for any f E y there does not exist a sequence of functions with metric decompositions of finite Hausdorff dimension that achieve arbitrarily small approximation error (i.e. / ((((x), y)). and knowledge continuity.
4.5 Connections to Lipschitz Continuity
We now demonstrate that our axiomization of robustness presented in Section aligns with the notion of robustness commonly prescribed in vision [18]. This unifies the certified robustness bounds with respect to the representation space derived in Thm. 4.1 with existing work certifying robustness with respect to the input space in continuous applications such as vision.
Our first result identifies conditions under which knowledge continuity, implies Lipschitz continuity. Proposition 4.6. Suppose that (X, dz.)))... dy) are metric spaces. Let the first metric decompo- sitions of f : x - y be K,-Lipschitz continuous, for E [n]. If is c-knowledge continuous with respect to the 1tth hidden layer and (x) fx')) S n (x',,, J (x',)) for all x,x' /' E &, y y, and
some 11 > 0, then f is Lipschitz continuous expectation. That is,
f(x'()),f('))
R
Fo dx(x,x') S En I
(4.9)
1=1
The proof is presented in Appendix D and follows easily through some algebriac manipulation. It is easy to see that if f is knowledge continuous with respect to some identity (or contractive) metric decomposition, then we can loose the repeated product. Analogous to Remark 1, the concepts of Lipschitz continuity and knowledge continuity become similar when we can assign metrics to the input-output spaces. Next, combining this proposition with an auxiliary result from [89], we directly yield a certification on the input space.
Corollary 4.7. Suppose that assumptions of Prop. 4.6 are true. And also assume that (2, dx)
(R", Cp). (V,dy) = (RT", Lp). for 1 S p S co. Define a classifier from f R" - R g, where g(x) := arg max for any x E R.. Then, with probability 1 - 2n I K,, g(x) = g(x +8) for all ||ö11, < mm)rggin (x)) and > O. fx(x) is the thth coordinate of fxx and margin(f (x)) denotes the difference between the largest and second-large output logits.
We present the proof in Appendix D. Our second result identifies conditions under which Lipschitz continuity, implies knowledge continuity.
X Proposition 4.8. Let dy). Odyy) be a metric spaces. Let f : - y be e-Lipschitz continuous and L(f(x),) y) be n n-Liscchitz continuous with respect to both coordinates. If the first n metric decompositions of f are Li-Liccchitz continuous, then f is knowledge continuous with respect to the Th hidden layer. That is,
E(xy)> of S en II K
(4.10)
"Small perturbations on the input result in small changes in performance which implies small changes in output when the loss function Lipschitz continuous.
8

---

0.0
0.2
y 0-009910x 0.2011
of Su Aeherttihherrtttt Atocks
-2.5
0.35
0.0
0.25
Normalzod
0.20
-25
0.25
08
0.20
-24
y- 0.0585x 0.3236
0.
-2
0.8
(8) Norralzeed n-
(6) Relative Depth
Radatio
Figure (a) The average percentage of successful adversarial attacks by TextFooler [35] on host of models [58, 57, 16, 44] and the IMDB [48] dataset regressed with the average of knowledge continuity coefficients across all hidden layers (R² 0.35). (b) k-Volatility as k is varied across model's relative depth. (c) Correlation between k- k-volatiytty and adversarial vulnerability (averaged across all
models shown in (b)) with respect to TextFooler [35] as k varies.
We detail the proof of this proposition in Appendix D. We note that in continuous applications such computer vision, the assumptions of both propositions generally met (i.e. input-output spaces are metric spaces, all hidden layers are Lipschitz, and loss functions are locally Lipschitz). Furthermore, common architectures such fully connected networks, CNNs, RNNs, and even vision transformers are Lipschitz continuous [7 55J. This implies that our notion of robustness is indeed an appropriate generalization that transcends domain modality since in continuous settings can recover the strong bounds of Lipschitz continuity while expanding into new discrete and non-metrizable territory.
5 Practical Applications
In addition the theoretical guarantees given by knowledge continuity in Section also demon- strate that knowledge contimuity can be easily applied in practice. First, we find that knowledge continuity, similar to Lipschitz continuity, lines, our measure of volatility (see Def. 2) can be used to isolate particularly vulnerable hidden representations, These applications then directly motivate regulation of knowledge continuity means to enforce robustness.
be used gauge adversarial robustness. Along these
Unless otherwise specified, we run all of our experiments on the IMDB dataset [48] (a sentiment classification task) using host language models from different model families (encoder, decoder, encoder-decodee We also present additional experiments vision tasks. These experiments can be found in the Appendix G.
Knowledge continuity can predict adversarial robustness. For given model, f. with n hidden representations, choose some & E [n]. Then, consider the hidden representation index by k. For this fixed k, we determine its k-volatility by directly estimating Def. through naive Monte-Carlo algorithm (see Appendix for more details). Repeating this for all & E [n], we yield a collection of k-volatilities which we denote {E (e...... e,), one for each hidden layer. When we regress simple average of these coefficients, n Σi= cg. with the empirical adversarial robustness (estimated using TextFooler [35]), strong correlation is observed. This shown in Fig. 2(a). In particular, knowledge continuity alone is able to explain 35% of the variance in adversarial attack success rate. When we combine &-volatility with other model propertics like size, model family, even more variance can be explained (R2 0.48). Thus, knowledge continuity may be used as computationally efficient method to estimate adversarial vulnerability with respect to the input space as compared to iteratively applying real adversarial attacks. Moreover, when the adversary is unknown a priori, knowledge continuity can also be used in this way as a diagnostic tool. A detailed discussion of these experiments presented in Appendix E.
Knowledge continuity can localize vulnerable hidden representations. plot the relationship between the k-volatility, éx. and the relative depth of the model (i.e. k/n). We find that language models belonging to different model families (encoder, decoder, encoder-decoder) admit different volatility trajectories. This is shown in Fig. 2(b). In this way, knowledge continuity may provide more

---

Table 1: Comparison of our knowledge continuity algorithm to existing works across various model families and adversarial attack methods. TF, BA, ANLI denote adversarial attacks [35], [40], and [52], respectively. Regulating knowledge continuity to improve robustness is superior across almost all tasks and attacks.
Arch.
Method
IMDB
1MDB
IMDB BA
ANLIRI
ANLI22
ANLI
Base
93.6
47.9
45.2
44.5
45.6
33.8
BERT [16]
TF [35]
93.3
69.2
62.5
~110M params
ALUM [43]
93.5
56.9
45.2
46.7
46.3
KCReg (ours)
94.8
75.1
84.9
45.6
46.9
45.3
Base
93.6
63.9
54,9
42.7
44.9
43.4
GPT2 [57]
TF [35]
92.0
64.5
51.3
X
X
~1.5B params
ALUM [43]
94.9
49.4
27.5
43.8
45.2
44.6
KCReg (ours)
94.9
87.8
90.6
47.1
48.1
44.7
Base
93.7
53.9
39.3
46.1
44.7
46.0
T5 [58] -200 params
TF [35]
96.8
60.6
ALUM [43]
95.1
67.1
51.9
44.5
44.8
44.4
KCReg (ours)
94.9
89.3
91.3
48.2
45.0
44.3
nuanced picture of model's s inductive biases and robustness beyond scalar value like "accuracy under adversarial attack." " We present a detailed analysis of this in Appendix F. Further, these dynamics may act as a diagnostic tool and offer starting point for designing model-specific robustness interventions or adversarial defenses. For example, when insights from Fig. 2(b) are combined with a knowledge continuity regularization algorithm, this yields superior empirical robustness compared to existing methods. This shown
next subsection and Appendix G. In addition, knowledge continuity can also quantitatively characterize adversarial attack against host of models which is useful for online adaptive defenses [84, 64, 14). This is shown in Fig. 2(c), where TextFooler [35] largely exploits the knowledge continuities in middle/final layers of the model to decrease performance.
Regulating knowledge continuity. Motivated by the theoretical results in Section 4, we augment the loss function during training to mitigate knowledge continuity. Specifically, on each training iteration (batch), we start by choosing hidden layer at random according to a Beta distribution determined priori: X ~ Beta(a, ()) and let & = [nX] J. Here, a. f are chosen according to Fig, 2(b,c). We assign larger sampling probability to layers where both k-volatility high and where knowledge continuity is highly correlated with adversarial robustness. In this way, our regularization objective is both model and attack specific (if the attack method unknown, then we only apply the former). Then, we devise Monte-Carlo algorithm to estimate this layer's k-volatility, Eg. (see Appendix G) on this minibatch. And so, the augmented loss function becomes L'f((x) y) = ((f(), y) + deş with À 2 hyperparameter, controlling the regularization strength. In contrast existing adversarial training methods that perform inner-optimization steps [50, 43, 85], our method requires only additional heroth-order computations. As result, it outperforms existing works in training speed (up to 2x for TextFooler [35] and 3x for ALUM [43]), while improving robustness. We present a discussion the results, ablation studies, and training details in Appendix G.
Certifying robustness with knowledge continuity. We present an algorithm based on Thm. to certify robustness during test-time. Similar to [12], we estimate the probability of there existing an adversarial example within some fixed radius (in the representation space, according pre-defiend distance metric) through bootstrapping one-side confidence interval. Applying these methods to our regularization results, we show that regularizing knowledge continuity increases the certified robustness. The certification algorithm, its proof of correctness, and certifications of our regularized models are presented in Appendix H.
Conclusion
In this paper, we propose a novel definition, knowledge continuity, which addresses some of the key limitations of Lipschitz robustness. We demonstrate that our definition certifies robustness across domain modality, distribution, and norms. We also show that knowledge continuity, in contrast to Lipschitz continuity, does not affect the universal approximation property of neural networks. We also establish conditions under which knowledge continuity and Lipschitz continuity are equivalent. Lastly, we present several practical applications that directly benefit the practitioner. The broader impacts, reproducibility, and limitations of our work can be found in Appendix I, J, K, respectively.
10

---

7 Acknowledgements
Alan Sun thanks Fengwen Sun for the helpful feedback on early drafts of the work as well as Jeffrey Jiang and Andrew Koulogeorge for thoughtful discussions.
References
[1] M. Alzantot, Y. Sharma, A. Elgohary, B.-J. Ho, M. Srivastava, and K.-W. Chang. Generating natural language adversarial examples. Riloff, D. Chiang, Hockenmaier, and Tsujii, editors, Proceedings of the 2018 Conference on Empirical Methods Natural Language Processing, pages 2890-2896, Brussels, Belgium, Oct.-Nov. 2018. Association for Computational Linguistics.
[2] C. Anil, J. Lucas, and R. Grosse. Sorting out lipschitz function approximation. In International Conference on Machine Learning, pages 291-301 PMI., 2019.
[3] S. Arora, Y. Li, Y. Liang. T. Ma, and A. Risteski. A Latent Variable Model Approach to PMI-based Word Embeddings. Transactions of the Association for Computational Linguistics, 4:385 3995,,3990000 07 2016.
[4] P. L.. Bartlett, D. J. Foster, and M. J. Telgarsky. Spectrally- normalized margin bounds for neural networks. Advances in Neural Information Processing Systems, 30, 2017.
[5] S. Biderman, U. PRASHANTH, L. Sutawika, H. Schoelkopf, Q. Anthony, S. Purohit, and E. Raff. Emergent and predictable memorization in large language models. Advances in Neural Information Processing Systems, 36, 2023.
[6] D. Blackwell. Conditional expectation and unbiased sequential estimation. The Annals of Mathematical Statistics, pages 105-110, 1947.
[7] N. Carlini, M. Nast, C. A. Choquette- Choo, M. Jagielski, L Gao, P. W. W. Koh, D. Ippolito. F. Tramer, and L.. Schmidt. Are aligned neural networks adversarially aligned? In Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, a Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 11477-66500. Curran Associates, Inc., 2023.
[8] G. Casella and R. Berger. Statistical inference. CRC Press, 2024.
[9] P. Chao, A. Robey, Dobriban, H. Hassani, J. Pappas, and E. Wong, Jailbreaking black box large language models in twenty queries. In RO-FoMo:Robustee of Few-shot and Zero-shot Learning in Large Foundation Models, 2023.
[10] C. Chen, O. Li, D. Tao, A. Barnett, C. Rudin, and J. K. Su. This looks like that: deep learning for interpretable image recognition. Advances in Neural Information Processing Systems, 32, 2019.
[11] M. Cisse, P. Bojanowski, E. Grave, Y. Dauphin, and N. Usunier. Parseval networks: Improving robustness to adversarial examples. In International Conference on Machine Learning, pages 854-863. PMLR, 2017.
[12] J. Cohen, E. Rosenfeld, and Z. Kolter. Certified adversarial robustness via randomized smoothing, In K. Chaudhuri and R. Salakhutdinov, editors, Proceedings of the 36th International Conference Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 13 1310-1320. PMLR, 09-15 Jun 2019.
[13] Z. Cranko, Z. Shi, X. Zhang, Nock, and S. Kornblith. Generalised lipschitz regularisation equals distributional robustness. In International Conference on Machine Learning, pages 2178 2188. PMLR, 2021.
[14] F. Croce, S. Gowal, T. Brunner, E. Shelhamer, M. Hein, and T. Cemgil. Evaluating the adversarial robustness of adaptive test-time defenses. In K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, and S. Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 4421 4435. PMLR, 17-23 Jul 2022.
[15] G. Cybenko. Approximation by superpositions of sigmoidal function. Mathematics of Control, Signals and Systems, 2(4):303-33 -34, 1989.
[16] J. Devlin, M.-W Chang, K. Lee, and K. Toutanova. BERT; Pre-training of deep bidirectional transformers for language understanding. Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Volume I (Long and Short Papers), pages 4171 141866 Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
Burstein, C. Doran, and Solorio, editors, Proceedings of the 2019

---

[17] A. Dosovitskiy, L. Beyer. A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Min- derer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby. An image is worth 16x16 words: Transformers image recognition at scale. In International Conference on Learning Representations. 2021.
[18] N. Drenkow, N. Sani, I. Shpitser, and M. Unberath, systematic review of robustness in deep learning for computer vision: Mind the gap?, 2022.
[19] N. Elhage, Hume, C. Olsson, N. Schiefer. Henighan, S. Kravec, Z. Hatfield-DDdds, R. Lasenby, D. Drain, C. Chen, et al. Toy models of superposition. arXiv preprint arxiv: 200.22200. 2022.
[20] K. Eykholt, I. Eytimov, F. Fernandes, B. Li, A. Rahmati, C. Xiao, Prakash, T. Kohno, and D. Song. Robust physical- attacks on deep learning visual classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
[21] M. Fazlyab, A. Robey, H. Hassani, M. Morari, and G. Pappas. Efficient and accurate estimation of lipschitz constants for deep neural networks. Advances in Neural Information Processing Systems, 32, 2019.
[22] F. Gama, J. Bruna, and A. Ribeiro. Stability properties of graph neural networks. IEFF Transactions on Signal Processing, 68:5680-5695, 2020.
[23] S. Garg and G. Ramakrishnan. BAE: BERT based adversarial examples for text classifficct,,.. In B. Webber, T. Cohn, Y. He, and Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6174-6181, Online, Nov. 2020. Association for Computational Linguistics.
[24] J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. Proceedings of 3rd International Conference on Learning Representations, 2014.
[25] H. Gouk, E. Frank, B. Pfahringer. and M. J. Cree. Regularisation of neural networks by enforcing lipschitz continuity. Machine Learning, 1 10:3933-416,
2021. 2021.
[26] G. S. Halford, W. H. Wilson, and S. Phillips. Relational knowledge: the foundation of higher cognition. Trends in Cognitive Sciences, (4111):4775055 2010.
[27] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
[28] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770-778, 2016.
[29] Hein and M. Andriushchenko. Formal guarantees on the robustness of classifier against adversarial manipulation. In Guyon, U. V. Laxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gamett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
[30] D. Hendrycks. K. Zhao, S. Basart, J. Steinhardt, and D. Song. Natural adversarial examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages June 2021.
[31] K. Hornik, M. Stinchcombe, and H. White. Multilayer feedforward networks are universal approximators. Neural Networks, 2(5):359-366, 1989.
[32] Y. Huang. Zhang, Y. Shi. Kolter, and Anandkumar. Training certifiably robust neural networks with efficient local lipschitz bounds. Advances in Neural Information Processing Systems, 34:22745- 22757, 2021.
[33] A. Ilyas, S. Santurkar, D. Tsipras, L. Engstrom, B. Tran, and A. Madry. Adversarial examples are not bugs, they are features. Advances in Neural Information Processing Systems, 32, 2019.
[34] R. Jia, A. Raghunathan, K. Göksel, and P. Liang. Certified robustness to adversarial word substitutions. In K. Inui, J. Jiang, V. Ng, and X. Wan, editors, Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4129-41 4142, Hong Kong, China, Nov. 2019. Association for Computational Linguistics.
[35] D. Jin, Z. Jin, J. T. Zhou, and P. Szolovits. Is BERT really robust? a strong baseline for natural language attack on text classification and entailment, In Proceedings of the AAAI conference on artificial intelligence. volume 34, pages 8018 8025, 2020.
12

---

[36] H. Kim, G. Papamakarios, and A. Mnih. The lipschitz constamt of self-attention. In M. Meila and T. Zhang, editors, Proceedingy of the 38th International Conference Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 5562-5571, PMLR, 18-24 Jul 2021.
[37] M. Lecuyer, V. Atlidakis, R. Geambasu, D. Hsu, and S. Jana. Certified robustness adversarial examples with differential privacy. In 2019 IEEF Symposium on Security and Privacy (SP), pages 656-672. IEFE, 2019.
[38] K. Leino, Z. Wang, and M. Fredrikson. Globally- robust neural networks, In International Conference on Machine Learning, pages 6202-6222222 6222. PMLR, 2021.
[39] B. Li, C. Chen, W. Wang, and L. Carin. Certified adversarial robustness with additive noise. Advances in Neural Information Processing Systems, 32, 2019.
[40] L. Li, R. Ma, Guo, X. Xue, and X. Qiu. BERT-ATTACK: Adversarial attack against BERT using BERT. In B. Webber, T. Cohn, Y. He, and Y. Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP),, pages 6193-6202, Online, Nov. 2020. Association for Computational Linguistics.
[41] Lin, C. Song, He, L. Wang, and E. Hoperoft. Nesterov accelerated gradient and scale invariance for adversarial attacks. In International Conference on Learning Representations, 2020.
[42] Liu, Z. Shen, He, Zhang, R. Xu, H. Yu, and P. Cui. Towards out-of-distribution generalization: survey, 2023.
[43] X. Liu, Cheng, P. He, Chen, Y. Wang, H. Poon, and J. Gao. Adversarial training for large neural language models. arXiv preprint arXiv:2004. 2020.
[44] Y. Liu. M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov. RoBERTa; A robustly optimized BERT pretraining approach, 2020.
[45] Z. Lu, H. Pu, F. Wang. Z. Hu, and L. Wang. The expressive power of neural networks: A view from the width. Advances in Neural Information Processing Systems, 30, 2017.
[46] B. Lütjens, M. Everett, and J. P. How. Certified adversarial robustness for deep reinforcement learning. In L. P. Kaclbling, D. Kragic, and K. Sugiura, editors, Proceedings of the Conference on Robot Learning, volume 100 of Proceedingy Machine Learning Rexearch, pages 1328-1337. PMLR, 30 Oct-01 Nov 2020.
[47] C. Ma. Zhao, C. Chen, and Rudin. This Looks Like Those: Illuminating Prototypical Concepts Using Multiple Visualizations. Advances in Neural Information Processing Syxtems, 36, 2024.
[48] A. Maas, R. Daly, P. T. Pham, D. Huang, A. Y. Ng. and C. Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 440- Portland, Oregon, USA, June 2011. Association for Computational Linguistics.
[49] T. Mikolov, I. Sutskever. K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In C. Burges. L. Bottou, M. Welling, Z. Ghahramani, and K. Weinberger, editors, Advances in Nearal Information Processing Systems, volume 26. Curran Associates, Inc., 2013.
[50] T. Miyato, S.-i. Maeda, M. Koyama, and S. Ishii. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41 1889:1997-1993, 2018.
[51] M. Moor, M. Horn, B. Rieck, and K. Borgwardt, Topological autoencoders. In International Conference on Machine Learning, pages 7045-7054, PMLR, 2020.
[52] Y. Nie, A. Williams, E. Dinan, M. Bansal, J. Weston, and D. Kiela. Adversarial NLI: A new benchmark for natural language understanding. In D. Jurafsky, J. Chai, N. Schluter, and J. Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4885-4901. Online, July 2020. Association for Computational Linguisties.
[53] Y. Oren, S. Sagawa, T. B. Hashimoto, and P. Liang. Distributionally robust language modeling. In K. Inui, Jiang. V. Ng, and X. Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 0227-4331 Hong Kong, China, Nov. 2019, Association for Computational Linguistics.
13

---

[54] P. Pauli, A. Koch, J. Berberich, P. Kohler, and F. Allgöwer. Training robust neural networks using lipschitz bounds. IEEE Control Systems Letters, 6: 6:121-126, 2021.
[55] X. Qi, J. Wang, Y. Chen, Y. Shi, and L.. Zhang. Lipsformer: Introducing lipschitz continuity vision transformers. The Eleventh International Conference on Learning Representations, 2022.
[56] A. Radford, W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin,
J. Clark, ct al. Learning transferable visual models from natural language supervision. In International conference machine learning, pages 8748-8763. PMLR, 2021.
[57] A. Radford, K. Narasimhan, T. Salimans, I. Sutskever, et al. Improving language understanding by generative pre uve-training. 2018.
[58] C. Raffel,, N. Shazeer, A. Roberts, K. Lee. S. Narang, M. Matena, Zhou, Li, and P. Liu. Exploring the limits of transfer learning with unified text-to-fext transformer. Journal of Machine Learning Research, 21 1-67, 2020.
[59] W. Ruan, X. Huang, and M. Kwiatkowska. Reachability analysis of deep neural networks with provable guarantees. In Proceedings of the 27th International Joint Conference Artificial Intelligence, UJAA 18, page 2651 2659. AAAI Press, 2018.
[60] Salehi, H. Mirzaei, D. Hendrycks, Y. Li, H. Rohban, and M. Sabokrou. unified survey on anomaly, novelty, open-set, and out of-distribution detection: Solutions and future challenges. Transactions on Machine Learning Rexearch, 2022.
[61] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.C.. Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4510 4520, 2018.
[62] A. Shafahi, W. R. Huang, C. Studer, S. Feizi, and T. Goldstein. Are adversarial examples inevitable? In International Conference on Learning Representations, 2019.
[63] A. Shafahi, M. Najibi, M. A. Ghiasi, Z. Xu, J. Dickerson, C. Studer, L. S. Davis, G. Taylor, and T. Goldstein. Adversarial training for free! Advances in Neural Information Processing Syxtems, 32, 2019.
[64] C. Shi, C. Holtz, and G. Mishne. Online adversarial purification based on self superviee learning. In International Conference on Learning Representations, 2021.
[65] M. H. Stone. The generalized weierstrass approximation theorem. Mathematicy Magazine, 21(5):237-254, 1948.
[66] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing properties of neural networks, 2014.
[67] N. Tishby, F. C. Pereira, and W. Bialek. The information bottleneckmethod arXiv preprint physics/000000//7 2000.
[68] Y. Tsuzuku, Sato, and Sugiyama. Lipschitz- margin training: Scalable certification of perturbation invariance for deep neural networks. S, Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Biancle and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018
[69] M. Usama and D. E. Chang. Towards robust neural networks with lipschitz continuity, In Digital Forensics and Watermarking: 17th International Workshop, IWDW 2018, Jeju Island, Korea, October 22-24, 2018, Proceedings 17, pages 373-389. Springer, 2019.
[70] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 30, 2017.
[71] A. Virmaux and K. Scaman. Lipschitz regularity of deep neural networks: analysis and efficient estimation. Advances in Neural Information Processing Systems, 31, 2018.
[72] E. Wallace, S. Feng, N. Kandpal, M. Gardner, and S. Singh. Universal adversarial triggers for attacking and analyzing NLP. In K. Inui, J. Jiang, Ng. and X. Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-JJCNLP), pages 2153 2162, Hong Kong, China, Nov. 2019. Association for Computational Linguistics.
14

---

[73] B. Wang, S. Wang. Y. Cheng, Z. Gan, R. Jia, B. Li, and J. Liu. Info[bert): Improving robustness of language models from information theoretic perspective. In International Conference on Learning Representations. 2021.
[74] W. Wang. P. Tang. J. Lou, and L. Xiong. Certified robustness to word substitution attack with differential privacy. In K. Toutanova, A. Rumshisky, L. Zettlemoyer, D. Hakkani-Tur; I. Beltagy. S. Bethard, R. Cotterell, T. Chakraborty, and Y. Zhou, editors, Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1102-1112, Online, June 2021. Association for Computational Linguistics.
[75] A. Wei, N. Haghtalab, and J. Steinhardt. Jailbroken: How does Ilm safety training fail? In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 80079-80110 10. Curran Associates, Inc., 2023.
[76] L. Weng, H. Zhang, H. Chen, Z. Song. C.-J. Hsieh, Daniel, D. Boning, and Dhillon. Towards fast computation of certified robustness for ReLU networks. In J. Dy and A. Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 5276 5285. PMLR, 10-15 15 Jul 2018
[77] L. Weng, H. Zhang, H. Chen, Z. Song, C.-J. Hsich, L. Daniel, D. Boning, and I. Dhillon. Towards fast computation of certified robustness for relu networks. In International Conference on Machine Learning, pages 1276-52855 PMLR, 2018.
[78] T.-W. Weng, H. Zhang, P.-Y. Chen, J. Yi, D. Su, Y. Gao, C.-. Hsieh, and L. Daniel. Evaluating the robustness of neural networks: An extreme value theory approach. In International Conference on Learning Representations, 2018
[79] E. Wong and Z. Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In International Conference on Machine Learning, pages 5286-5295 PMLR, 2018
[80] E. Wong, L. Rice, and J. Z. Kolter. Fast is better than free: Revisiting adversarial training. In International Conference on Learning Representations, 2020.
[81] E. Wong, F. Schmidt, J. H. Metzen, and J. Z. Kolter. Scaling provable adversarial defenses. Advances in Neural Information Processing Systems, 31, 2018.
[82] X. Xu, L. Li, Y. Cheng. S. Mukherjee, A. H. Awadallah, and B. Li. Certifiably robust transformers with I-lipschitz self-attention, 2023.
[83] Yang, K. Zhou, Y. Li, and Z. Liu. Generalized aatoof-distribution detection: A survey. International Journal of Computer Vision, pages 1-28, 2024.
[84] C. Yao, P. Bielik, P. Tsankov, and M. Vechev. Automated discovery of adaptive attacks on adversarial defenses. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, Jnd W. Vaughan, editors, Advances Neural Information Processing Systems, volume 34, pages 26858-26870. Curran Associates, Inc., 2021.
[85] J. Y. Yoo and Y. Qi. Towards improving adversarial training of NLP models. In M.-F. Moens, X. Huang, L. Specia, and S. W.tt. Yih, editors. Findings of the Association for Computational Linguistics: EMNLP 2021, pages 945-956, Punta Cana, Dominican Republic, Nov. 2021. Association for Computational Linguisties.
[86] Y. Yoshida and T. Miyato. Spectral norm regularization for improving the generalizability of deep learning. arXiv preprint arXiv: 10941, 2017.
[87] A. 7hang, A. Chan, Y. Tay, J. Fu, S. Wang, S. 7hang, H. Shao, S. Yao, and R. K.-W. Lee. On orthogonality constraints for transformers. In C. Zong, F. Xia, W. Li, and R. Navigli, editors, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the l 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 375-382, Online, Aug. 2021. Association for Computational Linguistics.
[88] B. Zhang, D. Jiang, D. He, and L. Wang. Boosting the certified robustness of l-infinity distance nets. In International Conference on Learning Representations, 2022.
[89] B. Zhang, D. Jiang, D. He, and L. Wang. Rethinking lipschitz neural networks and certified robustness:
A boolean function perspective. Advances in Neural Information Processing Systems, 35:19398-111 19413, 2022.
15

---

[90] H. Zhang, H. Chen, C. Xiao, S. Gowal, R. Stanforth, B. Li, D. Boning, and C.-J. Hsieh. Towards stable and efficient training of verifiably robust neural networks. In International Conference on Learning Representations, 2020.
[91] A. Zou, Z. Wang. N. Carlini, M. Nast, Z. Kolter, and M. Fredrikson. Universal and transferable adversarial attacks on aligned language models, 2023.
16

---

Table of Contents
1 Introduction
2 Related Works
3
Preliminaries
Knowledge Continuity
4.1
Defining Perceived Knowledge
4.2
Defining Knowledge Continuity
4.3
Certification of Robustness
4.4
Expressiveness
4.5
Connections to Lipschitz Continuity
8
5
Practical Applications
6
Conclusion
10
7
Acknowledgements
11
More on Metric Decompositions
18
A.1 Metric Decompositions of Common Neural Architectures
18
A.2 Beyond Neural Networks: Inducing Metric Decompositions
19
Proof of Robustness
20
B.I Technical Lemmas
21
C. Proof of Expressiveness
22
C.I Technical Lemmas
Proof of Equivalence Between Lipschitz Continuity and Knowledge Continuity
23
E Predicting Adversarial Robustness with Volatility
25
F Localizing Volatile Hidden Representations
26
F.1 Layerwise Volatility...
26
F.2 Model-Specific Volatility
27
G Regularizing Knowledge Continuity
27
G.I Estimating Knowledge Continuity Algorithmically
27
G.2 Theoretical Guarantees of k- k-Volatility Estimation
................................
28
G.3 Computer Vision Results
................................
29
G.4 Ablation Studies
................................
29
G.5 Training Details
31
Certifying Robustness at Test-Time
32
I Broader Impacts
33
J Reproducibility
33
K Limitations
33
NeurIPS Paper Checklist
34
17

---

A More on Metric Decompositions
In Section 4.1, we introduced the notion of metric decomposition rigorously define the hidden representations of a neural network. Herein, we show that our notion of a metric decomposition well-describes host of neural architectures and also point to possible applications of this concept beyond just deep learning. Let us first consider possible metric decompositions of common neural architectures.
A.1 Metric Decompositions of Common Neural Architectures
Fully-Con Tully-Connected Neural Network. Suppose that f : RR - RR is fully-connected neural network with hidden layers. Each hidden layer indexed by i [n] has weight matrix W; E R4_116d, bias R R44+, where E N, d, d Define the
E R and activation function o hidden layers as
hg (x) og (W bg).
for all & E [n]. Clearly, f ht - o And our intermediate spaces are simply (R4) "=1"" It remains to define a metric on these hidden spaces. There are many ways of doing this. For example,
10
For any p 2 endow each intermediate space with the é, --norm. Define d(x, y) = cos(Ox.) x.y) where 0,,y is the angle between x, y. Then, if we choose o; to restrict the image of h, to be the unit sphere, may endow each intermediate space with this cosine distance.
Note here that there are two steps here: first identify what the intermediate spaces are, then assign metrics to them. The process identfying these intermediate spaces may be independent of the metrics we end of assigning them.
Convolutional Neural Network. For simplicity, we only consider the case of single 2d-convolution layer, a convolutional network with higher dimensions or more layers can be derived inductively. Let f Rexhxw - c'''''xu'' Suppose that this layer is parameterized by kernels W, E [R txk for i [c'] and some & N as well as bias b R² Then, it follows that
f(x), (Iwxae
Σ w, xxli,;;)) :l/.al..:)))
for E [c'] where f(x), E #'w' for h', w/ being the resulting dimension after convolution with k & kernel. Here, Ixx Rli'xu is a one matrix. To induce distance metrie on this output space, we can simply define matrix norm on each of the output channels and sum them. Let (II-I1))- be collection of matrix norms. Then, we define
d(f()).f(x)) = Σ |
It is easy verify that this is metric. Thus, the availability of a metric decomposition is not affected by parameter sharing.
Instead of incorporating every individual channel into our metric, we may also consider applying a pooling operation before passing the result through single matrix norm, ||-I]. For example,
d(f(x), f(x))
1 c²
Σ f(x);
Σ f(''
This, however, is no longer metric, as definiteness is not preserved. That is, there exists f(x) # f(x') where dxf(x), f(x)) 0. This issue be easily resolved by having d(.,)) operate quotient
space with respect to the equivalence relation fx) f(x') ) if and only if Σi f(x); = Σi f(x), This technique is further explored in the subsection.
Residual Connections. We present two distinct metric decompositions of residual network. Consider two culleccon layers with one residual connection. This is visualized below.
18

---

A(-
A(x)
B()
B(A(x))
B(A(x)) +x
x
+x
Let us assume that A [d R and R - R4., Here, the input x feeds back into the output layer B creating residual block (the set of layers between the input and the residual connection).
Trivially, can aggregate the entire residual block h(x) B(A(x) + x be our metric decompasition. Then, define a metric on the image of h, Rd1, analogous to the hidden layers of fully-connected neural network. This is the approach we throughout our practical applications section (Section 5), and it is the standard way to counter layers in computer vision [27] and natural langauge processing [16].
metric decomposition. That is, let
To operate at finer lever of granularity, we can also represent each layer within the residual block as part of metric decomposition. Let us redefine the residual block such that at every layer, we keep track of the input. The computational graph for this shown below.
A(-) o
(A(x),x)
B(-) (B(A(x)),))
+X (B(A(x)) +x,x)
Define A' : x), (A(x), (B(A())) x) and x' (B(A(x)), x) - (B(A(x))xxx) x). Then, it follows that A' B' - x' forms metric decomposition. Here, the metric in each layer with respect the quotient space where (a, aa,a') ~ (b, b') and only a = b. Therefore, we also recover the same vector space structure.
Transformers. By chaining our metric decompositions for the residual blocks with our metric decompositions for the fully-connected networks we can easily create a metric decomposition for any transformer. Throughout the paper, we use two distinct methods to generate representations of its hidden layers:
After each attention block which consists multiheaded attention and multilayered percep- trons, we retrieve the last token.
We average all of the tokens together.
In both of these methods, we are significantly reducing the dimension of the hidden layer. Thus, to formalize these metrics, we need to quotient out points that break the definiteness of our metric, as we have done before with the residual block.
A.2 Beyond Neural Networks: Inducing Metric Decompositions
We have shown that our notion of metric decomposition well-describe many deep learning architectures, but what about models that are not neural networks (like a decision tree)? Herein, we demonstrate that we can induce metric decompositions even when the model itself does not have explicit hidden layers.
Let us now consider an arbitrary function f :' + y. We can induce a metric decomposition on through an auxiliary function f' X X, for metric-decomee metric-decomposable f". If Idz, then, f = fof" and the metric decomposition of would be exactly the metric decomposition of f". This visualized below.
X
So (Z S
Sh (2n
Sh
f
y
Metric Decomposition of f'
Essentially, we have created autoencoder for X. This is common in many applications where neural network or some other method used feature extractor. this way, we can simply define metric with respect to these extracted features. However, this requires that either the autoencoder
19

---

to be exact or that our function f is invariant under representations that collide. Thus, this would allow models such as decision trees to also be metric decomposed.
B Proof of Robustness
Theorem (See Thm. 4.1). Let A C X such that Px,, [Y] P 0 and 8, > 0. Let A' = (x''x e X x (( A (x', ,') 1). f &
is e-knowledge continuous with respect to
(x,))A the hidden layer indexed by and (2 dx) is bounded by B > 0, then
P(s.))DDy IA' (x),
5)
(B.1)
"))
11 1 - exp
Q 6 V
log PIAT
where = a
Proof.
P Payy-Dx A (A' < P(ay)DDx | (1) <8
(B.2)
Pay Idd ((), <
We bound the numerator and denominator of Eq. B.2 separately. The denominator given by Cor. B.3. We upper-bound the numerator using Markov's inequality. Firstly, we find the expectation
of (x', y') over A' (x)) 6:
(x's²')
E(xy))Dxy (xx))) ((x EWADDDAY
(B.3)
d (x((((
(B.4)
d (x(((()
The previous inequality follows from Fubini's theorem, then
E(xy>ーDa of (x 2 E 5 (
(B.5)
f(((
Exxyo''' E --OxyxDD) (x')))))'))))
(B.6)
(KY)EA d(("),
E (xx 2
Ex.I 5- (X'')EA ((((6))
(B.7)
And by e-knowledge continuity.
de 2 Eu..nnss//--DDDx>... /> (K/))EA
Accr
(B.8)
(())))
This gives an bouer-bound expectation of ae (x(x)) y') over the set of all points that within 5-radius from (*). Since sc(x (x', y') 2 0 everywhere, by Markov's inequality,
P P(cy) IA' (x < <8
(B.9)
20

---

ôe
(B.10)
The last inequality follows from E(x,>-Dxy ox y) < by the definition of e-knowledge continuity. Now, by applying the complement of Lem. B.2, lower-bound the denominator and yield the following
POyyDD | (x)0")), < <6
éò
(B.11)
(6 - B log P[A]
2
exp
The proof is concluded by applying 22(-) notation to the denominator.
B.1 Technical Lemmas
Definition 6. A function 21 ... X, has bounded variation if there are Cp.... Cp E R such that for all < i S n and E E&1, EI.....,, E Xn
( Xx,...,.)) f(. X.. x/Ex,
sup
x,,1 S G₁.
(B.12)
Lemma B.1 (McDiarmid's Inequality). Assume that the function f & .... X & - R. satisfy the bounded differences property with bounds CI..... Cg. Consider the independent random variables X X2 where X; X, for all 1 < Then, for any E 0,
2E
PX 1, x ELS(X ... X cl 2exp
(B.13)
Σ
Lemma B.2. Suppose that (X,d) is a bounded metric space such that supx,''xx' d(x,x') B for some B > 0. Let A C X such that [[A] > O and e > 0. Then,
2 B²
2 P[A]
Pld(x, A) 2 c] S exp
E' B
log
2
Proof. For brevity, denote by Lem. B.1, -f
= inf dEA d(x, a). Since (2', d) is a bounded metric space,
26
(B.14)
B²
2c
(x) -ff((>) + (X (
-e] 2 2xxp
(B.15)
B2
(x) - (x) & - -el 2exp (-22²
(B.16)
B2
Let E = (x) Then,
2(E
Plf < 0] 2exp
(B.17)
B²
2(EfA(x))
P[А] 2exp
(B.18)
B2
B
2
Efa()) S
log
(B.19)
2
P[A]
The second inequality follows from Pxf 0] PI) P[A]. By Eq. B.15,
26a
-Ex 2c] ES4() S -c] 5 2exp
B2
21

---

20a
- 2|
B2
.2
& 2exp
B²
B²
2 P[A]
2E
p
fa(x) 2 c
log
S 2xxp
(by Eq. B.19.)
2
B²
for any 8 0, e 8
B-
log
2
And so,
2
P[A]
2
2
PIf 2 8] < 2 exp
B
log
B
2
[PA]
which is the desired expression.
2
Corollary B.3. (x) 6] 2 1 2exp
2
6
B
log
2 P[A]
B²
C Proof of Expressiveness
Proposition (See Prop. 4.4). Suppose (2, dx, (V.d)) (x == (8,dx) are compact metric spaces, F C yd is the set of all continuous functions from to y such that 'xx(xxx')' du, < co and L be Lipschitz continuous in both coordinates. Then, there exists universal function approximator V of F that is knowledge continuous (i.e. EE xxx < co for some k).
Proof. By Lem, C.3, the set of Lipschitz continuous functions L is dense in the set of all contin- uous functions & with respect to uniform metrie. By Lem. C.I, since [C(x, y) S Kd y), supxex d(f(x),g(x)) < then for any probability measure JP over X,
/ ((((()) dP d S
/ [I(((()).gX)) dP Ke.
where K is the Lipschitz constant of L. This implies that for any sequence E,, 62,62.... we can choose Lipschitz continuous functions f2... with Lipschitz constants Cj. Cz.... such that I [(f,(x),)) y) duy < cr- It remains to show that each of these functions are in fact knowledge continuous. Since is a metric space, we consider the trivial metric decompasition of our sequence of functions (see Remark 1). Specifically, denote h Idy and proceed to bound E (x.
AC (x fo dx(x,x') (duy x xdu))
Eo (x,y) II
(C.1)
$ 11 (E(,)())))) ((S,(x'), ((S,(x)),)))) ((,()))))))))),,)))) (dir x ddj)) (C.2)
xx(x,x)
< 1 |2(,,,)))))
((S,())) - v) dx(x,x)
d(u xP))
(C.3)
I (C(.(()))) - ((,((x)))')
(duy x xdu)).
(C.4)
d(x.x)
<Il (dx((((),1(1))) d(nz x x lf xx(x,x')
kdx(v.)) day ny))
(C.5)
dx(x,x))
By Lem. C.4, any compact metric space is bounded. So, let (X, d) be bounded by b > 0. It follows that dx y) S and
< $1 KC, dlu, x (4)) Kb / dx(x,x)) duy.
(C.6)
22

---

=CC, Kb / dx(x,x')'
(C.7)
By assumption dx x1) d <
and the statement of the proposition follows.
C.1
Technical Lemmas
Lemma C.1. [(. -) is Lipschitz continuous both coordinates, then any x' E E [6(xxx)1 S
Kd(x,x') x'), where K is the Lipschitz constant of L.
Proof. By Lipschitz continuity,
(((x,x)) ((x,x) S Kd(x,x'), |e(x,x)1 S Kd(xxx').
Lemma C.2. The set of all Lipschitz continuous functions from X - X' separates all points in X.
Proof. The identity function is 1-Lipschitz continuous and it also separates all points in &.
Corollary C.3. Let 6 C X be the set of all continuous functions from & & and L X² be the set of all Lipschitz continuous functions from & + X. If is compact, then L is dense in '6 with respect to the uniform metric: df g) = suPxex g())))((
Proof. This follows directly from Lem. C.2 and the Stone-Weierstee Weierstrass theorem [65].
Lemma C.4. Any compact metric space (R, d) also bounded.
Proof. By way of contraposition suppose that (X, d) is not bounded. Then, supx,x'ex d(x,x') Pick x1 X arbitrarily and pick x, for E Zt n > I such that x(x,.x1) > Clearly, there does not exist a convergent subsequence the sequence X1, Ny,...... Thus, (2, d) cannot be compact.
D Proof of Equivalence Between Lipschitz Continuity and Knowledge Continuity
Proposition. (See Prop. 4.6) Suppose that (X, (x,xx).O... 4y) are metric spaces. Let the first n metric decompositions of f X y be K,-lipschitz continuous, for [n]. If f is e-knowledge continuous with respect to the th hidden layer and (x))(x) f(x)) S n and some > 0, then f is Lipschitz continuous in expectation. That is,
(xx)) (x', y) all x, x' yY,yE
Euc.n dx(s(x),{(x)))
dx(x,x)) Sen I1
(D.1)
Proof. We proceed bound the knowledge continuity of f from below.
Acl (x',))
Ex E(xy)DDy EL'YDDD y=y
(D.2)
sc
2 FayyDDy FWYDDD /'yy
(D.3)
II - K,dz(x',x)
fd((((x)
2 Ex EWAAD /=y
(D.4)
IT K,dx(x,x))
(d((((x,,((()))
= =Ean)
(D.5)
!ー Kxdx(x,x))
23

---

Eq. D.2 comes from the fact that we take the expectation only over pairs of points (x. y), (x', y²) where y y and also because the summand always nonnegative. Then, we inductively apply the definition of K,-Lipschitz continuity to yield Eq. D.3. Eq. 1).4 follows directly from the assumption in the statement of the proposition. Since the expression in Eq. D.4 now has no dependence on the label distribution, we may expand the expectation which results in Eq. D.5. Lastly, by the definition of e-knowledge continuity,
xx)(((x)
e EZ (umill/ IT K,dx(x,x))
dy(f(x)((x))) dx(x,x))
en II K2 Ex. /=1
and this concludes the proof of the proposition.
To prove Cor. 4.7, we need the following auxiliary result from [89].
Proposition D.1 (See [89]). For a neural network f R" R with Lipschitz constant L under nonor, define the resulting classifier as g(x) := arg ma fx(x) for an input x. Then, is
8/2 provably robust under perturbations ||:1 marginl (x)), i.e.
4/2
g(x+ + 6) g(x)
for all ||öl| margin(f(x))
(D.6)
2L
Here, margin(f is the difference between the largest and second largeset output logit. Corollary (See Cor. 4.7). Suppose that assumptions of Prop. 4.6 are true. And also assume that (X, (Rt", e p). (V.d, dy) (Rt",, Lp). for 1 S S co. Define a classifier from RR" R. where g(x) == arg max fx(x for any x E R.. Then, with probability 1 s!! I K₁.
g(x) = g(x + 8) for all [| 2/ f and 1 > fx(x) is the thth coordinate of fxx and margin(f(x) denotes the difference between the largest and second rccond-largest output logits.
Proof. By Prop. 4.6, we have that
dy dx(f(x){{(x'))
Sen II Kj. f=1
(D.7)
dx(x,x))
By Markov's inequality,
dx(f(x),
P POC.MM////~-D
en
I K,
(D.8)
xx(x,x))
We yield the corollary by directly applying Prop. D.1 assuming that f t-Lipschitz continuous.
Next, we establish conditions under which Lipschitz continuity implies knowledge continuity.
Proposition (Prop. 4.8). Let (X, dx), (V, dy) be a metric spaces. Let f x - e- e-Lipschitz continuous and (xf(x), y) be n-Liscchitz continuous with respect both coordinates. the first metric decompositions of f are K,-Lipschitz continuous, then f is knowledge continuous with respect to the th hidden layer. That is,
E(x,)>Dxy (x S I
1 K
(D.9)
Proof. Let us start with the definition of e-Lipschitz continuity and lower-bound it. For any (x,y),x', () E X.y.
dy(f(x),f(x))
Se,
(D.10)
dx(x,x))
24

---

dx(((x)))())) Sc.
(D.11)
I K, (x)
(1((),)) ((x,5)) 6.
(D.12)
IT 2 K, (,((f(().
y) S on I1 dg(f(())
(D.13)
j=1 K
Eq. D.11 follows from inductively applying the definition of Lipschitz continuity on the metric decom- positions of f. Specifically, (f+1(f*+(x)) (x), fx)) < (xd,(f'(x), f'x)). Then, by the Lipschitz continuity of L in both coordinates we yield Eq. D.12. Since the Lebesgue integral preserves order, Eq. D.13 directly implies the statement of the proposition and this concludes the proof.
E Predicting Adversarial Robustness with Volatility
In this section, we detail the experimental methods and results that use knowledge continuity to predict adversarial vulnerability, briefly discussed in Section We focus on langauge models of various sizes and their ability to perform sentiment classification on the IMDB dataset [48]. Before computing any statistics of the model, we finetune against the IMDB dataset and reserve a test set on which we compute a vulnerability score and estimate the model's adversarial vulnerability.
Vulnerability Score. As described in the main text, given model with n hidden layers, we compute all of its k-volatility scores. This is done with naive Monto-Carlo algorithm which we present in Appendix G. This results list of k-volatility scores {cr.... C₂ 1, one for each hidden layer. Then, we perform simple average n Σx=1 Eg- Let denote this quantity as the vulnarability score.
Estimating Adversarial Robustness. remains to estimate the adversarial vulnerability of given model. We do this empirically by applying out-of-the-box adversarial attack (specifically, TextFooler [35]) on the given model with respect to the reserved test set. We then measure the number of successful adversarial attacks defined as
#Successful Adversarial Attacks Laadvessaailll
peorrect
Pccorrect]
where Peoorett is the set of examples in the test set that correctly classified by the model (after finetuning) without any intervention. And, aadversarial are the set of examples that are incorrectly classified after adversarial attack is applied. In other words, we only consider points where perturbation will worsen performance. In expectation, this estimate of adversarial robustness should be 1/2 factor of the notion of vulnerability we present in Thm. 4.1, where also consider point to be vulnerable perturbation increases its performance.
We then perform linear regression using vulnerability score and host of other model properties to predict the number of successful adversarial attacks. Concretely, we seek learn the relationship:
R
Successfuul Adversarial Attacks m (n--)
Σ
&
+
additional architectural vuriables
where m E Rd and b E R are the learnable regression parameters. We also incorporate d - size and architectural variables into our regression as we found that significantly increases predictiveness. And so, the input variables to our regression and their types are:
Feature
Type
Encoder Only
[0, 1]
Decoder Only
[0.
Encoder-Decodee
[0.
log(tParameterrs
R
Σ" k== EA
R
25

---

(L)
(2)
(3)
Variables
Coefficients
AR²
Coefficients
A
Coefficients
R
Encoder Only Decoder Only
X
1485
0.40
-548
0.07
X
X
-2816
0.71
-557
0.02
Encoder-Decoder
X
1332
0.29
1105
0.18
log(FParameters
X
66
-6.1 x 10-5
-363
0.04
Zk=1 Ck
49
0.35
96
2.57
x
X
R
0.35
0.48
0.28
Table 2: Regression results from our three previously described experimental settings. We regress the number of successful adversarial attacks against (1) only the vulnerability score (2) vulnerability score and model characteristics (3) only model characteristics. The coefficients for each of these regressions results shown in the column Coefficients. We also run permutation tests for each coefficient and the change in R is shown in the column A R² (higher the better).
For the regression itself, we perform a Ridge regression with a = 1. We test three experimental conditions where regress the model's adversarial robustness against: (1) only vulnerability score, (2) vulnerability score and model characteristics, (3) only model characteristics. We experiment with seven models: RoBERTa (Base/Large) [44], BERT-Uncased (Base/Large) [16), GPT2, and T5 (Small/Base) [58]. Our regression results are shown in Table. 2.
permutation tests to determinc the contribution of each feature to the explained variance. Specifically, for each feature, keeping all else constant, permute its values. If this feature is significant contributor to the explained variance, intuitively, we should see a large decrease in R² after this intervention. If s is the R² without any intervention and s, is the Rº after permuting the data by a,(-) [m] [m] (for dataset of data points). Then, define
After yielding initial bfestt fit (see Fig. 2(a)),
N Σ
A =
where N controls the number of permutations that we apply. For all experiments we choose N = 100. For formal theory on permutation tests, see [8].
We find that when our vulnerability score is added to the regression, contributes significantly to the explained variance. Moreover, in (2), importance among all regression variables.
that vulnerability score has the highest feature
F Localizing Volatile Hidden Representations
In this section, we localize adversarially vulnerable hidden representations in two ways. Firstly, we k-volatility gauge which layers vulnerable across selection of models. Then, we focus model-specific characterizations of robustness with respect to k-volatility. We present experiments on the same selection of models in Appendix the same dataset (IMDB [48]), and the same adversarial attack (TextFooler [35]) to empirically measure adversarial vulnerability.
F.1 Layerwise Volatility
As mentioned in the previous section (Section E), for a given model with n hidden layers, we can measure its k-volatility for k E [n] through Monte-Carlo algorithm. For each model, we then plot its k-volatility against its relative depth which defined as [k/n]- These curves are shown in Fig. 2(b). We see that models which have different architectures independent of size have very different k-voliiiyity curves.
We have already shown in the previous section that there is positive correlation between k-volatility and adversarial vulnerability. However, this correlation is derived from the simple average of volatility scores. Are the koloatilyyty scores in some layers more predictive of adversarial vulnerability
26

---

than others? If the k-volatility in some layers is more correlated with k-volatility in others, then it should suffice to minimize k-volatility these former layers. This would also speed up regularization and training.
We repeat the experiments in the previous settings. But, instead of collating k-volacility through simple average, we run one regression for each relative depth across all models (which we discretize into 9 bins). This result is shown in Fig. 2(c). Surprisingly, we find that the magnitude of k-volatility is not necessarily predictive of adversarial vulnerability. For example, in Fig. 2(b), almost all of the models exhibit low average k-volatility in the latter layers. However, the k- vollaiilyty of latter layers predict adversarial vulnerability the best.
F.2 Model-Specific Volatility
We start by exploring the k-volatility across each of our test models. We notice that k-volatility cannot be predicted by surface-level features such as size model type alone. This is shown clearly Fig. 3. Yet, as discussed Appendix E, it is still able to predict actual adversarial vulnerability with moderate power. Thus, conjecture that k- v-voltiility captures complex aspect of the model's vulnerability which cannot be solely attributed to its size or type.
1.0
Cavoner Brcader
0.8
0.7
Scaledit
0.6
0.5
0.4
0.3
18.0
18.5
19.0
18.0
19.0
log(/Parameters)
log(:Parmmtoos)
Figure 3: Average k-volatility plotted against the log of number of model parameters (left). We see that although there a strong negative correlation, the exactly relationship is nontrivial. Moreover, this negative correlation is also consistently observed across model families (right).
G Regularizing Knowledge Continuity
In this section, we provide a comprehensive overview of regulating knowledge continuity achieve robustness. We first show a simple algorithm that estimates k-volatility. Then, we demonstrate how this can incorporated into any loss function as a regularization term. We then prove guarantees that revolve around the unbiasedness of our estimation algorithm. Lastly, we present detailed discussion of the results shown in Table including training details and ablation studies over the hyperparameters.
G.1 Estimating Knowledge Continuity Algorithmically
We first present a method for estimating k-volatility. This is shown in Alg. ((ESTK In theory, one should choose M = N, as this will lead to a most accurate estimate. This is similar to contrastive learning methods where it is desirable to make the minibatch sizes as large possible [56]. However, N > 1, this can become quickly intractable. In practice, during regularization we keep N to be the same as if we were doing normal finetuning (i.e. 32/64) and set M N. This works well, and, anecdotally, we find that in contrast to contrastive learning increasing N or M past this threshold yields marginal returns. Further work could examine this relationship in more detail.
As discussed in the main text, the choice of metric (or representation space) which we enforce knowledge continuity against is crucial as it determines the type of robustness we will achieve. Therefore, Alg. ((CCRG)) we incorporate this detail by sampling a hidden layer of interest using Beta distribution specified by hyperparameters a, B. Then, that minibatch, regularize k- k-volatility
27

---

Algorithm A Monte-Carlo algorithm for estimating k-volatility of some metric decomposable function f with n hidden layers (left). Augmenting any loss function regularize k-volatility (right), given some Beta distribution parameterized by a, P and regularization strength À 2 O.
procedure ESTK (X,, - M.
Sample (MI.....MM ) C [N] unitormly
oj -0 Losses - (L(f(x,,),,) ).,,;)
procedure KCREG(a, P. M.M.
Beta(a, ())
k - max(1 Xnj, 1) of - ESTK return ΣE (((()) + M
for (i.j) E [M] [M]
M,K)
Dist + (x(f"(xx,) (x (,,,))
(x,,))
Ao
of - of + [Losses, - Losses,|/Diss
return of
with respect to that sampled layer. Note that we choose the Beta distribution for simplicity, however, it can be replaced by any distribution like mixture of Gaussians.
In contrast to existing adversarial training methods such as [32] and [63] which only use the embed- dings, our algorithm gives the practitioner more control over which hidden layer (or distance metric) enforce smoothness. In this way, if the practitioner has some knowledge a priori of the attacker's strategy, they may choose to optimize against the most suitable metric decomposition. We present a brief discussion of the various tradeoffss when choosing a, B in the following section as well detailed empirical analysis in the following subsections.
is the weight we put on the regularizer in relation the loss function L. We provide detailed ablation study of the effects of 2 in the following subsections. We surprisingly find that even for 2 < can achieve significant edge in terms of robustness over existing methods. This is in contrast virtual ad advessaiial training methods such as [43] which requires applying a d-value magnitudes larger. Moreover, for larger 2, we find that the accuracy of the model not compromised. This provides some empirical support for Theorem 4.3.
G.2 Theoretical Guarantees of k-Volatility Estimation
In this subsection, we show that our Monte-Carlo algorithm presented in Alg. 1(ESTK VOL) is an unbiased estimator. The proof is simple and follows from some bookkeeping.
Proposition G.1 (Alg. I(ETTK VOL) is an Unbiased Estimator). Assuming that each data point in the batch, ((x,, 3;1 ~ Dx.y. is sampled i.i.d, then Alg. I(EsTKVOL) is an unbiased estimator for (x.)).
Proof. Let 0 be the random variable representing the output of Alg. 1. It suffices to show that [[
(x, y),
where the expectation on the left-hand side is taken over the set of all batches. By the definition of Alg. I(EsTKVoL),
(x Tny (X Yo.)
AC
El|]
E
Σ
(G.1)
2
(x (x()))
AC Jny (xx Fnn de(Fk(x,)) (x)))))
M
Σ
(G.2)
2
= E =Eo (x, y).
(G.3)
The second equality follows from the linearity of expectation.
We emphasize that our estimator very naive. Improving its efficiency could form the basis of possible future work. For example, Rao-Blackwer Boo-Blackwellizing [6] Alg. ((EsTKVoL) to yield an estimator with
28

---

(2014)
Ours
Base
0.-
0.8
A
0.3
0.7
0.6
0.6-
reanet50
0.5
mobilenet
vI810
Macro
0.0
0.2
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.6
0.0
0.2
0.6
1.0
(2020)
0.0
resnet50
VIE16
3.9
0.8
0.8
0.6
-03
0.7
0.4
Attack Strength
CNN-Based
CNN-Based
Transformer-Baee
Figure 4: Regularization k-volatility for a host of vision models. We apply two adversarial attacks FGSM [24] (top row) and SI-NI-FGSM [41] (bottom row) with various attack strengths. Attack strength measured in terms of maximum -n-ormm of the applied perturbation to the image.
smaller variance, applying rejection sampling to deal with the potential sparsity of the representation space discussed in Section 4.4, or adapting the regularization weight based on some bootstrapped confidence interval (if the estimate has higher variance then decrease weight regularization and vice versa). However, we see that even with this naive algorithm we achieve improvements in robustness well training speed.
G.3 Computer Vision Results
In addition to regulating language models, we also demonstrate that KCREG is effective for vision tasks. This provides empirical support for the equivalences we proved in Section 4.5. The exact same method of k-volatility estimation and loss augmentation is applied. We finetune three models ResNet50 [28], MobileNetV2 [61], and VIT16 [17] on the MNIST dataset both with and without regularization algorithm. We then apply two different adversarial attacks: FGSM [24] and SI-NI- FGSM [41]. We find that in both cases, regularization k- k-volatility improves/stabilizes robustness across attack strengths (see Fig. 4).
G.4 Ablation Studies
Herein, we present ablation studies for the crucial hyperparameters in our regularization algorithm (across the natural language tasks that we explored in the main text), Alg. 1(KCREG): which is the weight we assign the knowledge continuity regulation loss and (a, f)) which determines the sampling behavior of the index of the hidden representation space.
Ablation Study of 2 (Fig. 5(right)). The weight given to the regularizer (2) is ablated over, with the results shown in Fig. 5. For any positive 2, there is an immediate large improvement in adversarial is systematically increased by factors of 10, we do
robustness. Next, in the accuracy (not under attack). This corroborates Theorem. 4.3, as it demonstrates that regulating knowledge discontinuities (no matter how strongly) of our model. On the other hand, we also do not see a significant increase in adversarial robustness as à increases. This may imply that we have reached the threshold of adversarial robustness under TextFooler [35]. Specifically, the adversarial attacks generated by TextFooler may not be valid in that they have flipped the ground-truth label. Therefore, we believe that good for this particular application should lie somewhere between and x 14
a significant change
odds with minimizing the empirical risk
29

---

-
10
-K-
le-1
-4-
1e-2
--.
1e.3
1e.4
0.25
Attack Success Rate
0.20
85
Test Accuracy
0.15
0.10
0.05
70
0.00
IMD3+ ToxFFolee
0.0
0.2
0.8
10
10
10"
10"
Attack Strength
Regularization Strength
Figure 5: Ablation over the strength of regularization and its effect the attack strength-attack success rate curves (left). Ablation over the regularization strength (for fixed attack strength 0.3) and its effect on test accuracy (right). We see that moderate regularization significantly improves robustness across all attack strengths. This improvement does not come at the expense of test accuracy. The attack-strength is measured using the minimum angular similarity between the perturbed and original text. Both ablations are done with respect to GPT2 on the IMDB [48] dataset with respect to the TextFooler attack [35].
Ablation Study of Adversarial Attack Strength (Fig. 5(left)). For every value of 2, we also vary strength the adversarial attack. The adversarial attack strength is measured through the angular similarity of the embeddings between the original text and the perturbed text. Intuitively, if this constraint is loosened the adversary is allowed to find text that is semantically very different and vice versa. We see that moderate k-volatility regulation achieves the best adversarial robustness across all attack strengths.
Ablation Study of (a, () In this subsection, we briefly discuss how the a, hyperparameters which determine the shape the Beta distribution in Alg. I(KCREG) affect the final performance and robustness of our model on the IMDB dataset. Recall that the shape of the Beta distribution determines index of the hidden layers we are using the compute the knowledge continuity. Thus, they are crucial in determining the behavior of our regularizer.
We finetune (BERT, T5, GPT2) models on the IMDB dataset with the hyperparameters described in the next subsection. The results are displayed in Table 3. Across all models we observe a decrease in robustness for 1. 2. These values correspond to right-skewed distribution which places high sampling probability on the earlier (closer to the input) hidden layers. Intuitively, perturbations in the early layers should correspond textual perturbations with respect to some metric like the Levenshtein distance should be only loosely not completely (un)correlated with the actual labels of these inputs. Therefore, enforcing knowledge continuity with respect to this metric should not see increase robustness. Moreover, we also observe larger decrease in accuracy (not under attack) with the same parameters. This suggests that maintaining this sort of knowledge continuity in the earlier layers is harder to converge on and there may be "push-and -nd--lll behavior between optimizing knowledge continuity and accuracy (not under attack), Surprisingly, observe significant difference between the other a. values shown in the table.
proportional textual perturbations in the input text. Pure
We did formally benchmark other configurations of a, f such increasing their magnitude to impose sharper distribution. Anecdotally, during training, we noticed that using these sharper distributions both significantly slowed the model's convergence and decreased the model's accuracy (not under attack). It could be that though knowledge continuity itself is local property and the enforcement of this local property requires change on a global scale. In other words, one cannot simply reduce the knowledge discontinuities or uniformly converge with respect to one layer without participation from other layers. The extent which other layers are involved the regularization of specific one is an interesting question that we leave for future research.
30

---

Model
IMDB
IMDB π
BERT BASE
93.6
47.9
BERT BASE BASE +Rggan
94.8
75.1
BERT
BERT
BASE Regi22
89.2
74.1
BERT BASE Reg(12
87.0
68.2
GPT2
93.6
63.9
GPT2+Regg.11
94.6
85.0
GPT2+Regg.n
87.8
G1T2+Rog(12)
93.1
84.9
TA TS + Regann
93.7
53,9
95.0
88.9
TSRSS Regg22)
94.9
89.3
BASE Rega.2
94.6
88.1
Table 3: We train finetune (BERT, T5, GPT2) using knowledge continuity regularization, described in Alg. I(KCRG)). We varied the a. B hyperparameters for the Beta distribution as to determine the effect of these parameters on model performance and robustness. The rows of the table are labeled with the format: Model+Rega..py The bolded entries of the table correspond to the best performing metries out of the knowledge continuity regulated models.
Hyperparameter
Value
Optimizer
Adam
Adam &
0.9
Adam &z
0.999
Adam e
x 10-8
Max Gradient Norm
1.0
Learning Rate Scheduler
Linear
Epochs
20
Batch Size
32
Learning Rate
x 10-5
Weight Decay
x 10-9
Table 4: Training hyperparameters and optimizer configurations for finetuning models (BERT, GPT2, T5} on IMDB without any form of regularization adversarial training.
G.S Training Details
In this section, we describe in detail the training objectives, procedures, algorithms, and hyperparme- ters that used in the main and further experiments done the appendix.
Brute-Force Adversarial Training. For all models undergoing adversarial training, we first finetune the model against the training set. Then, attack it using the TextFooler [35] algorithm with examples from the training set. After the attacks are concluded, we then incorporate the text of successful adversarial attacks back into the training set and proceed to finetune again. This procedure iterati vely continues. For the sake of computational efficiency, for all models we applied this procedure once. The parameters we using during the adversarial attack is the same hyperparameters actually use at test-time. Specifically, we impose query budget of 300 queries.
Plain Finetuning IMDB. The IMDB dataset consist of 50,000 examples with 25,000 for training and 25,000 for testing. We split the test set 40%-60% to create validation and test set of 10,000 and 15,000 examples, respectively. Examples were sampled uniformly random during the splitting process. Since adversarial attacks were costly, we uniformly subsampled 5,000 examples from this 15,000 benchmark robustness in the experiments related the regularizer. However, for the experiments estimating the knowledge vulnerability score, we performed adversarial attacks on all 15,000 datapoints in the this 5,000 subsample versus and the entire 15,000 dataset.
We found significant difference between robustness estimation
We train all models using the hyperparameter and optimizer configurations shown in Table 4.
31

---

Knowledge Discontinuity Regulation on IMDB. To enforce the knowledge discontinuity on IMDB, for all models. As shown Table varied a. E [1,2) x [1,2) and displayed the best models in terms of robustness in Table. in the main text. We train all models for 50 epochs. Other than that all the other hyperparameters and optimizer configurations are the same as regular finetuning (see Table 4).
we use constant A I 10-2 -
Knowledge Discontinuity Regulation on ANLI Optimizing over the ANLI dataset was significantly harder than on IMDB. As result, for each model class { (BERT, GPT2, TSJ we performed a quick hyperparameter search over 2 (1 X 10=4), the learning rate (5 x 10-5), and weight decay (1 10-9) fixing the parameterization of the Beta distribution to be the best values on the IMDB dataset. That is, for T5: a 2, p 1: BERT-R---a BaRe-Base-aa Uncased:
$ = GPT2: a = 2, 2.
ALUM on IMDB and ANLI We train all ALUM models for 50 epochs (the same as knowledge discontinuity regularized models). For hyperpararmeters specific to the ALUM algorithm we choose all of the same ones as its authors, [43], with the exception of a (analogous to the our algorithm, essentially the weight put on the virtual adversarial training loss term). The authors of the original paper choose = 10. We, however, found that this applied to finetuning does not converge at all. Thus, with a rough grid search in the parameter space we found 1 x 10-3 to be the best with respect to both performance and robustness.
We keep the same hyperparameters ANLI, however, impose early stopping during the training process. That is, we choose the best model with respect to its performance on the dev set.
H Certifying Robustness at Test-Time
Herein, we present certification algorithm using Thm. 4.1 and our Monte-Carlo estimate of volatility I(ESTKVOL). Our algorithm (shown in Alg. 2) is based on the work of [12]. We upperbound the k-volatility by bootstrapping a 1 a confidence interval. Then, directly apply Thm. 4.1 using the 0-1 loss function. Thus, Cor. H.1 follows. We emphasize here that this certification algorithm may not be directly informative, especially in the discrete/nonn-metrzabb setting, unless we have an inverse map from the representation space back to the input space. This is discussed further in [82]. Nonetheless, it can be used as a method to verify whether or not certain intervention techniques are successful before deploying them in the wild.
Corollary H.1. Let A = {xx,, Di) and A' = 1(x'8)) E xX Exx,9-Dy A (x,')) > n). (x,))A
Then, with probability a, the output of Alg. 2 bounds (f(),, where E is the 0-1 loss.
Algorithm 2 Certifying robustness of a metric decomposable function f with respect to one hidden representation using Alg. I(ESTK VOL) and Thm. 4.1.
procedure CERTIFY(S: ((x,, y;) k, 8,
Let be the 0-1 loss function Ey UPPERCONTBOUNDD B max (x))) f((x))
[(xx [1)) k.jja
2
V 1 exp
-2/B²
8 BV log 2n
return CLIP(I-CO8/V 8,V/
procedure UPPERCONFBOUND(F..C ((x,, y;) (...
U - 0g
for 1 ... k do
5 - sample w/ replacement points from ())
U, - VOT f.J.)
return Σ E/=1 U, + -- + (a)std())
Along these lines, we apply our certification algorithm to our regularized models to verify that the certified robustness has indeed improved. These results are shown in Fig. 6.
32

---

1.0
1.0
Base
KCReg (Ours)
Change Absolute Accuracy
Certified Robustness
0.8
0.8
0.6
0.4
0.2
0.2
0.0
0.0
0.0
11.2
22.4
33.6 44.8 56.0 67.2 0.0 11.2 22.4 33.6 44.8 56.0 67.2
Perturbation Distance
Figure 6: Certification of robustness for GPT2, layer=6. We apply Alg. 2 certify robustness of the model before and after regularization with Alg. ((DREG)) Each line corresponds to the change in absolute accuracy for set of examples to be considered non-robust. The y-axis corresponds to the certified probability measure of the set of non-robust examples under this criterion and the x-axis corresponds to the maximum perturbation distance in the representation space.
I Broader Impacts
This contribution concerned with robust deep learning models. As deep learning becomes ubiquitous as the primary method for creating artificial intelligence, their applications in increasingly critical areas the lay and corporations alike demand not only both high inferential accuracy and confidence but also safety and trustworthiness guarantees. Robustness addresses this latter point. More specifically, our contribution unifies separate robustness efforts from continuous and discrete domains.
Reproducibility
All of our experiments were conducted four NVIDIA RTX A6000 GPUs well as four NVIDIA Quadro RTX 6000 GPUs. The rest of our codebase including implementations the algorithms and figures described in the manuscript can be found at https: /tt///gtttut com/ aal.auaannnlnniaa 1700//m
K Limitations
The certification guarantees our definition knowledge continuity is a probabilistic one. Specifically, this randomness is over the data distribution. However, this does not protect against out-of-distribune attacks that plague large language models such as [72, 91]. More work is needed to yield deterministic results that do not become vacuous in discrete settings. As mentioned in Section 4.4, our expressiveness bounds only apply under little restrictions to the metric decompositions of the estimator f. Though we see some empirical verification for this in Appendix G, it remains unclear whether not we can tighten these bounds.
33

---

L NeurIPS Paper Checklist
1. Claims
Question: the main claims made the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contrihutions made in the paper and important assumptions and limitations. A No or NA answer this question will be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can expected to generalize other settings.
It is fine include aspirational goals motivation long it is clear that these goals attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We present a detailed discussion of the limitations Section K.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well speesspcciiiaaiinn asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
For each theoretical result, does paper provide the full set assumptions and a complete (and correct) proof?
Question:
Answer: [Yes]
Justification: For all of the theoretical results in the paper, we include all of its assumptions. We include full proofs of each theoretical result in Appendices A, B, C, G. To the best of our knowledge. the proofs are correct.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross cross-referenced.
34

---

All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided appendix or supplemental material.
Theorems and Lemmas the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We present of the hyperparameters in the experiments that require training in Ap- pendix G. Additionally, our compute resources are detailed in Appendix J.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible important, regardless whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation. may be necessary either make it possible for others replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
While NeurIPS does require releasing code, the conference does require all submissions provide some reasonable avenue for reproducibility, which may depend on the nature of contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe architecture clearly and fully.
(c) If the contribution is a way to access this model for reproducing results a way to reproduce model (e.g.. with an open-source dataset or instructions for how to construct
model (e.g.. large language model), then there should either be
dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome to describe the particular way they provide reproducibility. In the case of closed- source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data code, with sufficient instructions to faithfully reproduce the main experimental results, as described supplemental material?
Answer: [Yes]
Jusúifcatiin: We attach all of the code used to generate the figures and the experimental results in the supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines httpp:://niis.cc/puutteee SttSSuamissioooooooooooo for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
35

---

The instructions should contain the exact command and env environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines
Soeem/CoooSummsssiooo Pyyicyy ) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce experimental results for new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g.. data splits, hyperparameters. how they were chosen, type of optimizer, etc.) necessary understand results?
Answer: [Yes]
Justification: For the experiments that require training, we discuss detail the hyperparametens in Appendix G. Moreover, we also attach the code used to generate all results and figures in the supplementary materials of the submission.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should he presented in the core of the paper to level of detail that is necessary appreciate the results and make of them.
The full details be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined other appropriate informa- tion about the statistical significance of the experiments?
Answer: [No]
Justification: The experiments in our paper serve as type of sanity check and demonstrate possible explorations rather than a benchmark against existing state-of-the- methods.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanicd by error bars, confidence intervals, statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error hars should be explained (closed form formula, call to library function, hootstrap, etc.)
The assumptions made should be given (c.g., Normally distributed errors).
should clear whether the error is standard deviation or the standard error of mean.
is to report I-sigma error bars, one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI. if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures
symmetric eтror bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer:
[Yes]
Justification:
We provide details on compute resources we use in Appendix J.
Guidelines:
36

---

The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, every respect, with the NeurIPS Code of Ethics attps://neuripp:
http://uuripp.ccppulliieeee cc/public/Ethi. cechhhhcciiieeeeeeee
Answer: [Yes]
Justification: The authors have reviewed NeurIPS Code Ethics and to the best our knowledge it does conform to this in every respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consideration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the broader impacts of the work in Appendix I.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (c.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploy- ment technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it not needed point out that generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition to attacks, mechanisms for monitor- ing misuse, mechanisms to monitor how system leams from feedback over time, improving the efficiency accessibility MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place responsible release of data or models that have a high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper is concerned with training more robust deep learning models. Thus, it does not pose such risks.
Guidelines:
The answer NA means that the paper poses such risks.
37

---

Released models that have high risk for misuse or dual-use should be released with necessary safeguards allow for controlled use of the model, for example by requiring that adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have heen scraped from the Internet could pose safety risks. The authors should describe how they avoided relcasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
12. Licenses for existing assets
Question: Are the ereators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: The paper does not existing assets.
Guidelines:
The answer NA means that the paper does not use existing asscts.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g.. CC-BY 4.0) should be included for each asset. For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. www.tttcooodc..t..aaaaeaeo.aaaa com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license a dataset.
existing datasets that are re-packaged, both the original license and license of the derived asset (if it has changed) should be provided.
this information not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We not introduce any new assets in paper.
Guidelines:
The answer means that the paper does not release new assets.
Researchers should communicate details the dataset/code/model as part of their submissions via structured templates. This includes details about training. license, limitations, etc.
The paper should discuss whether how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsoureing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: We do not perform any crowdsourcing experiments nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least minimum wage the country data collector.
15. Institutional Review Board (IRB) Approvals Equivalent for Research with Human Subjects
38

---

Question:
Does the paper describe potential risks incurred by study participants, whether such risks disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (oran equivalent approval/eeviww based on the requirements your country or institution) were obtained?
Answer: [NA]
We not perform any experiments that involve crowdsourcing nor research with human subjects.
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for human subjects research. If you obtained IRB approval, should clearly state this in the paper.
We rocognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do include any information that would break anonymity (if applicable), such as the institution conducting the review.
39


Reviews:
[{"review_id": "RfJqYShXpa", "reviewer": "Reviewer_88sL", "summary": "This paper proposes \"knowledge continuity\", a metric inspired by the Lipschitz continuity that aims to certify the robustness of neural networks. Compared to existing methods, the proposed metric yields certification guarantees that only depend on loss functions and the intermediate learned metric spaces of neural networks, which are independent of domain modality, norms, and data distribution. The authors further show that the knowledge continuity is not conflicted with the expressiveness of a model class. Several practical applications are provided.", "questions": "I don't have questions.", "limitations": "Yes, the authors have discussed the limitation in the main text.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The metric seems to be novel, has an elegant form and seems to be able to provide good explanation of robustness. \n- The proposed metric is supported by a good amount of theoretical proofs, stating that improving the knowledge continuity does not be conflicted with the representation power of the neural networks. \n- practical results indicate that regularizing the metric can help the robustness.", "weaknesses": "- A lot of details of the empirical results are deferred to the appendix which makes me hard to understand what are the experimental settings. The authors should consider reorganize the paper - right now it seems that the paper is actually compressed from a longer version (say journal) but not dedicated to the conference. For example, the regression test seems to be very interesting but there is no any actual result shown in the main text. \n- Some numbers are wrongly bolded in the table (or the meaning of bolded results is not clearly conveyed) since they are not the highest number.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer `88sL`,\n\nWe appreciate your thoughtful feedback. We are glad you noticed the elegance and novelty of our metric, and appreciated the way our theoretical results tie nicely into practical applications. We address your concerns as follows:\n\n> A lot of details of the empirical results are deferred to the appendix which makes me hard to understand what are the experimental settings.\n\nWe want to emphasize that the focus of our paper is to introduce the theoretical framework of knowledge continuity and its implications on certified robustness. As a result, the practical experiments are more so a proof-of-concept.\n\nIn the final manuscript, we will use the additional page to expand on our experimental methodology: including Alg. 1, Fig. 4, and the detailed regression results (see Fig. 2). \n\nTo further clarify the practical applications section, we first devise a simple Monte-Carlo method for estimating the knowledge continuity (Def. 2 and 4, Lines 178-179, Lines 197-198) of a network (see Alg. 1, Lines 794-795). Then, this quantity (for brevity, let's call this KVS) is used in three distinct experiments: \n\n1. We run a **linear regression using KVS to predict accuracy under adversarial attack.** We find that KVS does in fact predict accuracy under adversarial attack, explaining 35\\% of the variance which is statistically significant.\n2. We finetune a set of models (see Table 1) on the IMDB sentiment classification task and **append KVS directly to the cross-entropy loss function.** We find that **adding this term increases accuracy under adversarial attack without sacrificing inferential performance.** This corroborates our theoretical results in Theorem 4.1 (Lines 218-22) and Proposition 4.3-4 (Lines 247-263).\n3. Lastly, since KVS is a layer-specific metric, **we measure how KVS changes with network depth and perform a qualitative analysis.** In a nutshell, we find that **models which belong to different model families** (encoder, decoder, encoder-decoder) **have drastically different behavior** with respect to KVS. These plots are shown in Fig. 4 (Pg. 24).\n\n> Some numbers are wrongly bolded in the table (or the meaning of bolded results is not clearly conveyed) since they are not the highest number.\n\nThe bold numbers simply correspond to our proposed method within each model family. We understand that our bolding scheme was confusing, and this will be corrected in the final manuscript to only bold the highest numbers in each column. \n\nWe hope that these clarifications will boost your confidence in our paper. We would be happy to discuss any remaining concerns. \n\nThank you for your time.\n\nBest,\n\nAuthors", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ufyHr6lKLz", "reviewer": "Reviewer_nGPb", "summary": "This paper proposes a new concept inspired by Lipschitz continuity, aimed at certifying the robustness of neural networks across different input domains, such as continuous and discrete domains in vision and language. This certification guarantees based on the loss function and intermediate learned metric spaces, independent of domain modality, norms, and distribution. In this paper, the authors define knowledge continuity as the stability of a model's performance to its perceived knowledge of input-output relations. This approach focuses on the variability of a model's loss to its hidden representations rather than imposing arbitrary metrics on inputs and outputs.", "questions": "- See \"Weaknesses\"\n- The equation in line 252 is not numbered and contains a typo.", "limitations": "N/A", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The paper generalizes the Lipschitz continuity concept to continuous and discrete domains. \n- The paper provides definitions and proofs for knowledge continuity. \n- By providing a domain-independent approach to certified robustness.", "weaknesses": "- The authors claim that their method generalizes across modalities (continuous, discrete, and non-metrizable). However, they have not conducted experiments in other domains. The experiments in Table 1 are limited to the discrete text domain.\n- The paper does not provide specific implementation details. Section 5 directly presents the experimental results in Table 1, but the metrics in Table 1 are not clearly defined.\n- From the paper \"On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective,\" it is observed that model robustness improves with increased model capacity. It is necessary to demonstrate that the proposed method works on larger models like LLaMA3-70B/Qwen-1.5-70B to prove that the effectiveness of the method is not merely due to limited model capacity.\n- The proposed method does not consistently achieve better performance on ANLI. It remains competitive with previous methods rather than showing clear superiority.\n- The datasets used for comparison, IMDB and ANLI, focus more on word-level attacks, which are not particularly challenging for current large language models (LLMs). There is a lack of analysis on LLMs.\n- It is unclear how much benefit this method provides to bidirectional attention models like BERT and RoBERTa, which already have some error correction capabilities.\n- When dealing with out-of-distribution (OOD) data, the distributions themselves are different. It is uncertain whether this method will still be effective under such conditions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer `nGPb`,\n\nThank you for your time and review. We are glad you appreciated some of the key contributions of our paper, especially that our approach is a rigorous, domain-independent extension of Lipschitz continuity. We hear your concerns, and would like to address them here:\n\n> experiments in other domains\n\nTo support our claims (Proposition 4.6-8) and generalize Table 1, we run additional experiments on vision tasks across a set of models whose architectures vary from CNN-based to Transformer-based. (ResNet, MobileNet, ViT). We find that regulating knowledge continuity improves adversarial robustness on almost all of these models across attack methods. Please see our global response and Fig. 1 in the attached PDF for details. These results will also be included in the final manuscript.\n\n> The paper does not provide specific implementation details.\n\nWe understand this may have been unclear. So, here are some key areas where implementation details can be found: \n* For empirical model analysis, please see Appendices E-G\n* For estimating knowledge continuity and regularization (Def. 2 and 4, Lines 178-179), please see Alg. 1-2 (Lines 794-795). \n* For ablation studies and training details, please see Appendix G.3, G.4\n\nMoreover, we have now added additional ablations over the attack strength, supplementing Table 1. Please see Fig. 3 in the attached global response PDF. Our regression results and implementation are shown in Appendix E, Fig. 2. \n\nWe will use the additional page to expand on our experimental methodology in the main text. \n\n> metrics in Table 1\n\nMetrics in Table 1 are all accuracy since the datasets are class-balanced.\n\n> It is necessary to demonstrate that the proposed method works on larger models like LLaMA3-70B/Qwen-1.5-70B to prove that the effectiveness of the method is not merely due to limited model capacity.\n\nWe respectfully disagree that these experiments are necessary to validate the claims in our paper.  The main contribution of our paper is our theoretical framework which does not dependent on the capacity of the model. We show in Theorem 4.1, Proposition 4.3-8 that our robustness guarantees do not become vacuous as capacity goes to infinity. Along these lines, our experiments with BERT, GPT2, and T5 demonstrate that this framework is compatible with fundamental architectures used in language modeling that transcend scale. However, we agree that such experiments are important to the applicability of the framework and would form the basis of interesting future work.\n\n> The proposed method does not consistently achieve better performance on ANLI. It remains competitive with previous methods rather than showing clear superiority.\n\nANLI is a difficult task, even for LLMs [1,2,3]. Thus, we use this task to demonstrate that the performance of our regularization remains competitive and does not significantly degrade the model's performance as task difficulty scales. \n\n> IMDB and ANLI, focus more on word-level attacks, which are not particularly challenging for current large language models (LLMs). There is a lack of analysis on LLMs.\n\nBased on our extensive literature review, word-level adversarial attacks remain effective even as models scale. Please see results from [2] (last bullet point on Pg. 3), [3] (see Fig. 3.9 and Table 3.2), and [4] (see Table 4 and section 4.3). However, we agree that an important future goal is to explain and regulate complex phenomena such as jailbreaking, hallucination, bias, etc. \n\nTo this end, a key contribution of our method is that our bounds do not depend on the type of perturbation on the input, only its *task-dependent* semantic distance. Thus, we expect our framework to generalize to these more complex tasks and adversaries. We leave these topics for future work.  \n\n> It is unclear how much benefit this method provides to bidirectional attention models \n\nWe have included results for BERT in Table 1. Moreover, T5 also includes bidirectional attention modules for which our regularization method performs well.\n\n> OOD data\n\nWe acknowledge in our limitations section that our method does not protect against OOD attacks (see Lines 345-346). As this is not the focus of the paper, we leave the extension of our framework for future work. As in-distribution adversarial attacks are pervasive [5,6,7] in a host of domains, we believe that this limitation does not detract from the timeliness nor the contribution of this work. We would love to hear any of your suggestions on how to extend this work.\n\n> line 252 typo\n\nThanks for catching this error. This will be fixed in the final manuscript.\n\nThanks again for your review, and we hope that these clarifications addressed your concerns. We would be happy to discuss further.\n\nBest,\n\nAuthors\n\n[1] Chowdhery, Aakanksha, et al. \"Palm: Scaling language modeling with pathways.\" Journal of Machine Learning Research 24.240 (2023): 1-113.\n\n[2] Ye, Junjie, et al. \"A comprehensive capability analysis of gpt-3 and gpt-3.5 series models.\" arXiv preprint arXiv:2303.10420 (2023).\n\n[3] Brown, Tom, et al. \"Language models are few-shot learners.\" Advances in neural information processing systems 33 (2020): 1877-1901.\n\n[4] Wang, Jindong, et al. \"On the robustness of chatgpt: An adversarial and out-of-distribution perspective.\" arXiv preprint arXiv:2302.12095 (2023).\n\n[5] Chao, Patrick, et al. ‘Jailbreaking Black Box Large Language Models in Twenty Queries’. R0-FoMo:Robustness of Few-Shot and Zero-Shot Learning in Large Foundation Models, 2023.\n\n[6] Qi, Xiangyu, et al. ‘Fine-Tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!’ The Twelfth International Conference on Learning Representations, 2024.\n\n[7] Zou, Andy, et al. \"Universal and transferable adversarial attacks on aligned language models.\" arXiv preprint arXiv:2307.15043 (2023).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "brZU9nK3WS", "reviewer": "Reviewer_Cz6j", "summary": "This paper proposes a generalization of adversarial robustness and certified robustness for continuous, discrete, or non-metrizable input domains. The authors introduce _knowledge continuity_, a novel definition inspired by Lipschitz continuity which aims to certify the robustness of neural networks across input domains (such as continuous and discrete domains in vision and language, respectively). While existing approaches that aim to certify robustness (e.g., Lipschitz continuity) lie in the continuous domain with norm and distribution-dependent guarantees, the new proposed framework proposes certification guarantees that depend only on the loss function and the intermediate learned metric spaces of the neural network. \n\nThe current definition of robustness considers the stability of a model's performance with respect to its perceived knowledge of input-output relationships. However, the authors argue that robustness is better achieved by focusing on the variability of a model's loss with respect to its hidden representations, rather than imposing arbitrary metrics on its inputs and outputs.\n\nTo summarize the authors' contributions:\n- They introduce _knowledge continuity_, a new concept that frames robustness as variability of a model's loss with respect to its hidden representations.\n-  They show theoretically that knowledge leads to certified robustness guarantees that generalize across modalities (continuous, discrete, and non-metrizable), and show that this new robustness definition does not come at the expense of inferential performance.  \n- They present several practical applications of knowledge continuity such as using it  to train more robust models and to identify problematic hidden layers.", "questions": "See Weaknesses.", "limitations": "See Weaknesses.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The paper is clear and very well written.\n- The problem, i.e. defining a robustness framework for discrete or non-metrizable input domains, is very interesting and important as language models become more ubiquitous. \n- The new framework seems to generalize previous definitions of robustness, for example, robustness via Lipschitz continuity becomes a special case of knowledge continuity. \n- The authors show that their new definition of robustness \n- The authors propose experiments to demonstrate their approach on language tasks.", "weaknesses": "- While the method seems to be useful for robustness in the context of discrete input domains such as language, the new framework also seems to inherit drawbacks of current certified methods, namely Lipschitz continuity and randomized smoothing:\n  - Obtaining robustness from Lipschitz continuity requires having knowledge of the architecture of the network, and constraining the architecture to fit the definition; an example of this is that transformers are not _globally_ Lipschitz continuous, and thus certified robustness for transformer architectures via Lipschitz continuity is difficult. While knowledge continuity could in theoretically be used on any architecture where the hidden layers are metric spaces, I wonder if specific architectures might lead to vacius bounds. Similarly, quantized networks would not fall under the definition of having hidden layers that are metric spaces . \n  - As randomized smoothing, the robustness derived from knowledge continuity is not deterministic. This could lead to a high computation of robustness accuracy and certified accuracy.  \n- The implementation of knowledge continuity is not very clear, I understand that the authors have proposed a regularization scheme via Algorithm 1 for \"Estimating knowledge continuity\". Algorithm 1 estimates the the expected value that defines knowledge continuity, is this approximation accurate and how does the accuracy of the approximation affect the training speed?\n- While the authors have shown that there is theoretically no tradeoff between the robustness of knowledge continuity and natural accuracy, this does not mean that there is no tradeoff in practice. It would be interesting to see the tradeoff between between the strength of the attacks and the strength of the regularization, which seems to be missing from the paper.\n- The authors proposed a certified robustness approach, but no certification algorithm was provided. How to compute the certified accuracy with knowledge continuity. Given the non-deterministic approach of their framework, some statistical testing, as in randomized smoothing, may be necessary.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer `Cz6j`,\n\nThank you for your valuable feedback. We are glad you appreciated the timeliness of our work and its theoretical soundness. We hear your concerns, and hope to clarify the following:\n\n> While knowledge continuity could in theoretically be used on any architecture where the hidden layers are metric spaces, I wonder if specific architectures might lead to vacuous bounds.\n\nWe considered this, but determined it did not detract from our work:\n\nFirstly, we disambiguate *architecture* and *metric decomposition*. For a given architecture, there could exist many possible metric decompositions, as a decomposition is dependent not only on the topology induced by the metric but also the metric itself (see Def. 1). For example, consider a fully-connected network whose hidden representations are endowed with either the $\\ell_1$ or $\\ell_2$ metric; we would consider these to be distinct decompositions even though the architecture is the same. \n\nYou are correct in pointing out that there exists decompositions that yield trivial bounds. For example, consider the discrete metric ($d(x,y) = c$ if and only if $x=y$ and $0$ otherwise for some $c \\in [0,\\infty]$). As $c\\to\\infty$ knowledge continuity will approach 0 almost everywhere, assuming our loss is finite almost everywhere. \n\nIn general, we find these decompositions to be degenerate and unreasonable in practice since they also trivialize robustness. For example, if $c = \\infty$, robustness is inherently not well-defined since any perturbation would lead to an infinitely different point. Thus, it would make sense for the bounds to become vacuous anyways. \n\nWe conjecture that only metric spaces with infinite Hausdorff dimension behave this way (see Conjecture 4.5, Lines 275-278). Thus, any metric-decomposable neural architecture whose computation is tractable in practice, will lead a \"natural metric decomposition\" that has non-trivial bounds. \n\nHappy to discuss this more during the discussion phase.\n\n> quantized networks would not fall under the definition of having hidden layers that are metric spaces\n\nOne could equip the hidden layers of any quantized network the Hamming distance as its metric or use the $\\ell_p$ distances. The choice that makes sense depends on the coarseness of the quantization being performed.\n\n> robustness derived from knowledge continuity is not deterministic. This could lead to a high computation of robustness accuracy and certified accuracy.\n\nThough it would be nice to have deterministic bounds, one of the key insights of our work is that the stochasticity of our framework is necessary for generalizability across different domains.\n\nIn continuous domains such as computer vision, the map of the input is dense in any of the metric decompositions. For example, consider a model $f: \\mathbb{R}^n \\to \\mathbb{R}^m$. For any $x, a \\in \\mathbb{R}^n$ there exists some $x' \\in \\mathbb{R}^n$ such that $f(x') = f(x) + a$. The same cannot be said for most discrete or non-metrizable applications. Especially in natural language processing, where the correspondence between the learned representation space and the input is provably sparse. This means that any deterministic bound which depends only on the metric in the hidden space cannot capture this nuance.\n\nIn addition, we emphasize that for the notion of robustness to generalize across domains, it cannot be task invariant (see Lines 37-40). This necessitates dependence on both the input $x$ and its corresponding label $y$. Thus, we believe that any meaningful domain-independent certification must depend on the data distribution.\n\nWe appreciate the feedback and this discussion will be added in the final manuscript.\n\n> Accuracy of Algorithm 1 and effect on training speed?\n\nWe use a Monte Carlo approach to implement knowledge continuity: directly calculating the expression inside the expectation. We demonstrate that Alg. 1 is indeed accurate, as it is an unbiased estimator for knowledge continuity (see Proposition G.1). Further, we bound the number of samples needed to compute an arbitrarily accurate approximation of knowledge continuity in Proposition G.2 using a median-of-means approach. In practice, we find that this naive algorithm performs quite well and converges faster than methods of regularization such as ALUM or virtual adversarial training (see Lines 339-340).\n\n> tradeoff between between the strength of the attacks and the strength of the regularization\n\nWe run ablations over the strength of the regularization and observe the tradeoffs between test accuracy and accuracy under adversarial attack in Fig. 7 (on Pg. 29). We find that even a small amount of regularization dramatically boosts robustness. And as the magnitude of regularization increases, test accuracy is mostly uncompromised.\n\nWe run additional ablations over the adversarial-attack strength and find that our aforementioned results are consistent. For more detail, please see Fig. 3 in the attached pdf in the global response. These results will also be included in the final manuscript.\n\n> no certification algorithm was provided...Given the non-deterministic approach of their framework, some statistical testing, as in randomized smoothing, may be necessary.\n\nTaking inspiration from randomized smoothing as you suggested, we propose an algorithm to compute the certified robustness. This is shown in Alg. 3 in the global response. First, we upper-bound the knowledge continuity by bootstrapping a one-sided confidence interval similar to randomized smoothing, then directly apply Theorem 4.1. The proof of correctness for this algorithm follows as a corollary from Theorem 4.1 and an application of the central-limit theorem. Detailed results are shown in Fig. 2 in the attached global response.\n\nThanks again for the detailed review. We hope that we addressed all of your concerns, and would be happy to discuss more.\n\nBest,\n\nAuthors", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gh8l9xYm3E", "reviewer": "Reviewer_7Euo", "summary": "In this work, the authors propose a novel definition of \"continuity\": knowledge continuity, which measures the continuity of a model with respect to its hidden representations instead of the inputs. The knowledge continuity can be used to certify adversarial robustness across domain modality. Then, they provide theorems of robustness certification based on the new definition of continuity and demonstrate that in some cases the knowledge continuity implies Lipschitz continuity. Further, they prove that knowledge continuity doesn't affect the universal approximation property of neural networks, which means constraining a network to be knowledge-continuous may not hinder its expressiveness. Lastly, the authors provide some applications of knowledge continuity in improving robustness.", "questions": "In this work, the authors decompose the network $f^k$ as $h^k\\circ h^{k-1}\\circ \\dots \\circ h^1\\circ h^0$. However, some architectures such as ResNet and transformers, have residual structures. So, how can we decompose these networks as above?", "limitations": "As the authors said, this work provides some probability results that don't protect against OOD attacks. More deterministic results are needed in future studies.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The novel definition of continuity is more general and can be used in discrete domains such as natural language.\n* The authors provide solid and interesting theories to support the value of knowledge continuity.\n* Some applications are provided to further prove their new definition's effectiveness.", "weaknesses": "Some typos:\n* In Line 193, delete \"with respect to a function $f$\"? \n* In Line 218, $P_X[A]$ -> $P_{X\\times Y}[A]$.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer `7Euo`,\n\nThank you for your review. We appreciate your recognition of the theoretical depth of our work, as well as its applications. We would like to address your concerns as follows:\n\n> Some typos\n\nThank you for pointing these out! They will be fixed in the final manuscript. \n\n> In this work, the authors decompose the network $f^k$ as $h^k \\circ h^{k-1} \\circ \\ldots \\circ h^1 \\circ h^0$. However, some architectures such as ResNet and transformers have residual structures. So, how can we decompose these networks above?\n\nThere are many methods to decompose such structure. Let us consider two fully-connected layers $A, B$ such that $x \\to A(x) \\to B(A(x)) \\to B(A(x)) + x$. Here, the input $x$ forms a residual connection with the output of $B\\circ A$. For your reference, here we present two distinct methods. \n\nIn the simplest case, we can treat the residual block as a whole as a metric decomposition: $x \\to h(x)$ where $h(x) = B(A(x)) + x$. This is the approach we use in practice when dealing with the residual connections. Moreover, this is the standard way to count layers in computer vision and natrual language processing. \n\nOn the other hand, if we wish to decompose the residual block itself. Define \n\\begin{gather}\n    A' : x\\mapsto (A(x), x), \\\\\\\\\n    B' : x\\mapsto (B(x), x), \\\\\\\\\n    x' : (x,y) \\mapsto (x+y,y).\n\\end{gather}\nIt follows that $x' \\circ B' \\circ A'$ is equal to our residual block. Moreover, the image of each function in this decomposition: $x', B', A'$ forms a metric space. Here, the metric space is with respect to the quotient space where $(a,a') \\sim (b,b')$ if and only if $a=b$. In this way, we also recover the same subspace structure as the intermediate operations in the residual block. In other words, we duplicate the input vector, concatenate them together, and only operate on the first vector. \n\nThis example and others are included in Appendix A (Lines 570-596). *We want to emphasize that the non-uniqueness of the metric decompositions does not affect the validity of our theoretical results, as we only assume that such a decomposition exists.*\n\nThanks for your time, and we hope that these clarifications will boost your confidence in our work. We would be happy to discuss any remaining concerns. \n\nBest,\n\nAuthors", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear All Reviewers,

Thank you for your time and valuable insights into our work. In particular, we are glad that multiple reviewers appreciated the rigor of our theoretical work, the way our definitions generalize various robustness metrics across domains, and the importance of our practical experiments. We believe our work is especially pertinent as language models become more ubiquitous, as the reviewers pointed out. 

## Regularization of CV Models

Following the suggestion of Reviewer `nGPb` and Reviewer `Cz6j`, we have conducted additional empirical studies to strengthen our theoretical results. In the paper, we prove (Lines 285-313) that in continuous domains knowledge continuity is equivalent to Lipschitz continuity and recovers its tight robustness guarantees. To empirically test this, we regularize the knowledge continuity of several vision models and apply two adversarial attacks from [1] and [2]. We see that in most cases regularizing knowledge continuity improves adversarial robustness (see Fig. 1 in the attached pdf).  

## A New Empirical Certification Algorithm 

Next, we build off of our Monte-Carlo algorithm for estimating knowledge continuity (see Alg. 1, Lines 794-795), certification bound in Theorem 4.1 (Lines 218-221) and present an algorithm to empirically certify robustness. We apply this algorithm on GPT2 before and after regularization and demonstrate that our proposed method of regularization does improve knowledge continuity and results in a tighter certification bound. This algorithm is inspired by randomized smoothing and its proof of correctness follows directly from Theorem 4.1 as well as the central-limit theorem. The algorithm and its results can be found in Alg. 3 and Fig. 2 in the attached pdf, respectively. 

```
Algorithm 3: Certifying the robustness of a neural network

Function CERTIFY(f, dataset, k, j, alpha, delta, eta):
1. Define L as the 0-1 loss function
2. Calculate epsilon_U using UPPERCONFBOUND function
3. Find the maximum difference B between f^j(x_a) and f^j(x_b) for all 1<=a, b <= n
4. Calculate V using formula given in Theorem 4.1 (Lines 218-222)
5. Return the clipped value of (1 - epsilon_U * delta / V) between 0 and 1

Function UPPERCONFBOUND(f, L, dataset, k, j, alpha):
1. Initialize U as a zero vector of dimension k
2. For i from 1 to k:
   a. Create a sample S by randomly selecting n points from the dataset with replacement
   b. Calculate U_i using Algorithm 1 (Lines 794-794) with S, L, f, and j
3. Return the mean of U + the std of U multiplied by the inverse normal cdf applied to alpha
```

## Additional Ablation Studies Over Regularization and Attack Strength

Lastly, we perform ablations over the regularization strength and examine its effect on the attack strength-attack success rate curves and test accuracy. We find that moderate regularization significantly improves robustness across all attack strengths and this improvement does not come at the expense of test accuracy (see Fig. 3 in the attached PDF). 

All of these results, algorithms, and corresponding proofs will be included in the final manuscript.

We hope that these additional experimental results will improve your confidence in our work. We would be happy to discuss any remaining concerns.

Best,

Authors


[1] Goodfellow, Ian J., et al. ‘Explaining and Harnessing Adversarial Examples’. arXiv [Stat.ML], 2015, http://arxiv.org/abs/1412.6572. arXiv.

[2] Lin, Jiadong, et al. ‘Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks’. International Conference on Learning Representations, 2020, https://openreview.net/forum?id=SJlHwkBYDH.

Meta-review: The paper received all positive ratings before the rebuttal. The reviewers think the idea is novel and the empirical results are promising. After the rebuttal, most of the minor concerns have been addressed. The AC made the acceptance recommendation.
Decision: Accept (poster)