Adversarially Robust Out-of-Distribution Detection Using Lyapunov-Stabilized Embeddings
Authors: Hossein Mirzaei
Mackenzie W Mathis
Venue: ICLR 2025 Poster (2025)
Keywords: Out-of-Distribution Detection
Adversarial Robustness
Stability Theorem
Neural Ordinary Differential Equations
Lyapunov Theorem
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We propose AROS, a neural ODE-based OOD detector using Lyapunov stability and an orthogonal binary layer to enhance robustness and improve adversarial detection

Abstract: Despite significant advancements in out-of-distribution (OOD) detection, existing methods still struggle to maintain robustness against adversarial attacks, compromising their reliability in critical real-world applications. Previous studies have attempted to address this challenge by exposing detectors to auxiliary OOD datasets alongside adversarial training. However, the increased data complexity inherent in adversarial training, and the myriad of ways that OOD samples can arise during testing, often prevent these approaches from establishing robust decision boundaries. To address these limitations, we propose AROS, a novel approach leveraging neural ordinary differential equations (NODEs) with Lyapunov stability theorem in order to obtain robust embeddings for OOD detection. By incorporating a tailored loss function, we apply Lyapunov stability theory to ensure that both in-distribution (ID) and OOD data converge to stable equilibrium points within the dynamical system. This approach encourages any perturbed input to return to its stable equilibrium, thereby enhancing the model’s robustness against adversarial perturbations. To not use additional data, we generate fake OOD embeddings by sampling from low-likelihood regions of the ID data feature space, approximating the boundaries where OOD data are likely to reside. To then further enhance robustness, we propose the use of an orthogonal binary layer following the stable feature space, which maximizes the separation between the equilibrium points of ID and OOD samples. We validate our method through extensive experiments across several benchmarks, demonstrating superior performance, particularly under adversarial attacks. Notably, our approach improves robust detection performance from 37.8% to 80.1% on CIFAR-10 vs. CIFAR-100 and from 29.0% to 67.0% on CIFAR-100 vs. CIFAR-10. Code and pre-trained models are available at https://github.com/AdaptiveMotorControlLab/AROS.

Published as a conference paper at ICLR 2025
ADVERSARIALLY ROBUST OUT-OF-DISTRIBUTION DE- TECTION USING LYAPUNOV-STABILIZED EMBEDDINGS
Hossein Mirzaei & Mackenzie W.
École Polytechnique Fédérale de Lausanne (EPFL)
hossein, mww.aaiisdeeghlooeeeee ch, mackenzie wwt.hieeptthiiieh.ccccmom
ABSTRACT
Despite significant advancements in methods still struggle to maintain robustness against adversarial attacks, com- promising their reliability in critical real-world applications. Previous studies have attempted to address this challenge by exposing detectors to auxiliary OOD datasets alongside adversarial training. However, the increased data complexity inherent in adversarial training, and the myriad of ways that OOD samples can arise during testing, often prevent these approaches from establishing robust decision boundaries. To address these limitations, we propose AROS, a novel approach leveraging neural ordinary differential equations (NODEs) with Lyapunov stability theorem in order to obtain robust embeddings for OOD detection. By incorporating a tailored loss function, we apply Lyapunov stability theory to ensure that both in-distribution (ID) and OOD data converge to stable equilibrium points within the dynamical system. This approach encourages any perturbed input to return to its stable equilibrium, thereby enhancing the model's robustness against adversarial perturbations. To not use additional data, we generate fake OOD embeddings by sampling from low-likelihood regions of the ID data feature space, approximating the boundaries where OOD data are likely to reside. To then further enhance robustness, we propose the use of orthogonal binary layer following the stable feature space, which maximizes the separation between the equilibrium points of ID and OOD samples. We validate our method through extensive experiments across several benchmarks. demonstrating superior performance, particularly under adversarial attacks. Notably, our approach improves robust detection performance from 37.8% to 80.1 % on CIFAR-10 vs. CIFAR-100 and from 29.0% to 67.0% % on CIFAR-100 vs. CIFAR- 10. Code and pre-trained models are available at https://githun www.ha..ccomaaaaaa http://wwww....aaaeeeeeeeoooottt
(OOD) detection, existing
INTRODUCTION
Deep neural networks have demonstrated remarkable success in computer vision, achieving significant results across wide range of tasks. However, these models vulnerable adversarial examples subtly altered inputs that can lead to incorrect predictions 2233. As result, designing a defense mechanism has emerged as a critical task. Various strategies have been proposed, and adversarial training has become one of the most widely adopted approaches 5 6). Recently, Neural Ordinary Differential Equations (NODEs) have attracted attention as a defense strategy by leveraging principles from control theory. By leveraging the dynamical system properties of NODEs, and imposing stability constraints, these methods aim to enhance robustness with theoretical guarantees. However, they have been predominantly studied in the context of classification tasks and not in distt-iiittinttii (OOD) detection.
14 15
OOD detection is safety-critical task that is crucial for deploying models in the real world. In this task, training is limited to in-distribution (ID) data, while the inference task involves identifying OOD samples, i.e., samples that deviate from the ID data [7). Recent advancements have demonstrated impressive performance gains across various detection benchmarks (88 20 2D. However, a
significant challenge arises concerning the robustness of OOD detectors against ad adversarial attacks. An adversarial attack on a detector involves introducing minor perturbations to test samples, causing the detector to predict OOD as ID samples or vice versa. Yet, robust OOD detector is imperative,

---

Published as a conference paper at ICLR 2025
AROS (Onrs)
RODEO
AID
ALOE
CSI
VOS
DHM
(A) ImageNet vs. Texture
(B) CIFAR-10 vs. CIFAR-100
(C) CIFAR- 100 vs. CIFAR-10
100
100
100
80
a0
80
AUROC(%)
AUROC(%)
AUROC(%)
60
60
60
40
40
20
Perturbation Radius
Perturbation Radiun
Perturbation Radius (6)
Figure 1:
OOD detection performance for various models under different perturbation magni- tudes. The perturbations
generated using PGD 1000 (foo) attack targeting both test ID and OOD samples. (A) ImageNet is used as the ID dataset, while the Texture dataset is used as the OOD during
test time. (B) CIFAR-10 is utilized the ID, with CIFAR-100 as the OOD. (C) CIFAR-100 is used as the ID, with CIFAR-10 10 as the OOD. A perfect detector achieves an AUROC of 100%, a random detector scores 50%, and fully compromised detector under attack scores 0%. Notably, no other model achieves detection performance above random (i.e., greater than 50% AUROC) at 255
especially in scenarios like medical diagnostics and autonomous driving 22 23 24
25
26 Recently,
several approaches have sought to address this challenge by first demonstrating that relying solely on ID data is insufficient for building adversarially robust detectors (27 28
29 23
30
26 31 32
33
36 37 38). Consequently, new methods propose incorporating copious amounts of auxiliary OOD data in conjunction with adversarial training to improve the detector's robustness.
34
35
While effective, a significant gap remains between detector performance on clean data and their robustness against adversarial attacks (see Figure
Tables
2a and 26).
This performance gap primarily arises from the wide variety of potential OOD samples encountered during testing. Relying exclusively on an auxiliary dataset to generate perturbed OOD data can bias the model toward specific OOD instances, thereby compromising the detector's 's ability to generalize to unseen OOD data during inference (16] 39 41 43). This limitation is particularly pronounced in adversarial settings, where adversarial training demands a higher level of data complexity compared 5 47). Additionally, the collection of auxiliary OOD data is costly
to standard training (44 45 46
process, as it must be carefully curated to avoid overlap with ID semantics to ensure that the detector is not confused by data ambiguities (39; 41). Finally, as our empirical analysis reveals, existing OOD detection methods are vulnerable even non-adversarial perturbations a concerning issue for open-world applications, where natural factors such as lighting conditions or sensor noise can introduce significant variability (48) (see Table
We propose AROS (Adversarially Robust OOD Detection through Stability), a novel approach that leverages NODES with the Lyapunov stability theorem (Figure 2). This constraint asserts that small perturbations near stable equilibrium points decay over time, allowing the system state to converge back equilibrium. By ensuring that both ID and OOD data are stable equilibrium points of the detector, the system's dynamics mitigate the effects of perturbations by guiding the state back to its equilibrium. Instead of using extra OOD image data, we craft fake OOD samples in the embedding space by estimating the ID boundary. Additionally, binary layer increases the separation between ID and OOD equilibrium points, enhancing robustness. We evaluate AROS under both adversarial and clean setups across various datasets, including large- scale datasets such as ImageNet (49) and real-world medical imaging data (i.e., ADNI (22), and compare it to previous state-of-the-ant methods. Under adversarial scenarios, we apply strong attacks, including PGD 1000 (4), AutoAttack (50), and Adaptive AutoAttack (51).
Our Contribution:
show that adding an orthogonal
2 PRELIMINARIES
Out-of- catt-f-Distribution Detection. In an OOD detection setup, it is assumed that there are two sets: an ID dataset and an OOD dataset. We denote the ID dataset Din which consists of pairs (xi", n)) where xt represents the ID data, and yd E yinn := {1...., K} denotes the class label. Let Dour represent the OOD dataset, containing pairs (xo", ), where your E yout
2

---

Published as a conference paper at ICLR 2025
{K I,...,, K + O}. and ) nm 0 (8)18)) In practice, different datasets are often used for D' and D Alternatiely, another scenario is called open-set recognition, where a subset of classes within dataset is considered as ID, while the remaining classes considered as OOD 16 54 55). A trained model F assigns an OOD score Sy to each test input, with higher scores indicating greater likelihood of being OOD.
Adversarial Attack on OOD Detectors. Adversarial attacks involve perturbing an input sample x to generate an adversarial example x² that maximizes the loss function ((x";)) y)). The perturbation magnitude is constrained by to ensure that the alteration remains imperceptible. Formally, the adversarial example is defined as
arg max, {(2'; y). subject to || 12==="1, IIp S e, where p denotes the norm (e.g., = 2, oc) (56, 3 44). A widely used attack method is Projected Gradient Descent (PGD) 44), which iteratively maximizes the loss by following the gradient sign of ((x";;) :) with a step size a. For adversarial evaluation (28) 34 37), we adapt this approach by targeting the OOD score (z))) Specifically, the adversarial attack aims to mislead the detector by increasing the OOD score for ID samples and decreasing it for OOD samples, causing misprediction:
26 - x. xi+1 = xi + a sign (I(y) Vz))(=))), x² = T
where n is the number of steps, and I(y)) = +f E ymh and - y E )
Neural ODE and Stability. In the NODE framework, the input and output are treated as two distinct states of a continuous dynamical system, whose evolution is described by trainable layers parameterized by weights and denoted as ha. The state of the neural ODE, represented by Z, evolves over time according to these dynamics, establishing a continuous mapping between the input and output (57 58 59). The relationship between the input and output states is governed by the following differential equations:
dz(f) dt ht)) 2(0) Pippus
2 z(T)
Coupuut-
RELATED WORK
OOD Detection Methods. Existing OOD detection methods can be broadly categorized into post- hoc and training- based approaches. Post-hoc methods involve training classifier ID) data and subsequently using statistics from the classifier's outputs or intermediate representations to identify OOD samples. For instance, Hendrycks et al. (52) propose using the maximum softmax probability distributions (MSP) as a metric. The MD method (60) leverages the Mahalanobis Distance in the feature space, and OpenMax (61) recalibrates classification probabilities to improve OOD detection. Training- based methods, modify the training process to enhance OOD detection capabilities. Such modifications can include defining additional loss functions, employing data augmentation techniques. or incorporating auxiliary networks. Examples of training-based methods designed for standard OOD detection include VOS (39), DHM 19), CATEX (62), and CSI (63). On the other hand, ATOM (30), ALOE (28), ATD 344, and RODEO (37) have been developed specifically for robust detection. For detailed descriptions of these methods, please refer to Appendix AI
Stable NODE for Robustness. TiSODE (64) introduces time-invariant steady NODE to constrain trajectory evolution by keeping the integrand close to zero Recent works employ Lyapunov stability theory develop provable safety certificates for neural network systems, particularly in classification tasks. PeerNets ( was among the first to use control theory and dynamical systems to improve robustness. Kolter al. (65) designed Lyapunov function using neural network architectures to stabilize a base dynamics model's equilibrium. ASODE (66) uses non-autonomous NODEs with Lyapunov stability constraints mitigate adversarial perturbations in slowly time-varying systems. LyaDEQ (67) introduces a new module based on ICNN (68) into its pipeline, leveraging deep equilibrium models and learning a Lyapunov function to enhance stability. SODEF (69) enhances robustness against adversarial attacks by applying regularizers to stabilize the behavior of NODE under the time-in invariant assumption. In Table 4aj we analyze these stability-basee classifiers as OOD detectors and highlight the potential of Lyapunov's theorem as a framework for robust OOD detection, and show our method's s ability to improve performance over these excellent baselines.
PROPOSED METHOD
Motivation. A robust detector should be resistant to shifting ID test samples to OOD, and vice versa, under adversarial attack. A common approach for developing robust OOD detectors involves
3

---

Published as a conference paper at ICLR 2025
ID Samples
Fake OOD Embeedinng
ID Embecdinge
Test Sample
fo
((0)
=(0)
N(H Etrain) )
NODE
NODE
d dx(()) ho( ((t)) d t
d x(()) b+(2(t))
f
x(T)
((T)
h FC Layer
Adversarial Training
Orthogonal Binary Layer
Orthogonal Binary Layer
fg
Embedding
Crafting ODD Embeddings
Extraction
OOD Score
An illustration of AROS. (A) To obtain robust initial features for OOD detection, we perform adversarial training on a classifier using only ID samples. (B) We estimate the ID distribution within the embedding space and generate fake OOD embeddings as proxy for real OOD data. This enables the creation of two balanced classes of samples: ID and fake OOD. (C) The model incorporates NODE layer h& and an Orthogonal Binary Layer B,, Using these two classes, train the pipeline with the loss function LsL to stabilize the system dynamics. (D) During inference, an input passes through the feature extractor fo. NODE has and Orthogonal Binary Layer B., and the resulting likelihood from B, serves as the OOD score. The complete algorithmic workflow of AROS can be found in Appendix A2
Figure 2:
employing adversarial training on ID data, combined with an auxiliary real OOD dataset, expose the detector to potential vulnerable perturbations. The core intuition is that adversarial training on ID data alone, without accompanying OOD dataset, leaves the detector susceptible
perturbations that
alter the boundary between ID and OOD data during testing (2 29 30 23 34 26 Beyond the unsatisfactory performance of the prior approach, there are further challenges with this strategy. A key issue is the cost of preparing an auxiliary dataset disjoint from the ID data, along with ensuring that the selected OOD images adequately cover the boundary between ID and OOD samples-a a critical factor for such frameworks (70) 37 30: 24). Moreover, adversarial training of neural networks is notably more data-intensive than standard setups, further increasing complexity (45) 46: 5 (4).. There is also the concern that exclusively relying perturbed OOD data may introduce biases toward specific OOD examples (1)) To address these challenges, we propose AROS, which utilizes provable stability theorems in the embedding space to develop robust OOD detector without requiring exposure to perturbed OOD image data.
31
33 36 37).
Overview of AROS. AROS ensures that perturbed input samples remain close to their non-perturbed counterparts in the feature space by leveraging the Lyapunov stability theorem (71) 72 73 (7). By using NODE, we consider the model dynamical system and design it that ID and OOD samples converge to distinct stable equilibrium points of that system. This approach prevents sigmificant deviations the output when adversarial perturbations are applied. However, since OOD data is unavailable, we craft fake OOD samples in the embedding space by estimating the boundaries of the ID distribution and sampling from the corresponding low-likelihood regimes. To further avoid any misprediction between OOD and ID data caused by perturbations, we maximize the distance between their equilibrium points by leveraging an orthogonal binary layer for classification. In the following, we will thoroughly explain each proposed component, highlighting the benefits of AROS.
4.1 FAKE EMBEDDING CRAFTING STRATEGY
There have been efforts to utilize synthetic features, primarily under clean scenarios (39;
75 70).
However, for adversarial settings, prior work has often relied on a large pre-trained model and additional data. In contrast, approach limits information to ID samples, proposing to craft OOD data from ID data in the embedding space. These generated OOD samples are subsequently utilized in the training step.
4

---

Published as a conference paper at ICLR 2025
We employ well-trained encoder to transform ID training data into robust embedding spaces. To achieve these embeddings, we first adversarially train a classifier on ID training samples using cross-entropy loss LCE and the (l," attack. By removing the last fully connected layer from the classifier, we utilize the remaining encoder, denoted as fo, to extract ID emheddings r, where fo(x)) from an IDD training sample (Figure ( Specifically, by considering Dinn irain with K classes, we estimate their distribution as a K class-conditional Gaussian distribution, a well-known approach in the detection literature (39) 76 77] 78 79 80). We then select fake embeddings from the feature space corresponding to class j such that r Maa. En) satisfies:
(2m)</2 1/2 exp (- ( (iy)
(4) < <B,
where. 8 serves as a likelihood threshold, and we set that to small value (e.g., 0.001) (Figure 2 Additionally, we conduct an ablation study to evaluate the impact of different values of / and discuss practical considerations (see Appendix A3.3). .uur comprehensive ablation experiments demonstrate the consistent performance of AROS across varying values. Note, d is the dimensionality of the feature vectors r, and j = 1..... K. The terms f. and Ê, represent the mean vector and covariance matrix of the j-th class of ID training samples in feature space, respectively:
A 1 ny E
Σ 1 m--1 (fo.,((f(zz)) (ocioi)))))(((a A))o(()))))))) -
(2)
where nj is the number of samples in class j. By sampling equally across each class of Din we generate a set of synthetic, "fake" OOD embeddings (Figure 2C), denoted as TOOD- We then construct a balanced training set by taking the union of the embeddings of ID samples and the OOD
train
embeddings, defining as: X = { (Diini)) itain) rron] We define the labels y for this set as 0 for ID and I for fake OOD embeddings.
4.2 LYAPUNOV STABILITY FOR ROBUST OOD DETECTION
As mentioned, several approaches have been proposed to apply Lyapunov's theorem to deep networks in practice, including methods such as LyaDEQ (67), ASODE (66), and SODEF (69). Here, we utilize their framework to define the objective function and also benchmark our approach to these baselines.
Amongst them, SODEF adopts a time-invariant (69) 64 assumption, which makes stability analysis
more practical, as the behavior of the neural ODE depends solely on the state 2(t), independent of the specific time that the state is reached. This assumption implies that the equilibrium points of the NODE remain constant over time, facilitating more tractable analysis of how perturbations evolve around these points (64 81 69). This is supported by our experiments in Table 4a which highlight SODEF's superior robustness. Consequently, we adopt the time-invariant framework and use their approach to define the loss function. In order to gain intuition for our approach, we provide the basic mathematical overview of how we leverage the Lyapunov theorems. In this study, as a practical consideration, we assume that the networks utilized have continuous first derivatives with respect to the input 20), which has been shown to be a reasonable assumption (82).
d (=()),,,, state z" is an equilibrium point of system if z* satisfies h((**) ) = O. An equilibrium point is stable if the trajectories starting near =* remain around all the time. More formally:
For given dynamic system dz(t)
Definition 1: (Lyapunov stability (83) An equilibrium 2" is said to be stable in the sense of Lyapunov if, for every > O, there exists 6 > 0 such that, if [|z(0) 201 < 8, then [| - -+]| < e for all 20000 0. If =f is stable, and lim [|=(t) 2* = 0, x is said to be asymptotically stable.
Theorem 1: (Hartman-Grobman Theorem 884 Consider a time-invariant system with continuous
first derivatives, represented by di h(z(t)). For a fixed point z", if the Jacobian matrix Vh evaluated at z" has no eigenvalues with real part equal to zero, the behavior of the original nonlinear dynamical system can be analyzed by studying the linearization of the system around this
dx(tt)
fixed point. The linearized system is given by dx'(t) Az' (t), where A is the Jacobian matrix evaluated at z². This allows for a simplified analysis of the local dynamics in the vicinity of zt
dt
Theorem 2: (Lyapunov Stability Theorem (3)) The equation
dz'(t))
Az (t), is asymptotically ds
stable and only if all eigenvalues of A have negative real parts.
5

---

Published as a conference paper at ICLR 2025
Theorem 3: (Levy- Desplaaslss Theorem (5) Let A [oss] be an n-dimensiooal square matrix and suppose it strictly diagonally dominant, i.e., |asi 2 (gg) |aiyl and as I 0 for all i. Then every eigenvalue of A has a negative real part.
Definition I introduces the concept of asymptotic stability. Building on this, Theorem 1 demonstrates that the behavior of a nonlinear, time-invariant system near a fixed point can be effectively analyzed through its linearization. Theorem 2 then establishes a key condition for the asymptotic stability of linear systems: all eigenvalues of the system matrix must have negative real parts. To facilitate the verification of this stability condition, Theorem 3 provides practical criterion based on the matrix's eigenvalues. In the subsequent section, we will introduce an objective function designed to adhere to these stability criteria.
4.3 ORTHOGONAL. BINARY LAYER AND TRAINING STEP
We propose incorporating an orthogonal binary layer (86) denoted as B, after the NODE he in our pipeline to maximize the distance between the equilibrium points of ID and OOD data. Intuitively, this layer prevents the misalignment of convergence between perturbed OOD) data and ID data by maximizing the distance between their equilibrium points. Given the output & from the he. the orthogonal binary layer B, applies a transformation using weights w such that T ensuring orthogonality. Although Lyapunov stability encourages perturbed inputs to converge to neighborhoods of their unperturbed counterparts, the infinite-depth nature NODE (87) makes them susceptible to degraded activations due to exploding or vanishing gradients (88). The introduction of an orthogonal layer mitigates this risk. Moreover, encouraging orthogonality within neural networks has demonstrated multiple benefits, such as preserving gradient norms and enforcing low Lipschitz constants- both of which contribute to enhanced robustness (89, 90, 991).
To satisfy the aforementioned conditions, we optimize the following empirical Lagrangian LsL with training data (Xmain: :):
min Lst.
(Xomi))))) 7 (X + 72exp
(hh((Xman))
9-7
Xtrin
n Σ
+ 73exp
(X
Σ I[ has (Xmin))
Note that here, Xuaan serves as the initial hidden state, i.e., =(0), for the NODE layer. The first term, lCF, a cross-entropy loss function. The second term forces 20) to be near the equilibrium points, while the remaining terms ensure strictly diagonally dominant derivatives, as described in Theorem monotonically increasing function with a minimum bound to
The exp(.) function is selected limit the unbounded influence of the two regularizers, preventing them from dominating the loss. We set 7 = to balance the first regularization term with fCE, and 72 = 7 0.05 to assign small, equal values that effectively enforce stability without overpowering the other terms. By setting 72 and 7/3 equal, we ensure that both stability conditions contribute equally. Details of the ablation study on these hyperparameters, along with other training step specifics, are provided in Appendices -3. A3.3.2 and A4, By optimizing this objective function, the model learns Lyapunov-stable representations where ID and OOD equilibrium points are well-separated in the feature space after the NODE. The B, captures the probability distribution over the binary classes (ID vs. fake OOD), and for the OOD) score of an input v, use its probability assigned to the OOD class (Figure(D)).
EXPERIMENTS
Here present empirical evidence to validate the effectiveness of method under various setups, including adversarial attacks, corrupted inputs (non-adversanl) perturbations), and clean inputs (non-perturbed scenarios). We note that the backbone architecture for the methods considered is the same as described in [
First, adversarially train classifier on ID data and then use it to map the data into robust embedding space. Gaussian distribution fitted around these embeddings, and low- likelihood regions of the distribution are sampled to create fake OOD data proxy for OOD test samples. We

---

Published as a conference paper at ICLR 2025
Table I. Performance of OOD detection methods under clean evaluation, random corruption (Gaussian noise), and PGD (z adversarial attack with 1000 steps and 255. well as AutoAttack and Adaptive AutoAttack (AA), measured by AUROC (%). A clean evaluation is one where no attack made on the data. For corruption evaluation, Gaussian noise from the ImageNet-C (48) benchmark was used. The best results are highlighted in bold, and the second-best results are underlined each row.
These methods loveraged auxiliary datascts and these used large peetrained models as of their pipeline.
Dataset
Attack
Method
VOS (Reeen
DIM (evelloo
CATEX
ATD (WiidtWNi))
RODEO (OLIADD
AROS
Dm
Pont
(Rehhr
(DeseNer)
Clean
87.9
100.0
88.3
92.2
94.2
78.8
82.0
75.6
88.2
CIFARIO
CIFARIOO
Corruption
56.2
57.7
60.4
54.7
57.3
54.5
59.2
58.6
84.3
PGD 10c0
4,2
1.8
1.6
37.1
37,8
80.1
AutoAttack
0.0
1,2
0.0
0.4
0.5
14.8
36.2
35.9
78.9
AdaptiveAA
0.0
0.0
0.0
11.5
34.8
32.3
76.4
Clean
71.3
100.0
85.1
53.2
57.5
43.6
57.5
61.5
74.3
CIFARIO0
CIEAR10
Corruption
53.8
58.2
57.4
50.1
55.3
56.1
56.0
54,9
71.8
PGD uo0
5.4
2.0
12.1
29.0
67.0
AutoAttack
2.6
0.0
0.3
0.9
0.0
0.0
10.5
28.3
66.5
AdaptiveAA
0.0
9.4
26.7
then demonstrate that time invariance, which establishes that the NODE's behavior does not explicitly depend on time, leads to more stable behavior of the detector under adversarial attacks (see Section 5). Consequently, we leverage Lyapunov stability regularization under a time-invariant assumption for training. However, potential challenge arises when ID and OOD equilibrium points located near each other. As remedy, we introduce an orthogonal binary layer (86) 92; (3) that enhances the separation between ID and OOD data by increasing the distance between their neighborhoods of Lyapunov-stable equilibrium. This enhances the model's robustness against shifting adversarial samples from OOD to ID and vice versa. Finally, we use the orthogonal binary layer's confidence output as the OOD score during inference.
Experimental Setup & Datasets. We evaluated OOD detection methods under both adversarial and clean scenarios (see Tables [ and 2a). Each experiment utilized two disjoint datasets: one as the ID dataset and the other as the OOD test set. For Table [ CIFAR-10 or CIFAR- (94) served as the
ID. Table 2a extends the evaluation to ImageNet-I 1k as the ID, with OOD being comprised of Texture
(55)
SVHN (96), iNaturalist (97), Places365 (98), LSUN (99), and iSUN (1OO).
An OSR (101) setup was also tested, in which each experiment involved single dataset that was randomly split into ID (60%) and OOD (40%) subclasses, with results averaged over 10 trials. Datasets used for OSR included CIFAR- 10, CIFAR-100, ImageNet- Ik, MNIST (02)), FMNIST (103,, and Imagenette (104) (Table 2b). Additionally, models were evaluated on corrupted data using the CIFAR-10-C and CIFAR- TO0-C benchmarks (48) (see Table Specifically, both the ID) and OOD data were perturbed with corruptions that did not alter semantics but introduced slight distributional shifts during testing. Further details on the datasets are provided in Appendix (A5]
Evaluation Details. For adversarial evaluation, all ID and OOD test data were perturbed by using a fully end-to-end PGD (loo) attack targeting their OOD scores (as described in Section 2). We used
255 for low-resolution images and 255 for high-resolution images. The PGD attack steps denoted as M were set to 1000, with 10 random initializations sampled from the interval (-c,c). The step size for the attack was set to = 2.5 x A7 (4). Additionally, we considered AutoAttack and Adaptive AutoAttack (Table TD. Details on how these attacks are tailored for the detection task can be found in Appendix A4 As the primary evaluation metric, used AUROC, representing the area under the receiver operating characteristic curve. Additionally, we used AUPR and FPR95 as supplementary metrics, with results presented in Table 4b AUPR represents the area under the precision-recall curve, while FPR95 measures the false positive rate when the model correctly identifies 95% of the true positives.
4
Reported and Re-Evaluater Results. Some methods may show different results here compared to those reported in their original papers (30 28) due to use of stronger attacks we incorporated
7

---

Published as a conference paper at ICLR 2025
Table 2a. Performance of OOD detection methods under clean evaluation and PGD measured by AUROC (%). The perturbation budget e to & for low-resolution datasets and for high-resolution datasets. The table cells denote results in the 'Clean/PGD format.
265
Dataset
Method
AROS
Din
Dunt
VOS
DHM
CATEX
CSI
ATOM
ALOE
ATD
RODEO
(Ours)
CIFAR-100
87.9/4.2
100.0/188
88.3/0.8
92.2/3.6
94.2/116
78.8/16.1
82.0/37.1
75.6/37.8
88.2/80.1
CIARR-10
SVHN
93.3/2.8
100.0/ 4.5
91.6/2.3
97.4/1.7
89.2/4.7
83.5/26.6
87.9/39.0
83.0/38.2
93.0/86.4
Places
89.7/5.2
99.6/0.0
90.4/
93.6/0.1
98.7/5.6
85. 1/219
92,5/59.8
96.2/20.2
90.8/33.5
LSUN
98.0/7.3
100.0/2.6
9/.8/08
97.7/0.0
99. 1/1.
98.7/50.7
96.0688.1
99.00 85.1
90.6/ 82.4
iSUN
94.600.5
99.1/2.8
93.2/
95.4/3.6
99.5/2.5
98.3/49.5
94.8/65.9
97.7/78.7
88.9/81.2
CIEAR-10
71.3/54
100.0/2.6
85.1/
87.5/2.0
43.6/1.3
57.5/12.1
61.5/29.0
CIFAR-100
SVHN
92.6/3.2
100.0/0.8
94.6/5.7
90.5/4.2
92.8/5.3
74.0/18.1
72.5/27.6
76.9/314
81.5/70.6
Places
75.5/0.0
100.0/3.9
87.3/1.4
73.6/0.0
94.8/3.0
75.0/12.4
83.3/40.0
93.0/66.6
77.0/69.2
LSUN
92.9/5.7
100.0/1.6
94.0/8.9
63.4/1.8
96.6/1.5
98.7/50.7
96.0/68.1
98.1/63.1
74.3/68.1
ISUN
70.2/4.5
99.6/3.6
81.2/0.0
81/4/3.0
96.4/1.4
98.3/49.5
94.8/65.9
1/66/
72.8/67.9
Texture
86.7/0.8
82,4/0.0
92.7/0.0
85.8/0.6
88,9/7.3
76.2/21.8
74.2/15.7
71.3/19.4
78.3/69.2
ImugeNet-
iNaturalist
94.5/0.0
80.7/0.0
97.9/2.0
85.2/1.7
83.6/10.5
78.9/19.4
72.5/12.6
72.7/15.0
84.6/75.3
Places
90,2/0.0
76.2/0.4
90.5/0.0
83,9/0.2
84.5/12.8
78.6/15.3
75.4/17.55
69.2/18.5
76.2/68.1
LSUN
91,9/0.0
82.5/0.0
92.9/0.4
78.4/1.9
85.3/11.2
77.4/ 16.9
68.3/15.1
70.4/662
79.4/69.0
iSUN
92.8/2.7
81.6/0.0
93.7/0.0
77.5/0.0
80.3/14.1
75.3/11.8
76.6/15.8
72.8/17.3
80.3/71.6
Meon
88, 1/2.8
93.4/1.6
91.2/2.3
83.3/1.5
91.4/5.6
81,4/25.5
81.6/37.4
1/44/
82.0V74.0
Table 2h. Performance (Clcan/PGDI000 1000) by AUROC (%), on the OSR setup, which splits one dataset's classes randomly to create Din and Dout.
OOD detection methods under clean PGD 1000 (1 measured
Datasel
Method
AROS (Ours)
VOS
DHM
CATEX
CSI
ATOM
ALOE
ATD
RODEO
MNIST
86.3/4.8
92.6/0.4
92.3/19
93.6/6.1
74.8/4.1
79.5/37.3
68.7/56.5
97.2/85.0
94.4/86.3
EMNIST
78.1/2.0
85.9/0.0
87.0/0.4
84.6/1.2
64.3/4.2
72.6/28.5
59.6/42.1
87.7/65.3
84.1/72.6
CIFAR-10
74.7/0.0
90.8/0.0
95.1/0.0
91.4/0.6
68.3/5.0
52.4/25.6
49.0/32.4
79.6/62.7
78. 78.85
CIFAR-100
63.5/0.0
78.6/0.0
91.9/0.0
86.7/1.9
51.4/2.6
49.8/18.2
50.5/36.1
64. 1/3..3
67.0 67.0/58.2
Imagenette
76.7/0.0
84,2/0.0
96.4/1.6
92.8/0.0
63.5/8.2
61.7/14.2
63 8/28.8
70.6/39.4
78.2/67.5
ADNI
73.5/4.1
69.4/5.2
86.9/0.1
82.1/0.0
66.9/2.3
64.0/11.0
68.3/33.9
75.5/24.6
80.9/61.7
Mean
75.5/1.8
83.6/0.9
91.6/0.7
88.5/1.6
64.9/4,4
63.3/22.5
60.0/38.2
79.1/52.1
80.6/69.3
for evaluation, or the more challenging benchmarks used. For example, ALOE (28) considered
lower perturbation budget for evaluation (i.c.. and the ATD (34) and RODEO (37) benchmarks
used CIFAR-10 vs. union of several datasets, rather than CIFAR-IO vs. CIFAR-100. The union set included datasets such as MNIST, which is significantly different from CIFAR-10, leading to higher reported robust performance.
Results Analysis. Without relying additional datasets or pretrained models, AROS significantly outperforms existing methods in adversarial settings, achieving up to a 40% improvement AUROC and demonstrating competitive results under clean setups (see Table 2b Specifically, AROS also exhibits greater robustness under various corruptions, further underscoring its effectiveness in OOD detection. We further verify approach through an extensive ablation study of various components in AROS (see Section(6).
We note the superiority of AROS compared to representative methods in terms of robust OOD detection. Notably, AROS, without relying on pre-trained models or extra datasets, improves adversarial robust OOD detection performance from 45.9% to 74.0 %. In the OSR setup, the results
8

---

Published as a conference paper at ICLR 2025
Table 3. Performance of OOD detection methods under various types of non-adversarial perturbations, referred to as image corruptions, as introduced in the CIFAR- 10-C and CIFAR- 100- datasets (48), measured by AUROC (%). Specifically, test inputs, including both ID and OOD, are perturbed with particular corruption in each experiment.
Dataset
Mothods
Corputtin
Moon
Gares
Shoe
Empulto
Delacus
Glanc
Mation
Zarm
Snow
Froz
Peg
lliggtt
Contract
Klasti
Piuel
JPEG
56.
76.5
71.7
19:77
72.0
77.0
58.5
79.1
72.5
DHM
57.7
78.7
72.4
75.4
75.6
73.0
77.5
758
70.8
56.8
74.5
58.0
77.4
76.4
80.6
72.2
CHARIIC
CIPAR 199
CATEX
73.0
254
18.6
222
26.2
CSI
54.7
58.0
58.7
62.9
61.7
09.0
65.9
77.2
09.2
74,8
91.9
65.8
74.2
62.6
74.9
68.1
ATOM
57.3
63.6
70.7
7222
69.9
74.6
55.3
$0.3
74.7
77.4
71.7
54.5
76.4
64.9
73.0
70.9
56.3
54.0
76.9
71.5
ATD
59.2
29.2
71.0
76.7
76.9
75.6
20.5
78.2
14.9
59.5
77.8
39.5
79.0
90.8
$2.9
73.7
RODRO
68.5
57.7
79.5
ABBS
84.3
76.5
79.2
$3.8
77.3
$2.0
81.3
83.4
$4.0
$4.0
$4.7
83.3
80.7
79.6
82.5
81.8
Vos
53.8
53.7
65.6
58.2
47.1
51.4
57.6
53.9
59.0
57.2
56.5
54.8
48.2
59.4
51.1
55.3
58.0
58.1
57.1
CHEAR- DOHC
CATEX
57.4
00.2
65.2
59.6
64.9
02.9
59.3
67.
51.4
59.8
00.0
64.2
56.8
57.5
58.6
61.0
CHAM-IR
99.6
52.2
XTOM
55.5
51.2
53.1
50.2
49.9
49.2
49.6
55.1
52.8
54.6
52.4
58.8
45.0
50.4
50.8
$1.5
ALOB
56.1
62.8
5188
54.9
54.8
52.7
56.4
51.7
ATD
56.0
57.4
63.5
57.5
44.8
57.1
54.2
56.9
58.3
55.2
53.7
57.5
49.3
50.8
56.0
55.1
RODEO
54.9
58.1
60.6
56.4
31.0
60.5
58.9
58.4
57.9
54.5
57.4
52.3
52.7
53.5
51.2
55.9
67.7
Table 4a. Comparison of post-hoc OOD detection meth- ods using different classifiers trained with various strate- gies and evaluated with multiple scoring functions. The comparison (Clean/PGD 1000) is conducted under clean and PGD conditions, measured by AUROC (%).
Table 4b. Performance OOD detection methods under clean and PGD 1000 measured by AUPR (%)
and FPR95 ((%) metrics. The perturbation budget
is
The table cells present results in the
'Clean/PGD 1000 format.
Clossifier
Posthoc Mathodd
CIFAR-100
CIFAR-10
SYHN
CIFAR-10
CIFAR-100
SVHN
Method
Metric
CIFAR-100
CIFAR-10
SVHN
CIFAR-10
CIFAR-100
SVHN
MSP
87.9/0.0
91.8/14
75:4/02
71.4/366
Standard
OpenMax
MD
88.54.33
86.4000
99.1/06
96.2/2.8
75.0/19
77.600
98.4066
93.9442
VOS
[PR95)
AUPRI
35:2/100.0
85.8/0.0
38.2/993
90.4/6.2
48.7/100.0
75.8/0.0
41.5/98.2
93.9/7.6
AT'
MSP
MD
81,4/25.6
79.3/160
85.1/19.7
88.2/215
2188/500
67.2/10.7
81.5/19.
74.6/11
DHM
F9895)
AUFRI
100,0:0.3
0.29992
100.0/4.8
0.099.1
100.0:0.0
1.1/10600
100.0/3.2
0.4/99.7
OpenMax
82.4/27.8
86.5/26.9
80.0/16.4
75.4/22.
CATEX
FPR95L
AUPRI
36.6/99.1
89.5/3.4
27.3/95.6
93.1
42.8/100.0
84.
$7.1/98.4
MSP
84.2/106
89.3/15.4
69.7/12.5
76. 1/23.8
ODENet
OpenMax
MD
85.8/14.2
80.79.1
#7.4/209
84.6/13.0
66.4/14.8
70.3/15.6
75.6/18.2
72.9/16.4
CSI
AUPRI
4066/000
$7.49991
65.2/100.
42.697.5
$2.9/0.4
LyaDEQ
MSP 77.5/56.5
MD
1/669
83.7/58.5
82.0/56.5
60.3/ 53.4
3
453.3
ATOM
FPR95L
AUPRI
25.0
97.95.8
96.6
127193.1
38.0/98.0
$9.35.1
29.2/97.9
94.6/72
OpenMax
76.04674
77.5/56.5
67.8/57.1
73.3/ 58.0
ALOE
FPROS
AUPRI
38.68992
804/21.7
1/31/
86 5/2.3
72.896.1
54 8492
85 1/16/
57 4/84.8
ASODE
OpenMux
MSP
MD
74,9449.5
72.6/44.2
76. 36.3/56.3
75.9/57.9
80.5/62.5
76.1/54.4
59.3/ 52.00
64 6144 9
66.1/52.1
80.5/ 50.4
72. 1/5.1
64.6/58.9
ATD
EPRUS
AUPRI
47.3/86.2
$1.9444
42.4 83.0
85.3/53.7
61.4/27.2
68.2 94.8
59 19.0991.9
68 3/26.1
MSP
83.56619
86-4/65.
67.2/53.1
73.7/60.4
RODEO
FPROS
AUPRI
63.5/47.0
42
88.2/51.6
49.6/25.4
65.3/8900
72 72.8/26.5
81 7/42.9
61 8:33
SODEF
MD
75.4/57.7
81 81,9/64.2
65.8/58.4
71.862.5
AROS
OpenMax
N/A
82.8663.
88.2 80.1
86.4999.
93.0/86.4
66.3/6666
74.3/67.0
82.5/70.6
15.2/64.
AROS
FPR95L
AUPRI
67.2
39. 39.3/45.2
97.2.11.4
15.5/27.0
71.0/65.3
$4.2/67.8
16.7
72. 72.1/6688
increased from 52.1% to 69.3%. Similar gains are observed in robustness against corruptions, as shown in Table
For instance, performance improved from 72.5% to 81.8% % on the CIFAR- vs. CIFAR- setup, and from 61.0% to 67.9% on the CIFAR-100-C vs. CIFAR- -O-C benchmark. Meanwhile, AROS achieves competitive results in clean scenarios (82.0%) compared to state-of-the-art methods like DHM (93.4%), though should be noted that DHM performs near zero under adversarial attacks. The trade-off between robustness and clean performance is well-known in the field (5) [05] 44), and
AROS offers the best overall balance among existing methods. Furthermore, we demonstrate that by using pre-trained models auxiliary data, AROS's clean performance can be further improved (see Appendix A3). Moreover, we provide additional experiments in Appendix [A3 to support our claims.
Classifier Training Strategies for Robust OOD Detection. We assessed the impact of different training strategies on the robust OOD detection performance of various classifiers, including those trained with standard training, adversarial training (AT), and NODE-based methods such ODENet,

---

Published as a conference paper at ICLR 2025
Table 5. An ablation study (Clean/PGD 1,00), measured by AUROC (%), on our method with the exclusion of different components while keeping all others intact. The left side is the configurations.
Components
CIFARI00
ImtggeNtt-1l
Contig
Adv: Trained
Bockhone
Sampling
Binary Layer
Orthogonal
Dara
fez
CIEARIOO
SVHN
CIFARIO
SVHN
Texture
INaturalist
61A/1766
86.92335
68.4/2277
79.0/16.2
76.4/18.8 8
82.7/20.3
90.1/56.7
93.8/1.5
75.0/41.8
82.0/47.5
81.9/3600
84.9148
88.2/74.6
66.9157.1
78.0/63.3
75.4600.
85.3/76.5
89.4/78.1
70.5661.3
74.0662.5
76.1667.4
81.3/72.7
Rownr
f
B8 7/1.1
93.0866.4
74. 74.367.00
81.5/20.6
78. 1699
84.6/75.3
F -aaedou
90.4816
94.2/87.9
75.7 68.1
82.2
79.2
85.1/76.8
LyaDEQ, SODEF, and ASODE. To utilize these classifiers as OOD detectors, various post-hoc score functions were applied, described in Section 3 The results are presented Table 4a In brief, adversarially trained classifiers exhibit enhanced robustness compared to standard training but still fall short of optimal performance. Furthermore, the inveiinvaiannce assumption in SODEF leads to improved robust performance relative to ODENet, LyaDEQ, and ASODE by effectively constraining the divergence between output states, which motivated us to explore similar frameworks. Notably, AROS demonstrates superior performance compared to all these approaches.
Implementation details. We use a WideResNet-70. model as fa (106) and train it for 200 epochs on classification using PGD For the integration of hgg an integration time of T 5 is applied. To implement the orthogonal layer B,, we utilize the geotorch ort hogonal library. Training with the loss CSL is performed over 100 epochs. We used SGD as the optimizer, employing cosine learning rate decay schedule with an initial learning rate 0.05 and batch size of 128. See Appendix A3 for more details and additional ablation studies different components of AROS.
6 ABLATION STUDY
AROS Components. To verify the effectiveness of AROS, conducted ablation studies across various datasets. The corresponding results are presented in Table each experiment, individual components were replaced with alternative ones, while the remaining elements were held constant. In Config A, we ignored the designed loss function CSL and instead utilized the cross-entropy loss function LCE for binary classification. Config B represents the scenario in which train the classifier in the first step without adversarial training on the II) data, instead using standard training. This I& becomes
reduces robustness making the final detector more vulnerable to attacks. In Config C, the orthogonal binary layer was replaced with a regular binary layer. In Config rather than estimating the ID distribution and sampling OOD data in the embedding space, we substituted this process by creating random Gaussian noise in the embedding space as fake OOD data. This removes the conditioning of the fake OOD distribution on the ID data and, as a result, makes them unrelated. This is in line with previous works that have shown that related and nearby auxiliary OOD samples are more useful (43 24). Config E represents default pipeline. Finally, in Config F, extended AROS by augmenting the fake OOD embedding data with additional OOD images (i.c., Food-101 (107) alongside the proposed fake OOD strategy. Specifically, transformed these additional OOD images into the embedding space using fe and combined them with the crafted fake embeddings, which led to enhanced performance.
susceptible perturbations within ID classes, ultimately
CONCLUSIONS
In this paper we introduce AROS, a framework for improving OOD detection under adversarial attacks. By leveraging Lyapunov stability theory, AROS drives ID and OOD samples toward stable equilibrium points to mitigate adversarial perturbations. Fake OOD samples are generated in the embedding space, and tailored loss function is used to enforee stability. Additionally, orthogonal binary layer is employed to enhance the separation between ID and OOD equilibrium points. Limitations and future directions be found in the Appendix A6
10

---

Published as a conference paper at ICLR 2025
8 ACKNOWLEDGMENTS
The authors thank the SNSF (Grant No. 320030-227871) and EPFL for funding. MWM is the Bertarelli Foundation Chair of Integrative Neuroscience.
REFERENCES
[1] Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In 2016 IEEE European symposium on security and privacy (EuroS&P), pages 372-387. IEEE, 2016.
[2] Nicholas Carlini and David Wagner. Towards evaluating the robustness neural networks, In 2017 ieee symposium on security and privacy (sp), pages 39-57. Icee, 2017.
[3] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, lan Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
[4] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018
[5] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference machine learning, pages 7472-7482 PMLR, 2019.
[6] Florian Tramer, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. Ensemble adversarial training: Attacks and defenses. In International Conference Learning Representations (ICLR), 2018
Yuri Ledyaev and Eduardo D Sontag. A Iyapunov characterization of robust stabilization. Nonlinear Analysis-Serice A Theory and Methods and Series B Real World Applications, 37(7):813-847 1999.
[8] Fabio Carrara, Roberto Caldelli, Fabrizio Falchi, and Giuseppe Amato. On the robustness to adversarial examples of neural ode image classifiers. In 20/9 IEEE International Workshop on Information Forensics and Security (WIFS), pages 1-6. IEEE, 2019.
[9] Jan Svoboda, Jonathan Masci, Federico Monti, Michael Bronstein, and Leonidas Guibas. Peernets: Exploiting peer wisdom against adversarial attacks. In International Conference Learning Representations, 2019.
[10] Arash Rahnama, Andre T Nguyen, and Edward Raff. Robust design of deep neural networks against adversarial attacks based Iyapunov theory. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 8178-8187, 2020.
[1]] Mingjie Li, Lingshen He, and Zhouchen Lin. Implicit euler skip connections: Enhancing ad- versarial robustness via numerical stability. In International Conference on Machine Learning, pages 5874-5883, PMLR, 2020.
[12] Ivan Dario Jimenez Rodriguez, Aaron Ames, and Yisong Yue. Lyanet: A Iyаpunov framework for training neural odes. International conference on machine learning, pages 18687-18703. PMLR, 2022.
[13] Zonghan Yang, Tianyu Pang, and Yang Liu. A closer look at the adversarial robustness of deep equilibrium models. Advances in Neural Information Processing Systems, 15:44441-0 2022.
[14] Sergey Dashkovskiy, Oleksiy Kapustyan, and Vitalii Slynko. Robust stability of nonlinear ode-pde system. SIAM Journal on Control and Optimization, 6103) 2023.
[15] Mustafa Zeqiri. Mark Niklas Müller, Mare Fischer, and Martin Vechev. Efficient certified training and robustness verification of neural odes. arXiv preprint 2002:22303 2023.
11

---

Published as a conference paper at ICLR 2025
[16] Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distribution detection: A survey. arXiv preprint 1.kiv:2110.113322 2021.
[17] Abhijit Bendale and Terrance Boult. Towards open world recognition. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), pages 1893-1902, 2015.
[18] Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan. Exploring the limits of out-of- distribution detection. Advances in Neural Information Processing Systems, 34, 2021.
[19] Senqi Cao and Zhongfei Zhang. Deep hybrid models for out out-of-distribution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4733-4743, 2022.
[20] Feng Xue, Zi He, Chuanlong Xie, Falong Tan, and Zhenguo Li. Boosting out-of-distribtioon detection with multiple pre-trained models. arXiv preprint arXiv:2212.127200 2022.
[21] Xuefeng Du, Zhen Fang, Ilass Diakonikolas, and Yixuan Li. How does unlabeled data provably help out-of-distribution detection? arXiv preprint arXiv:2402.03502, 2024.
[22] Ronald Carl Petersen, Paul S Aisen, Laurel A Beckett, Michael C Donohue, Anthony Collins Gamst, Danielle J Harvey, CR Jack Jr, William J Jagust, Leslie M Shaw, Arthur W Toga, et al. Alzheimer's ' disease neuroimaging initiative (adni) clinical characterization. Neurology, 74(3):201-209, 2010.
[23] Adam Goodge, Bryan Hooi, See Kiong Ng. and Wee Siong Ng. Robustness of autoencoders for anomaly detection under adversarial impact. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pages 1244-1250, 2021.
[24] Shu Kong and Deva Ramanan. Opengan: Open-set recognition via open data generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2021.
[25] Emily Roberts and Rajesh Gupta. Applying novelty detection techniques for quality assurance in manufacturing. Journal of Industrial and Production Engineering, (8(4))2511263) 2021.
[26] Shao-Yuan Lo, Poojan Oza, and Vishal M Patel. Adversarially robust one-class novelty detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
[27] Dan Hendryeks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. arXiv preprint arXiv:1812.04
2018.
[28] Jiefeng Chen, Yixuan Li, Xi Wu, Yingyu Liang, and Somesh Jha. Robust out-of-distribution detection for neural networks. arXiv preprint arXiv:2003.0m 097/ 2020.
[29] Rui Shao, Pramuditha Perera, Pong C Yuen, and Vishal M Patel. Open-set adversarial defense. In Computer Vision-ECCVV 2020: 16th European Conference. Glasgow, UK, August 23-28, 2020, Proceedings, Part XVII 16. pages 682-698. Springer, 2020.
[30] Jiefeng Chen, Yixuan Li, Xi Wu, Yingyu Liang, and Somesh Jha. Atom: Robustifying out-of- distribution detection using outlier mining. In Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13-17, 2021, Proceedings. Part III 21, pages 430-445. Springer, 2021.
[31] Alexander Meinke, Julian Bitterwolf, and Matthias Hein. Provably adversarially robust detec- tion of out-of-distribution data (almost) for free Advances in Neural Information Processing Systems, 35:30167-30188 2022.
[32] Stanislav Fort. Adversarial vulnerability of powerful near out-of-distributioo detection. arXiv preprint arXiv:2201. 0000020200702,.. 2022.
[33] Rui Shao, Pramuditha Perera, Pong C Yuen, and Vishal M Patel. Open-set adversarial defense with aaversaalaaa mutual learning. International Journal of Computer Vision, 1070 2022.
12

---

Published as a conference paper at ICLR 2025
[34] Mohammad Azizmalayeri, Arshia Soltani Moakhar, Arman Zarei, Reihaneh Zohrabi, Moham- mad Manzuri, and Mohammad Hossein Rohban. Your out-of-distributtio detection method is not robust! Advances in Neural Information Processing Systems, 35:4887-4901, 2022,
[35] Nicola Franco, Daniel Korth, Jeanette Miriam Lorenz, Karsten Roscher, and Stephan Guenne- mann. Diffusion denoised smoothing for certified and adversarial robust out-of-distribution detection. arXiv preprint arXiv:2303. 14961, 2023.
[36] Louis Béthune, Paul Novello, Thibaut Boissin, Guillaume Coiffier, Mathieu Serrurier, Quentin Vincenot, and Andres Troya-Galvis. Robust one-class classification with signed distance function using 1-lipschitz neural networks. arXiv preprint (xXix:2333.01978 2023.
[37] Hossein Mirzaei, Mohammad Jafari, Hamid Reza Dehbashi, Ali Ansari, Sepehr Ghobadi, Ma- soud Hadi, Arshia Soltani Moakhar, Mohammad Azizmalayeri, Mahdieh Soleymani Baghshah, and Mohammad Hossein Rohban. Rodeo: Robust outlier detection via exposing adaptive out-of-distribution samples. In Forty-first International Conference on Machine Learning, 2024.
[38] Peter Lorenz, Mario Fernandez, Jens Müller, and Ullrich Köthe. Deciphering the definition of adversarial robustness for post-hoc ood detectors. arXiv preprint 155544 2024.
[39] Xuefeng Du, Zhaoning Wang, Mu Cai, and Yixuan Li. Vos: Learning what you don't know by virtual outlier synthesis. arXiv preprint arXiv:2202.01 197, 2022.
[40] Peyman Morteza and Yixuan Li. Provable guarantees for understanding out-of-distribttion detection. Proceedings of the AAAI Conference Artificial Intelligence, 36(7):7831-7840, 2022.
[41] Yifei Ming, Ying Fan, and Yixuan Li. Poem: Out-of-distribution detection with posterior sampling. International Conference on Machine Learning, pages 15650-15665. PMLR, 2022.
[42] Xin Zou and Weiwei Liu. On the adversarial robustness of out-of-distributinn generalization models. Advances in Neural Information Processing Systems, 36, 2024.
[43] Hossein Mirzaei, Mohammadreza Salehi, Sajjad Shahabi, Efstratios Gavves, Cees GM Snock, Mohammad Sabokrou, and Mohammad Hossein Rohban. Fake it till you make it: Near-distribution novelty detection by score-based generative models. arXiv preprint vrXiv:2205.14297, 2022.
[44] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arviv: (oo.::706.00.. 2017.
[45] Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Adversarially robust generalization requires more data. Advances in neural information processing systems, 31, 2018.
[46] Preetum Nakkiran. Adversarial robustness may at odds with simplicity. arXiv preprint PrXiv:1901.00532. 2019.
[47] Sravanti Addepalli, Samyak Jain, et al. Efficient and effective augmentation strategy for adversarial training. Advances in Neural Information Processing Systems, 35:1488-155 2022.
[48] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXii:1903: 12261, 2019.
[49] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pages 248-254 255. leee, 2009.
[50] Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. CoRR, (axs/2003.0199) 2020.
13

---

Published as a conference paper at ICLR 2025
[51] Ye Liu, Yaya Cheng, Lianli Gao, Xianglong Liu, Qilong Zhang, and Jingkuan Song. Practical evaluation of adversarial robustness via adaptive auto attack, 2022.
[52] Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of- distribution examples in neural networks. In International Conference on Learning Rep- resentations, 2017.
[53] Jingkang Yang, Pengyun Wang, Dejian Zou, Zitang Zhou, Kunyuan Ding, Wenxuan Peng, Haoqi Wang, Guangyao Chen, Bo Li, Yiyou Sun, et al. Openood: Benchmarking generalized out-of-distribution detection. Advances in Neural Information Processing Systems, 35:32598-
3,611, 2022.
[54] Mohammadreza Salehi, Hossein Mirzaei, Dan Hendrycks, Yixuan Li, Mohammad Hossein Rohban, and Mohammad Sabokrou. A unified survey on anomaly, novelty, open-set, and out- of-distribution detection: Solutions and future challenges. arXiv preprint arXiv:2110.14051, 2021.
[55] Chuanxing Geng, Sheng-jun Huang, and Songean Chen. Recent advances in open set recogni- tion: A survey. IEEE transactions on pattern analysis and machine intelligence, 43(10))3614- 3631, 2020.
[56] lan Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adver- sarial examples. arXiv preprini arXiv: 1412.657, 2014.
[57] Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. In Advances in Neural Information Processing Systems, volume 31, pages 6571-6583, 2018.
[58] Emilien Dupont, Arnaud Doucet, and Yee Whye Teh. Augmented neural odes. In Advances in Neural Information Processing Systems (NeurIPS). 2019.
[59] Will Grathwohl, Ricky T. Q. Chen, Jesse Bettencourt, Ilya Sutskever, and David Duvenaud. Ffjord: Free-form continuous dynamics for scalable reversible generative models. In Interna- tional Conference on Learning Representations (ICLR), 2019.
[60] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks, Advances in neural information processing systems, 31, 2018.
[61] Abhijit Bendale and Terrance E. Boult. Towards open set deep networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1563-1572, 2016.
[62] Kai Liu, Zhihang Fu, Chao Chen, Sheng Jin, Ze Chen, Mingyuan Tao, Rongxin Jiang, and Jieping Ye. descriptions. Advances in Neural Information Processing Systems, 36, 2024.
out-of-distribution detection via hierarchical context
[63] Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin. Csi: Novelty detection via contrastive learning on distributionally shifted instances. Advances in neural information processing systems, 33:1 11899-11852, 2020.
[64] Hanshu Yan, Jiawei Du, Vincent YF Tan, and Jiashi Feng. On robustness of neural ordinary differential equations. arXiv preprint w9Xiv:1900.055.1 2019.
[65] Zico Kolter and Gaurav Manck. Learning stable deep dynamics models. Advances in neural information processing systems, 32, 2019.
[66] Xiyuan Li, Zou Xin, and Weiwei Liu. Defending against adversarial attacks via neural dynamic system. Advances in Neural Information Processing Systems, 35:6372-6383, 2022.
[67] Haoyu Chu, Shikui Wei, Ting Liu, Yao Zhao. and Yuto Miyatake. Lyapunov-stable deep equi- librium models. Proceedings of the AAAI Conference on Artificial Intelligence, 38(10): 16155- 1622, 2024.
14

---

Published as a conference paper at ICLR 2025
[68] Brandon Amos, Lei Xu, and J Zico Kolter. Input convex neural networks. In International conference on machine learning, pages 146-155. PMLR, 2017.
[69] Qiyu Kang, Yang Song, Qinxu Ding, and Wee Peng Tay. Stable neural ode with lyapunov-stable equilibrium points for defending against adversarial attacks. Advances in Neural Information Processing Systems, 34:14925-20 1925-14937, 2021.
[70] Xuefeng Du, Yiyou Sun, Xiaojin Zhu, and Yixuan Li. Dream the impossible: Outlier imagina- tion with diffusion models. arXiv preprint 2309::2333.131//. 2023.
[71] Charles Dawson, Zengyi Qin, Sicun Gao, and Chuchu Fan. Safe nonlinear control using robust neural lyapunov-bare -arrier functions. In Conference on Robot Learning, pages 1724-1735. PMLR, 2022.
[72] Nur Uddin, Hendra G Harno, and Rianto Adhy Sasongko. Altitude control system design of bicopter using Iyapunov stability approach. In 2021 International Symposium on Electronics and Smart Devices (ISESD), pages 1-6. IEEE, 2021.
[73] Aman Sharma and Narendra Kumar. Lyapunov stability theory based non linear controller design for standalone pv system. In 2020 JEEE International Conference for Innovation in Technology (INOCON), pages 1-7. IEEE, 2020.
[74] Ya-Chien Chang, Nima Roohi, and Sicun Gao. Neural Iyapunov control. In Advances in Neural Information Processing Systems, pages 3240-3249, 2019.
[75] Lue PJ Sträter, Mohammadreza Salehi, Efstratios Gavves, Cees GM Snock, and Yuki M Asano. Generalad: Anomaly detection across domains by attending to distorted features. arXiv preprint arXiv:2407. 12427, 2024.
[76] Matan Jacob Cohen and Shai Avidan. Transformaly two (feature spaces) are better than one. arXiv preprint 1rXiv:2112.04185. 2021.
[77] Swaminathan Venkataramanan et al. Gaussian latent representations for uncertainty estimation using mahalanobis distance. In Proceedings of the IÉEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2023.
[78] Xun Jin et al. Shape your space: A gaussian mixture regularization approach latent space geometry optimization. In Advances in Neural Information Processing Systems, pages 12688-12701, 2021.
[79] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting outti--distributtin samples and adversarial attacks. In S. Bengio, H. Wallach, Larochelle, K. Grauman, N. Cesa-Bianchi, and Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
[80] Jie Ren, Stanislay Fort, Jeremiah Liu, Abhijit Guha Roy, Shreyas Padhy, and Balaji Laksh- minarayanan. A simple fix to mahalanobis distance for improving near-ood detection. arXiv preprint arXiv:2106.09000 2021.
[81] Stefano Massaroli, Atsushi Yamashita, and Hajime Asama. Dissecting neural odes. In Advances in Neural Information Processing Systems, 2020.
[82] Kurt Hornik. Approximation capabilities of multilayer feedforward networks. Neural networks, 4(2):251-257, 1991.
[83] Chi-Tsong Chen. Linear system theory and design. Saunders college publishing, 1984.
[84] DK Arrowsmith and CM Place. Differential equations, maps and chaotic behavior. Champman and Hall, London, 1995.
[85] Roger A Horn and Charles R Johnson. Matrix analysis. Cambridge university press, 2012.
[86] Shuai Li. Kui Jia, Yuxin Wen, Tongliang Liu, and Dacheng Tao. Orthogonal deep neural networks. IEEE transactions on pattern analysis and machine intelligence, 1352)::352-11366 2019.
15

---

Published as a conference paper at ICLR 2025
[87] Mufan Li, Mihai Nica, and Dan Roy. The future is -og-gaussian: Resnets and their infinite- depth-and-width limit at initialization. Advances in Neural Information Processing Systems, 34:7852-7864, 2021.
[88] R Pascanu. Understanding the exploding gradient problem. arXiv preprint 2012.
[89] Patricia Pauli, Anne Koch, Julian Berberich, Paul Kohler, and Frank Allgöwer. Training robust neural networks using lipschitz bounds. IEEE Control Systems Letters, 6:121-126, 2021.
[90] Yujia Huang. Huan Zhang, Yuanyuan Shi, J Zico Kolter, and Anima Anandkumar. Training certifiably robust neural networks with efficient local lipschitz bounds. Advances in Neural Information Processing Systems, 14:22745-227555 2021.
[91]
Maximmnan Zühlke and Daniel Kudenko. Adversarial robustness of neural networks from the perspective of lipschitz calculus: A survey. ACM Computing Surveys, 2024.
[92] Cong Xu, Xiang Li, and Min Yang. An orthogonal classifier for improving the adversarial robustness of neural networks. Information Sciences, 591:251-20 1222 2022.
[93] Sima Behpour, Thang Long Doan, Xin Li, Wenbin He, Liang Gou, and Liu Ren. Gradorth: a simple yet efficient -iittii-iitiitttinnn detection with orthogonal projection of gradients. Advances in Neural Information Processing Systems, 36, 2024.
[94] Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
[95] Mircea Cimpoi, Subhransu Maji, lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1606-3613, 2014.
[96] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop Deep Learning and Unsupervised Feature Learning, volume 2011, page 5, 2011.
[97] Grant Van Horn, Oisin Mac Aodha, Yang Song. Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8769-8778, 2018
[98] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(6):1452-1464, 2017.
[99] Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. In arXiv preprint arXiv:1506 03360000033..... 2015.
[100] Jia-Bin Xu and Jianxiong Xiao. Isun: Large-scale scene understanding. In arXiv preprint urXiv:1502.03500 2015.
[101] Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. Open-set recognition: A good closed-set classifier is all you need? ICLR, 2022.
[102] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 1):2278-2224 1998.
[103] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. In arXiv preprint arXiv: 2017.
[104] Jeremy Howard. Imagenette. httpas//ggihaab.cco////saaaa//gp cessed: 2024-09-27.
Ac-
[105] Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. arXiv preprint arXiv: 100.. 2018.
16

---

Published as a conference paper at ICLR 2025
[106] Sergey Zagoruyko. Wide residual networks. arXiv preprint arXiv:1605.07146. 2016.
[107] Lukas Bossard, Matthieu Guillaumin, and Lue Van Gool. Food-101-mining discriminative components with random forests. Computer vision-ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part VI 13, pages 446-461. Springer, 2014.
[108] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. International conference on machine learning.
pages 8748 8763. PMLR, 2021.
[109] Hossein Mirzaei, Mojtaba Nafez, Mohammad Jafari, Mohammad Bagher Soltani, Mohammad Azizmalayeri, Jafar Habibi, Mohammad Sabokrou, and Mohammad Hossein Rohban. Univer- sal novelty detection through adaptive contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 229 11011-2222222 2024.
[110] Hossein Mirzaei, Ali Ansari, Bahar Dibaei Nia, Mojtaba Nafez, Moein Madadi, Sepehr Rezaee, Zeinab Sadat Taghavi, Arad Maleki, Kian Shamsaie, Mahdi Hajialilue, et al. Scanning trojaned models using out-of-distribution samples. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[111] Arshia Soltani Moakhar, Mohammad Azizmalayeri, Hossein Mirzaei, Mohammad Taghi Manzuri, and Mohammad Hossein Rohban. Seeking next layer neurons' attention for cmorrbackkrropggationniii like training in multi-agent network framework. arXiv preprint arXiv:2310.09952, 2023.
[112] Hossein Mirzaei, Mohammad Jafari, Hamid Reza Dehbashi, Zeinab Sadat Taghavi, Moham- mad Sabokrou, and Mohammad Hossein Rohban. Killing it with zero-shot: Adversarially robust novelty detection. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7415-7419, IEEE, 2024.
[113] Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, and Anima Anandkumar. Diffusion models for adversarial purification. arXiv preprint arXiv:2205.07466 2022.
[114] Yong Chen, Xuedong Li, Xu Wang, Peng Hu, and Dezhong Peng. Diffilter: Defending against adversarial perturbations with diffusion filter. IEEE Transactions on Information Forensics and Security, 2024.
[115] Kaiyu Song, Hanjiang Lai, Yan Pan, and Jian Yin. Mimicdiffusion: Purifying adversarial perturbation via mimicking clean diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14665-246677 2024.
[116] Minjong Lee and Dongwoo Kim. Robust evaluation of Liffusion-based adversarial purification. In Proceedings the IEEE/CVF International Conference on Computer Vision, pages 134 144, 2023.
[117] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems, 14:8780-8794, 2021.
[118] Runpeng Yu, Songhua Liu, Xingyi Yang, and Xinchao Wang. Distribution shift inversion for out-of-distribution prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3592-3602, 2023.
[119] Yuanpu Cao, Lu Lin, and Jinghui Chen. Adversarially robust industrial anomaly detection through diffusion model. arXiv preprint arXiv:2408. 04839, 2024.
[120] Xiangyuan Yang, Jie Lin, Hanlin Zhang, Xinyu Yang, and Peng Zhao. Improving the trans- ferability of adversarial examples via direction tuning. arXiv preprint 23315:2303.15009 2023.
17

---

Published as a conference paper at ICLR 2025
[121] Qinliang Lin, Cheng Luo, Zenghao Niu, Xilin He, Weicheng Xie, Yuanbo Hou, Linlin Shen, and Siyang Song. Boosting adversarial transferability across model genus by deformation- constrained warping. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages 3459-3467, 2024.
[122] Han Wu, Guanyan Ou, Weibin Wu, and Zibin Zheng. Improving transferable targeted adver- sarial attacks with model self enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24615-24624, 2024.
[123] Yansong Gao, Change Xu, Derui Wang, Shiping Chen, Damith C Ranasinghe, and Surya Nepal. Strip: A defence against trojan attacks on deep neural networks. Proceedings of the 35th annual computer security applications conference, pages 113- 2019.
[124] Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adversarially robust imagenet models transfer better? Advances in Neural Information Processing Systems, 33:3533-3545, 2020.
[125] Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with a fast adaptive boundary attack. In International Conference on Machine Learning, pages 2196-2205. PMLR, 2020.
[126] Matteo Croce and Matthias Hein. Reliable evaluation of adversarial robustness with autoattack. In International Conference Machine Learning, pages 2701-2711 PMLR, 2020.
[127] Maria Andriushchenko, Yang Song, and Zachary C Lipton. Square attack: query-efficient black-box adversarial attack via random search. In International Conference on Learning Representations, 2020.
[128] Mu Zhou, Lucas Stoffl, Mackenzie Weygandt Mathis, and Alexander Mathis. Rethinking pose estimation in crowds: Overcoming the detection information bottleneck and ambiguity. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 14689-14699, October 2023.
[129] Shaokai Ye, Anastasiia Filippova, Jessy Lauer, Maxime Vidal, Steffen Schneider, Tian Qiu, Alexander Mathis, and Mackenzie W. Mathis. Superanimal pretrained pose estimation models for behavioral analysis. Nature Communications, 15, 2024.
[130] Zijiao Chen, Jiaxin Qing, Tiange Xiang, Wan Lin Yue, and Juan Helen Zhou. Seeing beyond the brain: Conditional diffusion model with sparse masked modeling for vision decoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22710-22720, 2023.
[131] Scott Zimmerman.
Guide to the Martman-grobman and poincar e-bendixon theorems, 1ath.
https://dt //dl. watest.orr/ppffo.
2008. MATH181HM: Dynami-
cal Systems.
[132] Philip Hartman. Ordinary differential equations. SIAM, 2002.
[133] Steven H Strogatz. Nonlinear dynamics and chaos: with applications to physics, biology. chemistry, and engineering. CRC press, 2018
[134] Lawrence Perko. Differential equations and dynamical systems, volume 7. Springer Science & Business Media, 2013.
[135]
Mathukumalli Vidyasagar. Nonlinear systems analysis. SIAM, 2002.
[136] Nam Parshad Bhatia and Giorgio Szegö. Stability theory of dynamical systems. Springer Science & Business Media, 2002.
[137] Edward James McShane. Extension of range of functions. Bulletin of the American Mathemat- ical Society, 1934.
18


Reviews:
[{"review_id": "wKnbtHnr5t", "reviewer": "Reviewer_dAkz", "summary": "This paper proposes a new out-of-distribution detector that is robust against adversarial attacks. Specifically, the authors point out the problems of adversarial training with auxiliary datasets and propose a novel approach based on neural ordinary differential equation (ODE) and Lyapunov stability. The authors force the neural ODE model’s dynamical system to have stable equilibrium points on both in-distribution (ID) and out-of-distribution (OOD) samples. The authors also present a theoretical background that supports the proposed method. With experiments, the authors present the performance of the proposed method and some ablation studies.", "questions": "1. I’m curious about the sampling methods. Do you have some justification that the low-likelihood part would represent the embeddings of the realistic OOD samples, or do you just not care about the realisticness of the embeddings?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The proposed method's theoretical background is solid, and the main trick (i.e., the Levy-Desplanques Theorem) seems useful for other neural ODE models.\n2. With an orthogonal layer, the authors effectively overcame practical problems, such as exploding gradients and vanishing gradients.\n3. The experiments are well done. The authors used very strong attacks and many baseline models to compare. The proposed method's performance is really impressive.", "weaknesses": "1. The proposed method generates fake OOD data in the embedding space, which might expose other vulnerabilities in ID embedding extraction. For example, the adversary can try poisoning the training ID data by injecting OOD samples so that the crafted OOD data does not include those injected OOD samples.\n2. If I understand correctly, the fake OOD embedding is sampled from the low-likelihood space. This makes sense because OOD samples will be embedded in the low-likelihood part. However, this could be an ineffective method to generate embeddings of “realistic” OOD samples because even random noise would be mapped to the low-likelihood part.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VZ6pRvJazY", "reviewer": "Reviewer_e2wJ", "summary": "This paper introduces a novel adversarially robust out-of-distribution (OOD) detection method, AROS, which leverages Neural Ordinary Differential Equations (NODE) and Lyapunov stability theory to enhance model stability against adversarial perturbations. AROS employs a pseudo-OOD data generation strategy, sampling low-likelihood regions in the embedding space to create pseudo-OOD samples, thereby improving the model's generalization capability. Additionally, it incorporates an orthogonal binary layer to strengthen the separation between in-distribution (ID) and OOD samples. Experiments on CIFAR-10, CIFAR-100, and ImageNet-1k demonstrate that AROS maintains high detection accuracy across various attacks, including strong adversarial PGD and adaptive AutoAttack. Ablation studies indicate that Lyapunov stability regularization and pseudo-OOD embedding generation are particularly important for enhancing robustness.", "questions": "See Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper proposes an OOD detection method based on Lyapunov stability and NODE, representing an innovative attempt to integrate control theory with OOD detection to improve model robustness under adversarial perturbations.\n2. The proposed orthogonal binary layer is effective in enhancing the separation of ID and OOD samples.", "weaknesses": "1. In the formula for the pseudo-embedding generation strategy, $p(r | y = j)$ denotes the conditional probability of generating pseudo-OOD embeddings. Here, the conditional probability notation $p(\\cdot | \\cdot)$ is used to describe feature vectors that meet a given condition, but in reality, it defines an inequality rather than a complete probabilistic expression. To avoid misunderstandings, it is recommended to rewrite this as an inequality, preventing the impression that a probability density function is defined here.\n\n2. The definition and application of the orthogonal binary layer $B_{\\eta}$ are somewhat complex. This layer aims to enhance the separation of ID and OOD samples by enforcing orthogonality in specific directions of the feature space. However, in the formula $L_{SL}$, directly using $B_{\\eta}$ as a layer with multiple parameters may cause confusion, especially when multiple hyperparameters are involved. The expression might be clearer if $B_{\\eta}$ is decomposed or its function is clearly annotated.\n\n3. The model architecture used in the experimental section is not clearly described in the paper. Please clarify the model architecture.\n\n4. Most of the models used in the experiments are based on traditional CNN architectures. Could the authors consider using Transformer-based models to evaluate performance differences between different methods?\n\n5. It seems the experiments use untargeted attacks. Has the author considered the impact of targeted attacks on OOD detection?\n\n6. In Section A3.2, \"Extended Ablation Study,\" the authors provide results from ablation experiments but do not analyze the reasons for choosing the default parameters in the paper or discuss how different parameters might affect the results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AO1KPtxMSI", "reviewer": "Reviewer_w7tq", "summary": "This paper proposes the Adversarially Robust OOD Detection through Stability (AROS) method, aimed at enhancing robustness in out-of-distribution (OOD) detection, particularly under adversarial conditions. The authors base their approach on Neural Ordinary Differential Equations (NODE) and Lyapunov stability theory, designing a time-invariant dynamic system to train the model for stability in perturbed environments. The AROS method includes adversarial training, pseudo-OOD embedding generation, Lyapunov stability regularization, and an orthogonal binary layer to construct a robust feature space that improves the separation between in-distribution (ID) and OOD samples. Experimental results demonstrate that AROS outperforms other OOD detection methods across multiple benchmark datasets (e.g., CIFAR-10, CIFAR-100, and ImageNet-1k), especially under strong adversarial attacks such as PGD, AutoAttack, and Adaptive AutoAttack.", "questions": "1. In the experiments in Table 1, the authors used different model architectures. What is the reason for this choice? Could a unified model architecture be used to ensure fairness in method comparison?\n2. Tables 2, 3, and 4 do not specify the model architectures used. Only OOD detection performance on different datasets is shown, which reduces the reproducibility of the paper.\n3. In the text, $\\epsilon$ is set to 8/255 and 4/255, but in Table 8a, the authors set $\\epsilon$ to 128/255 and 64/255. Such high perturbation rates would cause noticeable distortion in samples. Please explain the significance of this setting. Could results for perturbation rates of 16/255 and 32/255, especially 16/255 (the default parameter in most adversarial attack methods), be added?\n4. The authors simply list the experimental results for $\\beta$ without providing in-depth analysis. For instance, in Table 11a, why are the results for $\\beta = 0.001$ identical to those for $\\beta = 0.025$? The three hyperparameters $\\gamma_1$, $\\gamma_2$, and $\\gamma_3$ in the loss function $L_{SL}$ are also listed without further analysis.\n5. Could some generated pseudo-OOD data samples be provided for readers to view?\n6. I am curious whether the proposed method in the paper can be applied to the detection of backdoor attack samples.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper is clearly written and easy to understand.\n2. The authors conducted extensive experiments to validate the effectiveness of their method.", "weaknesses": "1. The loss function $L_{SL}$ includes multiple hyperparameters, $\\gamma_1$, $\\gamma_2$, and $\\gamma_3$. The choice of these hyperparameters affects the model's focus on different regularization terms, but directly showing them in the formula can seem verbose.\n2. A detailed explanation of the elements in $[\\nabla h_{\\phi}(X_{\\text{train}})]_{ii}$ \n\nand $[\\nabla h_{\\phi}(X_{\\text{train}})]_{ij}$ is needed to clarify their computation.\n\n3. In the section on the orthogonal binary layer, the equation $z^T z = I$ expresses the orthogonalization condition. However, it is unclear if $z$ has been normalized and whether its inner product yielding $I$ is the required condition.\n4. The adversarial attack methods used in experiments are not the latest. The authors are encouraged to add 2023 and 2024 adversarial attack methods, particularly those with stronger transferability and generalization. See this work [1] for reference.\n5. The authors did not provide the code for the experiments.\n\n**Reference:**\n\n[1] https://github.com/Trustworthy-AI-Group/TransferAttack", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UwLwydwwGy", "reviewer": "Reviewer_x9hq", "summary": "This paper introduces AROS, a method for enhancing the robustness of OOD detection under adversarial attacks. Using Lyapunov-stabilized embeddings within a NODE framework, AROS aims to stabilize the decision boundaries between ID and OOD samples by promoting equilibrium convergence through adversarial training. The approach includes generating synthetic OOD data in the feature space and applying an orthogonal binary layer to maximize separation between ID and OOD samples.", "questions": "1. Could the authors clarify the theoretical guarantees of Lyapunov stability when applied within the NODE framework, particularly for high-dimensional embeddings? Specifically, could they expand on how the equilibrium points are stabilized across varying adversarial perturbation magnitudes, especially under different norm constraints? Additionally, the derivations and stability proofs related to Theorems 1 and 2 are complex. Would the authors consider including expanded derivations or references in the appendix to enhance understanding for readers without a control theory background?\n\n\n2. The introduction of the orthogonal binary layer aims to enhance ID-OOD separation. Could the authors provide more empirical insights or visualizations to demonstrate the layer's effectiveness in maintaining separation, particularly in adversarial scenarios? Furthermore, an analysis of the sensitivity of this layer to different perturbation levels would strengthen the practical understanding of its robustness.\n\n\n3. The authors propose generating synthetic OOD embeddings by sampling from low-likelihood regions in the ID feature space. However, how well does this synthetic data represent real-world OOD instances, which may be more complex in structure? Could the authors provide empirical evidence or a quantitative comparison to validate the representativeness of these synthetic OOD samples? Additionally, would the authors consider including an ablation or alternative generation methods to explore the impact of different OOD data synthesis strategies?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. **Innovative Use of Stability Theory:** The integration of Lyapunov stability within NODEs for OOD detection is novel and well-motivated. The application of stability principles to mitigate adversarial perturbations shows theoretical grounding.\n\n2. **Comprehensive Evaluation:** The method undergoes extensive testing across datasets (CIFAR, ImageNet, and medical data) under both adversarial and clean conditions. Performance metrics like AUROC and AUPR are presented alongside ablations, providing a thorough examination.\n\n3. **Orthogonal Binary Layer:** The addition of an orthogonal layer to further separate equilibrium points of ID and OOD data adds robustness and addresses alignment issues in perturbed scenarios.", "weaknesses": "1. **Complexity of Mathematical Justifications:** The paper’s reliance on Lyapunov stability and NODEs introduces significant mathematical complexity. Equation derivations and stability proofs (such as those around equilibrium points) are not fully explained, making it challenging for readers without a control theory background.\n\n2. **Synthetic OOD Data Generation:** While the fake OOD generation in the feature space is practical, it lacks clarity on how well it approximates real-world OOD samples. Further analysis or empirical validation of this approximation would strengthen the argument for its effectiveness.\n\n3. **Loss Function Ambiguity:** The custom loss function, $L_{SL}$, is somewhat ambiguous in its practical effects on training stability. Further detail on parameter tuning (e.g., effects of $\\gamma$-values) and ablation studies on these would clarify its role in achieving robustness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)