Universal and Efficient Detection of Adversarial Data through Nonuniform Impact on Network Layers
Authors: Furkan Mumcu
Yasin Yilmaz
Venue: Accepted by TMLR ()

Abstract: Deep Neural Networks (DNNs) are notoriously vulnerable to adversarial input designs with limited noise budgets. While numerous successful attacks with subtle modifications to original input have been proposed, defense techniques against these attacks are relatively understudied. Existing defense approaches either focus on improving DNN robustness by negating the effects of perturbations or use a secondary model to detect adversarial data. Although equally important, the attack detection approach, which is studied in this work, provides a more practical defense compared to the robustness approach. We show that the existing detection methods are either ineffective against the state-of-the-art attack techniques or computationally inefficient for real-time processing. We propose a novel universal and efficient method to detect adversarial examples by analyzing the varying degrees of impact of attacks on different DNN layers. Our method trains a lightweight regression model that predicts deeper-layer features from early-layer features, and uses the prediction error to detect adversarial samples. Through theoretical arguments and extensive experiments, we demonstrate that our detection method is highly effective, computationally efficient for real-time processing, compatible with any DNN architecture, and applicable across different domains, such as image, video, and audio.

Published in Transactions on Machine Learning Research (06/2025)
Universal and Efficient Detection of Adversarial Data through Nonuniform Impact on Network Layers
Furkan Mumcu
furkan Busf. edu
Deportment of Electrical Engineering University of South Florida
Yasin Yilmaz
yasing Drssf. edu
Department of Electrical Engineering University of South Florida
Reviewed on OpenReview: oppnrevvee. net/ net/forum? id= OCY5APPFFI
Abstract
Deep Neural Networks (DNNs) are notoriously vulnerable to adversarial input designs with limited noise budgets. While numerous successful attacks with subtle modifications to origi- nal imput have been proposed, defense lechniques against these atlacks are relatively under- studied. Existing defense approaches either focus on improving DNN robustness by negating the effects of perturbations or use a secondary model to detect adversarial data. Although equally important, the attack detection approach, which studied in this work, provides more practical defense compared to the robustness approach. We show that the exist- ing detection methods are either ineffective against the state-of-the-art attack techniques or computationally inefficient for real-time processing. We propose a novel universal and efficient method to detect. adversarial examples by analyzing the varying degrees of im- pact of attacks on different DNN layers. Our method trains a lightweight regression model that predicts deeper-layer features from early-layer features, and uses the prediction error to detect adversarial samples. Through theoretical arguments and extensive experiments, demonstrate that our detection method is highly effective, computationally efficient for real-time processing, compatible with any DNN architecture, and applicable across different domains, such as image, video, and audio.
1 Introduction
Deep neural networks (DNNs) are known to be vulnerable subtle and manipulative noise for input data instances that can be designed by adversaries to cause erroncous outputs. Notably, Goodfellow et al. (2014) proposed a simple and effective method, called the Fast Gradient Sign Method (FGSM), to craft such adversarial instances by adding or subtracting small perturbation to each input dimension based on the sign of the gradient. After FGSM, various adversarial sample generation methods were demonstrated across different domains. However, compared to the vast diversity among attack techniques, there are not enough studies that effectively and efficiently detoct the increasing number of attacks various domains.
There are two main defense strategies. The more ambitious one aims to mitigate the effects of attacks (e.g.. correctly classifying adversarial images) by developing robust DNNs that are increasingly less vulnerable to adversarial data. As opposed to this idcalistic objective, the more practical second strategy aims to detect and discard the adversarial data. This binary classification approach (natural vs. adversarial), which we use in this work, naturally provides a more tractable defense strategy than trying to solve the original DNN problem with adversarial data.
One of the oldest and most effective defense techniques adversarial training for both the robustness Goodfellow et al. (2014) and detection Grosse et. al. (2017) in which the DNN is simply retrained using

---

Published in Transactions on Machine Learning Research (06/2025)
also the known adversarial instances. Although effective against known attacks, adversarial training suffers from high computational cost and unseen attacks. Similarly, other detection methods are typically only effective against some attacks. For instance, Metzen et al. (2017). in a supervised learning setup, trains a binary classifier for attack detection, which performs well on the attacks seen in training, but fails for the unseen allacks. recently proposed delector, EPSSAD Zhang et al. (2023), can detect unseen altacks by training only on natural data in a semi-supervisod anomaly detection setup at the expense of significant computational cost. Another recent detection method, VLAD Munnu & Yilmaz) (2024c), can also detect wide range of attacks by training a secondary model only natural data, however it is vulnerable to transferable attacks that also affect its secondary baseline model.
The existing adversarial data detectors in the literature are proposed for a particular application and do not naturally extend to other applications as they are based on domain-specific DNNs, e.g., adversarial image detection using convolutional neural networks Metzen et al. (2017) or diffusion models ZZhang et al. (2023). Motivated by the lack of a computationally efficient detector which can accurately detect a wide range of allacks in real-time, we propose universal and efficient detector, called Layer Regression (LR), that works together with DNNs in a variety of applications, including image recognition, video action recognition, and speech recognition. Our contributions can be summarized as follows:
We propose the first universal and efficient adversarial data detection method, which takes advantage of the nomuniform impacts of adversarial samples on different DNN layers.
We conduct extensive experiments for image recognition, and show that LR significantly outperforms existing elficient methods. Universality of LR is shown with its superior performance in detecting action recognition attacks and speech recognition attacks.
In addition to its high performance across a wide range of domains, models, and attacks, LR is also the most lightweight defense method. It is orders of magnitude faster than the existing defenses, making it ideal for real-time attack detection and resourco-constrained systems.
2
Related Work
2.1
Adversarial Attacks
The robustness of deep neural networks and their vulnerability to adversarial examples have been investigated since the introduction of FGSM (Goodfellow et al. 2014p. Numerous adversarial attacks have been proposed
to generate effective adversarial examples in recent years (Madry et al. 2017 Croce & Hein 2020b Kurakin
et al. 2018 Chen et al.) 2017 llyas et al.) 2018 Mumeu & Yilmaz| 2024a
Wang & He 2021 Fang et al.,
2024
Gao et al.) 2020). There are two main adversarial attack settings, namely white-box and black-box.
While it is assumed that the allacker has access to the larget model in the white-box setting, in the black-box setting, the attacker does not have any prior information about the target model.
White box attacks, including FGSM (Goodfellow al., 2014). PGD (Madry et al.) 2017 APGD Croce & Hein 2020b), generate adversarial examples by maximizing the target model's loss function and they are usually ly referred to as gradient-based attacks. BIM (Kurakin et al. 2018 tries to improve gradient-based attack by applying perturbations iteratively. Transferability based black-box attacks, which were introduced in Paapenno et al. (2017), among one of the most common black-box approaches. The idea is to use an attack on a substitute model for generating adversarial examples for unknown target models, utilizing the transferability of adversarial examples to different DNNs. While adversarial examples generated by these attacks are most effective when the substitute model exactly matches the target model as in a white-box attack setting, their success is shown to be transferable even when there is significant architectural differences between the substitute and target models. Wang & He (2021) introduced VMI and VNI to further extend iterative gradient-based al Lacks and try to achieve high transferability by considering the gradient variance of the previous iterations. PIF (Gao et al., 2020) uses patch-wise iterations to achieve transferability. ANDA (Fang et al. 2024) aims achieve strong transfcrability by avoiding the overfitting of adversarial examples to the substitute model. In addition, there are approaches that combine multiple attacks, such as Attakk Croce & Hein| 2020a), to test the robustness of models against diverse set of adversarial perturbations.
2

---

Published in Transactions on Machine Learning Research (06/2025)
2.2 Adversarial Defenses
Attempling to make changes on the input data for removing the effects of perturbalions from adversarial examples is the most common defense strategy. JPEG compression is studied in several works (Cucu et al. 2023 Aydemir et al. 2018 Das et al.) 2018), and it has been shown that compressing and decompressing helps to remove adversarial effects from input images. Xie et al. (2017) uses random resizing and padding on the inputs to eliminate the adversarial effects. Several denoising methods (Liao et al.] 2018: Xiong et al.,
2022 Salman et al., 2020) were proposed to remove adversarial perturbations from the inputs. Mustafa ct al.) (2019) uses wavelet denoising and image super resolution as pre- pre-processing steps to create a defense pipeline against adversarial attacks. Prakash et al. (2018) tries to eliminate adversarial effects by redistributing the pixel values via a process called pixel deflection. Adversarial training is another method which is studied to increase robustness of DNNs, however adversarial training often fails to perform well especially under various attack configurations (Bai et al.] 2021). Papernot & McDaniel (2018) proposes to increase the robustness of the model by applying k-nearest neighbors (kNN) classification to the feature representations at different layers of deep neural network. This method also implicitly utilizes the nonuniform impact of adversarial data on network layers, however their deep kNN method is not universally applicable to any deep neural network and not computationally efficient, unlike proposed LR method. On the other hand, several detection methods were proposed in recent years. Xu (2017b) introduces feature squeezing where they use bit reduction, spatial smoothing, and non-local means denoising to detect adversarial examples. Zhhan et al. 2023) tries to detect adversarial samples by computing an Expected Perturbation Score (FPS), which averages a sample's behavior over multiple perturbations generated using a pre-trained diffusion model. Yang ctt al. (2022) detects adversarial examples by identifying semantic contradictions by using a generator that reconstructs the input based on the network's internal feature representations, Pang al. (2018) proposed a training-time modification using reverse cross-cntropy to enforce separable feature representations for clean and adversarial inputs, which improves detection performance but requires modifying the training process. Similarly, Tian et al. (2018) introduced detection method based on prediction consistency under image transformations, leveraging the instability of adversarial samples under rotation or translation, whereas our method operates on internal fcature inconsistencies without needing imput perturbations or model retraining. A recent defense strategy which utilizes a baseline model, e.g., Vision Language Models (VLMs), with the assumption that the output of target model and baseline model are close to each other for clean input, but are far away from each other for adversarital input. A recent example is demonstrated by Mumeu & Yilmaz 2024c where they used CLIP (Radford et al. 2021 to detect adversarial video examples.
3 Detecting Adversarial Examples
Consider a Deep Neural Network (DNN) model g(-) that takes an input x and predicts the target variable y with g(x). As discussed in Section 2.2 there are three main defense strategies against adversarial attacks: adversarial training, modifying input, and detecting adversarial samples by monitoring changes in output with respect to a baseline. While the former two focuses on the changes in the input (Ir vs. zddr), the latter utilizes the changes in the output (g(z) vs. g(addu)). Our approach differs from these existing approaches by leveraging the nonuniform changes among different DNN layer activations. Instead of analyzing the input addv or the output g (( the proposed defense method analyzes the intermediate steps between x ado and g(xnde).
To develop universal detector that works with any DNN and against any attack, we start with the following generic observation. Although there are numerous attacks with different approaches to generate adversar- ial examples, all attacks essentially aim to change the model's prediction by maximizing the loss L (c.g.. cross-entropy loss) between prediction ((aad)) and the one-hot encoded ground truth y while limiting the perturbation Fang et al. b024
max L(g(zadu),) s.t. ol gade
x01100 SC
(1)
Considering this common aim of attack methods, "start with a small perturbation and end up with a big one", and the sequential nature of DNNs, we hypothesize that the impact of adversarial examples on the final layer
3

---

Published in Transactions on Machine Learning Research (06/2025)
is higher than the initial layer. Let us first define generic DNN g()) consisting of layers a {a1. Q2, ... ) In DNNs, including CNNs, transformers, etc., layers incrementally process the information from the previous layers to compute their respective outputs to the next layer. For example, for a model g where each layer is connected to the previous one, the final output of the model can be formulated as follows:
.(x) On (an-ann
(x))).
(2)
For simplicity, we will denote a layer's is output vector with (z))). Note that, the output of the last layer (x) = (x)) is typically the class probability vector in classification tasks, and 12) is referred as the leature vector of the model.
Our Conjecture and Remark below to build a universal and efficient detector are based on the assumption that, for an adversarial sample gddu the first layer output ai (2 remains close the clean version (2) since the change in the input is required to be unnoticeable by design, i.e,, I| 2: S...
Conjecture: Assume a loss function L(g(z), y) that is monotonic with (19(z) - yyloo Assuming highly ac- curate target model g that is trained by optimizing the weights w to effectively minimize the loss (( (()w(x))))
i.e.
g(x) = Iw- (x); where w" arg min L(gw (1).
(3)
w
we can rewrite Equation !
as
max ((((a)de)) s.t. I| - x1100 S g9d0
(4)
Note that (x) g(x), an (2ad)) = g(oadv), and L(g(zade), ) g(x)) is monotonic with lla(2adu) Hence, ||a, (z0dV) - on (2) is maximized while limiting |i 011o0 by a small number.
Since the first layer is right next to the imput, for which the perturbation is minèmized, and for away from the final layer, for which the impact is mazimized, we conjecture that, for n > 1,
d = (I - ai (x) < dn = =lan(zade) - a, (o)
(5)
According to this conjecture, the impact of an adversarial sample is higher on the final layer output than the first layer output. We next present how to utilize this for detecting adversarial samples.
Remark: Consider a function f which maps (1() to (x(x) and stable in the sense that (I
{(1 (z + +l) I 8 for small and 8. Since ||æade - =2 S from Equation [ assuming a (2ade) stays close to on (Ir), f(a (xdde)) )) is close to [(a(x)) due to the stability of f. From Equation 5 On (z is far away
from an (r) compared to the distance between (I()) and a1 (x=d1), Thus, the estimation error for adversarial samples is larger than the error for clean samples:
ea = (as (x - an (n )oo > Ec === - an (x)
(6)
3.1 Layer Regression
While Equation 5 provides the theoretical motivation, the result in Equation offfers a mechanism to detect adversarial samples if we can train a suitable function f (Figure D. We make four approximations to obtain practical algorithm based on Equation g First, for computational efficiency and real-time detection even in resource-constrained systems, we propose use a multi-layer perceptron (MLP) to approximate f. Second, since (z) denotes the predicted class probabilitics, we choose the feature vector an-1, which takes unconstrained real values, the target to train MLP as regression model. Third, approximate non con-differentiable Iloo with l12 to train the MLP using the differentiable mean squared error (MSE) loss. To empirically validate of Equation 5 under these three approximations, we use the ImageNet validation dataset, ResNet-50 He et al. (2015) as target model, and PGD attack Madry et al. (2017) to compute the (at""))))- mcan and standard deviation of normalized change in layer 1, dr [ea[ and layer n 1, d [a,, 1 (20d0)- - (m))la
I 1 1
4

---

Published in Transactions on Machine Learning Research (06/2025)
Ni (9)
f--
0.0344
n.7268
0,0918
Rocursey subility
Rg
(())()))
4:
010133
0.005
((N(A)
Fc
0.0747
0.450
prosimity
Eq. (9) d
(
4e (4
(G
Figure 1: (Left) Equation conjectures that the impact of adversarial samples is higher on the final layer than the first layer. Equation uses this to show that the error of a stable estimator is higher for adversarial samples compared to clean samples. (Right) The performance of approximations in proposed detector to Equation depend on two conflicting objectives: proximity of input vectors for clean and adversarial samples, and training an accurate and stable estimator. (Table) Empirical confirmation of the agreement between the proposed detector with approximations and the theoretical arguments.
(1(x).. ... an 2(xr
(a) Qg (2))
(mm(v), an-1 (x))
train
m
m(t)
0g
(x)
test i is adversarial if l is greater than threshold
(x)) (2((2()))) ... (om-1(0mm-110m 8m (a)m())))
Figure 2: (Left) Layer selection and slicing operations to form the input vector. (Right) Training and testing procedures of the proposed detector.
In deep neural networks with n > 1, training a suitable to estimate the feature vector dn-1 using the first layer output a₁ the input is challenging task due the highly nonlinear mapping in layers. To develop a lightweight detector via MLP, the fourth approximation, early layer outputs the input to the regression model instead of using only the first layer. Using mixture of 5th, 8th, and 13th convolutional layers in ResNet-50 under the same setting used for di and d by computing the mean and standard deviation of MSE é = IIf (x){)))))--))))))))()(()))) (z))(z) for clean images and é₂ = (f(a())"))) an-1(2"d=)) llz for adversarial images, where an MT.P with two hidden layers used for f. Results shown Figure I corroborate Equation 5 and Equation under the approximations. Input selection for MLP is further discussed in this section and Supplementary D
propose selecting mixture of
the input, empirically check Equation t
Utilizing the four approximations to Equation G discussed above, we propose universal and lightweight detection algorithm, as shown in Figure 2 with the following steps: (i) select subset of the first n layer vectors and generate a new vector v from the selected subset, (ii) feed v into a regression model m to predict the feature vector (x), (iii) train m by minimizing the mean squared error (MSE) loss f(m((v) (x)) the clean training set devoid of adversarial samples, (iv) use f(m(v), (x)) as the detection score. is expected to produce low scores for clean inputs and high for adversarial inputs. A pseudo-code of LR is given in Supplementary
5

---

Published in Transactions on Machine Learning Research (06/2025)
The formation of vector can be formed in various ways, such as using only the ith layer vector v = az(z) mixture of several layer vectors. To enable larger estimation error ea for adversarial samples than estimation error ee for clean samples, the choice for u should strike a balance between two competing goals, as illustrated in Figure 1
proximity of clean v(z) and adversarial v(z"d"), and accuracy and stability of cstimation function ʃ. While training an accurate and stable regression model is more feasible when v is selected from the layers closer to the target layer n 1,1, c.g., = an-2(a), such a detector might be less sensitive to adversarial samples since both (x) and (z) are expected to be impacted significantly by the attack, i.e, an-2(x) and (2-2(2)dr) will not be proximal. On the other hand, while selecting (1(x) ensures reasonably small perturbation in v, it also makes obtaining an accurate and stable estimator more challenging. As result, we propose to select a subset of layer vectors
= (r2()) (12) 4rm (rm(
(7)
where a, E a and m < n is the number of selected layers. From the selected layer vectors, we aim to generate a new vector v. However, since the layer vectors are often large due to the operations like convolutions or attentions, to get a specific portion of the selected layer vectors, we define a unique slicing function
= {s1, 82, 8 for each layer vector in dy. Then, each slicing function is applied to the corresponding layer vector in a, to get the sliced vectors
8, = (si (2(ar2(1)), Sm (arm (I))}-
(8)
Finally, the vector is generated by coucatenating the vectors in 8p:
v = (a.1 (ari 62(()2(x)). 8m (Arm (x))]
(9)
The order of (x())(xx)) in V can be randomized in inference to counteract adaptive attacks, as explained in Scction 4.4 The proposed layer selection and slicing process is summarizod in Figure 2 During the training, only the clean input samples are used. After the training, the loss is expected to be low for clean inputs and high for adversarial inputs. The implementation of Layer Regression publicly available
4 Experiments
Datascts and Evaluation: For the ovaluation ImageNet, 10,000 images from the validation set were chosen. For CIFAR-100, the test set, which also contains 10,000 images, was used. The commonly used area under the receiver operating characteristic (AUROC) curve is used to evaluate the attack detection performance of defense mothods. Similarly to existing work (Zhang et al.] 2023), the evaluation is conducted on adversarial and clean sets. The clean set comprises all images that are correctly classified by the target models. An adversarial set is formed for each attack-target model combination by gathering the attack's 's adversarial images misclassified by target model. More details on the evaluation methods and datasets are given in Supplementary
Baselines: We benchmark our method against a diverse range of baseline approaches, including six popular computationally efficient methods: JPEG compression (JPEG) (Das et al. 2018), Randomization (Ran- (Xie et al.) 2017), Deflection (Deflect) (Prakash et al. 2018), Feature Squeezing (FS) Xu 2017a)
Wavelet denoising (Denoise) and super resolution (WDSR) (Mustala et al., 2019), and two most recent state-of-the-art detection methods: VLAD (Mumeu Yilmaz) (024c) and EPS-AD (Zhang et al. 2023)
Since FS, VLAD and EPS-AD were proposed as detectors in their respective papers, we use their official implementations. The remaining methods were proposed to increase robustness by altering the inputs to remove perturbations. Therefore, we derive detection method from them by comparing the predictions before and after the methods are applied. The case the predictions match indicates no attack and a mismatch indicates an attack.
Target Models: The experiments are conducted using the validation set of ImageNet datasot (Russakovsky et al., 2015)) and the CIFAR- 100 dataset (Krizhessky et al.) 2009). To represent the different architectures we used three CNN-based image classification models, VGG19 SSmonyyn & Zisserman (2014), ResNet50
6

---

Published in Transactions on Machine Learning Research (06/2025)
He et al. 2015), InceptionV33 Szegedy et al. (2015); and three transformer-based models, ViT Dosovitskiy 2020 DeiT Touvron et al. 22021) LeViT Graham et al. (2021).
Attack Methods: In this section, we use the untargeted loo attack setting, however we also report our detector's performance under targeted and lz attack settings in Supplementary B We consider two different threat models in the experiments. The first one is the white-box static attack setting which the attacker has complete knowledge of the classifier but not the detector. Strong attacks from the literature, namely the BIM (Kurakin et al., 2018). PGD (Madry et al. 2017). PIF (Gao et al.] 2020), APGD (Croce & Hein 2020b), ANDA (Fang et al. 2024), VMI and VNI (Wang & He 2021) attacks are used as white-box attacks. We also test against an ensemble white-box attack, called AutoAttack (AA) (Croce & Hein 2020a
The second threat model is the adaptive atlack setting in which the attacker also has the full knowledge of defense mechanism. We analyze the performance of our detector against a specially crafted adaptive attack Yang al. (2022) which is trained to simultancously deccive the target model and bypass our detector in Section 4.4
LR training: For each model, an MLP with 2 hidden layers is trained as the LR. detector to demonstrate that highly effective detector can be built using a very lightweight neural network with minimal computational overhead. After experimenting with different layer selection and slicing strategies, we found that selecting only from early or final layers reduce the performance, but randomly selecting three layers and slicing the middle 60% portion each selected layer yields good results. The selection procedure of layers a, and slicing functions 8y, and training parameters are detailed in Supplementary F
4.1 Detecting White-Box Static Attacks
We conducted extensive comparisons with the existing conputationaall--ffiiien defenses. In Table | we report the AUROC scores for JPEG, Random, Defiect, Denoise, WDSR, FS and our method against BIM, PGD, PIF, APGD, ANDA, VMI and VNI attacks targeting 7 models, namely VGG19, ResNet50, Incep- tionV3, ViT, DeiT, LeVIT, on two datasets, ImageNet and CIFAR-100. In every experimental sctting, with different attacks, target models, and datasets, our proposed method outperforms these defense methods by wide margin. Compared to LR's average AUROC score of 0.98, the best performance among the existing efficient methods remains at 0.64. More importantly, our method is robust across varying target models and attack types, as indicated by its small standard deviation. While the other defense methods are effective against certain larget-attack combinations, they fail to generalize this to wide range of settings. For in- stance, on ImageNet, JPEG, Random, and FS perform better with transformer models, however they rarely exceod the random guess performance with the CNN models.
Due to the high computational requirements of VLAD and EPS-AD, we benchmark these methods in a smaller setting, where 1,000 images and 3 target models are used with the same 7 attacks and the results are given in Table 2 In all of the test cases, VLAD has an average AUROC score of 0.88. It performs worst against the attacks targeted ResNet50 where its performance varies between 0.82 and 0.77. On the other hand, similarly to the results demonstrated in Table [ LR proves its robustness to different attack and target combinations with the average AUROC score of 0.99. EPS-AD has a similar performance to our method where it successfully detects attacks with an average AUROC score of 0.99. However, compared to LR, both VLAD and EPS-AD have considerably large computational cost, which limits their real-world usage. In the next section, we discuss the computational efficienty and real-world applicability of all defense methods considered in this sention.
4.2 Computational Efficiency
Real-time attack detection is crucial aspect of many real-world systems. A detection mechanism must operate consistently alongside the DNN model to ensure timely identification of adversarial examples. Con- sequently, an ideal detector should be computationally efficient. In this section, we compare the computa- tional costs of the defense methods evaluated in our experiments. For each defense method, we processod 1,000 samples from ImageNet and calculated the average processing time per sample. Figure g presents the processing time per sample (PTS) in seconds vs. AUROC for each defense method. Our method is
7

---

Published in Transactions on Machine Learning Research (06/2025)
ImageNNo
CTFAR-100
JPEG
Roudoo
Delles
Denoise
WDSR
FS
LR [Ouns]
JPEG
Randon
Dellect
Denoise
WDSR
FS
LR (Ours)
HIM
0.42
0.43
0.48
0:45
0.39
0.13
0.99
0.47
0:31
0.40
0.41
0:01
0.991
PGD
0.50
0.46
0.48
0.45
0.46
0.22
0.99
0.48
0.31
0.49
0.48
0.44
0.02
0.99
VOG19
PIF
0.33
0:41
0.18
0:45
0.31
0.08
0.99
D:17
0.31
0.19
0.15
0:11
0.01
0.99
APGD
0.53
0.46
0.48
0.45
0.49
0.23
0.99
0.48
0.32
0.49
0.48
0.42
0.02
0.99
ANDA
0.39
0.47
0.19
0.47
0.38
0.26
0.95
0:18
0.36
0:19
0.19
0:43
0.11
0.99
VMI
0.36
0.42
0.48
0.45
0.34
0.11
0.99
0.47
0.32
0.49
0.48
0.41
0.01
0.99
VNI
0.43
0.45
0,18
0:46
0,38
0.19
0.99
D:47
0.32
0:49
0.98
0:42
0.02
0,99
RM
0.67
0.065
0.44
0:48
0.55
0.23
0.39
0.822
0:081
D:46
0.82
0.73
0.31
0.98
PGD
0.77
0.49
0.68
0.53
0.98
0.87
0.82
0.40
0.99
ReaNe150
PIF
0.51
0.6L
0.19
0:48
0.11
0.29
0.96
0.73
0.72
0.45
0.18
0.61
0.63
0.97
APGD
0.75
0.70
0.49
0.49
0.64
0.47
0.97
0.84
0.65
0.46
0.62
0.78
0.44
0.96
ANDA
0.45
0.19
0,41
0.14
0.96
0.36
0.45
0:45
0.99
VMI
0.49
0.52
0.49
0:48
0.45
0.14
0.99
0.61
0.58
0.45
0.50
0.50
0.23
0.99
VNI
0.53
0.54
0.49
0.49
0.47
0.21
0.97
0.62
0.35
0.45
0.50
0.51
0.28
0.97
RIM
0.55
0.53
0.50
0.77
0.24
0.98
0.88
0.43
0.74
0.81
0.36
1.00
PGD
0.62
0.58
0.50
0.49
0.50
0.36
0.97
0.86
0.66
0.43
0.76
0.81
0.32
0.99
Ircepiie
PIF
0.48
0.50
0.49
0.67
0.11
0.99
0.80
0.44
0.72
0.34
1,00
APGD
0.60
0.56
0.50
0.50
0.79
0.36
0.96
0.88
0.82
0.43
0.77
0.81
0.38
0.99
ANDA
0.19
0.30
0.50
0:49
0.52
0.23
0.92
0.71
0.57
0:43
0.50
0.59
0.29
0.99
VMI
0.48
0.50
0.67
0.12
0.98
0.80
0.35
1.00
VNI
0.32
0.53
0,50
0.50
0,70
0.25
0.96
0.86
0.79
0.43
0.60
0.81
0.31
1,00
BIM
0.87
0.90
0.53
0.64
0.82
0.93
0.99
0.50
0.66
0.50
0.50
0.49
0.20
0.98
PGD
0.85
0.53
0.79
0.91
0.99
0.48
0.27
0.97
PIF
0.79
0.82
0.52
0.51
0.67
0.87
0.99
0.49
0.53
0.50
0.49
0.48
0.15
0.98
VII
APGD
0.86
0.90
0.53
0.63
0.50
0.92
0.99
0.50
0.71
0.50
0.50
0.50
0.32
0.95
ANDA
0.52
0.64
0.97
0:48
0:26
0.85
VMII
0.77
0.81
0.51
0.57
0,70
0.86
0.99
0.49
0.58
0.50
0.19
0.48
0.15
0.99
VNI
0.78
0.83
0.52
0.61
0.73
0.91
0.99
0.50
0:60
0.50
0.50
0.49
0:27
0.95
BIM
0.86
0.52
0.78
0.88
0.99
0.50
0.64
0.50
0.48
0.36
0.92
PGD
0.56
0.88
0.53
0.58
0.50
0.90
0.99
0.50
0.66
0.50
0.49
0.48
0.34
0.87
PIF
0.77
0.51
0.71
0.62
0.99
0.47
0.16
0.97
Delf
APGD
0.85
0.52
0.57
0,78
0.68
0.99
0.50
0.65
0.50
0.50
0.48
0.36
0.91
ANDA
0.77
0.76
0.53
0.58
0.70
0.75
0.99
0.49
0.51
0.50
0.49
0.48
0.25
0.91
VMI
0.79
0.51
0.09
0.99
0.48
0.96
VNI
0.80
0.85
0.52
0.55
0.72
0.85
0.99
0:49
0.57
0.50
0.50
0:48
0:28
0.94
BIM
0.68
0.73
0.50
0.50
0.62
0.64
0.99
0:53
0.83
0.68
0.98
0.86
0.98
0.99
PGD
0.50
0.62
0.66
0.37
0.85
0.885
0.93
LAVIT
PIF
0.54
0.65
0.50
0.49
0.49
0.51
0.99
0.86
0.82
0.50
0.50
0.80
0.62
0.93
APGD
0.75
0.50
0.50
0.51
0.69
0.76
0.98
0.93
0.83
0.68
0.58
0.86
0.98
0.99
ANDA
0.49
0.51
0.50
0.49
0.43
0.94
0.84
0.71
0.52
0.57
0.81
0.65
0,93
VMII
0.53
0.39
0.50
0.19
0.48
0.39
0.99
0.93
0.83
0.66
0.66
0.88
0.98
0.99
VNI
0.00
0.00
0.50
0.51
0.54
0.59
0.99
0.522
0.83
0.67
0.71
0.85
0.90
0.99
Anerage
0.63
0.50
0.51
0.61
0.50
0.99
0.65
0.62
0.50
0.56
0.00
0.35
0.97
Table 1: Comparison between 7 computationally efficient defenses in terms of detection AUROC against 7
attack methods targeting 6 models using ImageNet and CIFAR-100 datasets.
BIM
PGD
PTF
APGD
ANDA
VMI
VNI
RosNet VGG19
VLAD
0.94
0.94
0.96
0.96
0.93
0.93
0.95
EPS-AD
0.99
0.97
0.97
0.99
0.98
0.99
0.99
LR (Ours)
0.99
0.99
0.99
0.99
0.99
0.99
0.99
VLAD
0.81
0.81
0.81
0.82
0.77
0.80
0.50
EPS-AD
0.99
0.98
0.98
0.99
0.99
0.99
0.99
LR (Ours)
0.99
0.99
0.98
0.97
0.96
0.99
0.97
VLAD
0.94
0.92
0.89
0.93
0.85
0.93
0.91
VIT
EPS-AD
0.99
0.97
0.98
0.99
0.99
0.99
0.99
LR (Ours)
0.99
0.99
0.99
0.99
0.99
0.99
0.99
Table 2: Comparison between state-of-the-arr defense methods in terms of detection AUROC on ImageNet.
the fastest, with a processing time of just 0.0004 seconds. In contrast, VLAD, WSDR, and EPS-AD are the slowest, with PTS values of 0,1431, 0.2611, and 2.9998 seconds, respectively. Our method demonstrates exceptional detection performance and also the fastest. Notably, our mothod runs 1.3 x 10 times faster than EPS-AD, which achieves similar detection performance to ours. The processing times are measured using a desktop computer with an NVIDIA 4090 GPU, AMD Ryzen 9 7950X CPU, and 64 GB RAM.
8

---

Published in Transactions on Machine Learning Research (06/2025)
1.0
AUROC
Deflect
0.7
VLAD
Denoise
WDSR
EPS-AD
Ours
10
10
10a
Processing Time
Figure 3: The proposed detector both universal (high AUROC across all scenarios) and lightweight (fastest among all cxisting defenses).
0.9
AUROC
Ours
EPS-AD
0.7
JPEG
Random
Deflect
VLAD
Denoise
0.5
WDSR
FS
4
8
16
32
64
128
E
Figure 4: Detection performance attack strength.
4.3 Performance under Varying Attack Strength
Adversarial attacks often use a parameter e to adjust the amount of perturbation on adversarial examples. For the experiments in Section [ we used the default attack settings that are proposed by the authors. In this study, investigate the performance of defense methods against PGD attack with different values. In the original implementation of PGD, attack strength parameter is set to 8. In addition to the original C, we also generated attacks by setting to 4, 16, 32, 64 and 128. Figure plots the performances of defense methods for ViT-PGD model-atlack pair. Our method performs the same for every e value, since LR depends on the perturbation's effects on DNN layers, a significant performance drop is not expected due to
the changes attack strength. E.S--D performs lower on weaker attacks, especially when their performance drops considerably. In remaining defense methods, while VLAD and Deflect experience only small changes under stronger attacks, whereas the others greatly affected by the value.

---

Published in Transactions on Machine Learning Research (06/2025)
Model
VGG19
ResNet
IncV3
ViT
DeiTT 0.99
LeViT 0.98
AUROC
0.99
0.97
0.96
0.99
Table 3: LR's performance against AutoAttack.
4
8
16
32
64
128
AUROC 0.81
0.78
0.79
0.80
0.81
0.81
Table 4: LR's performance under varying strengts of PGD-based adaptive attack where À is set to 1.
4.4 Performance against Adaptive Attacks
AutoAttack is an ensemble-based method combining multiple attacks. In recent works, it is used as an adaptive atttack (Zhang et al. 2023) to evaluate detectors. In Table 3 we provide LR's performance against AutoAttack targetting 6 models. Similarly to the results in Section 4.1 LR performs with an average of 0.98% AUROC score.
Since AutoAttack does not use any knowledge of the detector, we design PGD-based targeted adaptive atlack, following a similar approach to (Yang et al. 2022), with the objective function
max LClussifier A LLR-
(10)
where Cilassifier and LLR denote the loss functions of the target model and LR detector, respectively. The sign of LR's loss function is (-) since LR identifies an input clean when the loss is low. Note that, since this threat model has the complete knowledge of the detector, it is extremely challenging and destructive for the detector. To cope with this challenge, we propose randomly concatenating the layer vectors described in Equation equation g at inference time.
In Table i with VIT as the target model and the experimental settings proposed in YYan et al. 2022) where number of iterations are set as 200 and À is set as 1, we report LR's performance under adaptive PGD attack with varying attack budgets E [4, 8, 16, 32, 64, 128). LR achieves an average AUROC score of 80%, even under this challenging adaptive threat model
5 Applicability in Other Domains
LR a universal detector applicable in various domains where DNNs are used. Here, we demonstrate its performance in additional domains, video action recognition and speech recognition. In Supplementary
A and we provide more information about the experimental details used in this section. We also present additional results on another application, traffic sign recognition, in Supplementary H
5.1 LR for Video Action Recognition
In this section, we implement TRR against video action recognition attacks and compare its performance to existing defenses designed for action recognition models, namely Advit (Xiao et al. 2019 Shuffle ([wang
let al. 2023). and VLAD (Mumeu & Yilmaz 2024c). In the experiments, PGD-v atlack (Mumnu & Yilmas 2024c and Flick attack (Pony et al. 2021 are used to target MVIT (Fan et al. 2021) and X3D (Feichtenhofer
2020). The experimental settings in Mumcu & Yilmaz (2024c) on Kinetics-400 (Ky et al. 2017] dataset
are followed. The results in Table 5 show that LR outperforms the other defenses with an average AUROC
of 0.93%, followed by VLAD which achieves 0.91%.
5.2 LR for Speech Recognition
To demonstrate LR's wide applicability beyond computer vision, we also evaluate its performance against a speech recognition attack. As target model, we use Wav2vec (Schneider et al. 2019) model, which is trained on LibriSpeech (Panayotov al. 2015) dataset. Model attacked with FGSM [Goodfellow al. 2014). LR achieves average AUROC of 0.99. Figure ratessraaes some attacked and clean samples from LibriSpeech,
10

---

Published in Transactions on Machine Learning Research (06/2025)
Advit
Shuffle
VLAD
LR (ours)
PGD-v
0.93
0.98
0.93
0.99
MVIT
Flick
0.34
0.65
0.87
0.89
PGD-v
0.92
0.76
0.97
0.95
X3D
Flick
0.54
0.59
0.90
0.92
Average
0.68
0.74
0.91
0.93
Table 5: AUROC scores against attack methods targeting vidco action recognition models.
ADVERSARIAL
CLEAN
EEXPECT HAVE VERY ANDELLA SINCE
THEY WERE NOW PLAYING WITH THEIR DOLLS IN THE PARLOR
|EXPECT YOU HAVE BEEN VERY GOOD GIRLAND WITH YOUR LOL
AND MAMA THINKIN
THEY WERE NOW PLAYING WITH THEIR DOLLS IN THE PARLOUR
WER: 0.4
LR soore: 1.60
THE MANAGING CASE WAY PARTLY ON CONVINCING THE REASON OF THE CHILD AND PARTLY ON AN APPEAL TO HER AFFECTION
LADY LARKSPUR STARTS SUDDENLY AND TURNS TOWARDS HIM
LADY LARKESBOUR SUDDENLY AND TURNED TOWARD HIM
THE MOTHER E VANGIN EE TE PWAY FIN POTEON TE E EINE OF THECI AND POT ON THFIL PERFETEN
WER:
LR score: 4,42
WER: 5
LR score: 0.13
Figure 5: Clean and adversarial samples for speech recognition with ground truth in box, recognized text, word error rate (WER), and MSE of proposed LR.
along with the MSE values of LR. As shown in the figure, LR. even detect stealthy attacks which cause minimal increase in word error rate (WER) while distorting the recognized speech. Remarkably, while the WER of the first adversarial example is lower than that of the second clean example (0.4 vs. 0.5), the LR loss for this stealthy adversarial sample is more than ten folds greater than that the second clean example (1.6 vs 0.13).
6 Conclusion
Although there are effective defense methods for specific model-attack combinations, their success does not generalize to all popular models and attacks. In this work, filled this gap by introducing Layer Regression (LR), the first universal method for detecting adversarial examples. In addition to its universality, I.R is much faster than the existing defense methods. By analyzing the common objectives of attacks and the sequential layer-based nature of DNNs, we showed that adversarial samples have greater impact on the final layer than on the initial layers. Motivated by this, LR trains a lightweight multi-layer perceptron (MLP) on clean samples estimate the feature vector using a combination of outputs from early layers. The estimation error of LR for adversarial samples typically much higher than the error for clean samples. Through extensive experiments,
showed that LR outperforms the existing defenses in image recognition by wide margin in terms of speed and detection performance, and also delivers highly effective results in distinct domains, namely vidco action recognition and speech recognition. While inference time randomization of input order in LR is shown to be a promising technique for addressing adaptive attacks that are fully aware of LR, further research is needed to explore its full potential in this challenging scenario.
References
Ayse Elvan Aydemir, Alptekin Temizel, and Tugba Taskaya Temizel. The effects of jpeg and jpeg2000 compression attacks using adversarial examples. arXio preprint Xrxi 1003:803. 10418, 2018.
11

---

Published in Transactions on Machine Learning Research (06/2025)
Tao Bai, Jinqi Luo, Jun Zhao, Bihan Wen, and Qian Wang. Recent advances in adversarial training for adversarial robustness. or Xio preprind or arXin:2102.0133 2021.
Pin- Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order optimization based black-bos box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM workshop on artsficial intelligence and securily, pp. 26, 2017.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference machine learning, pp. 2206 2216. PMLR, 2020a.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pp. 2206 2216. PMLR, 2020b.
Adelina- VaalinaVaaleninaa Cucu, Giuseppe Valenzise, Daniela Stänescu, loana Ghergulescu, Lucian lonel Gäină, and Bianca Guşiţä, Defense method against adversarial attacks using jpeg compression and one-pixel attack for improved dataset security. In 2023 27th International Conference on System Theory, Control and IEEE, 2023.
Computing (ICSTCC), pp. 523-52 533-27..22
Nilaksh Das, Madhuri Shanbhogue, Shang- Tse Chen, Fred Hohman, Siwei Li, Li Chen, Michael E Kounavis, and Duen Horng Chau. Shield: Fast, practical defense and vaceination for deep learning using ipeg compression. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, pp. 196 204, 2018.
Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition scale, arXiv preprint ar (rXt:22000119920 2020.
Haoqi Fan, Bo Xiong, Karttikeya Mangalam, Yanghao Li, Zhicheng Yan, Jilendra Malik, and Christoph Feichtenhofer. Multiscale vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 1824-6835, 2021.
Zhengwei Fang, Rui Wang, Tao Huang, and Liping Jing. Strong transferable adversarial attacks via en- sembled asymptotically normal distribution learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14841-24850, 2024.
Christoph Feichtenhofer. X3d: Expanding architectures for efficient video recognition. In Proccedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 203-213, 2020.
Lianli Gao, Qilong Zhang, Jingkuan Song, Xianglong Liu, and Heng Tao Shen. Patch-wise attack for fooling deep neural nelwork. In Computer Vision ECCV 2020: 16th European Conference, Glasgow, UK, August 23 23-28, 28, 2020, Proceedings, Port XXVIII 16, pp. 307 322. Springer, 2020.
Tan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arxiv preprint
2014.
Benjamin Graham, Alaaeldin El-Nouby, Hugo Touvron, Pierre Stock, Armand Joulin, Hervé Jégou, and Matthijs Douze. Levit: a vision transformer in convnet's clothing for faster inference. In Proceedings of the IEEE/CVE international conference on computer vision, pp. 12259 12269, 2021.
Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, and Patrick McDaniel. On the (statistical) detection of adversarial examples. arXiv preprint arXiv: 702. 762.008000000 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. arxiv e-prints. arXiv preprint arXf::1122003388 10, 2015.
Tang-FFag Hsiao, Bo-Lun Huang, Zi- Xi-Xiang Ni, Yan- San-Ting Lin, Hong- Han Shuai, Yung Hui Li, and Wen- Wen-Huang Cheng. Natural light can also be dangerous: Traffic sign misinterprotation under adversarial natural light attacks. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3915-3924, 2024.
12

---

Published in Transactions on Machine Learning Research (06/2025)
Jachui Hwang, Huan Zhang, Jun-Ho Choi, Cho-Jui Hsich, and Jong-Scol Lec. Temporal shuffling for defending deep action recognition models against adversarial attacks. Neural Networks, 2023.
Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-box adversarial attacks with limited queries and information. In International Conference Machine Learning, pp. 2137-111222221 2146. PMLR, 2018,
Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics human action video dataset. arXiv preprint crXtt:705500695 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
A Alexey Kurakin, Tan J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. Tn Artificial intelligence safety and security, pp. 99 112. Chapman and Hall/ 2018.
Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu. Defense against adversarial attacks using high-level representation guided denoiser. In Proceedings of the IEEE conference on computer vision and pottern recognition, pp. 1778-1755 2018.
Aleksander Madry, Alcksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial atlacks. arXiv preprint 1rXic:1706 06080, 2017.
Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff. On detecting adversarial pertur- bations. XrXiu preprinl arXiv: 172::172.0000 04267, 2017.
Furkan Mumeu and Yasin Yilmaz. Sequential architecture-aggostic black-box attack design and analysis. Pattern Recognition, 147:110066, 2024a.
Furkan Mumeu and Yasin Yilmaz. Fast and lightweight vision-language model for adversarial traffic sign detection. Electronics, (11))21117, 2024b.
Furkan Mumcu and Yasin Yilmaz. Multimodal attack detection for action recognition models. In Proceedings of the IEFE/CVE Conference on Computer Vision and Puttern Recognition, pp. 2967-2976, 2024c.
Furkan Mumcu, Keval Doshi, and Yasin Yilmaz. Adversarial machine learning allacks against video anomaly detection systems. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pottern Recog- nition, pp. 206 213, 2022.
Aamir Mustafa, Salman H Khan, Munawar Hayat, Jianbing Shen, and Ling Shao. Image super-resolution as a defense against adversarial attacks. IEEE Transactions on Image Processing, 29:1711-1224, 2019.
Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 5206-52111 TEEE, 2015.
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Towards robust detection of adversarial examples. In NewrIPS, 2018
Nicolas Papernot and Patrick McDaniel. Deep k-nearest neighbors: Towards confident, interpretable and robust deep learning. Xi preprint orXiu:1803. 765, 2018.
Nicolas Papernot, Patrick McDaniel, Tan Goodfellw,, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications securily, pp. 506 519, 2017.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
Roi Pony, Itay Nach, and Shie Mannor. Over-the-air adversarial flickoring attacks against video recognition networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 515-524 2021.
13

---

Published in Transactions on Machine Learning Research (06/2025)
Aaditya Prakash, Nick Moran, Solomon Garber, Antonella DiLillo, and James Storer. Deflecting adversarial attacks with pixel deflection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.
2018.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp. 2021.
PMLR,
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211-25.. 2015. doi: 10.1007/s1 110007/s1123--01-0881666
Hadi Salman, Mingjie Sun, Greg Yang, Ashish Kapoor, and Zico Kolter. Denoised smoothing: A provable defense for pretrained classifiers. Advances Neural Information Processing Systems, 33:21945-21957
2020.
Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli. wav2vec: Unsupervised pre-training for speech recognition. arXiv preprinl Xrxi:: 05862, 2019.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint 00t::::449.150000 2014.
J. Stallkamp, M. Schlipsing, Salmen, and C. Igel. Man vs. computer: Benchmarking machine learning al-
gorithms for traffic sign recognition. Neural Networks, (0):-, 2012. ISSN 0893-6080. doi: 10.1016/j. neunet. 2012.02.016. URL http::/www www.sscciiieicceeeeeeeeeeeeeeeee htt.://cwc................oooo/a
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanct, Scott Reed, Dragomir Anguclov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1-9, 2015.
Yu Tian, Xiaopeng Yang, and Yuanqing Cai. Detecting adversarial examples through image transformations. In AAAT, 2018.
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-cfficient image transformers distillation through attention. In International conference on machine learning, pp. 10347-100 10357. PMLR, 2021.
Du Tran, Heng Wang, Lorenzo Torresani, and Matt Feiszli. Video classification with channel-separated convolutional networks. In Proceedings of the IEEE/ CFF international conference on computer vision, pp. 5552 5561, 2019.
Xiaosen Wang and Kun He. Enhancing the transferability of adversarial attacks through variance tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1924-1933. 2021.
Ross Wightman. Pytorch image models. htttpe////hhhhbb.cm////gtaaaaare
2019.
Chaowei Xiao, Ruizhi Deng, Bo Li, Tacsung Lee, Benjamin Edwards, Jinfeng Yi, Dawn Song, Mingyan Tiu, and Tan Molloy. Advit: Adversarial frames identifier based on temporal consistency in videos. Tn Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 3968 3977, 2019.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. arXiv preprint arXiv: 1711 1711.01991, 2017.
Zikang Xiong, Joe Eappen, He Zhu, and Suresh Jagannathan. Defending observation attacks deep rein- forcement learning via detection and denoising. In Joint European Conference on Machine Learning and Knowledge Discovery in Datahases, pp. 235 250. Springer, 2022.
14

---

Published in Transactions on Machine Learning Research (06/2025)
W Xu. Feature squeezing: 1rXiv:104 155, 70.
Detecting adversarial exa mples in deep neural networks. arXiv preprint
W Xu. Feature squeezing: Detecting adversarial exa mples in deep neural networks. orXiv preprint arXiv:174..00 01155, 2017b.
Yijun Yang, Ruiyuan Gao, Yu Li, Qiuxia Lai, and Qiang Xu. What you see is not what the network infers: Detecting adversarial examples based on semantic contradiction. arXiv preprint urXiv:2201.09650, 2022.
Bin Ye, Huilin Yin, Jun Yan, and Wanchen Ge. Patch-based attack on traffic sign recognition. In 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pp. 164 171. IEEE, 2021.
Piotr Zelasko, Sonal Joshi, Yiwen Shao, Jesus Villalba, Jan Trmal, Najim Dehak, and Sanjeev Khudanpur. Adversarial attacks and defenses for speech recognition systems. arXiv preprint ar 17122, 2021.
Shuhai Zhang, Feng Liu, Jiahao Yang, Yifan Yang, Changsheng Li, Bo Han, and Mingkui Tan. Detecting adversarial data by probing multiple perturbations using expected perturbation score. In International conference on machine learning, pp. 41429 41451. PMLR, 2023.
Yue Zhao, Hong Zhu, Ruigang Liang, Qintao Shen, Shengzhi Zhang, and Kai Chen. Seeing isn't believing: Towards more robust adversarial attack against real world object detectors. In Proceedings of the 2019 ACM SIGSAC conference on compuler and communications security, pp. 1989 2004, 2019.
15

---

Published in Transactions on Machine Learning Research (06/2025)
A More Related Work from Other Domains
Adversarial examples are studied on several other domains including vidco action recognition (Pon et al. 2021j, traffic sign recognition (Hsiao et al.] 2024). object detection (Zhao et al. 2019, video anomaly detection (Murncu et al. 2022 and speech recognition (Zelasko et al. 2021). For instance, flickering attack
(Flick) (Pony al., 2021 tries to attack action recognition models by changing the RGB stream of the
input videos. |Hsiao al. 2024 investigates the effects of natural light on traffic signs and to generate
adversarial examples.
Advit (Xiao et al. 2019) is detection method introduced for vidcos. It generates pscudo frames using optical flow and evaluates the consistency between the outputs for original inputs and pseudo frames to detect attacks. Another defense method, Shuffle (Hwang et al. 2023), tries to increase the robustness of action recognition models by randomly shuflling the input frames. (Mumcu Yilmaz 2024b) proposed to train lightweight VLM and use it for adversarial traffic sign detection. In speech recognition, defenses like denoising or smoothing are studied against adversarial examples Zelasko al. 2021
B LR's Performance Against 12 and Targeted Attacks
PGD targeted
PGD1,
APGDeareeted
APGD
ResNet50
0.99
0.95
0.98
0.95
VIT
0.99
0.95
0.98
0.96
Table 6: LR's performance under la and targeted attack settings.
We evaluate LR's performance under la and targeted attack settings, specifically against PGD targeted:: PG APGDtargeted and APGD;, attacks using two different target models ResNet50 and ViT on Ima- geNet. The results are reported Table f Similar to the white-box atlack selting, LR performs an average AUROC score of 97% which further demonstrates the success of our detection method in different types of attacks.
C
More Details on Experimental Settings
From the validation set of ImageNct (Russakovsky et al., 2015), used 40,000 images to train the detectors and the remaining 10,000 for testing. In CIFAR-100 (Krizhevsky el al. 2009), there are 100 classes, 500 training images and 100 testing images per class, resulting in total of 50,000 training and 10,000 test images. During the tests, only the correctly classified images by the target models were used. Total number
for test images for each specific model and dataset is given in Table
VGG19
ResNet50
InceptionV3
VIT
DefT
LeVIT
ImageNet
4257
7701
7410
8457
7907
7639
CIFAR-100
7045
8315
8089
8544
8667
8359
Table 7: Number of test images used for each model.
VLAD Muuncu & Yilmaz 2024c was originally proposed for video recognition attacks, with the initial
implementation designed for videos consisting of 30 frames. In the experiments conducted in Section g we used the official VLAD implementation. However, instead of averaging the scores across 30 frames, we applied it to a single image.
Effects of Layer Vector Choice
For the experiments Section we used three layer vectors to form v, i.c., m = 3 in Eq. 9 Since the number of possible combinations for layer selection and slicing creates an intractably large search space, we limited our optimization to representative subset of options. Table summarizes the results of preliminary
16

---

Published in Transactions on Machine Learning Research (06/2025)
experiment CIFAR-100, which is conducted with DeiT as the target model and PGD and PIF the adversarial attacks. In this experiment, we considered 25 attention layers of the model and trained our detectors by forming in four different ways: (1) using only the first attention layer vector di, (2) using only the last attention layer vector 025, (3) combining the vectors from fifth, sixth, and seventh attention vectors (0s, (G 47], (4) and combining the vectors from eighth, thirleenth, and seventeenth attention vectors (as; 013; 017]. The best strategy turns out combining vectors from early layers (as, 0s 07] as it strikes a good balance in the trade-off between the two conflicting goals (Figure [ proximity of v(x) and v(2"d)), and accuracy and stability of estimator f. Additionally, having multiple layers and slicing functions can create randomness in each LR implementation, which makes developing attack against our detection system harder. The layer selection and slicing strategies that gave the best results reported in Table are listed in Supplementary F
Attack
01
[as, Go a7l
[as, 413; Ø17]
0n 2
PGD
0.697
0.867
0.751
0.567
PIF
0.744
0.972
0.746
0.521
Table 8
Impact of layer selection on detection score.
E Layer Vector Selection Procedure
According to our preliminary experiments, using vectors exclusively from either early or final layers nega- tively impacts performance. Therefore, we propose selecting three layer vectors randomly between 1/5 and 4/5 of all layers. The proposed procedure for selecting layer vectors is detailed in Algorithm
Algorithm Layer Vector Selection Procedure
1: Input: all layer vectors Vny
Output: selected layer vectors Um-
Get the layer size of all layer vectors Un len(v,)
Decide upper and lower bounds:
6: a nnn/
7: b + 4-/
8: Slice the vectors between the upper and lower bounds:
9: Um + Un [a :6]
10: Randomly pick vectors from Un:
11: Um <- randomm.choices((m:
12: Return Um
F Selected Layer Vectors and Slicing Functions
A specific subset. of layer vectors ar, as described in Equation equation 7 is chosen for each larget model that is used during the experiments in Section For the models, Pytorch Image Models (timm) (Wightman 2019) used.
We used the following layer subsets for the target models:
Resnet50: Layers with the name conv2 were filtered, then among 15 conv2 layers, 5th, 8th and 13th layers, for all ImageNet and CIFAR-100 tests
InceptionV3: Layers with the name comv were filtered, then among 94 cone layers, 15th, 25th and 35th layers, for all ImageNet and CIFAR-100 tests
VGG19:
Layers with the name features were filtered, then among 37 features layers, 8th, 13th and 17th layers, for all TmageNet and CIFAR-100 tests
17

---

Published in Transactions on Machine Learning Research (06/2025)
4. VIT: Layers with the name attn.proj were filtered, then among 23 attn.proj layers, 8th, 13th and
17th layers, for all ImageNet and CTFAR-100 tests
DefT: Layers with the name aton.proj were filtered, then among 24 attn.proj layers, 8th, 13th and
17th layers for ImageNet tests, 5th, 6th, 7th layers for CIFAR-100 tests
6. LeViT: Layers with the name proj. were filtered, then among 12 aton.proj layers, 3th, 5th and 7th layers for ImageNet tests, 5th, 6th, 7th layers for CIFAR-100 tests
Before concatenating the vectors, specific slicing function for each vector is applied as described in The specific slicing functions for each corresponding a, are detailed below:
Equation equation
Resnet50: 5, 28,: 28], 50, : 7, 7] and [: 10, 14, 14].
[: 3,; 17, 17]. 2. InceptionV3: 3 35, 35), [3 3, :, 35] and
3. Vgg19: [: 5,; 25, 25],
5, 25, 25] and [: 5, 25,; 25).
4. VIT: [;: 4: 200],
;:: 4 200] and
[:. 4 200).
DeiT: [5: 4, 200),
[+: 4, 200] and
[ 4, 200).
6. LeViT [: 4,: :, 14), [ 14, 7, 7] and |: 14, 7, 7].
The vector v is generated by concatenating the sliced or vectors as described in Equation equation After acquiring for model, an MLP with two hidden layers is trained to minimize the MSE loss between v and the feature vector 0n-1- Adam optimizer with 3 10 learning rate is used for the training.
G Pseudo-Code for LR
In Section
3.1
explain detection algorithm in detail. Here, in Algorithm 2 also provide PyTorch
Paszke et al.
2017 like pseudo code for LR.
Algorithm 2 Layer Regression (LR)
Input: input x, selected subset of vector layers ar, slicing functions 8, DNN model g, feature vector
an-1, LR detector m.
Output: loss I if not training mode.
DNN lakes the input as g(x)
4: Fach layer vector ar, process with corresponding slicing function in
5: for a; in a, do
6: S, t s (z))()))
end for
u u+ torch.cat(s,)
feed into a delection model m, such that: m(v) 10: calculate the MSE loss between m(v) and an-1
11: It MSE(m((V),c an))
12: if training then
13: Lbackward()
14: optimizer.step()
15: clsc
16: Return
17: end if
18

---

Published in Transactions on Machine Learning Research (06/2025)
H
Additional Study: LR in Traffic Sign Detection
FGSM
PGD
Patch
Light
Average 0.96
LR
0.97
0.99
0.95
0.94
Table 9: Detection AUROC of LR against four attacks targeting ResNet50 the traffic sign recognition task.
As additional experiment, we implement LR against attacks that target traffic sign recognition. In this study, ResNet50 is used target model and attacked with FGSM[Gooddelloo et al. 2014p, PGD(Madry ett al. 2017), Light (Hsiao et al.] 2024) and Patch (Ye et al. 2021) attacks. In Table 9 we show that LR achieves an average AUC score of 96%, which further proves the applicability and success of our detection method across different domains.
Details of Ablation Study & Other Domain Experiments
For Section 5.1 the experimental settings detailed in Mumcuu & Yilmaz (2024c] is followed: 'A subset of Kinetics-400 (Kay et al. 2017) ) is randomly selected for each target model from the videos that are correctly classified by the respective model. For cach subset, the total number of the vidcos are between 7700 and 8000 and each class has least
most instances. An adversarial version of the remaining 20% portion is generated with each adversarial attack, in way that they cannot be correctly classified by the models. Then the adversarial set is used for evaluation along with the clean versions." Similarly to main experiments, trained an MLP with 2 hidden layers with the same training hyper-parameters described in Supplementary For MVTT (Fan et al. 2021) and CSN (Tran et al. 2019) models, layers which have the name cone and proj were filtered respectively. While for both of the models 3th 5th and 7th layers used to form subset Oy; for CSN (Tran et al.] 2019) additional layers 4th, 6th and Sth were also used. [:, 4, 200] and i.: 4, 3, 7, 7] were used as slicing functions for MVIT (Fan et al. 2021 and CSN (Tran et al. 2019) respectively.
For Section 5.2 an MLP with 2 hidden layers with the same training hyper-parameterss described in Sup- plementary is trained. For Wav2vec (Schneider et al. 2019) model, layers which have the name atten- droport were filtered, and 3th layer used to form ar. As slicing function [ 4, 20, 10] is selected. While 500 sound clips from LibriSpeech dataset used for experiments, 2000 sound clips saved for training.
For Supplementary H we use the experimental settings as demonstrated in Mumeu Yilmaz (2024b) and use the GTSRB (Stallkamp al.) 2012) dataset which includes 43 classes of traffic sigus, split into 39, 39,209 176 training images and 12,630 test images. LR detector is trained with the same settings that are specified for Resnet50 in Supplementary
J Visualized adversarial examples
(Attack Strength)
clean
0.01
0.12
0.15
Figure 6: Effects of different attack strength values on a sample. The attack strength parameter e is increased in the direction of the arrow. In the first sample, there no attack. Samples are generated with adversarial attack PGD (Madry et al. 2017) and target model ResNet50 (He et al. 2015
19

---

Published in Transactions on Machine Learning Research (06/2025)
Clean
BIM
PGD
PIF
APGD
ANDA
VNI
Sy !
Sp
Figure 7: Adversarial examples generated with BIM (Kurakin et al. 2018 PGD (Madry et al. 2017), PIP
Gao et al. 2020). APGD (Croce Hein 2020b), ANDA (Fang et al. 2024 VMI and VNI (Wang & He 2021 by altacking larget model ResNet50
(He et al. 2015) First column represents the clean samples.
Clean
BIM
PGD
APGD
ANDA
VNI
Adversarial Perturbations
Figure 8: Visualization perturbations generated with BIM (urakin et al. 2018 PGD (aadry et al. 2017 PIF Gao et al. 2020), APGD (Croce & Hein 2020b), ANDA (Fang et al. 2024), VMI and VNT Wang & He 2021 by attacking target model ResNet50 (He et al. 2015). First row shows the clean sample and corresponding adversarial example for each attack. Second row demonstrates the noises added to the clean sample by each attack.
20


Reviews:
[{"review_id": "RlJDqrb9Dr", "reviewer": "Reviewer_SVYA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* The proposed (LR) approach exhibits universality, demonstrated through various experiments.\n*  The method offers reduced computational costs compared to existing approaches, making it more feasible for real-time applications.\n* The observations that motivated the LR framework contribute to the novelty of this work.\n* The experimental results are encouraging.\n\n\nWeaknesses:\n* While the paper presents a solid foundation, there is room for improvement in terms of presentation. Minor errors in grammar and consistency detract from the overall flow. I recommend revisions to address these issues.\n* To further strengthen the paper's contribution, I suggest exploring connections to existing work in this domain, such as Towards Robust Detection of Adversarial Examples by Pang et. al and Detecting Adversarial Examples Through Image Transformation  by Tian, Yang and Cai.\n* The authors demonstrate the LR method's versatility by applying it to video and speech recognition tasks. Considering its potential applicability to natural language processing (NLP) domains, particularly with the increasing adoption of large language models (LLMs), I encourage the authors to explore this area further.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses the challenge of identifying adversarial attacks in machine learning models. Specifically, it proposes a Layer Regression (LR) framework for detecting such examples. The LR method involves training a lightweight multi-layer perceptron (MLP) on clean data to estimate the output of each layer in a deep neural network (DNN). The authors demonstrate that adversarial samples have a disproportionate impact on the final layers of a DNN compared to earlier layers. By quantifying this difference, the LR approach can accurately identify adversarial examples. The authors then validate their method through extensive experiments and show its applicability to video action recognition and speech recognition tasks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I encourage the authors to address the first two weaknesses listed above. This should definitely strengthen this work further.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "EHBbtkB1y6", "reviewer": "Reviewer_942J", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "S1: I like this section on adaptive attacks. I think it's smart.\n\nS2: Figure 3 is compelling.\n\nS3: I think the paper is clear and well written. I like the way that it presents the conjecture before leading into the experiments.\n\nS4: I really like section 5, showing that this works on other domains.\n\nW1: this is a pretty minor concern. The paper I think the paper does a reasonable job of testing against baselines. However, in 2025, I admit I am a little bit surprised that stronger baselines don't exist or that something very similar to this hasn't already been done. In the related works section, I counted five mentions of papers that worked on prior detection method for Lp norm attacks. Consider me a little bit surprised if that's all there is. \n\nW2: I think the biggest limitation of this paper is that it just focuses on Lp norm attacks. In the vision domain, these attacks are the most studied, but not necessarily the most hazardous type of attack against vision systems in the real world. Meanwhile, I think section 5 of the paper is really good. But overall, I think it's worth questioning the extent to which this paper is going to be very practically useful and interesting in 2025. I think it can be. But I think that the most compelling type of work on detecting adversarial attacks in 2025 is work that focuses on language models. Overall, I think that this paper does a good job and I think it's probably worth accepting. But someone who works in the field, I don't feel as if I have learned something very unexpected or important from reading this paper.\n\nW2:", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a method to detect Lp norm attacks against neural networks. The motivation for this method is that Lp norm attacks makes small perturbations to inputs but caused large changes to outputs. This led the authors to hypothesize that the magnitude of changes to a network activations increase across the network layers. Based on this, they introduce they are regression to identify these attacks using this heuristic. In tests, it works well against baselines.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "No", "audience_explanation": "", "requested_changes": "I actually have very little to say here. I think the authors put things together very well. One thing that I would recommend would be to update the abstract to add a little bit more detail about how the method works. I just think that would be helpful.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "EvXmdWjbeE", "reviewer": "Reviewer_jSMN", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: The motivation is clear, and the perspective to detect adversarial examples via the conjecture that adversarial perturbations amplify as they propagate through the layers of a neural network seems interesting.\n\nWeaknesses: 1. The writing needs to be significantly improved. 2. Lack of rigor, the conjecture (that adversarial perturbations amplify as they propagate through the layers of a neural network) holds in most neural networks, but cannot be guaranteed. 3. Lack of justifications for/rationale behind the proposed method's setting, like the number of MLP layers and the subset of layers for the MLP layers' input.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces a universal and efficient defense method against adversarial attacks. The proposed defense method is based on the conjecture that adversarial perturbations amplify as they propagate through the layers of a neural network.\n\nSpecifically, latent features extracted by multiple layers from the target model are fed into this MLP layer, and then the 'distance' between the output of this MLP layer and the feature vectors extracted by the target model is computed. If this  'distance' is larger than a predefined threshold, then the input is an adversarial sample.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- In Eq.(3), the left-hand $g(x)$ denotes the vector of class probability (after softmax layer?), the right-hand $argmin_{w}$ returns the desired model parameters $w^\\ast$, so it is inappropriate to say $g(x)=argmin_{w}$.\n- In Eq.(4), what if $g(x)$ would not lead to the predicted label to be $y$? i.e., misclassification cases.\n- For Eq.(5), adversarial perturbations amplify as they propagate through the layers of a neural network, which holds in most neural networks, but cannot be guaranteed. For example, if we have a 2-layer linear network $g(x)=W_2 W_1 x$ and $||W_2||<1, ||W_2||<1$. It would be better to be more rigorous.\n- The rationale behind the selection of the mixture of early layer outputs (how to decide which layer should be selected) is not clear.\n- Why not use KL divergence distance to measure the 'distance'?\n- Why choose MLP with 2 hidden layers? What if the adversarial attacks consider to attack MLP as well?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)