Embodied Active Defense: Leveraging Recurrent Feedback to Counter Adversarial Patches
Authors: Lingxuan Wu
Xiao Yang
Yinpeng Dong
Liuwei XIE
Hang Su
Jun Zhu
Venue: ICLR 2024 poster (2024)
Keywords: Adversarial patches
active defense
embodied intelligence
Primary area: societal considerations including fairness, safety, privacy

Abstract: The vulnerability of deep neural networks to adversarial patches has motivated numerous defense strategies for boosting model robustness. However, the prevailing defenses depend on single observation or pre-established adversary information to counter adversarial patches, often failing to be confronted with unseen or adaptive adversarial attacks and easily exhibiting unsatisfying performance in dynamic 3D environments. Inspired by active human perception and recurrent feedback mechanisms, we develop Embodied Active Defense (EAD), a proactive defensive strategy that actively contextualizes environmental information to address misaligned adversarial patches in 3D real-world settings. To achieve this, EAD develops two central recurrent sub-modules, i.e., a perception module and a policy module, to implement two critical functions of active vision. These models recurrently process a series of beliefs and observations, facilitating progressive refinement of their comprehension of the target object and enabling the development of strategic actions to counter adversarial patches in 3D environments. To optimize learning efficiency, we incorporate a differentiable approximation of environmental dynamics and deploy patches that are agnostic to the adversary’s strategies. Extensive experiments demonstrate that EAD substantially enhances robustness against a variety of patches within just a few steps through its action policy in safety-critical tasks (e.g., face recognition and object detection), without compromising standard accuracy. Furthermore, due to the attack-agnostic characteristic, EAD facilitates excellent generalization to unseen attacks, diminishing the averaged attack success rate by 95% across a range of unseen adversarial attacks.

Published as a conference paper at ICLR 2024
EMBODIED ACTIVE DEFENSE: LEVERAGING RECURRENT FEEDBACK TO COUNTER ADVERSARIAL PATCHES
Lingxuan Wu', Xiao Yang"*, Yinpeng Dongi.2, Liuwei Xie', Hang Su', Jun Zhu 1 Dept. of Comp. Sci. and Tech., Institute for AI, Tsinghua-Bosch Joint MI Center, THBI Lab, BNRist Center, Tsinghua University. Beijing, 100084, China 2 RealAA
(x1x23, yo 9) @mammmm...ee.. to..ghh.......ooo....oo cdu ixteargouttlook
idpeny, suhangss, desz @mmail Lwwnghun..neee...............i..
ABSTRACT
The vulnerability of deep neural networks to adversarial patches has motivated nu- merous defense strategies for boosting model robustness. However, the prevailing defenses depend on single observation or pre-established adversary information to counter adversarial patches, often failing to be confronted with unseen or adaptive adversarial attacks and easily exhibiting unsatisfying performance in dynamic 3D environments. Inspired by active human perception and recurrent feedback mecha- nisms, we develop Embodied Active Defense (EAD), proactive defensive strategy that actively contextualizes environmental information to address misaligned ad- versarial patches in 3D real-world settings. To achieve this, EAD develops two central recurrent sub-modules, i.e., a perception module and a policy module, implement two critical functions of active vision. These models recurrently process a series of beliefs and observations, facilitating progressive refinement their comprehension of the target object and enabling the development of strategic actions to counter adversarial patches in 3D environments. To optimize learning efficiency, we incorporate differentiable approximation of environmental dynam- ics and deploy patches that are agnostic to the adversary's 's strategies. Extensive experiments demonstrate that EAD substantially enhances robustness against a variety of patches within just few steps through its action policy in safety-critical tasks (e.g., face recognition and object detection), without compromising standard accuracy. Furthermore, due to the attack-agnostic characteristic, EAD facilitates excellent generalization to unseen attacks, diminishing the averaged attack success rate by 95% across a range of unseen adversarial attacks.
INTRODUCTION
Adversarial patches (Brown et al., 2017) have emerged as a prominent threat to the security of deep neural networks (DNNs) in prevailing visual tasks (Sharif et al., 2016; Thys et al., 2019; Xu et al., 2020; Zhu et al., 2023). These crafted adversarial patches can be maliciously placed on objects within scene, aiming to induce erroneous model predictions real-world 3D physical environments. As result, this poses security risks or serious consequences in numerous safety- as identity verification (Sharif et al., 2016; Yang et al., 2022; Xiao et al., 2021), autonomous driving (Song et al., 2018; Zhu et al., 2023) and security surveillance (Thys et al., 2019; Xu et al., 2020).
applications, such
Due to the threats, multitude of defense strategies have been devised to boost the robustness of DNNs. Adversarial training (Madry et al., 2017: Wu et al., 2019; Rao et al., 2020) is one effective countermeasure (Gowal et al., 2021) that incorporates adversarial examples into the training data batch. Besides, input preprocessing techniques like adversarial purification aim to eliminate these perturbations (Xiang et al., 2021; Liu et al., 2022; Xu et al., 2023). Overall, these strategies predominantly serve passive defenses; they mitigate adversarial effects uncertain monocular observations (Smith & Gal, 2018) with prior knowledge of the adversary (Naseer et al., 2019; Liu al., 2022; Xu al., 2023). However, passive defenses possess inherent limitations. First, they remain susceptible to unseen or adaptive attacks (Athalye et al., 2018a; Tramer et al., 2020) that evolve to cireumvent the robustness, owing to their dependence on presuppositions regarding the
X. Yang and Zhu corresponding authors.

---

Published as a conference paper at ICLR 2024
adversary's capabilities. Second, these strategies treat each static 2D image independently without considering the intrinsic physical context and corresponding understanding of the scene and objects in the 3D realm, potentially rendering them less effective in real-world 3D physical environments.
As a comparison, human perception employs extensive scene precedents and spatial reasoning to discern elements that are anomalous incongruent. Some research (Thomas, 1999; Elsayed et al., 2018) illustrates that human perception can effortlessly pinpoint misplaced patches or objects within 3D environments, even when such discrepancies trigger errors in DNNs on individual viewpoints. Inspired by this, we introduce Embodied Active Defense (EAD), a novel defensive framework that actively contextualizes environmental context and harnesses shared scene attributes address misaligned adversarial patches in 3D woold settings. By separately implementing two critical functions of active vision, namely perception and movement, EAD comprises of two primary sub- modules: the perception model and the policy model. The perception model continually refines its understanding of the scene based on both current and past observations. The policy model, in turn, derives strategic actions based on this understanding, facilitating more effective observation collection. Working in tandem, these modules enable EAD to actively improve its scene comprehension through proactive movements and iterative predictions, ultimately mitigating the detrimental effects of adversarial patches. Furthermore, our theoretical analysis also validates the effectiveness of EAD in minimizing uncertainties related to target objects.
However, training an embodied model designed for environmental interactions presents inherent chal- lenges, since the policy and perception models interconnected through complex and probabilistic environmental dynamics. To tackle this, we employ a deterministic and differentiable approximation of the environment, bridging the gap between the two sub-modules and leveraging advancements in supervised learning. Moreover, to fully investigate the intrinsic physical context of the scene and objects, we deploy adversary-agmostic patches from the Uniform Superset Approximation for Adversarial Patches (USAP). The USAP provides computationally efficient surrogates encompassing diverse potential adversarial patches, thus precluding the overfitting to a specific adversary pattern.
Extensive experiments validate that EAD possesses several distinct advantages over typical passive defenses. First, in terms of effectiveness, EAD dramatically improves defenses against adversarial patches by substantial margin over state-of-the-art defense methods within few steps. Notably, it maintains or even improves standard accuracy due to instructive information optimal for perceiving target objects in dynamic 3D environments. Second, the attack-agmostic designs allow for exceptional generalizability in a wide array of previously unseen adversarial attacks, outperforming statc-of-the. art defense strategy, by achieving attack success rate reduction of 95% across large spectrum of patches crafted with diverse unseen adversarial attacks. Our contributions are as follows:
To our knowledge, this work represents the inaugural effort to address adversarial robustness within the context of embodied active defense. Through theoretical analysis, EAD can greedily utilize recurrent feedback alleviate the uncertainty induced by adversarial patches in 3D environments.
To facilitate efficient EAD learning within the stochastic environment, we employ a deterministic and differenntaable environmental approximation along with adversary-agnoste patches from USAP, thus enabling the effective application of supervised learning techniques.
Through exhaustive evaluations, we demonstrate that our FAD significantly outperforms contempo- rary advanced defense methods in both effectiveness and generalization based on two safety-critical tasks, including face recognition and object detection.
2 BACKGROUND
In this section, we introduce the threat from adversarial patches under 3D environment and their corresponding defense strategy. Given scene x E X' with its ground-truth label y E y, the perception model f - ) aims to predict the scene annotation y using the image observation 0 O. The observation o; is derived from scene conditioned on the camera's 's state 8; (e.g.. 's position and viewpoint). The function C(-) denotes a task-specific loss function (e.g.. cross-entropy loss for classification).
camera's
Adversarial patches. Though adversarial patches are designed to manipulate a specific region of an image to mislead the image classifiers (Brown et al., 2017), they have now evolved to deceive various perception models (Sharif et al., 2016; Song et al., 2018) under 3D environment (Eykholt al., 2018; Yang al., 2022; Zhu et al., 2023; Yang et al., 2023). Formally, adversarial patch p is
2

---

Published as a conference paper at ICLR 2024
introduced to the image observation og, and results in erroneous prediction of the perception model f. Typically, the generation of adversarial patches in 3D scenes (Zhu et al., 2023) emphasizes solving the optimization problem presented as:
max 3
(I)
where A(+) denotes the applying function to project adversarial patch p from the 3D physical space to 2D perspective views of camera observations og based on the camera's 's state s,, This approach provides more general 3D formulation for adversarial patches. By specifying the state s; as a mixture of 2D transformations and patch locations, it aligns with the 2D) formulation by Brown et al. (2017). To ensure clarity in subsequent sections, we define the set of adversarial patches Px that can deceive the model under scene x as:
Fx = {p E [o, 1).1)
EA,f, p; p;)) + y):
(2)
where H, and W denote the patch's height and width. In practice, the approximate solution of TPx is determined by applying specific methods (Goodfellow et al., 2014; Madry et al., 2017; Carlini & Wagner, 2017; Dong et al., 2018) solve the problem in Eq. (I).
Adversarial defenses against patches. A multitude of defensive strategies, including both empirical (Dziugaite et al., 2016; Hayes. 2018: Naseer et al., 2019; Rao al.. 2020; Xu al., 2023) and certified defenses (Li et al., 2018; Zhang & Wang. 2019; Xiang et al., 2021), have been suggested to safeguard perception models against patch attacks. However, the majority of contemporary defense mechanisms fall under the category of passive defenses, as they primarily rely on information obtained from passively-received monocular observation and prior adversary's 's knowledge to alleviate adversarial effects. In particular, adversarial training approaches (Madry et al., 2017; Wu et al., 2019; Rao et al., 2020) strive to learn a more robust perception model f. while adversarial purification- based techniques (Hayes, 2018; 8; Naseer et al., 2019; Liu et al., 2022; Xu et al., 2023) introduce an auxiliary purifier g : O - O to "detect and remove" (Liu et al., 2022) adversarial patches in image observations, subsequently enhancing robustness through the amended perception model f g-
Embodied perception. In the realm of embodied perception (Aloimonos et al., 1988; Bajesy, 1988), an embodied agent can navigate its environment to optimize perception or enhance the efficacy of task performance. Such concept has found applications in diverse tasks such as object detection (Yang al., 2019; Chaplot al., 2021; Kotar & Mottaghi, 2022; Jing & Kong, 2023), 3D pose estimation (Doumanoglou et al., 2016; Ci et al., 2023) and 3D scene understanding (Das et al., 2018; Ma et al., 2022). To our knowledge, our work the first integrate an embodied active strategy to diminish the high uncertainty derived from adversarial patches to enhance model robustness.
METHODOLOGY
We first introduce Embodied Active Defense (EAD) that utilizes embodied recurrent feedback to counter adversarial patches in Sec. 3.1. We then provide theoretical analysis of the defensive ability of EAD in Sec. 3.2. Lastly, the technical implementation details of EAD are discussed in Sec. 3.3.
3.1 EMBODIED ACTIVE DEFENSE
Conventional passive defenses usually process single observation o for countering adversarial patches, thereby neglecting the rich scenic information available from alternative observations acquired through proactive movement (Ronneberger et al., 2015; He et al., 2017). In this paper, we propose the Embodied Active Defense (EAD), which emphasizes active engagement within a scene and iteratively utilizes environmental feedback improve the robustness of perception against adversarial patches. Unlike previous methods, EAD captures a series of observations through strategic actions, instead of solely relying on a single massively-receivee observation.
EAD is comprised of two recurrent models that emulate the cerebral structure underlying active human vision, each with distinct functions. The perception model f(; 0). parameterized by 0, dedicated to visual perception by fully utilizing the contextual information within temporal observations from the external world. It leverages the observation o, and the prevailing internal belief 6,- 1 on the scene to construct better representation of the surrounding environment b, in recurrent paradigm and simultaneously predict scene annotation, as expressed by:
(ve, b.) s(oe, b;
3

---

Published as a conference paper at ICLR 2024
ERROR snl
ERROR
Task-specific Prediction
Progressive Defenso
y1
y2
y
br
2z
Belief Refinement
Perception Model
Internal Reflection
H P2)
Policy Model
Uncertainty Reduction
EAD
EAD
EAD
Ri
22
Og 0c-1
di--
External
S1
$2
52
St
Interaction
Continuous Interaction
Aiveserial
An overview of our proposed embodied activate defense. The perception model utilizes observation from the extemal world and previous internal belief b the scene refine representation of the surrounding environment be and simultancously make tasc-fpe prediction ye. The policy model generates strategic action de in response to shared environmental understanding h, As the perception process unfolds, initially high informative uncertainty (y (1,,,, 0e) caused by adversarial patch monotonically decreases.
Figure 1:
The subsequent policy model ((; Φ), parameterized by Φ. governs the visual control of movement. Formally, It derives a, rooted in the collective environmental understanding be sustained by the perception model, as formalized by a ~ [b;;; ().
Leveraging recurrent feedback from the environment. Formally, the EAD's interaction with its environment (as depicted in Figure 1) follows simplified Partially- -bbeellbbbbbe Markov Decision Process (POMDP) framework, facilitating the proactive exploration of EAD within the scene x. The interaction process is denoted by MMx) == A, 2) Here, each determines specific MDP by parameterizing a transition distribution T (81++1 [8t; de; x) and an observation distribution [(o,|st.)) x). At every moment t, EAD obtains observation ON [(, x) based current state s, It utilizes O₂ as an environmental feedback to refine the understanding of environment by through perception model f-; 0). Such recurrent perception mechanism is pivotal for the stability of human vision (Thomas, 1999; Kok et al., 2012; Kar et al., 2019). Subsequently, the EAD model executes actions a, T (b;;; )b), rather than remaining static and passively assimilating observation. Thanks to the policy model, EAD is capable of determining the optimal action, ensuring the acquisition of the most informative feedback from the scene, thereby enhancing perceptual efficacy.
Training EAD against adversarial patches. Within the intricately constructed EAD model, we in- troduce specialized learning algorithm tailored for EAD to counter adversarial patches. Considering a data distribution 2 with paired data (x, y) and adversarial patches
1xx crafted with unknown attack techniques, our proposed approach optimizes parameters @ and Φ by minimizing the expected loss amid threats from adversarial patches, where the adversarial patch contaminates the observation or and yields of A(o,, p; st). Consequently, the learning of EAD to mitigate adversarial patches is cast an optimization problem:
min B(x,)-D 0.6
Σ pEPx
C(ve. 8)) with - b(-10))))))) a ~ T(-bb;
(4)
s.t. o x(-)s,,)), 8; ~ 111
where / = ..... V. denotes the model's prediction at timestep t and represents the maximum horizon length of EAD. Importantly, the loss function L remains agnostic to tasks, underscoring the remarkable versatility of EAD. This flexibility ensures EAD to offer robust defenses across diverse perception tasks. We further delve into the efficacy of EAD through an information-theoreiic lens.
3.2 PERSPECTIVE FROM INFORMATION THEORY
Drawing inspiration from active human vision, we introduce EAD in Sec. 3.1 as a strategy to enhance model's robustness against adversarial patches. However, due the black-box nature of neural networks, the fundamental driver behind this defensive tactic remains elusive. To delve deeper into model behaviors, we examine generic instance of EAD in Eq. (4), where the agent employs the
4

---

Published as a conference paper at ICLR 2024
InfoNCE objective (Oord et al., 2018). This is succinetly reformulated" as: S(1(6) (0000))))))))) (0)
K
max 0.4
E ((xi))
log
Egor E 8(102 of 0),00)
(5)
where S(-) 3 x)>33 3 + R measures the similarity between predicted scene annotation and ground truth, and K denotes the size of data batch ((x(), ((x6),00))) y())) sampled from 'D. Under the realm of embodied perception, the target task with InfoNCE objective is to match the observations collected from given scene x with its annotation y other modalities like CLIP (Radford et al., 2021), where the annotations are usually text for tasks like captioning given scene (Jin et al., 2015) or answering the question about scene (Ma et al., 2022).
Theorem 3.1 (Proof in Appendix A.1). For mutual information between current observation and scene annotation y conditioned on previous belief bt denoted as (0 ylb, 1), we have:
go(y 10 (
of
[oo y/bb-11) 2 (x60) (00))
log
+ log(K),
(6)
Σ Egtn go 16
where go oolylor.... 0t) denotes variational distribution for conditional distribution p(ylo,, .t)) with samples ((x(i), y
Remark. To bridge the lower bound of conditional mutual information with the objective, can rewrite to (y ()) with the similarity term in Eq. (4) served as score function, and obtain qe (e(vbbr11. = p(b--1,00 (i0--,,,,e--S(eee ((f(bb-,,,0 Then, this term is equivalent to the negative In- foNCE objective for EAD in Eq. (4). indicates that the training procedure for EAD is actually to indirectly maximize the conditional mutual information, thereby leading the agent to learn an action policy aiming to collect an observation o₂ to better determine the task designesegd annotation y. And the bound will be tighter as the batch size K increases.
Definition 3.1 (Greedy Informative Exploration). A greedy informative exploration, denoted by T², an action policy which, any timestep I, chooses an action a that maximizes the decrease in the conditional entropy of random variable given new observation resulting from taking action ag Formally,
arg max (H *Ell
H( 1.
where H-) denotes the entropy, II is the space of all policies.
Remark. The conditional entropy H (y b.-1) quantifies the uncertainty of the target y given previ- ously sustained belief 6,-1, while H (y bt-1, 0c) denotes the uncertainty of y with extra observation or- Although not optimal throughout the whole trajectory, the greedy informative exploration serves as a relatively efficient baseline for agents to rapidly understand their environment by performing continuous actions and observations. The efficiency of the greedy policy is empirically demonstrated in Appendix D.7.
Given unlimited model capacity and data samples, the optimal policy model T in problem (5) a greedy informative exploration policy. The optimization procedure in Eq. (5) simultaneously estimates the mutual information and improves the action policy with its guidance. Owing to the mutual information being equivalent to the conditional entropy decrease (detailed proof in Appendix A.2), it leads the action policy approaching greedy informative exploration.
Therefore, we theoretically demonstrate the effectiveness of EAD from the perspective of information theory. A well-learned EAD model for contrastive task adopts greedy informative policy to explore the environment, utilizing the rich context information to reduce the abnormally high uncertainty (Smith& & Gal, 2018; Deng et al., 2021) of scenes caused by adversarial patches.
3.3 IMPLEMENTATION TECHNIQUES
The crux of EAD's learning revolves around the solution of the optimization problem (4). However, due to the intractability of the environment dynamics, namely the observation Z and transition T, is not feasible to directly optimize the policy model parameters using gradient descent. Concurrently,
Constraints in Eq. (4) and belief b. in the output have heen excluded for simplicity.
5

---

Published as a conference paper at ICLR 2024
Algorithm Learning Embodied Active Defense
Require: Training data D. number of epochs N. loss function L. perception model 0), policy model <(; differentiable observation function Z and transition function T. max horizon length uniform superset approximation for adversarial patches PP and applying function A.
Ensure: The parameters 0, of the learned EAD model.
I: for epoch + 1 N do
X. <- sample a batch from Dynin
3: P sample batch from from P:
t Au null, 1, Bu null, randomly initialize world state Su:
5: repeat
6:
S. + 1,,
X). O: + Z(8,, X), O A(O:, P;St); bo Compute observations
7:
D Compute beliefs, actions and predictions S.11 4 T(S:, X), O++1 2( 0 A(O S.1);
B. Y. < (B.., 0 0) A, f =B:;))
8:
9:
Bi+1, (+1 + 0 0 0), Act1 (B(+1i))
10:
Le+1 t L(YY+1,, 111,
11:
Update Φ using Ve+1i
12:
t + 2;
13:
until >>
14:
end for
the scene-speeific adversarial patches F., cannot be derived analytically. To address these challenges, we propose two approximation methods to achieve aecuraceurate solutions. Empirical evidence from experiments demonstrates that these approaches are effective in reliably solving problem (4).
Deterministic and differentiable approximation for environments. Formally, we employ the Delta distribution to deterministically model the transition T and observation Z. For instance, the approximation of T is expressed T8 1|c,, de, x) 5(s++1 - (st,, dz2 x)), where T x A X > S denotes the mapping of the current state and action to the most probable next state, and 8(-) represents the Delta distribution. Additionally, use advancements in differentiable rendering (Kato et al., 2020) to model deterministic observations by rendering multi-view image observations differentiably, conditioned on camera parameters deduced from the agent's 's current state.
These approximations allow us to create a connected computational graph between the policy and perception models (Abadi, 2016; Paszke et al., 2019), thereby supporting the use of backpropagation (Werbos, 1990; Williams & Peng. 1990) to optimize the policy model's parameter & via supervised learning. To maximize supervision signal frequency and minimize computational overhead, update model parameters 0 and & every second step, right upon the formation of the minimal computational graph that includes the policy model. This approach allows us seamlessly integrate the passive perception models into EAD to enhance resistance to adversarial patches.
Uniform superset approximation for adversarial patches. The computation of P. typically neces- sitates the resolution of the inner maximization in Eq. (4). However, this is not only computationally expensive (Wong al., 2020) but also problematic as inadequate assumptions for characterizing adversaries can hinder the models' capacity to generalize across diverse, unseen attacks (Laidlaw et al., 2020). To circumvent these limitations, we adopt an assumption-frre strategy that solely relies on uniformly sampled patches, which act as surrogate representatives encompassing broad spectrum of potential adversarial examples. Formally, the surrogate set of adversarial patches is defined as:
:= (p.)
Pi u(00, H),,
(8)
where N represents the size of the surrogate patch set IP. In the context of implementation, N corresponds to the training epochs, suggesting that as - co, we achieve TPx S P. By demanding the EAD to address various patches, the active perception significantly bolsters its resilience against adversarial patches (refer to Sec. 4.1). The overall training procedure is outlined in Algorithm 1.
EXPERIMENTS
In this section, we first introduce the experimental environment, and then present extensive experi- ments to demonstrate the effectiveness of EAD face recognition (FR) and object detection.
Experimental Environment. To enable EAD's free navigation and observation collection, a ma- nipulable simulation environment is indispensable for both training and testing. For alignment with

---

Published as a conference paper at ICLR 2024
Table I: Standard accuracy and adversarial attack success rates on models. denotes methods involved with adversarial training. Columns with Adpt represent results under adaptive attack, and the adaptive attack against EAD optimizes the patch with expected gradient over the distribution of possible action policy.
Dodging ASR (%)
Impersonation ASR (%)
Method
Acc. (%)
MIM
EoT
GenAP 3DAdv
Adpt
MIM
EoT
GenAP 3DAdv
Adpt
Undefended
88.86
100.0
100.0
99.0
98.0
100.0
100.0
100.0
99.0
89.0
100.0
JPEG
89.98
98.0
99.0
95.0
88.0
100.0
99.0
100.0
99.0
93.0
99.0
LGS
83.50
49.5
52.6
74.0
77.9
78.9
5.1
7.2
33.7
30.6
38.4
SAC
86.83
73.5
73.2
92.8
78.6
65.2
6.1
9.1
67.7
64.6
48.0
PZ
87.58
6.9
8.0
58.4
57.1
88.9
4.1
5.2
59.4
45.8
89.8
SAC!
80.55
78.8
78.6
79.6
85.8
85.0
3.2
3.2
18.9
22.1
51.7
PZ'
85.85
6.1
6.2
14.3
20.4
69.4
3.1
3.2
19.1
27.4
61.0
DOAT 79.55
75.3
67.4
87.6
95.5
95.5
95.5
89.9
96.6
89.9
89.9
EAD (ours)
90.45
0.0
0.0
2.1
13.7
22.1
4.1
3.1
5.1
7.2
8.3
given vision tasks, define the state composite of the camera's yaw and pitch and the action as camera's rotation. This sets the transition function, so the core of the simulation environment revolves around the observation function, which renders 2D image given the camera state. 1) Training environment. As discussed in Sec. 3.3, model training demands differentiable environ- mental dynamics. To this end, employ the cutting-edge 3D generative model, EG3D (Chan et al., 2022) for realistic differentiable rendering (see Appendix D.1 for details on simulation fidelity). 2) extend
Testing environment. In addition evaluations conducted in EG3D-based simulations, our testing for object detection within CARLA (Dosovitskiy et al., 2017), aiming to assess EAD in more safety-critical autonomous driving scenarios. The details can be found in Appendix C.
4.1 EVALUATION ON FACE RECOGNITION
Evaluation setting. We conduct our experiments on CelebA-3D, which we utilize GAN inversion (Zhu al., 2016) with EG3D (Chan al., 2022) to reconstruct 2D face image from CelebA into 3D form. For standard accuracy, we sample 2, 000 test pairs from the CelebA and follow the standard protocol from LFW (Huang al., 2007). As for robustness evaluation, we report the white-box box attack success rate (ASR) on 100 identity pairs in both impersonation and dodging attacks (Yang & Zhu, 2023) with various attack methods, including MIM (Dong et al., 2018), EoT (Athalye et al., 2018b), GenAP (Xiao et al., 2021) and Face (3DAdv) (Yang et al., 2022). Note that 3DAdy utilizes expectation over 3D transformations in optimization, rendering robust to 3D viewpoint variation (within +15., refer Appendix D.7). More details described in Appendix D.I & D.2.
Implementation details. For the visual backbone, we employ the pretrained IResNet-50 Arc- Face(Duta et al., 2021) with frozen weight in subsequent training. To implement the recurrent perception and policy, we adopt a variant of the Decision Transformer (Chen et al., 2021) to model the temporal process which uses feature sequences extracted by the visual backbone predict normalized embedding for FR. To expedite EAD's learning of efficient policies requiring minimal perceptual steps, configure the max horizon length 4. Discussion about this horizon length is provided in Appendix D.7 and the details for EAD are claborated in Appendix D.5.
Defense baselines. We benchmark EAD against a range of defense methods, including adversarial training-based Defense against Occlusion Attacks (DOA) (Wu et al., 2019), and purification based methods like JPEG compression (JPEG) (Dziugaite al., 2016), local gradient smoothing (LGS) (Naseer et al., 2019), segment and complete (SAC) (Liu et al., 2022) and PatchZero (PZ) (Xu et al., 2023). For DOA, we use rectangle PGD patch attacks (Madry et al., 2017) with 10 iterations and step size 2/255. SAC and PZ require patch segmenter to locate the area of the adversarial patch. Therefore, we train the segmenter with patches of Gaussian noise to ensure the same adversarial- agnostic setting EAD. Besides, enhanced versions of SAC and PZ involve training with EoT- generated adversarial patches, denoted as SACİ and PZi More details are in Appendix D.4.
Effectiveness of EAD. Table demonstrates both standard accuracy and robust performance against diverse attacks under a white-box setting. The patch is 8% of the image size. Remarkably, our approach outperforms previous tfate-of-the-an techniques that agnostic to adversarial examples in both clean accuracy and defense efficacy. For instance, EAD reduces the attack success rate of 3DAdy by 84.3% in impersonation and by 81. 31.8% in dodging scenarios. Our EAD even surpasses
7

---

Published as a conference paper at ICLR 2024
Target
Source
Step 1
Step 2
Step 3
Step 4
Dodging
0.43
0.05
0.22
0.27
0.32
Impersonation
-0.08
0.23
Figure 2: Qualitative results of EAD. The first two columns present the original image pairs, and the subsequent columns depict the interactive inference steps that the model took. The adversarial glasses generated with 3DAdv, which robust to 3D viewpoint variation. The computed optimal threshold is 0.24 from [-1,1] 1].
12%
Altcc Succes Rate (Ng)
Hecation
Decratiion
UndeStoded
JPSG
1G3
SAC
PZ
DCA
EAD
Figure 3: Comparative evaluation of defense methods across varying attack iterations with different adversarial patch sizes. The adversarial patches crafted by 3DAdv for impersonation.
the undefended passive model in standard accuracy, which facilitates the reconciliation between robustness and accuracy (Su al., 2018) through embodied perception. Furthermore, our method even outstrips baselines by incorporating adversarial examples during training. Although SACt and Pzt are trained using patches generated via EoT (Athalye al., 2018b), we still obtain superior performance which stems from effective utilization of the environmental feedback in active defense.
Figure illustrates the defense process executed by EAD. While EAD may fooled at first glance, its subsequent active interactions with the environment progressively increase the similarity between the positive pair and decrease the similarity between the negative pair. Consequently, EAD effectively mitigates adversarial effeets through the proactive acquisition of additional observations.
Effectiveness against adaptive attack. While the deterministic and differential approximation could enable backpropagation through the entire inference trajectory of EAD, the computational cost is prohibitive due to rapid GPU memory consumption as trajectory length 7 increases. To overcome this, resort an approach similar to USAP that approximates the true gradient with the expected gradient over a surrogate uniform superset policy distribution. This necessitates an optimized patch to handle various action policies. Our adaptive attack implementation builds upon 3DAdv (Yang et al., 2022) using 3D viewpoint variations. We can see that EAD maintains its robustness against the most potent attacks. further shows that EAD's defensive capabilities arise from the synergistic integration of its policy and perception models, rather than learning a short-cut strategy to neutralize adversarial patches from specific viewpoints. More details are in Appendix D.3.
Generalization of EAD. As illustrated in Table 1, despite no knowledge of adversaries, ours demonstrates strong generalizability across various unseen adversarial attack methods. It is partially attributed to EAD's s capability to dynamically interact with its environment, enabling it to adapt and respond to new types of attacks. Additionally, we assess the model's resilience across wide range of patch sizes and attack iterations. Trained solely on patches constituting 10% of the image, EAD maintains a notably low attack success rate even when the patch size and attack iteration increases, as shown in Figure 3. This resilience can be attributed to EAD's primary reliance environmental information rather than patterns of presupposed adversaries, thus avoiding overfitting specific attack types. The details are available in Appendix D.7.
Ablation study. We conduct ablation studies within EAD in Table 2. 1) Effectiveness of recurrent feedback. We initially demonstrate the role of recurrent feedback, ie., reflecting on prior belief
8

---

Published as a conference paper at ICLR 2024
Table Standard accuracy and white-box impersonation adversarial attack success rates ablated models. For the model with random action policy, we report the mean and standard deviation under five rounds.
Attack Success Rate (%)
Method
Acc. (%)
MIM
EoT
GenAP
3DAdv
Adpt
Undefended
88.86
100.0
100.0
99.00
98.00
98.00
Random Movement
90.38
4.17 2.28
5.05 + 1.35
8.33 + 2.21
76.77
3.34
76.77 1: 3.34 31.13 3.00
Perception Model
90.22
18.13=4.6
18.62 + 2.24
22.19
+
30.77 + 1.81
Policy Model
89.85
3.09
4.12
7.23
11.34
15.63
+ USAP 90.45
4.12
3.07
5.15
7.21
8.33
Ground Truth
Step
Step 2
Step
Step 4
Figure 4: Qualitative results of EAD object detection. The adversarial patches are generated using MIM and attached to the billboards within the scene, leading the "disappearing" of the target vehicle. The setting from CARLA MARLA-GEAR. These images illustrate the model's interactive inference steps to counter the patches.
with comprehensive fusion model, in achieving robust performance. EAD with only the perception model significantly surpasses both the undefended baseline and passive FR model with multi-view ensembles (Random Movement). Notably, the multi-view ensemble model fails to counteract 3DAdv, corroborating that EAD's defensive strength is not merely a function of the vulnerability of adversarial examples viewpoint transformations. 2) Utility of learned policy. We then investigate the benefits of the learned policy on robust perception by comparing Clean-Data EAD with the perception model adopting random action. The learned policy markedly enhances resistance to various attacks. The findings supporting the effectiveness and efficiency of the learned policy are furnished in Appendix D.7. 3) Influence of patched data. By employing EAD handle diverse set of patches sampled from surrogate uniform superset during training, the robustness of EAD is further augmented.
4.2 EVALUATION ON OBJECT DETECTION
Experimental setup. We further apply EAD to ob- ject detection to verify its adaptability. We train EAD with a simulation environment powered by EG3D (Chan et al., 2022), and evaluate the performance of EAD robustness evaluation API provided by CARLA-GEAA (Nesti et al., 2022). For the object detector, pretrained Mask-RCNN (He et al., 2017) on COCO (Lin et al., 2014). We assess model robustness 360 test scenes featuring patches attached to billboards and re- port the mean Average Precision (mAP). For attacks, we use methods including MIM (Dong et al., 2018), TIM (Dong et al., 2019) and SIB (Zhao et al., 2019) with iterations set at 150 and a step size of 0.5. Specifically, we augment MIM and TIM by incorporating EoT over 2D transformations to enhance patch resilience against viewpoint changes. Meanwhile, SIB demonstrates robustness with varying distances (1 25 m) and angles (++60%). (Zhao et al., 2019). The patch size varies with the billboard size, constituting 4 ~ 8% of the image size. Further details can be found in Appendix E.
Table 3: The mAP (%) of Mask-RCNN under different adversarial attacks.
Method
Clean
MIM
TIM
SIB
Undefended
46.6
30.6
34.2
31.2
LGS
45.8
36.5
37.8
36.4
the
SACT
46.5
33.3
35.1
32.5
PZ
46.3
33.2
35.1
32.9
EAD (ours)
46.6
39.4
39.3
39.5
Experimental results. As evidenced in Table 3, our approach surpasses the others in both clean and robust performances. Furthermore, EAD retains its robustness and effectively generalizes against variety of unencountered attacks, thus substantiating our previous assertions in FR. A visualization of the defense process is provided in Figure 4. More detailed results are provided in Appendix E.5.
5 CONCLUSION
In this paper, we introduce novel proactive strategy against adversarial patches, named EAD. Inspired by active human vision, EAD merges external visual signals and internal cognitive feedback via two recurrent sub-modules. facilitates the derivation of proactive strategic actions and continuous refinement of target understanding by recurrently leveraging environmental feedback. Experiments validate the effectiveness and generalizability of EAD in enhancing defensive capabilities.

---

Published as a conference paper at ICLR 2024
ETHICS STATEMENT
Embodied Active Defense (EAD) shows promise as protection against adversarial patch attacks under 3D physical environment. the field of adversarial defense, EAD presents promising direction for enhancing robustness. However, EAD could also impact the performance of some applications like face watermarking and facial privacy protection, which rely on generating adversarial perturbations. We advocate for responsible use, particularly in contexts where bias could occur. Although EAD effectively counters adversarial attacks, may require additional hardware or computational resources to enable dynamic interactions and process temporal information. We report no conflicts of interest and adhere the highest standards of research integrity.
ACKNOWLEDGEMENTS
This work was supported by the National Natural Science Foundation of China (Nos. 62276149, U2341228, 6206113600 62076147), BNRist (BNR2022RC00006, Tsinghua Institute for Guo Qiang. Tsinghua-OPPO Joint Research Center for Future Terminal Technology, and the High Perfor- mance Computing Center, Tsinghua University. Y. Dong also supported by the China National Postdoctoral Program for Innovative Talents and Shuimu Tsinghua Scholar Program. J. Zhu was also supported by the XPlorer Prize.
REFERENCES
Martín Abadi. Tensorflow:
learning functions scale. In Proceedings of the 21st ACM SIGPLAN International Conference on Functional Programming, pp. 1-1, 2016. 6
John Aloimonos, Isaac Weiss, and Amit Bandyopadhyay. Active vision. International journal of computer vision, 1:333-356, 1988.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International conference machine learning, pp. 274-283. PMLR, 2018a. 1, 20, 21, 24
Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing robust adversarial examples. In International conference on machine learning, pp. 284-293. PMLR, 2018b. 7, 8, 19
Ruzena Bajesy. Active perception. Proceedings of the IEEE, 76(8):966-1000 1988. 3
David Barber and Felix Agakov. The algorithmı: a variational approach to information maximiza- tion. Advances in neural information processing systems, 16(320):201, 2004. 15
Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating propagating gradients through stochastic neurons for conditional computation. arXiv preprint ar08v 3432, 2013. 20
Tom B Brown, Dandelion Mané, Aurko Roy, Martín Abadi, and Justin Gilmer. Adversarial patch. arXiv preprint arXiv: 1712.0667 2017. 2,
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017
ieee symposium security and privacy (sp), pp. 39-57. Icce, 2017. 3, 21
Eric R Chan, Connor Z. Lin, Matthew Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas Guibas, Jonathan Tremblay, Sameh Khamis, et al. Efficient geometry-aware 3d generative adversarial networks. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 16123-16133, 2022. 7, 9, 18
Devendra Singh Chaplot, Murtaza Dalal, Saurabh Gupta, Jitendra Malik, and Russ R Salakhutdinov. Seal: Self-supervised embodied active learning using exploration and 3d consistency. Advances in neural information processing systems,
13098, 2021. 3
Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence modeling. Advances in newral information processing systems, 34:15044-15099 2021. 7
10

---

Published as a conference paper at ICLR 2024
Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.06174, 2016.
Hai Ci, Mickel Liu, Xuehai Pan, Fangwei Zhong, and Yizhou Wang. Proactive multi-camera collaboration for 3d human pose estimation. arXiv preprint arXiv:2303.03767 2023. 3
Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra. Embodied question answering. In Proceedings of the IEEE conference computer vision and patter recognition, pp. 1-10, 2018. 3
Jiankang Deng, Jia Guo, Xue Niannan, and Stefanos Zafeiriou. Areface: Additive angular margin loss for deep face recognition. In CVPR, 2019. 18,
Zhijie Deng, Xiao Yang, Shizhen Xu, Hang Su, and Jun Zhu. Libre: A practical bayesian approach to adversarial detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 972-982, 2021. 5
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 9185-9193, 2018. 3, 7, 9, 19, 27
Yinpeng Dong, Tianyu Pang. Hang Su, and Jun Zhu. Evading defenses transferable adversarial ex- amples by translation-invariant attacks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 4312-4321, 2019. 9, 27
Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. Carla: An open urban driving simulator. In Conference on robot learning, pp. 1-16. PMLR, 2017.
Andreas Doumanoglou, Rigas Kouskouridas, Sotiris Malassiotis, and Tae-Kyun Kim. Recovering 6d object pose and predicting next-best-view in the crowd. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 3583-3592, 2016. 3
Ionut Cosmin Duta, Li Liu, Fan Zhu, and Ling Shao. Improved residual networks for image and video recognition. In 2020 25th International Conference Puttern Recognition (ICPR), pp. 9415-9422. IEEE, 2021. 7
Gintare Karolina Dziugaite, Zoubin Ghahramani, and Daniel M Roy. A study of the effect of jpg compression on adversarial images. arXiv preprint arXiv: 06053 2016. 3, 7
Gamaleldin Elsayed, Shreya Shankar, Brian Cheung, Nicolas Papernot, Alexey Kurakin, Ian Good- fellow, and Jascha Sohl-Dickstein. Adversarial examples that fool both computer vision and time-limited humans. Advances in neural information processing systems, 2018. 2
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical world attacks on deep learning visual classification, Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1625-1634, 2018.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv: 14266552, 2014.
Sven Gowal, Sylvestre-Alvise Rebuffi, Olivia Wiles, Florian Stimberg, Dan Andrei Calian, and Timothy Mann. Improving robustness using generated data. Advances in Neural Information Processing Systems, 34:4218-4233. 2021.
Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. Ms-celeb- A dataset and benchmark for large-scale face recognition. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14, pp. 87-102. Springer, 2016. 21
Jamie Hayes. On visible adversarial perturbations & digital watermarking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1597-1604, 2018
11

---

Published as a conference paper at ICLR 2024
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-enn. In Proceedings of the IEEE international conference on computer vision, pp. 2961-2969, 2017. 3,9
Gary B. Huang, Manu Ramesh, Tamara Berg, and Erik Leamnd Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst, October 2007. 7, 26
Junqi Jin, Kun Fu, Runpeng Cui, Fei Sha, and Changshui Zhang. Aligning where to see and what to tell: image caption with region-based attention and scene factorization. arXiv preprint 2015. 5
1.06v:5006066
Ya Jing and Tao Kong. Learning to explore informative trajectories and samples for embodied perception. arXiv preprint arXiv:2303. 10936, 2023. 3
Glenn Jocher, Alex Stoken, Jirka Borovec, Ayush Chaurasia, Liu Changyu, Adam Hogan, Jan Hajek, Laurentiu Diaconu, Yonghye Kwon, Yann Defretin, et al. ultralytics/ollo5: v5. 0-yolov5-p6 1280 models, aws, supervise. ly and youtube integrations. Zenodo, 2021. 27
Kohitij Kar, Jonas Kubilius, Kailyn Schmidt, Elias B Issa, and James J DiCarlo. Evidence that recurrent circuits are critical to the ventral stream's execution of core object recognition behavior. Nature neuroscience, 22(())974-995 983, 2019. 4
Hiroharu Kato, Deniz Beker, Mihai Morariu, Takahiro Ando, Toru Matsuoka, Wadim Kehl, and Adrien Gaidon. Differentiable rendering: A survey, arXiv preprint a7Xiv 2020.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 11Xiv:1412.6980, 2014. 19
Peter Kok, Janneke FM Jehee, and Floris De Lange. Less is more: expectation sharpens representa- tions in the primary visual cortex. Neuron, 75(2):265-22 2200, 2012. 4
Klemen Kotar and Roozbeh Mottaghi. Interactron: Embodied adaptive object detection. In Proceed-
ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14860-14869, 2022. 3
Cassidy Laidlaw, Sahil Singla, and Soheil Feizi. Perceptual adversarial robustness: Defense against unseen threat models. arXiv preprint arXiv:2006. 12655, 2020.
Yuezun Li, Xiao Bian, Ling-Ching Chang, and Siwei Lyu. Exploring the vulnerability of single shot module in object detectors via imperceptible background patches, arXiv preprint 1809. 2018. 3
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision- ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6- 12, 2014, Proceedings. Part V 13, pp. 740-755. Springer, 2014. 9
Jiang Liu, Alexander Levine, Chun Pong Lau, Rama Chellappa, and Soheil Feizi. Segment and complete: Defending object detectors against adversarial patch attacks with robust patch detection. In Proceedings of the IEEE/CV Conference Computer Vision and Pattern Recognition, pp. 14973-14982, 2022. 3, 7, 20, 21, 22, 26
Ziwei Liu, Ping Luo, Xiaogang Wang. and Xiaoou Tang. Large-scale celebfaces attributes (celeba)
dataset. Retrieved August, 15(2018):11, 2018. 18
Xiaojian Ma, Silong Yong, Zilong Zheng, Qing Li, Yitao Liang, Song-Chun Zhu, and Siyuan Huang. Sqa3d: Situated question answering in 3d scenes. arXiv preprint vrXiv:2210.07774, 2022. 3,5
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint (0Xiv:7706.00000 2017. 1, 3, 28
Muzammal Naseer, Salman Khan, and Fatih Porikli. Local gradients smoothing: Defense against localized adversarial attacks. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). pp. 1300-1307. IEEE, 2019. 1,3,
12

---

Published as a conference paper at ICLR 2024
Federico Nesti, Giulio Rossolini, Gianluca D'Amico, Alessandro Biondi, and Giorgio Buttazzo. CARLA-GeAR: a Dataset Generator for a Systematic Evaluation of Adversarial Robustness of Vision Models. arXiv e-prints, art. arXiv:2206.0453 June 2022. 9, 27
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprini arXiv: 1rXivv/887.03748, 2018. 5, 16
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, pigh-peeformance deep learning library. Advances in neural information processing systems, 32, 2019. 6
Yury Polyanskiy and Yihong Wu. Lecture notes on information theory. Lecture Notes for ECE563 (UIUC) and, 6(2012-2016):7, 2014. 17
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et Learning transferable visual models from natural language supervision. In International conference on machine learning, Pp. 8748-8763. PMLR, 2021.
Sukrut Rao, David Stutz, and Bernt Schiele. Adversarial training against location-optimized adver- sarial patches. In Computer Vision-ECCV 2020 Workshops: Glasgow, UK, August 23-28, 2020, Proceedings, Purt V 16, pp. 429-448. Springer, 2020. 1,3
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention-MICCAN 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part IIl 18, pp. 234-244-22222 Springer, 2015. 3
Sara Sabour, Yanshuai Cao, Fartash Faghri, and David J Fleet. Adversarial manipulation of deep representations. arXiv preprint
2015. 20
Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K Reiter. Accessorize to a crime: Real and stealthy attacks on bate-of-the-ant face recognition. In Proceedings of the 2016 acm sigsac conference on computer and communications security, pp. 1528-1540, 2016. 1,
Lewis Smith and Yarin Gal. Understanding measures of uncertainty for adversarial example detection. arXiv preprint 1003v:8803.8855 2018. 5,5
Dawn Song, Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Florian Tramer, Atul Prakash, and Tadayoshi Kohno. Physical adversarial examples for object detectors. In 12th USENIX workshop on offensive technologies (WOOT 18), 2018. 1,2
Dong Su, Huan Zhang. Hongge Chen, Jinfeng Yi, Pin- Yu Chen, and Yupeng Gao. Is robustness the cost of accuracy? ?-a comprehensive study on the robustness 18 deep image classification models. In Proceedings of the European conference computer vision (ECCV), pp. 631-648, 2018. 8
Nigel Thomas. Are theories of imagery theories of imagination? an active perception approach to conscious mental content. Cognitive science, 23(2):207-244 1999. 2,
Simen Thys, Wiebe Van Ranst, and Toon Goedemé. Fooling automated surveillance cameras: adversarial patches to attack person detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp. 0-0, 2019.
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. Advances in neural information processing systems, 33:1633-1645. 2020.
Paul J Werbos. Backpropagation through time: what it does and how to do it. Proceedings of the IEEE, 18100):5500-1560 1990. 6
Ronald Williams and Jing Peng. An efficient gradient- gradient-based algorithm for on-line training of recurrent network trajectories. Neural computation, 2(4):490-501, 1990.
13

---

Published as a conference paper at ICLR 2024
Eric Wong, Leslie Rice, and J Zico Kolter. Fast is better than free: Revisiting adversarial training.
arXiv preprint arXiv:2001.0390) 2020. 6
Tong Wu, Liang Tong, and Yevgeniy Vorobeychik, Defending against physically realizable attacks
on image classification. arXiv preprint M9X9::9999009555 2019. 1, 7, 21, 22
Chong Xiang, Arjun Nitin Bhagoji, Vikash Sehwag, and Prateek Mittal, Patchguard: A provably robust defense against adversarial patches via small receptive fields and masking. In USENIX Security Symposium, pp. 2237-2254, 2021. 3
Zihao Xiao, Xianfeng Gao, Chilin Fu, Yinpeng Dong, Wei Gao, Xiaolu Zhang, Jun Zhou, and Jun Zhu. Improving transferability of adversarial patches face recognition with generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp-
11845-1 11854, 2021. 1, 7,
Kaidi Xu, Gaoyuan Zhang, Sijia Liu, Quanfu Fan, Mengshu Sun, Hongge Chen, Pin- Yu Chen, Yanzhi Wang, and Xue Lin. Adversarial t-shirt! evading person detectors in physical world. In Computer AssonnEECV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part V 16, pp. 665-681. Springer, 2020.
Ke Xu, Yao Xiao, Zhaoheng Zheng, Kaijie Cai, and Ram Nevatia. Patchzero: Defending against adversarial patch attacks by detecting and zeroing the patch. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision pp. 46344442444 1642-664, 2023. 3, 7, 21, 22
Jianwei Yang. Zhile Ren, Mingze Xu, Xinlei Chen, David J Crandall, Devi Parikh, and Dhruv Batra. Embodied amodal recognition: Learning to move to perceive objects. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2040-2050, 2019. 3
Xiao Yang and Jun Zhu, Adversarial attacks on face recognition. In Handbook of Face Recognition, pp. 387-404. Springer, 2023. 7
Xiao Yang, Yinpeng Dong, Tianyu Pang, Zihao Xiao, Hang Su, and Jun Zhu. Controllable evaluation and generation of physical adversarial patch on face recognition. arXiv preprint wrXiv:203.004233
2022. 1, 2, 7, 8, 19
Xiao Yang, Chang Liu, Longlong Xu, Yikai Wang, Yinpeng Dong, Ning Chen, Hang Su, and Jun Zhu.
Towards effective adversarial textured 3d meshes on physical face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4119-4128, 2023.
Haichao Zhang and Jianyu Wang. Towards adversarially robust object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 421-434140 2019. 3
Yue Zhao, Hong Zhu, Ruigang Liang, Qintao Shen, Shengzhi Zhang, and Kai Chen. Seeing isn't believing: Towards more robust adversarial attack against real world object detectors. In Proceedings the 2019 ACM SIGSAC conference on computer and communications security, pp. 1989-2004, 2019. 9
Jun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman, and Alexei A Efros. Generative visual manipulation on the natural image manifold. In Computer Vision-ECC ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part V 14, pp. 597-613, Springer, 2016.7 7
Zijian Zhu, Yichi Zhang, Hai Chen, Yinpeng Dong, Shu Zhao Wenbo Ding, Jiachen Zhong, and Shibao Zheng. Understanding the robustness of 3d object detection with bird's serd'-eeye represen- tations in autonomous driving. arXiv preprint arXiv:2303. 2023. 1, 3, 18
14

---

Published as a conference paper at ICLR 2024
A PROOFS AND ADDITIONAL THEORY
A.1 PROOF OF THEOREM 3.1
Proof. For series of observations (01...00;) 0r} and previously maintained belief belief by- deter-
mined by the scene x, we expand the left-hand side of Eq. (6) as follows:
I(o:: 1
Ex.y log ((0.
p (1.1)10011
ply b-)
Ex,y log p(u b-1)
(9)
By multiplying and dividing the integrand in Eq. (9) by variational distribution (o 0z),
have:
Ex,y log ply|bi-1, Of 01)0 (00
(
Ex Dog go ( oz)
b-1)
Due to the non-negativity KL-divergence, we have lower bound for mutual information:
E Ex,y log (0(
p(y(b:-1)
xx log ( (1(1)6111,01)
p(y14--1,0-1 0t-1) + Hyybbe-),
Exy log
(10)
where ())b--1) denotes the conditional entropy of y given belief b and Eq. (10) well known as Barber and Agakov bound (Barber & Agakov, 2004). Then, we choose an energy-based variational family that uses critic Eg (b:-1, oz, y) and scaled by the data density p(b O:):
qa gv( p(y(br-1) (o(b)-1,013) 7(b,-1,0))
(11)
where ((br-1,0r) 01) = Eye Eob By substituting Eq. (11) distribution into Eq, (10), we have:
Hly(b-1) Ex,y EE (br-1. o1, or,9)] [log Z (b)
Ex,y log (y
(12)
which is the unnormalized version of the Barber and Agakov bound. By applying inequality 2(0)-1.0))
log ((b < a(b- _oo logla(br-1,:))] 1 for any a(b:-1,0t) 0, and the bound is tight when a(b;-1, or) = 2 (bx ot). Therefore, we have tractable upper bound, which is known tractable unnormalized version of the Barber and Agakov lower bound mutual information:
Ex,y (Eo v)]
Ex (log 2(b4-1,
E. E.((50((c-110)..))
> Ex, EE (b 0, w)] Exl
+ log[a(b(-11..))
(1)
Ex.y log
Ex a(bi-1,00)
(13)
a(b
To reduce variance, we leverage multiple samples (xl1), , from D to implement a low- variance but high-bias estimation for mutual information. For other observation trajectory which is
15

---

Published as a conference paper at ICLR 2024
originated from other scene (x(0), y()) (j f i), we have its annotations { R from (+()) and y we have:
1z=1,zzi independent
,(4 0x) = al 0z50 (1) (k))
Then, we are capable of utilize additional samples (2) , ا to build Monte-Carlo estimate of the function (()
a(b,-1, 01. yk) m(bb o .... (k)
Eoly (o
To estimate the bound over samples, the last term in Eq. (13) becomes constant 1:
E 1 ,00
E By ( e Bx ml m(h1-1, 01 (1) y(K)) Exi
=I 1.
(14)
ml(,,1,, 12 ,.)))....)))))))
By applying Eq. (14) back to Eq. (13) and averaging the bound over K samples, (reindexing x(1) as 2ifi for each term), we exactly recover the lower bound on mutual information proposed by Oord al. (2018) as:
E000/22
E Byen ef 1006" a(b( of ; 12(1) ,(K)) (.)))
1 + E Ex gor gon
-Exoo
(1), y(()))
Eo(ol of راور
= g00 flog
alb
1-1, (1) (1)) k))
(1)
Eo 50 ا00
-xxt أدبي (log
Egon Glll of goo
K E
By multiplying and dividing the integrand in Eq. (9) by the p(vlb(0) (011)) and extracting 1 out of the brackets, it transforms into:
qo (6(2) 10 (1) 90))
Ex0r, (log
+ log(K).
Egen go (b() 032) ,
A.2 MUTUAL INFORMATION AND CONDITIONAL ENTROPY
Given scene annotation y, we measure the uncertainty of annotation y at time step with conditional entropy of given series of observations {b.-1. oz J. denoted as H H b1-1. In this section, we prove that the conditional entropy decrease is equivalent to the conditional mutual information Eq. (6).
Theorem A.1. For any time step > 1, the following holds:
H (;-) H
1os ybb-1).
(15)
Proof. Initially, by replacing the conditional entropy with the difference between entropy and mutual information, the left-hand side becomes:
b( Hy [H(y) I(y; br [H()) (1r b,-1,
= 11 ( b-.
16

---

Published as a conference paper at ICLR 2024
By applying Kolmogorov identities (Polyanskiy & Wu, 2014), it transforms into:
[y: b1,, (1:) I(o ylb--1).
With Theorem A. can establish comnection between the mutual information in Eq. (6). The greedy informative exploration in Definition 3.1, thereby deducing the relationship between the policy model of EAD with the InfoNCE objective in Eq. (5) and greedy informative exploration.
B OVERVIEW OF REINFORCEMENT LEARNING FUNDAMENTALS
At its reinforcement learning entails an agent learning to make decisions, interacting with its environment, and employing Markov Decision Process (MDP) to model this interactive process. In the context of embodied perception, only three elements relevant: observation, action, and state. The agent is in a particular state and needs to obtain observations from the environment to better understand take actions, interact with the environment, and transition the next environment. To illustrate this with a robot, consider the procedure in which the robot captures an image from one specific position and then moves the next position.
Furthermore, the Embodied Active Defense (EAD) method extends these concepts in reinforcement learning. Unlike making a one-time decision based on single observation, EAD consistently monitors its environment, adjusting its understanding over time. It can be likened to security camera that doesn't capture a single snapshot A distinctive feature of EAD is its integration of perception (seeing and understanding) with action (taking informed actions based on that understanding). EAD doesn't 't merely passively observe; actively interacts with its environment. This proactive engagement enhances EAD's ability to acquire more precise and reliable information, ultimately resulting in more informed decision making.
rather continuously observes and adapts to its surroundings.
vecision-making.
C EXPERIMENT DETAILS FOR SIMULATION ENVIRONMENT
Envrionmental dynamics. Formally, define the state 8; (h2, 0t), combination of camera's yaw h4 E IR and pitch yt IR at moment t, while the action is defined as continuous rotation denoted by a (Ah, Av). Thereby, the transition function is denoted as T( f ar x) St + ag. while the observation function is reformulated with the 3D generative model O x) = R(s,, x), where R((-) renderer (e.g., 3D generative model or graphic engine) that renders 2D image observation Or given camera parameters determined by state st.
In practice, the detailed formulation for renderer in computational graphics is presented as:
o, = -R, I,x).
(16)
where E, R is the camera's extrinsic determined by state st, while I 3x3 the pre-defined camera intrinsic. That is to say, we need to calculate the camera's extrinsic E, E RI with to utilize renderer to render images. Assuming that we're using a right-handed coordinate system and column vectors, we have:
R
T
E,
(17)
O
Where R2
3x3 is the rotation matrix determined by st and T g3x1 is the invariant translation vector. The rotation matrices for yaw he and pitch it are:
cos(fr)
0
sin(h+)
(h,, =
0
1
O
(18)
sin (h+)
0
cos(he)
I
Rd (vx) =
cos (vr) sin vv))
sin(v:)
(19)
cos(vr)
The combined rotation R₂ would then be R, = Rd (hr) x R (vz). Then, the complete extrinsic matrix becomes:
17

---

Published as a conference paper at ICLR 2024
(he) x R (vz)
T 1
E,
(20)
0
Applying function. In experiments, the adversarial patch is attached to flat surface, such as eyeglasses for face recognition and billboards for object detection. Utilizing known corner coordinates of the adversarial patch in the world coordinate system, employ both extrinsic E, and intrinsic K parameters to render image observations containing the adversarial patch. We follow the projection process of 3D patch by Zhu et al. (2023) to construct the applying function, with the projection matrix Maa-2d E I2 14x4 specified as follows:
Mad-2a
x E.,
(21)
This process is differentiable, which allows for the optimization of the adversarial patches.
In summary, we propose deterministic environmental model applicable all our experimental environments (e.g., EG3D, CARLA):
State = (h,, vr) E RR
Action
(Ah, Ar) E TR:
Transition Function T(s x, 8t + ae:
Observation Function Z(s,,)) = R(s,,)).
The primary distinction between simulations for different tasks lies in the feasible viewpoint regions. These are detailed in the implementation sections for each task, specifically in Appendices D and E.
D EXPERIMENT DETAILS FOR FACE RECOGNITION
D.1 CELEBA-3D
We use unofficial implemented GAN Inversion with EG3D (attps://github. com/ oneThousandl D00/EG3D-pro
and its default parameters convert 2D images from CelebA (Liu et al., 2018) into 3D latent representation w+. For the 3D generative model prior, we the EG3D models pre-trained on FFHQ which officially released https: //catalog... ngc. nvidia a. com/orgs /nwiiiiiiii dia ttteeteas/ssss htte///eeeerccccreeeeeee lower computational overhead, we desert the super-resolution module of EG3D and directly render RGB images of 112 x 112 with its neural renderer.
s/e33d. For
We further evaluate the quality the reconstructed CelebA-3D dataset. We measure image quality with PSNR, SSIM and LPIPS between original images and EG3D-rendees images from the same viewpoint. We later evaluate the identity consistency between the reconstructed 3D face and their original 2D face with identity consistency (ID), which is slightly different from the one in (Chan et al., 2022), for we measure them by calculating the mean ArcFace (Deng et al., 2019) cosine similarity score between pairs of views of the face rendered from random camera poses and its original image from CelebA.
As shown in Table 4, the learned 3D prior over FFHQ enables surprisingly high-quality single-view geometry recovery. So our reconstructed CelebA-3D equipped with high image quality and sufficient identity consistency with its original 2D form for later experiments. And the selection of reconstructed multi-view faces is shown in Figure
The CelebA-3D dataset inherits annotations from the original CelebA dataset, which is accessi- ble at hitps: /mmlab. ie. cuhk. edu. wttp////oootss/eeeeeaae.. html. The release this dataset for public access is forthcoming.
18

---

Published as a conference paper at ICLR 2024
Table 4: quantitative evaluation for CelebA-3D. The image size 112 112.
PSNR1
SSIM
LPIPS
IDT
CelebA-3D
21.28
.6001
.1314
.5771
Original
Reconstruction
Figure 5: qualitative evaluation for CelebA-3D. The first column presents the original face from CelcbA, and the subsoquent columns demonstrate the rendered multiview faces from inverted w with EG3D. The image size is 112 112.
D.2 DETAILS FOR ATTACKS
Impersonation and dodging in adversarial attacks against FR system. In adversarial attacks against FR systems, two main subtasks are identified: impersonation and dodging. Impersonation involves manipulating an image to deceive FR systems into misidentifying an individual as another specific person. Dodging, conversely, aims to prevent accurate identification by the FR system. It involves modifying an image so that the FR system either fails to detect a face or cannot associate the detected face with any data of its authentic identity in its database. These subtasks pose significant challenges to FR system security, highlighting the need for robust countermeasures in FR technology.
Attacks in pixel space. The Momentum Iterative Method (MIM) (Dong et al., 2018) and Expecta- tion over Transformation (BoT) (Athalye al., 201 2018b) focus refining adversarial patches in RGB pixel space. MIM enhances the generation of adversarial examples by incorporating momentum into the optimization process, which enhances transferring of adversarial examples. In contrast, EoT considers various transformations, such as rotations and lighting changes, thereby strengthening the robustness of attacks under diverse physical conditions. For implementation, we adhered the optimal parameters as reported in (Xiao et al., 2021), setting the number of iterations at N = 150, the learning rate at a = 1.5/20 255, and the decay factor at = 1. These parameters were consistent across all experiments. The sampling frequency for HoT was established at M 10.
Attacks in latent space of the generative model. In the context of GenAP (Xiao et al., 2021) and Face (Yang et al., 2022), the focus was on optimizing adversarial patches within the latent space of a Generative Adversarial Network (GAN). GenAP utilizes the generative capabilities of GANs to develop adversarial patches, whereas Face3DAdy is specifically designed for attacking facial recognition systems, accounting for 3D variations. Additionally, employed the latent space of EG3D for patch optimization, opting for the Adam optimizer (Kingma Ba, 2014) with learning
rate of y 0.01 and iteration count of N 150. The sampling frequency was also M 10.
19

---

Published as a conference paper at ICLR 2024
D.3 DETAILS FOR ADAPTIVE ATTACKS
Adaptive attack for defense baselines. To launch adaptive attacks against parameter-free purification-base defenses such as JPEG and LGS, we employ Backward Pass Differentiable Approximation (BPDA) as proposed by Athalye et al. (201 8a). This method assumes that the output from each defense mechanism closely approximates the original input. For adaptive attacks on SAC and PZ. we use their official implementations (Liu et al., 2022), incorporating dtraightThhough Estimators (STE) (Bengio et al., 2013) for backpropagation through thresholding operations.
Adaptive attack with uniform superset policy. In adaptive attacks for EAD, we leverage uniform superset approximation for the policy model. Thus, we have the surrogate policy
क = (Hemin hmax) x U(Qiiin Umax),
(22)
where Gr E I/min |max] x [Umin, Umax] and [hmins /.ma]], [Umins Umax] separately denotes the pre-defined feasible region for horizontal rotation (yaw) and vertical rotation (pitch). The optimization objective is outlined as follows, with simplified sequential representation for clarity?:
max Es ((Tr:
i-1
with (gr = f(HA(onps + " aa))=i j=1
(23)
s.t. e 10,0,1] 10,
where P1 denotes the distribution of initial state and C is the task-specific loss function.
Adaptive Attack Against Sub-Modules. An end-to-end attack may not always be the most effective strategy, particularly against defenses with complex forward passes. Targeting the weakest component is often sufficient. Therefore, propose two separate adaptive attacks: one against the perception model and another against the policy model. The attack on the perception model aims generate adversarial patch that corrupts the internal belief by (Sabour et al., 2015). The optimization objective for this attack is to maximize the Euclidean distance between the corrupted belief br and the benign belief br+, formulated follows:
max E E bb b,7 1/2.
with (T,, = S(AA
Σ
(24)
(o. b f(1o.))=1:)) s.t. Pe [0, Hxx
For the attack against the policy model, the goal is to create an adversarial patch that induces the action a; π (b;; with an invariant state 8; = $1 and generating erroneous predictions V.-. While the original problem be challenging with policy output constraint, we employ Lagrangian relaxation to incorporate the constraint into the objective and address the following problem:
policy model to output
0, thereby keeping the EAD model stationary
max EaNp, L(D.. (: 3)
with br. (:A)((()
(25)
s.t. E [0, 1,,xx
where 0 is constant that yields an adversarial example ensuring the model outputs actions.
"The recurrent inference procedure is presented sequentially in this section for simplicity.
20

---

Published as a conference paper at ICLR 2024
Adaptive Attack for the Entire Pipeline. Attacking the EAD model through backpropagation is infeasible due to the rapid consumption of GPU memory as trajectory length increases (e.g., 4 steps require nearly 90 GB of video memory). To mitigate this, 2016) to reduce memory consumption. By selectively recomputing parts of the computation graph defined by the 7-step EAD inference procedure, instead of storing them, this technique effectively reduces memory costs at the expense of additional computation. Using this method, we successfully attacked the entire pipeline along 4-step trajectory using an NVIDIA RTX 3090 Ti.
gradient checkpointing (Chen et al.,
Regarding implementation, we adopted the same hyper- parametrrrstrrss as Face3DAdv and considered the action bounds defined in Appendix D.5. For the constant e in the adaptive attack against the policy model, we employed bisection search to identify the optimal value as per Carlini & Wagner (2017). 7), finding 100 to be the most effective. Additionally, the evaluation results from these adaptive attacks and analysis are detailed in Appendix D.7 with main results in Table 9.
D.4 DETAILS FOR DEFENSES
JPEG compression. we set the quality parameter to 75.
Local gradients smoothing. We adopt the implementation at https: https:///gihhub..e com/ Fabiobrau/loca www.diiaddnn....g. innooooiinn,nnnttttoooonnnnn and maintain the default hyper-parameters claimed in Naseer al. (2019).
Segment and complete. We use official implementation https::/gittieii com/ joellliu wtaiii//eemeeetnnoonnnoenn
and retrain the patch segmenter with adversarial patches optimized by EoT (Athalye et al., 2018a) and USAP technique separately. We adopt the same hyper-parameters and training process claimed in Liu et al. (2022), except for the prior patch sizes, which we resize them proportionally the input image size.
Patchzero. For Patchzero (Xu et al., 2023), we directly utilize the trained patch segmenter of SAC, for they share almost the same training pipeline.
Defense against Occlusion Attacks. DOA is adversarial training- based method. We adopt the DOA training paradigm to fine-tune the same pre-trained IResNet-50 and training data as EAD with the code at https: : //gitttttt ccm/P2333 Seaa-oo-Tree cos-for-AA.. As for hyper-parameters, we adopt the default ones in Wu et al. (2019) with the training patch size scaled proportionally.
D.5 DETAILS FOR IMPLEMENTATIONS
Model details. In the context of face recognition, we implement EAD model with a composition of a pre-trained face recognition feature extractor and a Decision transformer, specifically, we select IResNet-50 as the the visual backbone to extract feature. For each time step t > 0, we use the IResNet-50 to map the current observation input of dimensions 112 x 112 x into an embedding with dimensionality of 512. This embedding is then concatenated with the previously extracted embedding sequence of dimensions (t (-1) x 512, thus forming the temporal sequence of observation embeddings 512) for Decision Transformer input. The Decision Transformer subsequently outputs the temporal-fused face embedding for inference as well as the predicted action. For the training process, we further map the fused face embedding into logits using linear projection layer. For the sake of simplicity, we directly employ the Softmax loss function for model training.
In experiments, we use pre-trained IResNer-50 with ArcFace margin on MSIMV3 (Guo al., 2016) from InsightFatt (Deng et al., 2019), which is available at https: //t::/// /t////tttu/bb/// com/ beepinsight//nsiightt wttp://niiiggaaaaaeeaeeeeeeeeeee mawterasteeeeeeeee model 200.
Phased training. It's observed that the training suffers from considerable instability when simulta- neously training perception and policy models from scratch. primary concern is that the perception model, in its early training stages, is unable to provide accurate supervision signals, leading the policy network to generate irrational actions and hindering the overall learning process. To mitigate this issue, initially train the perception model independently using frames obtained from random
21

---

Published as a conference paper at ICLR 2024
action policy, namely offline phase. Once achieving stable performance from the perception model, we proceed to the online phase and jointly train both the perception and policy networks, employing Algorithm 1,thereby ensuring their effective coordination and learning.
Meanwhile, learning offline with pre-collected data in the first phase proves to be significantly more efficient than online learning through interactive data collection from the environment. By dividing the training process into two distinct phases offline and online, we substantially enhance training efficiency and reduce computational costs.
Training details. To train EAD for face recognition, we randomly sample images from 2, 500 distinct identities from the training set of CelebA-3D. we adopt the previously demonstrated phased training paradigm with hyper-parameters listed in Table 5.
Table 5: Hyper-parameters of EAD for face recognition
Hyper-parameter
Value
Lower bound for horizontal rotation (heai)
-0.35
Upper bound for horizontal rotation (feax)
0.35
Lower bound for vertical rotation (Umin)
0.25
Upper bound for vertical rotation (Umas)
0.25
Ratio of patched data (Fpucs)
0.4
Training epochs for offline phase (ITomine)
50
learning rate for offline phase (Iromiie)
IE-3
batch size for offline phase (buriie)
64
Training epochs for online phase
50
learning rate for online phase (Iroaline)
1.5E-4
batch size for offline phase (beatine)
48
D.6 COMPUTATIONAL OVERHEAD
This section evaluates our method's computational overhead compared to other passive defense baselines in facial recognition systems. The performance assessment is conducted on a NVIDIA GeForce RTX 3090 and an AMD EPYC 7302 16-Core Processor, using a training batch size of 64. SAC and PZ necessitate training a segmenter to identify the patch area, entailing two stages: initial training with pre-generated adversarial images and subsequent self-adversarial training (Liu et al., 2022; Xu al., 2023). DOA, adversarial training-based approach, requires retraining the feature extractor (Wu et al., 2019). Additionally, EAD's training involves offline and online phases, without involving adversarial training.
As indicated in Table 6, although differential rendering imposes significant computational demands during the online training phase, the total training time of our EAD model is effectively balanced between the pure adversarial training method DOA and the partially adversarial methods like SAC and PZ. This efficiency stems mainly from our unique USAP approach, which bypasses the need for generating adversarial examples. thereby boosting training efficiency. In terms of model inference, our RAD, along with P7. and DOA, demonstrates superior speed compared to LGS and SAC. This is attributed to the latter methods requiring CPU-intensive, rule-based image preprocessing, which diminishes their inference efficiency.
Regarding detailed training, the EAD model was trained following the configuration in Appendix D.5. The offline training utilized NVIDIA Tesla A100 GPUs for approximately hours (210 minutes). Due to the substantial memory demands of EG3D differential rendering, the online training phase required 8 NVIDIA Tesla 100 GPUs and extended to about 14 hours (867 minutes). Figure illustrates the training curves of our method.
22

---

Published as a conference paper at ICLR 2024
Table 6: omputational overhead comparison of different defense methods in face recognition. We report the training and inference time defense NVIDIA GeForce RTX 3090 and an AMD EPYC 7302 16-Core Processor with the training batch size as 64.
Overall Training Time (GPU
Training Time per batch (s)
Inference Time per Instance (ms)
Parametric Model
Training Epochs
Method # Params (M)
hours)
JPEG
non- parametric non- parametric
9.65
LGS
26.22
SAC
26.43
44.71
segmenter
50 + 10
0.152/4.018
104
PZ
11.88
feature extractor
DOA
43.63
100
1.732
376
8.10
policy and perception model
EAD
57.30
50
0.595/1.021
11.51
Lo0
Loos
Epoch
Epoch
(a) Offline training.
(b) Online training.
Figure The training curves EAD model face recognition.
D.7 MORE EVALUATION RESULTS
Evaluation with different patch sizes. To further assess the generalizability of the EAD model across varying patch sizes and attack methods, we conduct experiments featuring both impersonation and doding attacks. These attacks share similarities with the setup illustrated Table 1. Although with different patch sizes, the results in Table 7 and Table 8 bear considerable resemblance to those displayed Table 1. This congruence further supports the adaptability of the EAD model in tackling unseen attack methods and accommodating diverse patch sizes.
Evaluation with different attack iterations. Figure 7 shows that EAD generalizes well under wide spectrum of attack iterations, and the only ASR slightly increases when the patch gets larger. As for dodging attacks, Figure 8 demonstrates EAD's superior generalizability compared to other defense strategies when countering dodging attacks. This akin to the observations Figure 3, where EAD, trained exclusively with patches comprising 10% of the image, consistently exhibits a low attack success rate, even with increasing patch sizes and attack iterations. This further underscores FAD's exceptional generalizability. One subtle phenomenon to note is that the segmenter-basee method (i.e. SAC and PZ) degrades when the attack iterations become smaller. Because the patches become more imperceptible.
Evaluation with different adaptive attacks. As Table 9 demonstrates, our original adaptive attack using USP more effective than tracing the authentic policy of EAD (overall). This may be
23

---

Published as a conference paper at ICLR 2024
Table 7: The white-box impersonation attack success rates face recognition models with different patch sizes. denotes methods are trained with adversarial examples.
8%
10 %
12% %
Method
MIM
EoT
GenAP 3DAdv
MIM
EoT
GenAP 3DAdv
MIM
EoT
GenAP 3DAdv
Undefended
100.0
100.0
99.00
98.00
100.0
100.0
100.0
99.00
100.0
100.0
100.0
99.00
JPEG
99.00
100.0
99.00
93.00
100.0
100.0
99.00
99.00
100.0
100.0
99.00
99.00
LGS
5.10
7.21
33.67
30.61
6.19
7.29
41.23
36.08
7.21
12.37
61.85
49.48
SAC
6.06
9.09
67.68
64.64
1.01
3.03
67.34
63.26
5.05
4.08
69.70
66.32
4.17
5.21
59.38
45.83
2.08
3.13
60.63
58.51
4.17
3.13
60.63
58.33
SACT
3.16
3.16
18.94
22.11
2.10
3.16
21.05
16.84
3.16
4.21
15.78
18.95
PZ
3.13
3.16
19.14
27.37
2.11
3.13
20.00
30.53
5.26
5.26
18.95
28.42
DOA
95.50
89.89
96.63
89.89
95.50
93.26
100.0
96.63
94.38
93.26
100.0
100.0
EAD (ours)
4.12
3.09
5.15
7.21
3.09
2.06
4.17
8.33
3.09
5.15
8.33
10.42
Table white-hox dodging attack success rates (%) on face recognition models with different patch sizes. denotes methods are trained with adversarial examples.
8%
10
12%
Method
MIM
EoT
GenAP 3DAdv
MIM
EoT
GenAP 3DAdv
MIM
EoT
GenAP 3DAdv
Undefended
100.0
100.0
99.00
89.00
100.0
100.0
100.0
95.00
100.0
100.0
100.0
99.00
JPEG
98.00
99.00
95.00
88.00
100.0
100.0
99.00
95.00
100.0
100.0
100.0
98.00
LGS
49.47
52.63
74.00
77.89
48.93
52.63
89.47
75.78
55.78
54.73
100.0
89.47
SAC
73.46
73.20
92.85
78.57
80.06
78.57
92.85
91.83
76.53
77.55
92.85
92.92
PZ
6.89
8.04
58.44
57.14
8.04
8.04
60.52
65.78
13.79
12.64
68.49
75.71
SACT
78.78
78.57
79.59
85.85
81.65
80.80
82.82
86.73
80.61
84.69
87.87
87.75
PZ
6.12
6.25
14.29
20.41
7.14
6.12
21.43
25.51
11.22
10.20
21.49
30.61
DOAT
75.28
67.42
87.64
95.51
78.65
75.28
97.75
98.88
80.90
82.02
94.38
100.0
EAD (ours)
0.00
0.00
2.10
13.68
2.11
1.05
6.32
16.84
2.10
3.16
12.64
34844
Anack Siccess Rote (16)
Attakk Success Eate
Uskinded
RADESS
Iteration
Heration
(a) Impersonation.
(b) Dodging.
Figure 7: Performance EAD under varying attack iterations and patch sizes. The adversarial patches are
crafted by 3DAdv.
attributed to vanishing exploding gradients (Athalye et al., 2018a) that impedes optimization. This problem is potentially mitigated by our approach of computing expectations over a uniform policy distribution. In the meantime, The results reaffirm the robustness of EAD against a spectrum of adaptive attacks. It further shows that EAD's defensive capabilities arise from the synergistic integration of its policy and perception models, facilitating strategic observation collection rather than learning a short-cut strategy to neutralize adversarial patches from specific viewpoints.
Impact of horizon length. We later demonstrate how the horizon length (i.e., decision steps) influence the performance of EAD in Figure 9. Two key observations emerge from this analysis: 1) The standard accuracy initially exhibits an increase but subsequently declines as t surpasses the maximum trajectory length which is manually set in model training, detailed in Figure 9a. 2)
24

---

Published as a conference paper at ICLR 2024
8%
10%
12%
Attick Stuccss BLace
Ieeraion
Iteration:
Iteration
Undelcnied
JP5G
LGS
DOA
Figure 8: Comparative evaluation of defense methods across varying attack iterations with different adversarial patch sizes. The sizes of the adversarial patches for each subfigure, from left to right, are 8%, 10%, and 12%, respectively. The adversarial patches crafted by 3DAdv for dodging.
Table Evaluation adaptive attacks on EAD Model. Columns with USP represent results obtained by optimizing the patch with expected gradients over the Uniform Superset Policy (USP). And perception and policy separately represent adaptive attack against single sub-module And overall denotes attacking EAD by following the gradients for along the overall (4 step) trajectory with gradient-checkpointnng
Dodging
Impersonation
USP
Perception
Policy
Overall
USP
Perception
Policy
Overall
ASR (%)
22.
10.11
16.84
15.79
8.33
1.04
9.38
7.29
The changes in the similarity between face pairs affected by adversarial patches consistently decline. Specifically, for impersonation attacks (Figure 9b). the change in similarity implies an increase, while decrease is noted for dodging attacks (Figure 9c). This trend indicates that more information from additional viewpoints accumulated during the decision process, effectively mitigating the issues of information loss and model ballucination caused by adversarial patches.
Efficiency of EAD's policy. Although the efficiency of EAD's policy is theoretically demonstrated it has proven be greedy informative strategy in Sec. 3.2, we further validate the superiority of the EAD's policy empirically by comparing the performance of EAD and EAD integrated with random movement policy, hereafter referred as EADRAND. Both approaches share identical neural network architecture and parameters. Figures 9b and 9c illustrate that the EADRAND is unable to mitigate the adversarial effect with random exploration, even when more action are employed. Consequently, exploration efficiency of the random policy significantly lags behind the approximate solution of the greedy informative strategy, which is derived from parameter learning.
Coisee Sealiuity Cloage
Cootee Sww. www.oott Chasse
Sepp
Stop
Sxy
(a) Standard accuracy.
(b) Similarity change in imperson- ation attack.
(c) Similarity change in dodging attack.
Figure 9: Model's performance variation along the number of decision steps.
Evaluation under various viewpoints. To further assess the versatility of defenses over multiple viewpoints, we examine model performance under 3D viewpoint variation. For all the experiments, utilize facial images requiring specific movement ranges for the eruciform rail, spanning from
25

---

Published as a conference paper at ICLR 2024
-15e to 15", corresponding to yaw and pitch respectively. Moreover, these conditions linearly combined to form a new category termed as mixture. We employ Face as the adversary. provided that solely adversarial examples crafted with Face3DAdv hat do not exhibit performance degradation when encountering viewpoint variations. Table 10 demonstrates that EAD consistently upholds superior performance,
under varied viewpoint conditions.
Table 10: Standard accuracy (%) and attack success rates (%) of Face3DAdv face recognition models under various testing viewpoint protocols. "Acc." denotes the accuracy best threshold calculated with standard protocol from LFW (Huang et al., 2007). "Imp." and "Dod." respectively represents the attack success rate of impersonation and dodging attacks.
Yaw
Pitch
Mixture
Method
Acc.
Imp.
Dod.
Acc.
Imp.
Dod.
Acc.
Imp.
Dod.
Undefended
98.64
94.74
87.24
98.87
91.95
$6.04
98.39
88.35
83.65
JPEG
98.88
91.48
86.43
98.94
89.26
84.83
98.39
84.94
82.60
LGS
93.12
29.32
72.39
92.42
26.23
69.32
91.92
24.54
68.50
SAC
95.47
57.22
75.21
96.30
59.83
74.84
95.24
53.83
74.04
PZ
97.81
41.07
51.91
97.77
39.99
49.66
97.39
36.35
48.54
EAD (ours)
99.28
13.68
7.21
D.8
MORE QUALITATIVE RESULTS
Qualitative comparison of different version of SAC. SAC is preprocessing-basee method that adopts segmentation model to detect patch areas, followed by "shape completion" technique to extend the predicted area into a larger square, and remove suspicious area (Liu al., 2022). As shown in Figure 10, the enhanced SAC, while exhibiting superior segmentation performance in scenarios like face recognition, inadvertently increases the likelihood of masking critical facial features such eyes and noses. This leads reduced ability of the face recognition model to correctly identify individuals, thus impacting its performance dodging attacks.
Regular SAC
Enhanced SAC
Adversarial Image
Image Processed with Segmenter- Mask
Image Processed with Completed Mask
Figure 10: Qualitative results of SAC trained with different data. The first column present the adversarial image processed by regular SAC which trained with patch filled with Gaussian noise, while the subsequent column demonstrate the one processed by enhanced SAC. The adversarial patches generated with 3DAdv and occupy 8% of the image.
More qualitative results of EAD. As shown in Figure 11, EAD model prioritizes distinct viewpoint to improve target understanding while simultancously maintaining a perspective where adversarial patches minimally effective.
26

---

Published as a conference paper at ICLR 2024
Step
Step
Step
Step
State
(1.15, -6.88)
(10.43, -9.16)
(-9.21, 2.81)
(-14.89, 9.03)
Action
(9.28, -2.28)
(-19.64, 11.97)
(-5.68, 6.22)
Figure Qualitative results of EAD. The represents the yaw and pitch of camera, and the action indicates the camera rolation predicted by EAD's policy model. The adversarial patches are generated with 3DAdv and occupy 8% of the image.
E EXPERIMENT DETAILS FOR OBJECT DETECTION
E. DETAILS FOR EXPERIMENTAL DATA
Details on CARLA. For model testing, we use the scene basis bil billboardds from CARLA- GEAR (Nesti al., 2022). For each testing scene, we randomly place different objects within the scene to ensure the diversity of testing scenarios. In this setting, the adversarial patch are affixed to billboards on the street. scene examples are represented in Figure 12.
Figure Scene examples from bil Iboard05 CARLA- GEAR. The first two images belong to different scenes. The first image is benign image, while the second image is contaminated with adversarial patches attached to the billboards. For the last two images, they rendered from different viewpoints.
Details on EG3D. We use a pre-trained EG3D model on ShapeNet Cars at https: nge. wwiinviiii viviaaa... com/ orgs/ http///rgg//iiiiiieeee. /iviiiiiiaaaa a teams//////////aaa///////// researchh /mwo/oomlsall/ /eg3d generate multi- view car images with the resolution of 128 x 128. And the seed is 114514. As for data annotation, We select the pre-trained YOLOv5x model which is the largest and performs the best (Jocher al., 2021). We fine-tune it the generated car images for 500 iterations and then utilize it as an annotation model label bounding boxes for later generated images.
E.2 DETAILS FOR ATTACKS
As for MIM (Dong et al., 2018), set the decay factor μ 1.0, while the size of the Gaussian kernel is 21 for TIM (Dong et al., 2019).
Details CARLA. For attacks in CARLA, utilize the attack against Mask-RCNN imple- mented by CARLA-GEAR at httph: /ttp:://gttttb/// http:///c/ceeeeaaaaaaaaateaeeeee its default hyper-parameters.
and
Details on EG3D. For attacks in environment powered by EG3D, we set the number of iterations as N 100 and the learning rate 10/255.
27

---

Published as a conference paper at ICLR 2024
E.3 DETAILS FOR DEFENSE
For SAC" and PZ", We use official implementation and its pre-trained patch segmenter checkpoint for object detection at https:/ /pes://githine com/ Wteellllu/eeeie htt.i///eeeennnnnooollleeeeeeeee AndComplete, which is trained with adversarial examples generated with PGD (Madry et al., 2017).
E.4 DETAILS FOR IMPLEMENTATION
Model details. For experiment conducted on simulation environment based on EG3D, We imple- EAD for object detection with combination of YOLOOvn and Decision Transformer. For each time step t > 0, given the current observation of dimensions 640 640 x 3 as input, the smallest feature maps (4 4 512) within the feature pyramid are utilized. These maps are extracted via the YOLOv5 backbone and reshaped into sequence with dimensions 16 x 512. we utilize the smallest 512) in the feature pyramid which extracted with the YOLOv5 backbone and reshape it into sequence (16 x 512). To concatenate it with the previous extracted observation sequence 16(( - 1) 512, we have a temporal sequence of visual features as the input of Decision Transformer, and it output the temporal-fuse action. To predict the bounding boxes and target label which is required in object detection, we reshape the cenporal-fused visual sequence back to its original shape and utilize it part of the feature pyramid in the later stage for object detection in YOLOv5n. As for experiment on CARLA, utilize combination of Mask-RCNN and Decision Transformer with similar implementation technique.
feature maps
visual feature sequences (16 512) and predicted
We use the official implementation and pre-trained model checkpoints for both YOLOv5n and YOLOv5x at https:: /ttp:://///tttta////e/ee thub. commoo//c///oooooo wuluulllllll.eo.. wewttralyyissyyllllooo As for Mask-RCNN, we adopt the implementation from torchvision.
Training details. We adopt similar training paradigm EAD for face recognition and set the hyper-parameterd of EAD for object detection as follows:
Table 11: Hyper-Parameters of EAD for object detection
Hyper-Parameter
Value
Lower bound for horizontal rotation (hein)
-T
Upper bound for horizontal rotation (ftmx))
x/2
Lower bound for vertical rotation (Vmin)
0
Upper bound vertical rotation (Fmax)
Ratio of patched data (rpucs)
0.4
Training iterations for offline phase
500
learning rate for offfine phase (Iromine)
2E-5
batch size for offline phase (bwmine)
64
Training iterations for online phase lcarning rate for online phase (Iroaline)
80
1E-5
batch size for online phase (boatine)
32
Figure 14: Adversarial examples with patches of dif- ferent shapes. From left to right: rectangle, ellipse and diamond.
Figure Different shapes used for evaluation. From left to right: rectangle, ellipse and diamond.
28

---

Published as a conference paper at ICLR 2024
E.5 MORE EVALUATION RESULTS ON EG3D
Evaluation with different patch shapes. Credits to other shapes of the patch are either too skeptical in face recognition non-rigid which causes 3D inconsistency, conduct corresponding experiments in the object detection scenario to evaluate the generalizability of EAD across different shapes of the patch. Specifically, we evaluate vectaggle-taiine EAD with adversarial patches of varying shapes, while maintaining fixed occupancy of the patch area. The shapes used for evaluation are depicted in Figure 13, with their respective adversarial examples presented in Figure 14. We present the iterative defense process in Figure 15.
Benign Result
Adversarial Result
Step
Stop
Step
Step
ner 2.75
Qualitative results of EAD simulation environment powered by EG3D. The first two columns present the detection results be nigh and adversarial images, and the subsequent columns demonstrate the interactive inference steps that model took. The adversarial patches are generated with MIM and occupy of the image.
Figure 15:
As demonstrated in Table our model surpasses other methods in terms both clean and robust accuracy, even when faced with adversarial patches of diverse unencountered shapes. These results further underscore the exceptional generalizability of EAD in dealing with unknown adversarial attacks.
Table The mAP (%) YOLOv5 under different white-box adversarial attacks and patch shapes.
denotes
methods are trained with adversarial examples.
Rectangle
Ellipse
Diamond
Method
Clean
PGD
MIM
TIM
PGD
MIM
TIM
PGD
MIM
TIM
Undefended
63.4
49.5
48.8
50.2
56.4
55.9
56.8
58.9
59.0
59.7
JPEG
62.7
51.1
50.6
57.8
57.5
57.4
57.5
60.1
59.9
59.9
LGS
64.7
64.8
55.4
60.7
65.0
66.6
61.9
65.6
66.7
63.7
SAC!
63.4
49.9
47.9
48.5
52.9
51.9
52.7
57.3
56.2
56.6
Pz
63.4
50.5
50.8
50.6
56.6
57.2
57.1
59.1
59.9
59.9
EAD (ours)
75.9
64.1
58.8
67.9
70.8
67.3
72.8
71.5
69.6
74.1
29


Reviews:
[{"review_id": "uyTGiJc76w", "reviewer": "Reviewer_iMPU", "summary": "The paper proposes a defense against patch-based adversarial attacks on visual recognition models. It utilizes the concept of emodied perception (wherein the model is allowed to interact with its environment to refine its belief) to mitigate the effect of 'out-of-place' patches in the scene and, in turn, improve the model's robustness against patch-based attacks. The authors implement embodied perception as a partially observable markov decision process. The overall system comprises of two new models: a perception model and a policy model. The perception model maintains an 'understanding' of the scene which it progressively updates based on new observations. The policy model dictates a transition process that is focused on improving the quality of observations to improve recognition. Given an initial observation, the system progressively refines its belief using the perception and policy models. Through two visual recognition tasks: face recognition and object detection, authors demonstrate the effectiveness of emodied perception based defense to not only improve robustness against seen and unseen attacks, but also improve standard performance.", "questions": "1. How does the computational cost of the proposed method compares to the baselines?\n2. How does the method fare against an adaptive attack targeting the weakest component in the pipeline (see further details in the Weaknesses section)? I'd love to increase my rating further if you can better convince me that the defense is robust against adaptive attacks.\n3. Why is the enhanced version of SAC less robust against dodging attacks than regular SAC (in table 1)?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "1. Overall, the writing quality and presentation of the paper is excellent. The authors lay out the motivation behind their method and challenges behind implementing it adequately. All necessary background is provided so that a person unfamiliar with the topic can follow along. The figures and tables present information effectively.\n2. The authors propose a novel application of emodied perception, using it as a defense against patch-based adversarial attacks, making it the first defense of its kind.\n3. The design of the method is clever and intricate, and is intuitively grounded in the understanding of human perception.\n4. The defense is task agnostic, and should conceptually work with any visual recognition task.\n5. The authors provide a theoretical understanding for the effectiveness of EAD using information theory.\n6. The ablation presented in the paper is thorough, and covers all the different components of the method as well as newly introduced hyperparameters.\n7. The method improves both standard and adversarial performance relative to prior defenses. Interestingly, standard performance is improved even over undefended baseline. Overall, improvements introduced by proposed method appear substantial.", "weaknesses": "1. There is no discussion regarding the computational cost of the proposed method and how it compares to the undefended baseline as well as prior defenses.\n2. An end-to-end attack is not necessarily the strongest adaptive attack, especially for defenses with complicated forward passes. Attacking the weakest component should be sufficient. To identify the weakest component, authors should try independently attacking perception and policy models to make them less effective in their respective tasks. For example, attacking perception model would involve generating an input that forces the perception model to output a corrupted internal belief vector b_t (using a latent space attack [a]).\n3. There are no theoretical results establishing how the approximations used in Sec 3.3 relate to the actual quantities. If possible to obtain, this would be nice to have.\n\n**References**\n\n[a] Sabour, S., Cao, Y., Faghri, F., and Fleet, D. J. Adversarial manipulation of deep representations. International Conference on Learning Representations, 2016.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xoefIk8UTS", "reviewer": "Reviewer_V4oi", "summary": "The paper proposes a model robustness approach Embodied Active Defense (EAD) for perception models against adversarial patches. EAD comprises of perception module to extract the image observation features and policy module to interact with the environment (modeled by decision transformer). The paper hypothesizes that passive robustness methods (i.e without temporal information or scene contextual information) would not be sufficient for unseen adversarial attacks and needs active feedback from the environment to achieve better model robustness. EAD training is a two stage learning approach performed in a dynamic environment (i.e. the generative space of faces using EG3D or CARLA simulation)  - 1) training the perceptual model for the specific task (for eg face recognition or object detection) using random policy 2) co-training the perception module along with the policy. The paper shows robustness to different types of patch based adversarial attacks and adaptive adversarial attacks. The paper also proposes to introduce uniformly sampled noise as adversarial examples while training the model that perturbs the observation. The paper empirically shows adversarial robustness improvement across unseen type of attacks and also shows improvement on clean samples (i.e overall test accuracy of a task in scenarios where adversarial samples are not introduced).", "questions": "It would be helpful if the paper could answer/clarify the following questions:\n1. How much training time/ wall clock time and memory resources does it take for the EAD defense (both at training and inference phase).\n2. The general assumption about adversary is that it is not bound by computation resources. For example the statement in the paper for adaptive attacks “While the deterministic and differential approximation could enable backpropagation through the entire inference trajectory of EAD, the computational cost is prohibitive due to rapid GPU memory consumption as trajectory length τ increases”, here for Face recognition can we do a white box attack by following the gradients for all the 4 -step trajectory?\n3. In the paper the description for Figure 3 mentions that “subsequent active interactions with the environment progressively reduce the similarity between the positive pair”. Is this a typo, should this be “reduce the dissimilarity” or “increase the similarity”?\n4. In Algorithm 1, the perturbed observation O’t is not being used, is there a typo there ?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper explores an adversarial robustness technique that helps the model to achieve better adversarial robustness in a dynamic environment. The proposed approach “Embodied Active Defense (EAD)” achieves improvement on various patch adversarial attack methods for the tasks of face recognition and object detection. The paper provides ablation experiments to analyze the various components of EAD algorithm. Ablation experiments regarding the strength of adversarial attacks (by increasing the patch size or increasing the iteration of iterative adversarial attacks) sheds light on the robustness of the proposed model against strong version of adversarial attacks. The supplementary material contains code for reproducibility", "weaknesses": "Overall, the paper writing style and organization needs improvement to allow the reader to easily understand the paper.\nIt would be helpful for the reader to get a better understanding for the following:\n1.  Some general suggestions regarding paper organizing. It would be great to introduce the problem statement for each of the tasks:\n    1.  For eg. definition/explaining of the subtasks of FR : Impersonation and dodging.\n    2.  Environmental model definition (State, Action, Transition Function and Observation Function similar to Face Recognition task) for object detection on CARLA simulator or ShapeNet EG3D.\n    3. In the related section, a brief explanation of the attacks used for evaluation.\n    4. Annotation definitions before introducing in the paper (for eg y^{_}_t prediction of the model) \n    5. (Optional) a readme for the codebase, to understand the outline of the codebase.\n2. The paper makes a claim that passive adversarial defense approach are not sufficient for dynamics environment. This claim should be supported by a passive defense used as a baseline. Also, it would be great to see the extra amount of training resources used to train the active embodied model vs a passive defense approach (for eg Madry's PGD adversarial training). It would also help the readers if some other active defenses would be used as a baseline in order to establish the efficacy of the proposed approach.\n3. The experimental (both training and evaluation) setting does not seem sufficient and scalable enough to make conclusions that it would general. For example “To train EAD for face recognition, we randomly sample images from 2, 500 distinct identities from the training set of CelebA-3D.” and “we report the white-box attack success rate (ASR) on 100 identity pairs in both impersonation and dodging attacks with various attack methods”.\n\nMinor Typo/ suggestions  \nEAD comprises two primary submodules: -> EAD comprises of two primary submodules:\n\nNotely, it maintains or even improves standard accuracy due to -> Notably, it maintains or even improves standard accuracy due to\n\nFormally, It derives a strategic action -> Formally, it derives\n\n“they have now developed to perceive various perception models”\n\n“It is noteworthy this presents a more versatile 3D formulation for adversarial patches”\n\nfrom given scene x with its annotation y from another modal like CLIP -> from given scene x with its annotation y from another model like CLIP\n\n“In D.5 MORE EVALUATION RESULTS ON EG3D, We present the iterative defense process in Figure ??.”  \n\nIn the statement “EAD presents a promising direction for enhancing robustness without any negative social impact”, it might be helpful to limit this statement as the one of the tasks being used is facial recognition that could have some unwanted impact .", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ua7uoChCVX", "reviewer": "Reviewer_u8yn", "summary": "In this work, the authors propose an active defense strategy that leverages active movements and recurrent perceptual feedback from the environment to defend against arbitrary adversarial patch attacks. The experiment results show that the proposed strategy can outperform SOTA passive defense strategies in terms of effectiveness and generalizability.", "questions": "1. For those adversarially-trained passive defense models, would it be beneficial to enhance them with 3D-augmented data, e.g., feeding multiple views of the same patch-attacked human face during training, if the lack of 3D environment awareness is the problem here?\n2. There is one part I feel confused about in the algorithm box: $O_t′ \\gets A(O_t, P; S_t)$ and $O_{t+1}'\\gets A(O_{t+1},P;S_{t+1})$, where do $O_t'$ and $O_{t+1}'$ go? They are not explicitly used or referenced anywhere else post-assignment. Are clean observations $O_t$ and $O_{t+1}$ overwritten by $O_t'$ and $O_{t+1}'$ after applied with the adversarial patch by default?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The authors present a pioneering approach in the realm of adversarial robustness by introducing the first proactive defense strategy with embodied perception. It adds to the current defense strategies against patch attacks, which were predominantly passive.\n2. The experiment findings clearly indicate the superiority of the proposed strategy over previous SOTAs. The comprehensive experiments demonstrate that an embodied model with environmental interaction ability can not only mitigate the uncertainty posed by adversarial attacks easily but also be trained with weak attacks like uniform-perturbed patches.", "weaknesses": "1. In Section 4.2, the authors mention the effectiveness against adaptive attack; however, I find it hard to understand the reason why a surrogate **uniform** superset policy distribution would necessitate an optimized patch to handle various action policies, as this uniform surrogate may not contain any useful information about the policy model in EAD for the adversary to attack.\n2. The paper could benefit from improved clarity, especially for readers with a foundational understanding of adversarial robustness but limited exposure to RL/Robotics. As it stands, the document is dense with jargon, making it challenging to navigate and comprehend upon the initial read.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JaQ2MLlmFE", "reviewer": "Reviewer_YXUb", "summary": "It develops Embodied Active Defense (EAD), a proactive defensive strategy that actively contextualizes environmental information to address misaligned adversarial patches in 3D real-world settings. To achieve this, EAD develops two central recurrent sub-modules, i.e., a perception module and a policy module, to implement two critical functions of active vision. These models recurrently process a series of beliefs and observations, facilitating progressive refinement of their comprehension of the target object and enabling the development of strategic actions to counter adversarial patches in 3D environments.", "questions": "see the weakness.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "To optimize learning efficiency, it  incorporates a differentiable approximation of environmental dynamics and deploy patches that are agnostic to the adversary’s strategies.\n\nExtensive experiments demonstrate that EAD substantially enhances robustness against a variety of patches within just a few steps through its action policy in safety-critical tasks (e.g., face recognition and object detection), without compromising standard accuracy.\n\nFurthermore, due to the attack-agnostic characteristic, EAD facilitates excellent generalization to unseen attacks, diminishing the averaged attack success rate by 95% across a range of unseen adversarial attacks.\n\nIt theoretically demonstrates the effectiveness of EAD from the perspective of information theory. A well-learned EAD model for contrastive task adopts a greedy informative policy to explore the environment, utilizing the rich context information to reduce the abnormally high uncertainty of scenes caused by adversarial patches.", "weaknesses": "It mentions the policy model with actions, and states. It is better to provide more details or examples to specify what the actions and states look like or their physical meanings if any. \n\nIt is better to discuss the complexity for the training and inference of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)