Diffusion Policy Attacker: Crafting Adversarial Attacks for Diffusion-based Policies
Authors: Yipu Chen
Haotian Xue
Yongxin Chen
Venue: NeurIPS 2024 poster (2024)
Keywords: Diffusion Model
Adversarial Attacks
Robot Learning
Primary area: generative_models

Abstract: Diffusion models have emerged as a promising approach for behavior cloning (BC), leveraging their exceptional ability to model multi-modal distributions. Diffusion policies (DP) have elevated BC performance to new heights, demonstrating robust efficacy across diverse tasks, coupled with their inherent flexibility and ease of implementation. Despite the increasing adoption of Diffusion Policies (DP) as a foundation for policy generation, the critical issue of safety remains largely unexplored. While previous attempts have targeted deep policy networks, DP used diffusion models as the policy network, making it ineffective to be attacked using previous methods because of its chained structure and randomness injected. In this paper, we undertake a comprehensive examination of DP safety concerns by introducing adversarial scenarios, encompassing offline and online attacks, global and patch-based attacks. We propose DP-Attacker, a suite of algorithms that can craft effective adversarial attacks across all aforementioned scenarios. We conduct attacks on pre-trained diffusion policies across various manipulation tasks. Through extensive experiments, we demonstrate that DP-Attacker has the capability to significantly decrease the success rate of DP for all scenarios. Particularly in offline scenarios, we exhibit the generation of highly transferable perturbations applicable to all frames. Furthermore, we illustrate the creation of adversarial physical patches that, when applied to the environment, effectively deceive the model. Video results are
put in: https://sites.google.com/view/dp-attacker-videos/.

Diffusion Policy Attacker: Crafting Adversarial Attacks for Diffusion-based Policies
Yipu Chen Georgia Institute of Technology ychen3302 @ gatech.edu
Haotian Xue
Georgia Institute of Technology htxue.aiee gatech.edu
Yongxin Chen
Georgia Institute of Technology yongchen @ gatech.edu
Abstract
Diffusion models (DMs) have emerged as a promising approach for behavior cloning (BC). Diffusion policies (DP) based DMs have elevated BC perfor- mance to new heights, demonstrating robust efficacy across diverse tasks, coupled with their inherent flexibility and ease of implementation. Despite the increas- ing adoption of DP as foundation for policy generation, the critical issue of safety remains largely unexplored. While previous attack attempts have targeted deep policy networks, DP used diffusion models as the policy network, mak- ing it ineffective be attacked using previous methods because of its chained structure and randomness injected. In this paper, we undertake a comprehen- sive examination of DP safety concerns by introducing adversarial scenarios, encompassing offline and online attacks, and global and patch-basee attacks. We propose DP-Attacker, suite of algorithms that can craft effective adversarial attacks across all aforementioned scenarios. We conduct attacks on pre-trained diffusion policies across various manipulation tasks. Through extensive experi- ments, we demonstrate that DP-Attacker has the capability to significantly decrease the performance of DP for scenarios. Particularly in offline scenarios, DP- Attacker can generate highly transferable perturbations applicable to all frames. Furthermore, we illustrate the creation of adversarial physical patches that, when applied to the environment, effectively deceive the model. Video results are put in: https: htt///sieee //sites.....///////////////tttet .owgge.oo....oooooooooooo.o..... wwwogle.com//iee..oooooooooeoooo com/viewwdiffustttae.eee/e/ www.fffuuooooooicccoo. attacker.
Introduction
Behavior Cloning (BC) [40] is pivotal area in robot learning: given expert demonstration dataset, it aims to train a policy network in a supervised approach. Recently, diffusion models [16, 49] have become dominant BC, primarily due their strong capability distribution. The resulting policy learner, termed Diffusion Policy (DP) [9, 18], can generate the action trajectory from pure Gaussian noise conditioned on the input image(s). An increasing number of works are adopting DP as action decoder for BC across various domains such robot manipulation [12, 58, 7], long-horizon planning [35, 26) and autonomous driving [29].
modeling multi-modal
Adversarial attack [31, 14] has been haunting deep neural networks (DNN) for long time: a small perturbation the input image will fool the DNN into making wrong decisions. Despite the remarkable success of diffusion policies in BC, their robustness under adversarial attacks [31, 14]
indicates equal contribution. Correspondence to: ychen33020gatecch.ed
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

@i Diffasion Policy Allacker
(b) Advensriial Perturbation for DP
(c) Advensarial Patch for
DP-Attacker (!'0) [online]
Comtra Hacker
-P-Attacker (1P,
6' 8 DP-Attacker (f", [offline]
8
P + 5
Fd
Fy
00000 o
66
Fa
Pe (5gl1)
Po(t,
Figure I: Adversarial Attacks against Diffusion Policy: We aim to attack robots controlled with visual-based DP, unveiling hidden threats to the safe application of diffusion-based policies. (a) By hacking the visual inputs, we can fool the diffusion process into generating wrong actions T (in red). We propose Diffusion Policy Attacker(DP SottrrDDD-tttkkeeee Attacker), which effectively attack the DP by (b) hacking the global camera inputs using small visual perturbations under both online and offline settings (c) attaching adversarial patch into the environment. The online settings current visual inputs at t-th timestep I' to generate vire-nariant perturbations 5r, while the offline settings only offline data IP to generate time-invariant perturbations 5.
remains largely unexplored, posing potential barrier and risk to their broader application. While it is straightforward to attack an end-to-end DNN by applying gradient ascent over the loss function [31, 14], it is non-trivial to craft attacks against DP, due to its concatenated denoising structure and high randomness. Prior research [25, 24, 56, 55, 45] has focused on attacking the diffusion process of the text-to-image (T2I) diffusion models [42]. However, there distinct differences between attacking a T2I diffusion model and attacking Diffusion Policy. Firstly, they concentrate on attacking the diffused value while we aim at attacking the conditional image. In addition, they try to fool the editing process over the clean images (c.g. SDEdit [34]), while we are trying to fool the robot to make wrong actions step by step, each action is generated from the pure gaussian noise. Diffusion policies are also more interactive with the environment. A successful attack not only needs to fool a single inference output but also needs to continuously fool the model to decrease model performance.
In this paper, we focus on crafting adversarial attacks against DP. Specifically, we propose Diffusion Policy Attacker (DP (tt-Attacker), the first suite of white-box box-attackk algorithms that can effectively deceive the visual-based diffusion policies. We investigate two hacking scenarios as illustrated in Figure 1: (1) digital attack-hacking the scene camera, which means that we can add imperceptible digital perturbations to the visual inputs DP, and (2) physical -hackin the scene by attaching small adversarial patches [4] to the environments (e.g. table). Furthermore,
consider both offline and online settings, for online settings, we can generate time-variant perturbations based on the current visual inputs, on the opposite, for the offline settings can only add one fixed perturbation across all the frames.
We conducted extensive experiments on DP pre-trained on six robotic manipulation tasks and demonstrated that DP-Attacker can effectively craft adversarial attacks against DP. For digital attacks, DP-Attacker system's performance. For physical attacks, DP-Attacker capable of creating adversarial patches tailored for each task, which can be put into the physical environment to disrupt the system. Also, we reveal that the non-robust image encoder makes the DP easy to attack.
generate both online and offline attacks that significantly degrade the DP
2 Related Works
Diffusion-based Policy Generation Diffusion models [49, 16, 48] exhibit superior performance in multiple domains like high-fidelity image generation, video generation and 3D generation [42, 39, 44, 47, 53, 41, 27). Due to its strong expressiveness
modeling multi-modal distribution, diffusion models have also been successfully applied to robot learning areas such as reinforcement learning [54, 2], imitation learning [9, 58, 20, 38), and motion planning [43, 30, 18). Among them, Diffusion policy (DP) [9, 58, 23] has gained significant attention due its straightforward training

---

methodology and consistent, reliable performance. In this paper, we focus on crafting adversarial attacks against visual- risual-based DP, technology already integrated into various indoor robot prototypes like Mobile Aloha [12].
Adversarial Examples for Deep Systems Adversarial attacks have been widely studied for deep neural networks (DNNs): given small perturbation, the DNN will be fooled to make wrong predictions [51, 14]. For DNN-based visual recognition models, crafting adversarial samples is relatively easy task using gradient-based budget- budget-limited attacks [31, 57, 14, 5, 10, 3]. However, attacking diffusion models consisting of a cascade of DNNs injected with noise, poses a more complex challenge. Recent studies have demonstrated the feasibility of effectively crafting adversarial samples for latent diffusion models using meticulously designed surrogate losses [25, 59, 24, 46, 45, 56, 6]. However, these efforts have primarily focused on image editing or imitation tasks and are limited to working solely in latent space [55]. Here we hope to explore the adversarial attacks against DP under
various settings.
Adversarial Threats against Robot Learning Previous research has highlighted adversarial attacks as a significant threat robot learning systems [8], where small perturbations can cause chaos in applications such as deep reinforcement learning [22, 13, 28, 37, 50, 36], imitation learning [15], robot navigation [21], robot manipulation [19, 33], and multi-agent robot swarms [1]. Despite the rising popularity of policies generated by diffusion models, to the best of our knowledge, there have
been no prior efforts aimed attacking these models in the field of robotics.
Preliminaries
3.1 Diffusion Models for Behaviour Cloning
Diffusion models [49, 16] are one type of generative model that can fit a distribution q(zo). using a diffusion process and denoising process. Starting from FK, pure Gaussian noise, the denoising process can generate samples from the target distribution by K iterations of denoising steps (Here we use K, k represent steps in diffusion and T, L. for running timesteps of the robot scenarios):
Zg c Aaea k) + NN of)), k = 0,2, .... K-1
where OKg, Ag. ag are hyper-parameters for the noise scheduler. ey is a learned denoiser parameterized by 0, which can be trained by optimizing the denoising loss termed L &e + ex, k) ex We define the reverse process in Equation 1 as Fk (xx+1) for simplicity.
Diffusion policies [18, 9] noted ma apply the diffusion models mentioned above, resulting in To(s"), where 7' E IR D. is the planned action sequences at timestep / in the continuous space, s the current states, and Da. La are the action dimension and action length respectively. Accordingly, the learnable denoiser becomes eo (Tk, k s), and the denoised diffusion process remains the same. For visual DP, the states s¹ are usually images captured by the scene or wrist cameras, so we use I' throughout to represent the visual inputs at timestep t. Finally, the policy can be formulated
T1 +' (I') = R2 (R)... (R "IK, I')..""))..)) 19).
The equation above shows that the predicted action yf is the output of chained denoiser models residually conditioned on the current observation I'. In practice, while DP outputs long sequence of actions т, we only execute the first few actions of it in receding horizon manner to improve temporal action consistency [9].
3.2 Adversarial Attacks Against Diffusion Models
Adversarial samples [14, 31, 5] have been widely studied as threat the AI system: for a DNN- based image classifier = [z), one can easily craft imperceptible perturbations P to fool the classifier to make wrong predictions over P(r). In digital attack settings [51, 14], the perturbation should be small and always invisible to humans, which can be formulated by the -nom as (P(x) x 0loc < a where a small value (e.g. 8/255 for pixel value). Methods like FGSM [14]
and PGD [31] can be casily applied to craft such kinds of adversarial attacks. For physical-world
3

---

adversarial patches [4, 11, 57, 17]. P(x) is always crafted as attaching small adversarial patch to the environments, and the patch should be robust to physical woorld transformations such as position. camera view, and lighting condition.
Recent works [25, 45] show that it is also possible to craft such kind of adversarial examples to fool latent diffusion models [42] with an encoder & and a decoder D: adding small perturbation to a clean image, the denoising process will be fooled to generate bad editing or imitation results. The following Monte-Carlo-based adversarial loss to attack latent diffusion model:
Lade (z) E& |ea (Z( + Ck. k) <x11
the mechanism behind attacking latent diffusion models [56] turns out to be the vulnerability of the autoencoder and works only for the diffusion model the latent space [55]. Also, the settings above differs from our settings of attacking a DP which targets on attacking the conditional image without the ground-truth clean action to get the diffused input of eg in Equation 3. the following section, we show that we can still effectively craft different kinds of adversarial samples based on Equation 3 with some modification.
Methods
4.1 Problem Settings
DP-Attacker
Threat Model In this paper, we assume that we have white-box access to some diffusion pol- icy network. That is, we have access to its pa- rameters and also the data used train it. Given this trained network, we wish to find adversarial perturbations that, when added to the observa- tion I, will cause the trained diffusion policy to generate unwanted actions (either random or targeted) that impede task completion (lower the task score or success rate). We consider two types of perturbations detailed in Sec. 4.2 and Sec. 4.3.
Global Attack
Patched Attack
Offline
Online
Offiiee
Targeted
Untargeted
Targeted
Untargeted
Figure Design Space of DP- Attacker: the tree above shows the design space of DP-Attacker, which can adapted to various kinds of attack scenarios, including global attacks (hacking and cameras) vs patched attacks (hacking the physical environment); offline vs online; targeted vs untar- geted.
The most straightforward way to measure the quality of the attack is use the difference be- tween generated actions from the original ac- tions an end-to-end manner:
Cuntar -1.t)= 11111
where -4.* is known good solution sampled by wo given the observation image I, and P(-) is some perturbation on the observation image. could be generated either from the trained policy for online attacks or from the training dataset for offline attacks. One can minimize the negative L-2 distance between a generated action and a good action for untargeted attacks. For targeted attacks, the action loss becomes
end2eed (1, ((To((()))) Finggr
where Tie is some target bad action we wish the policy to execute (c.g. always move to left). We use PGD [31] to optimize for the best perturbation that minimizes this loss. However, due to the inherent long-denoising chain of the diffusion policy ny. the calculation of this gradient could be quite costly [45].
In practice, running the end-to-end attacks above is not effective especially when the model is large and when we need to hack the camera at a high frequency. Instead, borrowing ideas from recent works [25, 24, 56] on adv-samples for diffusion models, we propose to use the following optimization objectives:
Cuntar (T, = - Ex + te, k, P(T)) exll?
(5)
where l is the timestep of the diffusion process and is the timestep of the action runner. We add noise to the good solution fht and then calculate the L-2 distance between the predicted noise of the

---

denoise network and the added noise. Minimizing this loss leads to inaccurate noise prediction of the denoising network and, in turn, leads bad generated action of the diffusion policy. For targeted attacks, the noise prediction loss is:
náv (I, = co + ck. k. P(T)) cx1l²
(6)
Minimizing this loss would allow the denoising net favor the generation of the target action. The gradient of the noise prediction loss is easier to calculate compared to the action loss because of the short one-step chain. This makes more favorable for conducting attacks.
4.2 Global Attacks
A global attack injects adversarial perturbation into the observation image I by adding it on top 5. The adversarial noise ô is of the same shape as the original image. To make the attack imperceptible, the adversarial noise's ' absolute value is limited by some o. To find such an adversarial noise, we use PGD [31], an optimization-base search for an adversarial noise. The adversarial noise can be constructed online during inference or offline using the training dataset. The algorithm for conducting an online global attack is shown in Algorithm 1. The algorithm optimizes for loss in Equation 5 or Equation 6. The algorithm can be modified easily to construct an offline attack. Given the training dataset Dr = (((T,,1') T we can optimize for the loss Cuntar (I, - (xx))r (1) + éx, P([)) exll" or 2 (I,1)
of the observation image, i.e. P(I)
method to
(rs 11))(11))) [T + Ck. k P())) exll This algorithm is provided in the appendix.
Algorithm Global Adversarial Attack (Online)
Input:
given observation image T, diffusion policy no, noise prediction net co. attack budget o, step size a, number of steps N
Output: adversarial noise
>> initialize adversarial noise
for = 1 N do ck. k ~ NN I), randint(1, K) if targeted attack then
>> sample forward noise and timestep > forward sample, Funger should be given use diffusion policy to generate good solution > forward sample
-' Larg + Ek
s +-1
else if untargeted attack then (T)
x² - +
8 +
end if
6p
f- ( k, clip(1 + 6, 0, 1)) - 8 ||e 11
L
8 - clip(0 - a sign( V ln
V f)) -0,
P Projected- -rrrteent Descent
end for return 8
4.3 Patched Attacks
patched attack directly puts specifically designed image patch E Reexhxw into the environment. The camera later captures it and causes undesirable motion from the diffusion policy. The patch should be active under different scales, orientations, and observation views. During training, we apply some random affine transform (shift, rotation, scale, and shear) T E T. The affine transform uses the center of the image as the origin of coordinate system. The resulting patch replaces the original observation image using the replacement operator: replace(7, x)) again using the image's center as the origin of the coordinate system. To search for such a patch, we use the training dataset and optimize for the best patch using PGD. The algorithm is illustrated in Algorithm 2.
5

---

Algorithm 2 Patch Adversarial Attack
Input:
training dataset Dr {(T', I') e T). diffusion policy тe. noise prediction net cø, set of affine transforms T, step size a
Output: adversarial patch x
x ~ (O, T)
< -clip(æ 0,5,
initialize patch
repeat
(r', 1 k~ Dr, T,N(O, I), randint(1, K)
D sample from dataset, transform, forward noise, and time step
if targeted attack then + + tagat + Ex:
D forward sample, Taggrr should be given
s st1
else if untargeted attack then x 7' +
forward sample, use dataset entry
8 6--1
end if
Cp t co(TT, k, replace 1", Txx)))
L 8 8. |lex 6p 1|
x - clip(x
sign (V2
> Sogjeced-Gradiend Descent
until satisfied return x
Experiments
We test the effectiveness of DP-Attacker with various strengths and configurations on different diffusion policies. Our target models are vision-based diffusion policy models introduced by Chi et al. [9]. We aim manipulate the visual input so that the generated trajectory will not lead to task completion. We quantitatively evaluate the effectiveness of our attack methods by recording the result task completion scores/successful rate. We also provide scores without attacks for reference and random noise attacks (adding some Gaussian noise to the observation images) as baseline attack method. We foueus on the models released by Chi al. [9]. However, our attack algorithm applies to other variants of diffusion policies as well.
Environment Setup Our benchmark contains 6 tasks: PushT, Can, Lift, Square, Transport, and Toolhang. These tasks illustrated in Figure 7 in the Appendix. Robosuite provides all the simulation of these tasks except for PushT [52, 32, 60]. For evaluation, we attack the released checkpoints of diffusion policies trained by Chi et al. [9]. For tasks Can, Lift, Square, and Transport, each has two demonstration datasets: Multi-Human (MH) and Proficient Human (PH). The other two tasks (PushT and Toolhang) has only one PH dataset, respectively. This gives total of 10 datasets. In [9], each dataset is used to train two diffusion policies with different diffusion backbone architectures: CNN-based and Transformer-based
We take the best performing checkpoints for these 20 different scenarios released by Chi et. al [9] as our attack targets. For each attack method, we run 50 rollouts and collect the average score or calculate the success rate of the tasks. The rollout length uses the same length as the demonstration dataset [9, 32]. Besides our attack methods, we also run the rollout using clean images for reference and with random noise added as a baseline attack method. The evaluation is done using single machine with an RTX 3090 GPU and AMD Ryzen 9 5950X to calculate rollouts and run our attack algorithms.
5.1 Global Attack
We first present the results of global attacks. We evaluate both online attack algorithm (creating adversarial noise on the fly per inference) and offline algorithm (pre-generating a fixed noise that is used for every inference).
Online Attack For online attacks, we use attack parameters a 0.03, 0.001 875, N = 50. For targeted attacks, we use normalized target action vector of all ones. We report the performance of the transformer-bast models before and after the attack in Table 1. The results of global attacks

---

Clean
Rand Noisc
Untar
Tur
Clean
Rand Noise
Untar
Tar
Figure 3: Global Attack (Online): We visualize the global attacks in Algorithm 1 within both the PushT and Can environments. Specifically, we present action rollouts for four types of observations: clean observations, observations perturbed with random Gaussian noise, and our optimized perturba- tions (both untargeted and targeted). While the DPs show robustness to random perturbations, they are vulnerable to adversarial samples generated using DP- -tttkercke.
Rush Toriee
Figure 4: Physical Adversarial Patches: we show the patches optimized by Algorithm 2, attaching the physical scene will effectively lower the success rate of the target diffusion policy.

---

Tasks
PushT
Can
Lift
Square
Transport
Toolhang
Demonstration Type
PH
PH
MH
PH
MH
PH
MH
PH
MH
PH
Clean
0.75
0.92
0.92
1
1
0.92
0.72
0.86
0.46
0.86
Random Noise
0.66
0.88
0.98
1
1
0.82
0.74
0.84
0.48
0.82
Offeie
0.46
0.08
0.08
0.94
0.7
0
0
0
0.02
0
Untargeted-Offlin
0.39
0.1
0.46
0.8
0.62
0.04
0
0
0
0
Targeted-Onling
0.10
0
0
0.02
0
0
0
0
0
0
Untargeted-Onling 0.19
0.02
0.02
0.62
0.62
0
0
0
0
0
Table 1: Quantitative Results on Global Attacks: The table includes the attack result for all transformer based diffusion policy networks. Our DP-Attack can significantly lower the performance of the diffusion models.
Can Transformer
Lift
Square
Toolhang
Backbone Arch
CNN
CNN
Transformer
CNN
Transformer
CNN
Transformer
Clean
0.98
0.92
1
0.94
0.92
0.8
0.86
Random Noise Patch
0.9
0.94
1
0.9
0.8
0.54
0.56
0.12
Untargtted-Offlnn
0.16
0.44
1
0.82
0.72
0.34
0.48
0.02
Table 2: Quantitative Results on Patched Attacks
on all models are given in the appendix. Example rollouts and images used in the rollouts are shown in Figure 3.
Offline Attack For offline global attacks, we train the training dataset with batch size 64, = 0.0001, a = 0.03 for 10 epochs. The resulting trained adversarial noise is added to the input image for every inference. The results are shown in Table 1. Examples of rollouts and images used in the attack can be found on our website.
We find that diffusion policy is not robust noises introduced by our DP- Attacker. The performance of diffusion policies is significantly reduced after running global attacks. A disturbance of less than 3% is able to decrease the performance from 100% to 0%. The success of offline global attacks also shows attacks can be cheaply constructed and pose a significant threat to the safety of using diffusion policy in the real world.
5.2 Patched Attack
PH CNN Ouline Glibal Attack
Gooall
1.300
1.50-
Training vs. Evaluating Since patched attacks directly put at- tack image into the environment, we only consider offline attacks that pre-
1.40
1.40
1.20
1.20
generate some patch that is used
1.00
1.00
throughout the rollout. We train the
patch using Algorithm 2, where the patch is applied to the training image using some randomized affine trans- form. This allows the gradient to pass through for successful training. Since we have used random affine transforms during training, the patch should be transferable when used in the simulation environment, For eval- uation, we create a thin box object with the trained image patch as its tex- ture and put randomly onto the ta- ble.
0.50
0,80
0.00
0.60
0.40
0.40
0.20
0.20
0.00
0.00
Random Nolse 3e-Astacher
Random Notse DP-Actacher
Figure 5: Difference in Encoded Feature Vector: we cal- culate the distance between the clean feature vector and the attacked feature vector. DP-Attacker perturb the feature vector significantly compared to naive random noise attack.
8

---

Parameters (ar 0.03)
N 10
N 20
N 50
Success Rate
0.94
0.8
0.66
Parameters (NV 50) 0 0.01
0.03
0.05
Success Rate
0.68
0.32
Table 3: Different Parameters for DP-Attack: We did ablation study parameters and N, and we can see that smaller steps and budgets are not enough to fool a DP. Larger budgets will dramatically decrease the Sucess Rate (SR).
Results We construct patch of size that covers around 5% of the observation image using Algorithm The details of the training can be found in the appendix. We evaluate the effectiveness of our patch attack algorithm on total of 8 checkpoints, covering the PH dataset across four tabletop manipulation tasks (Can, Lift, Square, and Toolhang) using both CNN and Transformer diffusion backbones. The result success rate (SR) is shown in Table 2. Example rollouts are shown in Table 4.
Simpler tasks such as Can and Lift quite robust to random noise patch. Our DP-Attacker produces adversarial patches that perform better than random noise in terms of degrading the diffusion policy performance.
5.3 Quantitative Results on Targeted Attacks
We qualitatively evaluate the effectiveness of our targeted attacks. We use our DP-Attacker to run gobbal-online-trgeen attacks with varying strength on two model checkpoints: PushT (CNN) and CAN (PH CNN). The target in PushT task is 2D coordinate around 3323.875,00 328.75) (note the side length of the PushTT environment is 1 1024), and the target in the CAN task the target end-effector position around 1866, 0. 1049, 1.0848) (in meters). In 6, we report how close the actions generated by diffusion policy are to our attack targets during the rollout.
Target Action During Rellout (PishT CNN)
Moon Distaune Targei Action During (CAN CNN)
0.06
distance (ustes))
0.06
dirtance
200
0.07
0..17
0.2
150
200
250
100
150
200
250
300
330
400
stop
covironment step
Figure 6: We used DP-Attacker with different attack strengths to run rollouts. We report the average distance between the predicted action sequence and the target action sequence (a sequence of duplicate target coordinate). The metric is calculated at each inference during the rollout.
Our AP-Attacker is able to manipulate the generated action to within 20 units and cm of the attack targets, respectively, for the PushT task and CAN task, with attack strength of 8 0.06. Example rollouts of these two attack scenarios can be found in the Sec. D the appendix.
Ablation Study
To investigate the effectiveness of our attack method, evaluate how the attack parameter plays role in tP-kttacker. First, we investigate the effect the number of PGD steps N. We keep the = 0.03, and 25 3/- Second, we investigate the effect of the noise scale We keep N 50, and 20 We evaluate all six attacks on the transformer backbone DP trained on the Lift PH dataset. The result is summarized in table 3.
Attack Parameters
We perform a comparison with the end-to-end action loss 3. We evaluate both methods with the same attack parameters (a 0.03, a 0.001 875, N 50) on the dest-performing transformer backbone trained on PH dataset of the Lift task. Again, we evaluate 50 randomly initialized environments. The selection of end to end loss with DDPM [16]
End to End Loss vs. Noise Prediction Loss

---

Method
Attack Time
Model Success Rate
Clean
Random Noise
1
End to End DDPM (Untargeted)
70s
End to End DDPM (Targeted)
67s
0.52
End to End DDIM-8 (Untargeted)
6.5s
0.9
End to End DDIM-8 (Targeted)
5.88
0.24
DP-Attacker (Untargeted) DP-Attacker (Targeted)
11.88
0.62
~ 1.33
0.02
Table 4: Compared with End to End Attacks DP-Attacker runs significantly faster than the end-to-end attacks even if it is accelerated with DDIM. Our DP-Attacker also provides better attack results.
scheduler makes it infeasible for online attacks. In addition, provide results where replace the loss-calculating noise scheduler with a DDIM-8 step scheduler [48]. This provides speedup for calculating the end to end loss. The result SR after the attack and the average time used to perform the online attacks are shown in 4. The naive end-to-end loss is significantly lower than our attack algorithms and does provide better results. We suspect that since diffusion models introduce randomness during the sampling of a trajectory, it is better to attack the noise prediction loss rather than the end end action loss.
What Is Being Attacked Is the Encoder We try to investigate further what exactly attacked in our DP-Attacker. Other literature relating to text-to-image diffusion models shows that the encoder is the one being attacked [45, 56]. We suspect the same is happening for diffusion policy. To investigate this, we calculate the L2 distance between the encoded feature vector of clean and attacked images random noise attack, unsuccessful attack parameters, and successful parameters, respectively, The details of the calculation is in the appendix. We do this for 1000 images in the training dataset and plot the distribution of the distances using violin plot in Figure The significant difference shows that our attack method has drastically changed the representation of the conditional visual feature. This later affects the downstream conditional noise prediction net, causing make inaccurate noise predictions. We put details about it in the Appendix.
Conclusion and limitations
In this paper, we propose DP-Attacker, a suite of algorithms designed effectively attack diffusion-based policy generation, an emerging approach in behavior cloning. We demonstrate that DP-Attacker can craft adversarial examples across various scenarios, posing a significant threat to systems reliant on DP. Our findings highlight that despite the inherent randomness and cascaded deep structure of diffusion-baat based policy generation, remains vulnerable adversarial attacks. We emphasize the need for future research to focus on enhancing the robustness of DP to ensure its reliability in real-world applications. There are also some limitations for this paper: our experiments were conducted exclusively within simulation environment, and we did not extend our testing to real-world scenarios. Additionally, we did not develop or implement any defensive strategies for the proposed tasks, which remains an area for future research and exploration.
Acknowledgments and Disclosure of Funding
We thank the anonymous reviewers for their valuable feedback on this paper. This work is supported in part by grants NSF CAREER ECCS- and NSF FRR- PRR-240.
References
[1] M. Abouelyazid, Adversarial deep reinforcement learning to mitigate sensor and communication attacks for secure swarm robotics. Journal of Intelligent Connectivity and Emerging Technologies, 8(3):94 112,
2023.
10

---

[2] A. Ajay. Y. Du, A. Gupta, J. Tenenbaum, T. Jaakkola, and P. Agrawal. Is conditional generative modeling all you need for decision-making?? arXiv preprint arXiv:2211.16657 2022.
[3] A. Arna,, O. Miksik, and P. H. Torr. On the robustness of semantic segmentation models to adversarial attacks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 888-897, 2018.
[4] T. B. Brown, D. Mané, A. Roy, M. Abadi, and J. Gilmer. Adversarial patch. arXiv preprint orXiv:1712.0966. 2017.
[5] N. Carlini and D. Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp). pages 39-57. Iece, 2017.
[6] Chen, Dong, and X. Xie. Exploring adversarial attacks against latent diffusion model from the perspective of adversarial transferabilityy arXiv preprint arXiv:2401.00000 2024.
[7] L. Chen, S. Bahl, and D. Pathak. Playfusion: Skill acquisition via diffusion from language- annotated play. In Conference on Robot Learning, pages 2012-2029. PMLR, 2023.
[8] T. Chen, J. Liu, Y. Xiang, W. Niu, F. Tong, and Z. Han. Adversarial attack and defense in reinforcement learning-free from ai security view. Cybersecurity, 2:1-22, 2019.
19] C. Chi, S. Feng, Y. Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion, arXiv preprint 2rXiv:2303.04137, 2023.
[10] Y. Dong, F. Liao, T. Pang, H. Su, 7hu, X. Hu, and J. Li. Boosting adversarial attacks with momentum. In Proceedings of the IEFE conference on computer vision and pattern recognition, pages 9185-9193, 2018.
[11] R. Duan, X. Ma, Y. Wang. Bailey, A. K. Qin, and Y. Yang. Adversarial camouflage: Hiding physical- world attacks with natural styles. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1000-1008, 2020.
[12] Z. Fu, T. Z. Zhao, and C. Finn. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation. arXiv preprint arXiv:2401.02 0222107..111, 2024.
[13] A. Gleave, M. Dennis, C. Wild, N. Kant, S. Levine, and S. Russell. Adversarial policies: Attacking deep reinforcement learning. arXiv preprint arXiv:1905.10115. 2019.
[14] I. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.65722 2014.
[15] G. Hall, A. Das, J. Quarles, and P. Rad. Studying adversarial attacks behavioral cloning dynamics. In 2020 IEEF.3mm International Conference on Tools with Artificial Intelligence (ICTAI), pages 452-459. IEEE, 2020.
[16] Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
[17] Z. Hu, S. Huang, X. Zhu, Sun, Zhang, and X. Hu. Adversarial texture for fooling person detectors in the physical world. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patten Recognition, pages 13307-13316, 2022.
[18] M. Janner, Y. Du J. B. Tenenbaum, and S. Levine. Planning with diffusson for flexible behavior synthesis. arXiv preprint (:xxi:220500999) 2022.
[19] Y. Jia, C. M. Poskitt J. Sun, and S. Chattopadhyay, Physical adversarial attack on a robotic arm. IEEE Robotics and Automation Letters, 7(4):9334 9341, 2022.
[20] T.-W. Ke, N. Gkanatsios, and K. Fragkiadaki. 3d diffuser actor: Policy diffuusin with 3d scene representa- tions. arXiv preprint arXiv: 10885 2024.
[21] M. I. Khedher and M. Rezzoug. Analyzing adversarial attacks against deep learning for robot navigation. In ICAART (2), pages 1114-1121, 2021.
[22] M. Lechner, A. Amini, D. Rus, and T. A. Henzinger. Revisiting the adversarial robustness-accuracy tradcoff in robot learning. IEEE Robotics and Automation Letters, 8(3):1595-166) 1602, 2023.
[23] X. Li, V. Belagali, J. Shang, and M. S. Ryoo. Crossway diffusion: Improving diffusion-baase visuomotor policy via self- self-super leamnin, arXiv preprint arXiv:2307.0184, 2023.
11

---

[24] C. LLian and X. Wu. Mist: Towards improved adversarial examples for diffusion models. arXiv preprint arXiv:2305. 12683, 2023.
[25] C. Liang, X. Wu, Y. Hua, J. Zhang, Y. Xue, T. Song. Z. Xue, R. Ma, and H. Guan. Adversarial example does good: Preventing painting imitation from diffusion models via adversarial examples. In International Conference on Machine Learning, pages 10763-20788 PMLR, 2023.
[26] Z. Liang. Y. Mu, H. Ma, M. Tomizuka, M. Ding, and P. Luo. Skilldiffuser: Interpretable hierarchical planning via skill abstractions diffusion-based task execution, arXiv preprint arXiv:2312.11533 2023,
[27] C.-H. Lin, J. Gao. L. Tang, T. Takikawa, X. Zeng. X. Huang, K. Kreis, S. Fidler, M.-Y. Liu, and T.-Y. Lin. 2022.
Magic3d: High-resolution text-to-3d content creation. arXiv preprint arXiv:22
[28] Y.-C. Lin, Z.-W. Hong, Y.-H. Liao, M.-L. Shih, M.-Y. Liu, and M. Sun. Tactics of adversarial attack on deep reinforcement learning agents. arXiv preprint T.K3v::770306660 2017.
[29] Liu, P. Hang, X. Zhao, J. Wang, and J. Sun. Ddm-lag: autonomous vehicles with lagrangian safety enhancement. arXiv preprint arXiv:2401.0362) 2024.
certusion-basede decision makinng model for
[30] Y. Luo, C. Sun, J. B. Tenenhaum, and Du Potential hased diffusion motion planning. 2023.
[31] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant adversarial attacks. In International Conference on Learning Representations, 2018.
[32] A. Mandlekar, D. Xu, J. Wong. S. Nasiriany, C. Wang, R. Kulkarni. L. Fei-Fei, S. Savarese, Y. Zhu, and R. Martin Martin. What matters learning from offline human demonstrations for robot manipulation. In arXiv preprint arXiv: 00.0008...... 2021.
[33] Melis, A. Demontis, Biggio, G. Brown, G. Fumera, and F. Roli. Is deep learning safe robot vision? adversarial examples against the icuh humanoid. In Proceedings of the IEEE international conference on computer vision workshops, pages 751-759, 2017.
[34] C. Meng. Y. He, Y. Song, J. Song. Wu, J.-Y. Zhu, and S. Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations, 2021.
[35] U. A. Mishra, S. Xue, Y. Chen, and D. Xu. Generative skill chaining: Long-horizon skill planning with diffusion models. In Conference Robot Learning, pages 2905-2925. PMLR, 2023.
[36] K. Mo, W. Tang, J. Li, and X. Yuan. Attacking deep reinforcement learning with decoupled adversarial policy. IEEE Transactions Dependable and Secure Computing, 200 1)77587768, 2022.
[37] A. Pattanaik, Z. Tang, S. Liu, G. Bommannan, and G. Chowdhary. Robust deep reinforcement learning with adversarial attacks. arXiv preprint 171iv:1712.000 2017.
[38] T. Pearce. T. Rashid, A. Kanervisto, D. Bignell, M. Sun, R. Georgescu, S. V. Macua, S. Z. Tan, 1. Momenne- jad, K. Hofmann, et al. Imitating human behaviour with diffusion models. arXiv preprint 211777331.11677. 2023.
[39] D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhom, J. Müller, J. Penna, and R. Rombach. Sdxl: Improving latent diffusion models for high resolution image synthesis. arXiv preprint arXiv:2307.01952. 2023.
[40] D. Pomerleau. Alvinn: autonomous land vehicle in a neural network. Advances neural information processing systems, 1, 1988.
[41] B. Poole. A. Jain, J. T. Barron, and B. Mildenhall, Dreamfusion: Text-to-3d using 2d diffusion. In The Eleventh International Conference on Learning Representations, 2023.
[42] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and patter recognition, pages
2022.
[43] K. Saha, V. Mandadi, J. Reddy, A. Srikanth, A. Agarwal, B. Sen, A. Singh, and M. Krishna. Edmp: Ensemble-of-ccott--uu guided diffusion for motion plamning. arXiv preprint arXiv-reprint arXiv:2309. /1414, 2023.
[44] C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang. E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan. T. Salimans, et al. Photorealistic text-to-image -imte-imaee diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 01.36647936994 2022.
12

---

[45] H. Salman, A. Khaddaj, G. Leclerc, A. Ilyas, and A. Madry. Raising the cost of malicious ai ai-powered image editing, arXiv preprini arXiv:2302.06588, 2023.
[46] S. Shan, J. Cryan, E. Wenger, H. Zheng. R. Hanocka, and B. Y. Zhao. Glaze: Protecting artists from style mimiery by text-to-image models, arXiv preprint arXiv:2302.04222 2023.
[47] U. Singer, A. Polyak, T. Hayes, X. Yin, J. An, S. Zhang, Q. Hu, H. Yang, O. Ashual, O. Gafni, et al. Make-a-video: Texl-to-vidoo generation without text-video data, arXiv preprint vcXiv:229.14492, 2022.
[48] J. Song, C. Meng, and S. Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021.
[49] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:201 I. 1.4466,, 2020.
[50] J. Sun, T. Zhang, X. Xie, L. Ma, Y. Zheng, K. Chen, and Y. Liu. Stealthy and efficient adversarial attacks against deep reinforcement learning. In Proceedings of the AAAI Conference Artificial Intelligence. volume 34, pages 5883-5891, 2020.
[51] C. Szegedy, W. Zaremba, Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing properties neural networks. arXiv preprint arXiv:1312.6733 2013.
[52] E. Todorov, T. Erez, and Y. Tassa. Mujeco: physics engine for model-based control. In 2012 JEEE/SS international conference on intelligent rohots and systems, pages 5026-5033. IEEE, 2012.
[53] Y. Wang, Z. Wu, C. Li, and A. Wu. Extraction and recovery of spatio-temporal structure in latent dynamies alignment with diffusion models. Advances in Neural Information Processing Systems, 96:38988-39005 2023.
[54] Z. Wang. J. Hunt, and M. Zhou. Diffusion policies as an expressive policy class for offline reinforcement learning. arXiv preprint arXiv:2208.00 06/93, 2022.
[55] H. Xue Y. Chen. Pixel is barrier: Diffusion models are more adversarially robust than we think. arXiv preprint arXiv:2404. 13320, 2024.
[56] H. Xue, C. Liang, X. Wu, and Chen. Toward effective protection against diffusion- mimicry through score distillation. In The Twelfth International Conference on Learning Representations, 2023.
[57] Xue, A. Araujo, B. Hu, and Y. Chen. Diffusion-based adversarial sample generation for improved stealthiness and controllability. Advances in Neural Information Processing Systems, 36, 2024.
[58] Y. Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu. 3d diffusion policy. arXiv preprint arXiv:2403.03955 2024.
[59] B. Zheng, C. Liang, X. Wu, and Liu. Understanding and improving adversarial attacks on latent diffusion model. arXiv preprint arXiv:2310..0... 130,0:33 2023.
[60] Y. Zhu, J. Wong, A. Mandlekar, R. Martin-Martin, A. Joshi, S. Nasiriany, and Y. Zhu, robosuite: A modular simulation framework and benchmark for robot learning. In arXiv preprint arXiv:2009. 12293, 2020.
13

---

Appendix
We put more video results, including rollouts of the DP-based robots under various attacks crafted by DP-Attacker, in the following anonymous link:
https://sites. htt.g//..../////////ttttttttteaa
A
Broader Impact
Diffusion-based policies (DPs) have emerged promising candidates for integrating real robots into our daily lives. Even with just a few collected demonstrations, DPs exhibit strong performance across various tasks [12, 58]. However, despite their utilization of diffusion models, which distinguish them from other policy generators, our research highlights their vulnerability to adversarial attacks. We demonstrate practical attacks DP-based systems, such as hacking cameras to introduce fixed perturbations across all frames (global offline attack) and incorporating patterns into the scene (physical patched attack). It is critical to consider these threats, and we urge future research to prioritize the development of more robust DPs before their widespread application in the real world.
B Algorithms
We also provide the algorithm for training the offline global attacks.
Algorithm 3 Global Adversarial Attack (Offline)
Input:
training dataset Dr {(7', /9) e T), diffusion policy noise prediction net step size a, attack budget a Output: adversarial noise 0
> initialize adversarial noise
repeat
(r', (,, Dr,N((0,)) I), randint(1, K)) sample from dataset, forward noise and timestep
if targeted attack then
r' Target Eg:
forward sample, T should be given
8 +-1
else if untargeted attack then
r¹ + - + ej
> forward sample
s --1
end if
- Eo ( k, clip(1' ( + 6,0, 1)) L C |ee Ep 1|
Ep
8 - ( - a sign( V C), -0,
D Projected- Graddent Descent
until satisfied
return 5
C Experimental Details
C.1 Task Descriptions
investigate total of six different tasks: PushT, Can, Lift, Square, Transport, and Tool hang. The tasks illustrated in 7. Here are the descriptions for each task:
PushT: The simulation happens in 2D. The agent controls a rod (blue circle) to push the grey block into the targeted green area. The score calculated is the maximum percent of coverage of the green area by the grey T block during rollout.
14

---

Can
Lift
Square
Transport
Tool hang
Figure 7: Simulation renders of the six tasks
Can: The simulation environment is provided by robosuite [60]. The agent controls the 6-DoF end-effector position and gripper close or open. The goal is to move the randomly positioned can in the left bin into the corresponding bin (lower right) in the right bin.
Lift: The simulation environment is provided by robosuite. The agent controls the 6-DoF end-effector position and gripper close or open. The goal is to lift up the randomly positioned red block.
The simulation environment is provided by robosuite. The agent controls the 6-DoF end-effector position and gripper close or open. The goal is to put the randomly positioned square nut around the square peg.
Square:
The simulation environment is provided by robosuite. The agent controls 2 6-DoF end-effector positions and grippers close or open. The goal is transport the hammer inside the box one one side to the box on the other side.
Transport:
Tool hang: The simulation environment is provided by robosuite. The agent controls the 6-DoF end-effector position and gripper close open. The goal is to construct the tool tower by first inserting an L-shaped bar into the base and later hanging the second tool on the tip of the bar.
C.2
Training Details
Our goal is to construct noises for the observation images. The image encoder uses multiple views when constructing the conditional image feature vector. Below are details of how we construct the adversarial noises.
C.2.1 Global Online Attacks
For global online attacks, we construct noises for all observation views, i.e. suppose the encoder takes two camera views (one side view and one eye-in-hand), and the conditional state length is two, we will construct a total of four noises for adding onto input images, respectively, before passing it into the policy for action generation.
For the random noise attack, the noise selected is sampled from standard Gaussian and scaled by = 0.03 and clipped in the range [-a, a). For untargeted online attacks, we use PGD parameters N 50, o 0.03, a 0.001875. For targeted online attacks, the targeted selected is an action matrix (actim dim x action horizon) of all I's (in normalized action space). The PGD parameters for targeted online attacks are the same as the untargeted online attacks.
C.2.2 Global Offline Attacks
Similar to global online attacks, we also construct noise for all observation views. However, since this is an offline attack, we pre-generate (train) just one set of adversarial noises for each input, and is used throughout the rollout for the same task. The training parameters for untargeted and targeted attacks are the same: number of epochs = 10, a 0.0001, and batch size 64. For targeted attacks, we again use normalized action of all 1's.
C.2.3 Patched Attacks
For patch attack training, we only apply the patch on the most important camera view (side view). However, this is to maintain the consistency for the patch gradient pass. However, the patch is put into
15

---

Parameter
Range
shift shift rotation
0.4, 0.4)
percent of image, in centered coordinate.
| 0.4, percent of image, in centered coordinate.
[-45%, 45%)
scale
[1, 1]
shear
50% 50%
shear
-50% 50%
Table 5: Set of Affine Transforms T for Patched Attack
Figure Comparison of Physical Patch Training and Evaluation
the simulation environment for evaluation and can observed from multiple perspectives. For tasks Can, Lift, and Square, the observation image size is 84 x 84, and we choose the training patch size of 17 x 17 that covers around 4% the observation. For task Toolhang where the observation image size is 84 84, choose the training patch size of 50 50 that covers around 4.3% of the image. The set of transforms
summarized in the table 5. The training parameters are epochs 10, batch size = 64, 0.0001. For evaluation, make patch objects of size 0.06m x 0.06m (2.36in x 2.36in) and put it onto the table. The rotation angle is from [-450, 45%). For tasks Can, Lift, and Square, be anywhere
the position of the patch constrained to be on the top left the table so it can be captured by the camera. The size is about the same and we provide the comparison in the figure
the table. For Toolhang, the position of the patch is
D
More Results
Full Table for Global Attacks We provide the full table for the results of global attacks as an extension for 1. CNN-based models are harder to attack. Nevertheless, the result score still decreased significantly.
Targeted Attacks With more attack budget, we can manipulate the robot's action quite well. For this experiment, we increase the online global attack budget a to 0.05. With this increased budget, we could manipulate the generated action of DP. This shows the effectiveness DP Attacker proposed targeted noise prediction loss. See our website for details.
What Is Being Attacked Is the Encoder To investigate whether it is the encoder that is being attacked in DP-Attacker encoded feature vector the clean image {x)), clean image + random noise {(x + yand) and clean image adversarial noise E(x + 6adv) by our -P-tttacker. Next we calculate the L2 distance between the encoded clean image vs the encoded random noise attacked image [[(z) E(x + d )l and the L2 distance between the encoded clean image the encoded DP Ap--Attckker attacked images [E(x) {(r 8mav))3 We collect the these two distances for 1000 images in the training dataset and plot the distribution of the two sets using violin plots The attack we use are random noise attack with 0.03, N = 50, a = 0.001875, ntarget 1. We do this two dataset, one is the CAN ph with CNN backbone where our DP Attacker successfully performs the attack, and the other is the Lift ph with CNN backbone where our DP-At Attacker fails to construct successful attacks (see table 6). The difference in distribution shows that successful attacks correlated with the successful disturbance of the encoder.
perform the following comparison. For a given image, find the
0.03 and online targeted global attacks with
16

---

Can
Square
Trunsport
Toolhang
Demonstration Type
PH
PH
MH
PH
MH
PH
MH
PH
MH
PH1
Clean
0.75/0.83
0,920.98
0.920099
0.72X0.84
0.86/0.8
0.46/0.82
Random Noise
0.66/0.87
0.88/1
0.98/0.98
1/1
0.82/0944
0.74X0.76
0.84/0.84
0.48/0.68
0.82/0.72
Targeted-Oritine
0.46/0.68
0.08/0.2
0.08/0.16
0.94/1
Q/066
0V0.66
0.02/0.44
Unturgeted Oflline
0.39/0.73
0.1/0
0.46/0.34
0.8/1
0.62/0.98
0.0400662
000688
0
0/0
Targeted-Cnline
D. 1000.45
0/0
0.02/1
QV1
00.54
000.08
GA0
0/0
0/0
Untargetod-Online
0.19/0.48
0.02X0.02
0.02/0.02
0.62/1
0.62/1
0/0.38
000.88
070.04
090.04
0/0
Table 6: Quantitative Results on Global Attacks: The table includes the attack results for both CNN and buude diffusion policy networks. The format is transformer/CNN. Our DP- At-Attack can significantly lower the performance of the diffusion models.
Method
Attack Time
Model Score
Clean
0.75
Random Noise
0.66
End to End DDPM
44
0.11
End to End DDIM-8
~4.33
0.11
DP Attacker Prrgeted-Onling
~1.55
0.11
Table 7: Compared with End to End Attacks DP-Attacker runs significantly faster than the end-to-end attacks even if it is accelerated with DDIM. Our DP-Attacker also provides better attack results
Speed and Effectiveness Comparison With End to End Loss We perform another compari- son with the end to end loss to show the both the speed benefit and effectiveness of our DP tp-ckrr We conduct online targeted attacks on the Transformer- based DP for the PushT task. The PGD parameters for end to end attacks at N 50, = 0.03, a 0.001875. The result average model score 50 simulations and attack time shown in The evaluation done on a machine with RTX4080 mobile GPU, and Intel 19-1 1390090 CPU.
Rollouts of Quantitative Targeted Attacks The following the rollouts of videos corresponding to Sec. 5.3. Please open this PDF with Adobe Acrobat Reader to view the animated frames.
r
Figure 9: = 0.05 (PushT)
Figure 10: 5 = 0.06 (PushT)
Figure 11: 5 = 0.07 (CAN PH)
Figure 12: 8 = 0.05 (CAN PH) Figure 13: 5 = 0.06 (CAN PH) Figure 14: 8 = 0.07 (CAN PH)
17

---

Transferability of Offline Attacks Across Different Backbones We also evaluate the transfer- ability of DP- Attacker generated offline attacks across different diffusion backbones. We first test the transferability of offiin global attacks. We use DP-attacker to generate untargeted global offline attacks (6 = 0.03) on two checkpoints (CAN-MH-CNN and CAN-MH-TF).. Then, we evaluate these models using the two generated adversarial perturbations, and the success rate is listed in 8. We also test the transferability of patched attacks. We use DP- Attacker to generate adversarial patches on two checkpoints (CAN-PH-CNN and CAN-PH-TF), Then, we evaluate these models using the two generated adversarial perturbations, and the success rate listed Table
Attacked
Attacked
Model
Model
CNN
TF
Original SR
CNN
TF
Original SR
Runner
Runner
Model
Model
CNN
0.34
0.78
0.98
CNN
0.16
0.54
0..8
0.32
0.46
0.92
TF
0.42
0.44
0.92
Table 8: Model success rate of the CAN task trained on the MH dataset. We generated of- fline global attacks for each backbone first, and tested them on both models. The attack trans- fer case is marked in blue. We also report the model's original SR (without perturbation).
Table 9: Model success rate of the CAN task trained on the PH dataset. We generated phys- ical patched attacks for each backbone first, and tested them on both models. The attack transfer case is marked in blue. We also report the model's original SR (without perturbation).
Note that from 2, the random noise patch does not affect the performance DP on the CAN task very much. However, our DP-Attacker generated patches are able to decrease model performance in transfer settings (marked blue) in Tables and 9, This shows the effectiveness of our DP- Attacker and its potential ability in black-box attacks.
18

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
We made accurate claims in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We stated our limitation in the last paragraph the main paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
19

---

Justification:
We don't 't have theoretical results in the paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We show experimental details in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
20

---

Answer: [Yes]
Justification:
We will release the code to the public.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips) cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //niss. ct/ccc/pull wwtppblcc/guuieeeeeeeeeeeee/eeee / ssisonolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
We have the training details in the appendix.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We report the averaged metries different datasets, and the difference is significant.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
21

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We put the settings in the main paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: http://eeerr.pe.eee httpss///aurpps..cp/bbl///////// httpllc//thissuuieeeeeee www.iiisscieeelaaeeee ines?
Question:
Answer: [Yes]
Justification:
This paper conforms with the ethics code.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the broader impact in the main paper.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
22

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: all the models and datasets used in this paper are properly credited and licenses are respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
23

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [NA]
Justification: No new datasets/models proposed in this paper.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer:
answerNA
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.


Reviews:
[{"review_id": "TBlgBeBPG3", "reviewer": "Reviewer_kPb5", "summary": "Diffusion policy is used to generate the action trajectory from a pure Gaussian noise conditioned on the input images, applied in many applications such as autonomous driving. This paper proposes white-box adversarial attacks against diffusion policy, which aim to generate a target bad action or an untargeted action by attaching global perturbation or patch-based perturbation on the observation image. To this end, they formulate the training loss for the perturbation as minimizing the distance between the generated action and the target bad action (in targeted attack) or maximizing the distance between the generated action and a sampled good solution (in untargeted attack). Empirically, they validate the effectiveness of the proposed adversarial attack on six robotic manipulation tasks.", "questions": "1. Threat model could be clarified with more clearness in problem settings. It appears to me it is a white-box attack where the attacker requires knowledge of all parameters of the diffusion policy.\n2. The novelty of the proposed adversarial attack remains unclear to me. On the one hand, there are existing works [25, 43] which propose adversarial attacks against latent diffusion models, introducing  the Monte-Carlo-based adversarial loss. On the other hand, the difficulty of computing gradients due to the inherent long-denoising chain of the diffusion policy is solved by [25, 24, 52]. So, it seems that the novelty over these existing works is that this paper is modifying the training loss as the difference between actions. Clarification of the novelties compared with existing works would be appreciated.\n3. Is there any other evaluation metric in existing works? It seems that there is only one evaluation metric used (i.e., the drop of the result task completion scores) in the experimental results to demonstrate the effectiveness of the attack. For me, this one single metric would be fine for the untargeted attacks; however, for targeted attacks, it might be better to show the success rate of generating the target bad actions.", "limitations": "See weakesses part.", "rating": 5, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "1. They explored adversarial attacks in a new setting, i.e., against diffusion policy.\n2. They have good visualizations of experimental results.", "weaknesses": "1. As mentioned in the limitation section, there is no intuitive defensive strategy or experimental results in real-world scenarios included in the paper.\n2. The proposed attack only considers the white-box setting, i.e., the attacker requires knowledge of all parameters of the diffusion policy. Discussion on black-box setting could also be included in the limitation section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable comments. We are glad that the reviewer found our attack scenario novel and liked our visualizations. Below is our response to some of the question raised.\n\nFirst, we would like to clarify the loss used in our DP-Attacker is not the distance between generated actions. Instead, we use a Monte-Carlo-based estimate with a loss between predicted noise and actual noise. In this way, we were able to significantly speed up the adversary construction process while maintaining good attack abilities. Below is our response to address the weaknesses and questions raised in the review.\n\n> W1 As mentioned in the limitation section, there is no intuitive defensive strategy or experimental results in real-world scenarios included in the paper.\n> \n\nWe refer the reviewer to sections **2** and **3** of our **general response**. These are interesting directions for future research.\n\n> W2 The proposed attack only considers the white-box setting, i.e., the attacker requires knowledge of all parameters of the diffusion policy. Discussion on black-box setting could also be included in the limitation section.\n> \n\nWe are not considering black-box attacks in this work. We will add a discussion of black-box attacks in the limitation section.\n\n> Q1 Threat model could be clarified with more clearness in problem settings. It appears to me it is a white-box attack where the attacker requires knowledge of all parameters of the diffusion policy.\n> \n\nIn **general response 4**, we have clarified the threat model. We will also incorporate this into the final paper.\n\n> Q2 The novelty of the proposed adversarial attack remains unclear to me. On the one hand, there are existing works [25, 43] which propose adversarial attacks against latent diffusion models, introducing the Monte-Carlo-based adversarial loss. On the other hand, the difficulty of computing gradients due to the inherent long-denoising chain of the diffusion policy is solved by [25, 24, 52]. So, it seems that the novelty over these existing works is that this paper is modifying the training loss as the difference between actions. Clarification of the novelties compared with existing works would be appreciated.\n> \n\nWe have a detailed subsection addressing this in the **general response 1**.\n\n> Q3 Is there any other evaluation metric in existing works? It seems that there is only one evaluation metric used (i.e., the drop of the result task completion scores) in the experimental results to demonstrate the effectiveness of the attack. For me, this one single metric would be fine for the untargeted attacks; however, for targeted attacks, it might be better to show the success rate of generating the target bad actions.\n> \n\nThe success rate is the most direct way for imitation learning to evaluate the effectiveness of a model. Although other continuous reward functions can also be used, the decrease in success rate will be enough to show whether the attacks have largely affect the system.\n\n The second point for evaluating targeted attacks is interesting. Previously, we only evaluated the targeted attacks qualitatively (see the videos at the bottom of our video website). Here, we have added an experiment that measures the distance between the model's output actions after the attack and target actions. Diffusion policy outputs a sequence of actions for the robot to execute. We use DP-Attacker with different strengths to attack it with some target action sequences and measure the average L2 distance between the output action and targeted action over the length of the action. Using predicted action instead of actual agent position is justified by the fact that the low-level controller will execute these actions without any checking. We report the results in a line graph that shows the distance over the environmental steps during the whole rollout. All attacks have step number $N=50$ and $\\alpha=\\frac{2\\delta}{N}$.\n\nPlease see the **newly added PDF Figure 1**. for the results.\n\nIn the first scenario, we used DP-Attacker on the PushT (CNN) task to run global online attacks. The target is set to a 2D coordinate around (323.875, 328.75). Note the side length of the PushT task action space is 1024. See the figure in the added pdf. We were able to manipulate the generated action to be within 20 units of the target coordinate with attack strength $\\delta=0.06$.\n\nIn the second scenario, we used DP-Attacker on the CAN (PH CNN) task to run global online attacks. The end-effector (EE) target is set to a 3D coordinate around (0.1686, 0.1049, 1.0848). The unit is meters. To simplify the metric, we did not set a target for the EE pose or the gripper opening or closing (or else we would need some distance calculating scheme for the 7D output). With attack strength $\\delta=0.06$, we could manipulate the generated action within 5 centimeters close to the target position. Both experiments show the effectiveness of our method.\n\nExample attacked frames of different strengths have been added to the pdf as well (please open it with Adobe Acrobat Reader to view the animated frames). Please see **Figure 2~7**.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vpGHDOAlZq", "reviewer": "Reviewer_69xg", "summary": "The paper analyzes the security vulnerabilities of the diffusion strategy and proposes possible attack scenarios.\nA set of algorithms called DP-Attacker is proposed, which can successfully reduce the performance of the diffusion strategy in different adversarial scenarios (including online and offline attacks).", "questions": "What is unique about this approach compared to a large number of previous black-box and white-box attacks?", "limitations": "N/A", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper proposes a novel adversarial attack framework, specifically targeting diffusion model-based strategies. It not only analyzes the security vulnerabilities of diffusion strategies, but also successfully implements effective attacks on these strategies, demonstrating powerful attack capabilities.\n\nThe paper verifies the effectiveness of the attack algorithm through extensive experiments. These experiments not only include attacks on existing diffusion strategy models, but also cover attack scenarios under different settings, such as online and offline attacks, and provide comprehensive experimental results and analysis.", "weaknesses": "1. The method used is not specific to the diffusion model.\n2. Novelty is limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the questions raised. Please see our response below. We sincerely ask the reviewer to refer to the general response for possible concerns.\n\n> Q1: The method used is not specific to the diffusion model. Novelty is limited. What is unique about this approach compared to a large number of previous black-box and white-box attacks?\n> \n\nWe refer the reviewer to **general response 1** for the novelty of our work. Our algorithm is specifically designed for diffusion policy to ensure online fast adversary generation. To achieve this, we have used a Monte Carlo approximation with the noise prediction loss rather than the end-to-end loss.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "y3qVKV2msa", "reviewer": "Reviewer_7zbu", "summary": "This paper studies the adversarial attack to diffusion policy. Two attack scenario settings are introduced. One is to attack the scene camera by adding imperceptible digital perturbations to the visual observation. The other is to attack the scene by adding small adversarial patches to the environment. Experiments show promising results.", "questions": "1. Is there any specific challenge for applying the existing attack methods to diffusion policy?\n2. Technically, what is the technical difference between attacking the conditional images of diffusion policy and those conditional images of diffusion models for other tasks, i.e., anomaly detection?", "limitations": "n/a", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper is easy to follow and well-organized.\n2. The motivation to attack diffusion policy is interesting.\n3. Two hacking settings are studied.", "weaknesses": "1. The technical novelty is marginal. The proposed framework directly applies the existing attack method ([25, 24, 52] in the Reference) to diffusion policy in two settings. This paper is more like an experimental report by applying adversarial attack methods to diffusion policy in different settings.\n2. The investigation on diffusion policy attack is very shallow. Deeper analysis is needed for improving the contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their insightful comments and feedback. Below is our response to address some of the questions raised in the review. We hope that it will address some of your concerns:\n\n> W1 The technical novelty is marginal. The proposed framework directly applies the existing attack method ([25, 24, 52] in the Reference) to diffusion policy in two settings. This paper is more like an experimental report by applying adversarial attack methods to diffusion policy in different settings.\n> \n\nYes, previous works like [24, 25] propose attacks for LDMs by perturbing the diffused image, and [25] further point out that the mechanism of why LDM is vulnerable is because the noise are scaled up by the encoder of the diffused latents.\n\nHowever, Diffusion Policy is different from LDMs since the attacked item is the condition instead of the diffused latent; as far as we know, no previous works have conducted attacks against the conditional image of diffusion models.\n\nAlso, the diffusion policy is a decision model that includes iterative interaction with the environments; attacking such a diffusion-based system remains an open field before our work.\n\nWe refer the reviewer to our **general response 1** for more details about the novelty of our work. Attacking diffusion policy is fundamentally different from attacking LDMs for T2I tasks. Our proposed DP-Attacker successfully dramatically decreases the performance of DP in simulation.\n\n> W2 The investigation on diffusion policy attack is very shallow. Deeper analysis is needed for improving the contribution.\n> \n\nWe are the first to investigate adversarial attacks against diffusion policy. We focus on providing a set of effective attack algorithms and verifying their effectiveness. We also provide some insights into why diffusion policy can be easily attacked. \n\n> Q1 Is there any specific challenge for applying the existing attack methods to diffusion policy?\n> \n\nWe have mentioned a few challenging aspects of attacking diffusion policy in **general response 1**.\n\n> Q2 Technically, what is the technical difference between attacking the conditional images of diffusion policy and those conditional images of diffusion models for other tasks, i.e., anomaly detection?\n> \n\nThis is an interesting question. We found the following paper on using diffusion models for anomaly detection with conditional images [1].  In this work, the image condition is used directly to guide the generation process by modifying the score with a mathematical formula. This differs from the diffusion policy, where the image condition is used as an input of DNN to predict the noise/score.\n\nNevertheless, attacking diffusion policy is a new task.  DP is a fundamental algorithm for embodied AI, we are the first to evaluate the robustness of it.\n\n[1] A. Mousakhan, T. Brox, and J. Tayyub, “Anomaly Detection with Conditioned Denoising Diffusion Models,” Dec. 03, 2023, arXiv: arXiv:2305.15956. Accessed: Aug. 05, 2024. [Online]. Available: http://arxiv.org/abs/2305.15956", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qccIvseEE3", "reviewer": "Reviewer_fJsu", "summary": "This paper presents strategies to attack visual-based diffusion policy networks. The authors investigated two attacking scenarios: hacking the scene camera by adding imperceptible digital perturbations and hacking the scene by attaching small adversarial patches to the environments.", "questions": "1. Is there a reason authors have considered PGD and patch-based attacks? Did they consider crafting realistic and novel attacks?", "limitations": "I appreciate that the authors have clearly mentioned their work's limitations. But my biggest concern is the use of already-existing (very common) visual perturbations. I suggest the authors craft attacks that are more realistic, challenging to implement, and have a higher probability of being seen in the real world.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is very well-written.\n2. Authors have shown experimental evidence that diffusion-based policy networks are vulnerable to adversarial attacks in digital and physical-domain settings.", "weaknesses": "1. Although authors have demonstrated that diffusion-based policy networks are not robust and susceptible to adversarial perturbations in visual inputs, their attacks are not novel. PGD and patch-based attacks are very common. From this perspective, this work does not contribute substantially. \n2. The authors mentioned in the related work section that physical-world attacks are always based on patches. This claim is not true. Many existing physical adversarial attacks are not based on patches. (e.g., Adversarial laser, adversarial shadows, etc.). The \n3. The authors mentioned in the limitations section that they have not evaluated this method in real-world settings, so it is hard to assess its practicality. \n4. The authors' main contribution is the demonstration that diffusion policy networks are vulnerable to adversarial attacks. Whenever the input is corrupted, it affects the model's output. It is unclear how likely a policy network or robot (in case of physical attacks) is to face these perturbations in real life. It would be great if authors could focus on crafting application-specific attacks with a high chance of occurring in real life.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for the reviewer's comments and valuable feedback. We are delighted that the author liked our writing and is convinced by our algorithms. We appreciate that the reviewer is convinced that diffusion-based policy networks are susceptible to adversarial attacks. Here’s our response to the reviewer’s concerns:\n\n> W1 Although authors have demonstrated that diffusion-based policy networks are not robust and susceptible to adversarial perturbations in visual inputs, their attacks are not novel. PGD and patch-based attacks are very common. From this perspective, this work does not contribute substantially.\n> \n\nThe focus of our work is to derive a suite of tools that could successfully hijack diffusion-based imitation learning and reveal that there is still a long barrier to making this algorithm safe for deployment. We have successfully reviewed this by developing fast and efficient online attacks, transferable offline attacks, and physically realizable patch attacks. We list some points in the **general response 1** about the comparison of our work with existing methods and why it is challenging.\n\n> W2 The authors mentioned in the related work section that physical-world attacks are always based on patches. This claim is not true. Many existing physical adversarial attacks are not based on patches. (e.g., Adversarial laser, adversarial shadows, etc.).\n> \n\nThank you for pointing this out to us. Adversarial patches are widely used in real-world attacks [3, 4, 5], especially in robotics applications [6, 7]. Future work can be done on exploring the robustness of DP against different types of realistic attacks.\n\nAdversarial lasers and shadows are interesting forms of physical attacks. We have found two works that have used them in physical world attacks. [1] focuses on adversarial lasers, and [2], which focuses on adversarial shadows. However, we believe patch-based attacks will be more suitable for conducting physical attacks for **diffusion policy**. [1] and [2] focus on attacking **DNNs for classification**. As mentioned by [1], the effectiveness of these attacks either creates new visual cues that lure DNNs to misclassify or perform as a dominant feature of a set of classes of objects. This logic does not transfer that well to diffusion policy, which is a visual motor policy that uses images as conditions to predict continuous actions. \n\n\n\n\n> W3 The authors mentioned in the limitations section that they have not evaluated this method in real-world settings, so it is hard to assess its practicality.\n> \n\nWe refer the reviewer to our **general response 2** for discussion on real-world attacks.\n\n> W4 The authors' main contribution is the demonstration that diffusion policy networks are vulnerable to adversarial attacks. Whenever the input is corrupted, it affects the model's output. It is unclear how likely a policy network or robot (in case of physical attacks) is to face these perturbations in real life. It would be great if authors could focus on crafting application-specific attacks with a high chance of occurring in real life.\n> \n\nTo answer the lack of real-world experiments, we refer to the answer for **general response 2**. In this work, we focus more on exposing the security risk of diffusion policy, so we have chosen patched attacks. As answered in W3 we believe other forms of physical attacks to be less effective for attacking DP. We will discuss the limitation that we have not thought of making physical attacks more realizable and will point in the direction of making less conspicuous patches as attacks.\n\n>Q1 Is there a reason authors have considered PGD and patch-based attacks? Did they consider crafting realistic and novel attacks?\n>\n\nSince our attack scenario is white-box, PGD is one of the best tools for gradient-based adversary construction. As we have mentioned in response to W2 and W4, we believe that physical patches are good candidates for attacking diffusion policies.\n\n\n[1] R. Duan et al., “Adversarial Laser Beam: Effective Physical-World Attack to DNNs in a Blink,” Mar. 11, 2021, arXiv: arXiv:2103.06504. doi: 10.48550/arXiv.2103.06504.\n\n[2] Y. Zhong, X. Liu, D. Zhai, J. Jiang, and X. Ji, “Shadows can be Dangerous: Stealthy and Effective Physical-world Adversarial Attack by Natural Phenomenon,” Mar. 22, 2022, arXiv: arXiv:2203.03818. doi: 10.48550/arXiv.2203.03818.\n\n[3] Y. Mirsky, “IPatch: a remote adversarial patch,” Cybersecurity, vol. 6, no. 1, p. 18, May 2023, doi: 10.1186/s42400-023-00145-0.\n\n[4] T. B. Brown, D. Mané, A. Roy, M. Abadi, and J. Gilmer, “Adversarial Patch,” May 16, 2018, arXiv: arXiv:1712.09665. doi: 10.48550/arXiv.1712.09665.\n\n[5] Y.-C.-T. Hu, J.-C. Chen, B.-H. Kung, K.-L. Hua, and D. S. Tan, “Naturalistic Physical Adversarial Patch for Object Detectors,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada: IEEE, Oct. 2021, pp. 7828–7837. doi: 10.1109/ICCV48922.2021.00775.\n\n[6] A. Tanev, S. Pavlitskaya, J. Sigloch, A. Roennau, R. Dillmann, and J. M. Zollner, “Adversarial Black-Box Attacks on Vision-based Deep Reinforcement Learning Agents,” in 2021 IEEE International Conference on Intelligence and Safety for Robotics (ISR), Tokoname, Japan: IEEE, Mar. 2021, pp. 177–181. doi: 10.1109/ISR50024.2021.9419509.\n\n[7] Y. Jia, C. M. Poskitt, J. Sun, and S. Chattopadhyay, “Physical Adversarial Attack on a Robotic Arm,” IEEE Robot. Autom. Lett., vol. 7, no. 4, pp. 9334–9341, Oct. 2022, doi: 10.1109/LRA.2022.3189783.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "INlbXPBnWQ", "reviewer": "Reviewer_FStp", "summary": "The paper proposes an adversarial example construction method targeting Diffusion-based Policies, aiming to create malicious observation inputs to diffusion policies that cause them to fail in generating correct actions, leading to robot errors. The authors present attack methods including both untargeted and targeted attacks. The main idea is to construct adversarial examples by optimizing a malicious denoising loss and updating the image according to the gradient. Experiments demonstrate that the generated adversarial examples can lower the success rate of the target diffusion policy.", "questions": "Overall, this paper is interesting and presents the first adversarial attack against diffusion-based visuomotor policies. However, I have the following concerns:\n\n- The proposed method seems to be a straightforward extension of prior work [1], with the main modification being the adjustment of the adversarial loss to accommodate the differences between diffusion policy and standard T2I models.\n\n- Unlike related work that uses adversarial examples to prevent painting imitation, this paper attacks the system directly. Can the authors provide a clear threat model, including a well-defined attacker's goal and capabilities? It would be helpful to provide corresponding scenarios.\n\n- The authors claim that DP-Attacker can generate highly transferable perturbations. How is this demonstrated in the experiments? For example, the authors claim that it is actually the encoder being attacked. The diffusion policy uses ResNet-18 as the encoder [2], can the adversarial examples generated by DP-Attacker transfer to other encoders?\n\n[1] Adversarial example does good: Preventing painting imitation from diffusion models via adversarial examples, 2023\n\n[2] Diffusion Policy: Visuomotor Policy Learning via Action Diffusion, 2023", "limitations": "One major limitation is the lack of real-world demonstrations for patched attacks. The authors have acknowledged this in the paper. Additionally, I have some questions regarding the authors' claim about the transferability of the generated adversarial example.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- This is the first adversarial attack against a diffusion-based visuomotor policies.\n- The proposed attack framework is comprehensive, including targeted, untargeted, online, and offline attacks.\n- The paper is well-written and easy to understand.", "weaknesses": "- The proposed method seems to be a straightforward extension of a previous approach applied to images.\n- The threat model is not well-defined.\n- The attack method makes strong assumptions on attackers, and its effectiveness in real-world scenarios, especially for patched attacks, is unclear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the useful valuable comments and feedback! We are glad that the reviewer liked our writing and acknowledges our comprehensive attacks. We would like to address the reviewer’s questions below.\n\n> The proposed method seems to be a straightforward extension of a previous approach applied to images.\nThe proposed method seems to be a straightforward extension of prior work [1], with the main modification being the adjustment of the adversarial loss to accommodate the differences between diffusion policy and standard T2I models.\n> \n\nYes, previous works like [24, 25] propose attacks for LDMs, and [25] further point out that the mechanism of why LDM is vulnerable is because the noises are scaled up by the encoder of the diffused latents.\n\nHowever, Diffusion Policy is different from LDMs since the attacked item is the condition instead of the diffused latent; as far as we know, no previous works have conducted attacks against the conditional image of diffusion models.\n\nWe refer the reviewer to our **general response 1** for more details about the novelty of our method.\n\n> The threat model is not well-defined. Unlike related work that uses adversarial examples to prevent painting imitation, this paper attacks the system directly. Can the authors provide a clear threat model, including a well-defined attacker's goal and capabilities? It would be helpful to provide corresponding scenarios.\n> \n\nWe have added a detailed definition of the threat model in general response 4. We will also add this clarification to the final paper.\n\n> The attack method makes strong assumptions on attackers, and its effectiveness in real-world scenarios, especially for patched attacks, is unclear.\n> \n\nWe refer the reviewer to **General Response 2** for attacks in the real world.\n\n> The authors claim that DP-Attacker can generate highly transferable perturbations. How is this demonstrated in the experiments? For example, the authors claim that it is actually the encoder being attacked. The diffusion policy uses ResNet-18 as the encoder [2], can the adversarial examples generated by DP-Attacker transfer to other encoders?\n> \n\nThat's a good question. We conduct experiments in **general response 5** for the transferability of the attacks across different model structures (CNN and Transformer). For the visual encoder, while there are no pre-trained DP to use, we think there is still some level of transferability according to previous investigations into different model structures.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RQeQBdgFu5", "reviewer": "Reviewer_7kiH", "summary": "This paper introduces DP-Attacker, a suite of algorithms designed to generate adversarial attacks against diffusion-based policies (DPs). The paper explores two attack scenarios: (1) hacking the scene camera by adding imperceptible digital perturbations to the visual inputs, and (2) hacking the scene by attaching small adversarial patches to the environment. The authors demonstrate the effectiveness of DP-Attacker through extensive experiments on pre-trained diffusion policies across various robotic manipulation tasks, showing significant degradation in performance for both online and offline attacks.", "questions": "• Can the authors provide a more detailed analysis of the transferability of the generated adversarial perturbations across different environments and robot models?\n\n• How does the performance of DP-Attacker compare to existing adversarial attack methods designed for deep neural networks? Are there specific advantages or disadvantages of using DP-Attacker?\n\n• What specific defensive strategies can be implemented to mitigate the identified vulnerabilities of diffusion policies? Can the authors provide a theoretical analysis or simulation results to demonstrate the effectiveness of these defenses?\n\n• What are the potential limitations of the proposed attack method in terms of computational complexity and scalability to large-scale systems?", "limitations": "The authors acknowledge the limitations of their work and discuss the broader societal impacts of their research.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Novelty: The paper presents the first suite of white-box attack algorithms specifically designed for visual-based diffusion policies. The proposed approach, based on noise prediction loss, effectively circumvents the challenges posed by the chained denoising structure and high randomness of diffusion models.\n\n- Significance: By highlighting the vulnerability of diffusion policies to adversarial attacks, the paper raises important safety concerns for real-world applications. This research serves as a crucial step towards developing more robust DP systems and ensuring their reliability in practical scenarios.", "weaknesses": "- Lack of Defense Strategies: While the paper demonstrates the effectiveness of DP-Attacker, it does not explore or propose any defensive strategies to mitigate the identified vulnerabilities. This omission hinders the development of robust DP systems and leaves open the question of how to protect against such attacks.\n- Limited Attack Scenarios: The paper focuses on two specific attack scenarios: hacking the camera and attaching adversarial patches. Exploring a broader range of attack scenarios, such as manipulating the robot's sensors or exploiting other weaknesses in the DP system, would provide a more comprehensive understanding of the system's vulnerabilities.\n- Computational Complexity: The paper mentions the computational complexity of gradient calculations for the end-to-end action loss. While the proposed noise prediction loss mitigates this issue, further analysis and comparison with other attack methods in terms of computational efficiency would be beneficial.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable feedback and comments. We are glad that the author finds our work novel and impactful for developing more robust DP systems. Below is our response to some of the questions raised by the reviewer.\n\n> Lack of Defense Strategies: While the paper demonstrates the effectiveness of DP-Attacker, it does not explore or propose any defensive strategies to mitigate the identified vulnerabilities. This omission hinders the development of robust DP systems and leaves open the question of how to protect against such attacks.\n> \n> \n> What specific defensive strategies can be implemented to mitigate the identified vulnerabilities of diffusion policies? Can the authors provide a theoretical analysis or simulation results to demonstrate the effectiveness of these defenses?\n> \n\nWe refer the reviewer to **general response 3** on defense strategies. \n\n> Limited Attack Scenarios: The paper focuses on two specific attack scenarios: hacking the camera and attaching adversarial patches. Exploring a broader range of attack scenarios, such as manipulating the robot's sensors or exploiting other weaknesses in the DP system, would provide a more comprehensive understanding of the system's vulnerabilities.\n> \n\nAttacking the robot’s sensors (proprioception) in the DP framework is an interesting direction for future research. However, we did not include this because we believed hackers were more likely to manipulate the image input (especially in physical-world attacks).\n\n> Computational Complexity: The paper mentions the computational complexity of gradient calculations for the end-to-end action loss. While the proposed noise prediction loss mitigates this issue, further analysis and comparison with other attack methods in terms of computational efficiency would be beneficial.\nWhat are the potential limitations of the proposed attack method in terms of computational complexity and scalability to large-scale systems?\n> \n\nThe computational complexity and scalability of our attack are interesting research directions. DP models are often quite small now to satisfy the requirement of fast inference as a visual-motor policy. Our proposed optimization is based on single-step noise prediction loss, which ensures fast gradient calculation.\n\n> Can the authors provide a more detailed analysis of the transferability of the generated adversarial perturbations across different environments and robot models?\n> \n\nWe refer the reviewer to **general response 5** on the transferability of our attacks.\n\n> How does the performance of DP-Attacker compare to existing adversarial attack methods designed for deep neural networks? Are there specific advantages or disadvantages of using DP-Attacker?\n> \n\nOur DP-Attacker is the first proposed suite of methods to attack diffusion policy. One can view the end-to-end loss comparison as an existing method. As shown in Table 4 in our paper, E2E loss with DDPM is dramatically more computationally heavy and impossible to use for online attacks. Our proposed DP-Attacker is faster and performs better in terms of degrading the policy performance.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers for their valuable comments and feedback on our paper. We are delighted that all the reviews find our method a novel and effective attack against diffusion policies. We are glad that the **reviewers kPb5 and 69xg** are convinced by our experimental visualizations and results. **Reviewers 7zbu, fJsu, and FStp** found our paper well-written and easy to follow.

In the following section, we would like to address some of the common questions raised by the reviewers:

> 1. Comparison of DP-Attacker with existing methods.

We are the first to present a set of algorithms (DP-Attacker) that can successfully attack diffusion policy, one of the most popular methods for imitation learning in robotics. We focus on exploring the attackability of diffusion policy and devising new adaptive attacks for this special task and special structure, which is an open question. Our method is based on previous adversarial attacks for DNNs and LDMs. However, none of them **can be directly used** to attack diffusion policy effectively since:

1. Diffusion policy is a “dynamic” policy network, whereas LDMs are more “static” models in their uses. The policy network is invoked at a high frequency, requiring fast adversary construction for online settings. Our method successfully achieved this with carefully designed loss and algorithms. We also devised offline attacks that can function across the whole policy rollout.

2. Although methods have been developed for attacking diffusion models (DMs) for image generation [24, 25, 43, 52], the problem differs from how diffusion policy is formulated. In attacking image DMs, which are mostly latent DMs, where the image generation is processed through SDEdit, the attacked image is the denoising input of the LDM network. However, in diffusion policy, the image is the conditional input of the diffusion model, and the denoising input is the robot's actions. 

3. Randomness is injected during the denoising process while generating an action in diffusion policy. This might make the policy more robust to adversarial inputs.


> 2. Lack of real world experiments.

First, the digital attacks (e.g. online attacks in our settings) will not be affected by real-world settings, since the attacks are generated to be directly added to the image.

For adversarial physical patches, we try our best to make it as close as the real settings in the real world by evaluating our attack with a "physically" stuck patch in the simulation:

The patch is put into the photo-realistic simulation environment provided by robomimic and robosuite and has been shown to decrease the performance of diffusion policy successfully. Real-world data is more complex and is challenging to learn for diffusion policy. We conducted our evaluation in simulation to show the effectiveness of our attack method better. Models trained on real-world data may also be more sensitive to adversarial perturbations. Thus, we believe our proposed DP-Attacker will work in the real world as well.

For further research, we will try to apply it to the real world.

> 3. What could be the defense strategy?

While the defense of DP-Attacker is not the focus of this work, we provide some possible defenses:

- (1) purify the observation using diffusion-based purification methods

- (2) apply adversarial training to increase the robustness of the image encoder

> 4. Could you clarify the threat model?

In our work, we only consider white-box attacks on diffusion policy (DP) in which the attacker can access the model, its parameters, and the data used to train it. The attacker's goal is to decrease the performance of DP (task score or success rates). In targeted attacks, we also wish to be able to control the model’s generated trajectory. Two attacking scenarios are explored (Fig. 1 in the manuscript). In the first scenario, the hacker is allowed to modify every pixel of the image with some budget $\delta$. Afterward, the modified image is used for DP inference and rollout. We develop two types of perturbations; one is calculated online and is generated per inference. The second type is pre-generated and used throughout the rollout. In the second scenario, the attacker puts a pre-generated colored patch in the camera's view. The patch undergoes a physical process of reflection and camera imaging before being used for inference in the model.

> 5. What is the transferability of your attacks?

Originally, we used transferability to indicate that our devised offline attacks (global or patched) can function to disrupt the performance of the DP across the rollout of the policy where the input image is consistently changing. However, since the reviewers are also interested in the transferability of our attacks across models, we conducted the following experiment. We believe it is more reasonable to test the transferability of offline attacks rather than online attacks because online attacks are often very specific to the model with white-box access and will not transfer well to other networks.

We first tested the transferability of offline global attacks. We used DP-attacker to generate untargeted global offline attacks ($\delta=0.03$) on two checkpoints (CAN-MH-CNN and CAN-MH-TF). Then, we evaluated these models using the two generated adversarial perturbations, and the success rate is listed in Table 1 of the attached PDF.

| Runner Model\Attacked Model | CNN | TF | Original SR |
| --- | --- | --- | --- |
| CNN | 0.34 | 0.78 | 0.98 |
| TF | 0.32 | 0.46 | 0.92 |

We also tested the transferability of patched attacks. We used DP-Attacker to generate adversarial patches on two checkpoints (CAN-PH-CNN and CAN-PH-TF). Then, we evaluated these models using the two generated adversarial perturbations, and the success rate is listed in Table 2 of the attached PDF.

| Runner Model\Attacked Model | CNN | TF | Original SR |
| --- | --- | --- | --- |
| CNN | 0.16 | 0.54 | 0.98 |
| TF | 0.42 | 0.44 | 0.92 |

Meta-review: The paper proposes a new attack against "diffusion policy" (DP) models, whose underlying goal is to craft tiny perturbations that can mislead these models so that the device receiving the model's output takes a wrong action. 

Six reviewers reviewed the paper. Three reviewers provided positive scores (7, 6, 5), while three reviewers are more on the negative side (proposing a 4). No flaws were identified during the reviewing phase, indicating that the results are likely to be correct and representative of real scenarios. The reviewers all acknowledged that the paper tackles a relatively underinvestigated aspect (from a security standpoint) of diffusion-based policies, which have plenty of real-world relevance. Some ethical concerns were identified, but have been deemed to be addressable by the Ethical reviewers. However, the major concerns identified were due to a limited novelty of the proposed attack from a methodological perspective. 

Indeed, the proposed attack seems to borrow its fundamental principles from prior work, and the results appear to be also somewhat expected, especially given that the paper envisions a "white-box" attacker.

While this is a noteworthy concern (as white-box attacks are tough to stage in the real world), I believe that the paper should be accepted. This is due to various factors, such as: 

* the paper is well written and presented
* the results show that the attack is effective
* the paper does consider a "novel" setting (from an adversarial ML perspective)
* the paper carries out an experiment envisioning a "physical" adversarial patch which aligns with a real-world environment
* the interactive discussion shed more light on two important properties of the attacks: their transferability as well as potential defenses

Given the above, the paper is likely to create interest in the NeurIPS audience (due to the widespread usage of DP) and while not providing groundbreaking results, it may represent a stepping stone for future research (which can, e.g., consider more constrained threat models; or develop ad-hoc countermeasures).
Decision: Accept (poster)