C3DM: Constrained-Context Conditional Diffusion Models for Imitation Learning
Authors: Vaibhav Saxena
Yotto koga
Danfei Xu
Venue: Accepted by TMLR ()

Abstract: Behavior Cloning (BC) methods are effective at learning complex manipulation tasks.  However, they are prone to spurious correlation - expressive models may focus on distractors that are irrelevant to action prediction - and are thus fragile in real-world deployment. Prior methods have addressed this challenge by exploring different model architectures and action representations. However, none were able to balance between sample efficiency and robustness against distractors for solving manipulation tasks with a complex action space. We present Constrained-Context Conditional Diffusion Model (C3DM), a diffusion model policy for solving 6-DoF robotic manipulation tasks with robustness to distractions that can learn deployable robot policies from as little as five demonstrations. A key component of C3DM is a \emph{fixation} step that helps the action denoiser to focus on task-relevant regions around a predicted \emph{fixation point} while ignoring distractors in the context. We empirically show that C3DM is robust to out-of-distribution distractors, and consistently achieves high success rates on a wide array of tasks, ranging from table-top manipulation to industrial kitting that require varying levels of precision and robustness to distractors.

Published in Transactions on Machine Learning Research (07/2024)
C3DM: Constrained-Conter Conditional Diffusion Models for Imitation Learning
Vaibhav Saxena
vsamena33@@atech.ccoo edu
School of Interactive Computing Georyia Institute of Technology
Yotto Koga
yotto.koga Bautodekk.oooccoommmo com
Robotics Lab Autodesk Research
Danfei Xu
donfei@gatech. edu
School of Interactive Computing Georgia Institute of Technology
Reviewed on OpenReview: /ttp::///penrevuiie net/ forum? id- jelexdRRAA
Abstract
Behavior Cloning (BC) methods are effective at learning complex manipulation tasks. How- ever, they are prone to spurious correlation corveltion--cxpressiiv models may focus on distractors that are irrelevant to action prediction and are thus fragile in real-workd deployment. Prior methods have addressed this challenge by exploring different model architectures and action representations. However, none were able to balance between sample efficiency and robustness against distractors for solving manipulation tasks with complex action space. We present Menstrraied-Contter Conditional Dilfusion Model (C3DM), a diffusion model policy for solving 6-DoF robotic manipulation tasks with robustness to distractions that can learn deployable robot policies from as little as five demonstrations. A key component of C3DM firation step that helps the action denoiser to focus on task-relevant regions around a predicted fination point while ignoring distractors in the context. We empirically show that C3DM is robust out-of-distributiin distractors, and consistently achieves high success rates wide array of tasks, ranging from table-top manipulation to industrial kitting that require varying levels of precision and robustness to distractors
1 Introduction
Behavior cloning (BC) is a simple yet elfective method for learning from offline demonstrations when such data is available. However, with limited training data and high-capacity neural network models, a BC policy trained to map high-dimensional input such as images to actions often degencrates to focusing on spurious features the environment instead of the task-relevant ones (De Haan et al. 2019 This leads to poor generalization and fragile exccution in real-world applications such as kitting and assembly, where acting scenes with distractors is inevitable. This challenge is especially prominent for continuous-control problems, where instead of committing to target, the policy often collapses to the "average" of a mix of correct and incorrect predictions due to distractions. Our research aims to develop a class of sample-efficient BC methods that are robust to distractions in the environment.
Prior work attempts to address this challenge through different model architectures and action rep- resentations. A prominent line of research (Zeng et al. 2021 Shridhar et al. 2022a b) uses fully convolutional networks (FCNs) with residual connections that learn mappings from visual input
Project website: https://sites.. google. http://tw....eeeee////aaaaaaaaaa http:////cc.ciiiiittttiaaaaaaaaa

---

Published in Transactions on Machine Learning Research (07/2024)
to spatially-qantizedd action space, and 2) attend to local features in addition to global view of the input. In these models each possible action gets assigned with probability mass, which avoids the action averaging effect making the model less prone to bad precision due to distractions.
While effective for 2D tasks such as pushing piles and transporting objects, this strategy prohibits lasks with complex 6-DoF action spaces as the quantization space grows exponentially with action dimensions. For model- ing actions in continuous space, generative policy mod- els (Florence et al. 2021: Weng et al.) 2023 Chi et al.,
Corntraieed osseraatio
C3DM
Fiation point
GoST action
Greea
2023) have shown that flexible probabilistic models such as Diffusion Models (Ho et. al. 2020 Chi et al. 2023) allow for modeling complex actions. However, the underlying high-capacity neural network models are still prone to learning spurious input-oulput correlations when training data is limited.
In this work, we argue that these generative policy models can be made robust to distractions if we imbue them with the ability to attend to inferred local features, like an FCN, and share parameters across the processing of such constrained contezts. Hence, we build a conditional generative model that controls what it sees in the input context by inferring action-relovant parts of the scene using a "fixation point" parameterization, and train it to iteratively refine its action prediction as it also refines its input context. Our method, Constrained- model policy that solves 6-DoF robotic manipulation tasks with high sample efficiency and robustness to distractions. Our model learns a distribution over actions given a global input context as either an RGB image or depth map. Similar Gaussian noising process, but in addition to that also infer constrained observations that are fixated around the largel action. The learned generalive distribution ut ulilizes an iteralively refining denoising process on both the observation and action variables. Our novel denoising process implements "fixation" by zooming into a part of the imput image, or masking out pixels, around iteratively denoising iteration. This procedure, that we call the fixation-while werioon-wiil--eeee
Figure 1: Illustrating 6-DoF action prodiction for table-top manipulation using our diffusion model (C3DM), which implicitly learns to fixate rele- vant parts of the input and iteratively refines its prediction using action-relevant details about the observation.
Conditional Diffusion Model (C3DM), is a diffusion
(Chi et al.) 2023), our inference distribution over actions is fixed a
"fixation points" at each whilo-----i--- Diffusion Process (fDDP) facilitates invariance to distractions during action denoising. We illustrate this process in Figure with the detailed procedure for implementing the proposed method in Figure 2 Our key contributions are:
We propose theoretically-grouneed fixation-while Denii-DDo-in- Diffusion Process (fDDP) framework that enables Diffusion Policy (Chi ct al., 2023) to progressively "zoom into" action-relevant input throughout the ileralive denoising process.
2.
We show that our method, C3DM, outperforms existing generative policy methods wide array of simulated tasks, such as sweeping, sorting, kitting and assembly, deploy our model on real robot to perform sorting and insertion after training on just 20 demonstrations, and demonstrate robust sim-to-real transfer on a kitting and cup hanging task from 5 demonstrations.
2 Related Works
Visual Imitation Learning. Imitation learning (IL) has been proven effective for robotic manipulation
tasks [Schaal 1999 Billard et al. 2008
Englert & Toussaint 2018 Recent works (Zhang et al. 2018 Finn
el al. 2017 Mandlekar et al. 2020
2021 Brohan et al. 2022 use deep neural networks to map directly from
image observations to action, and demonstrated visuomotor learning for complex and diverse manipulation tasks. However, these policies tend to generalize poorly to new situations due to spurious connections between
pixels and actions (Wang et al.) 2021 De Haan al. 2019) and requires extensive training data to become
robust. To address the challenge, recent works (Abolghasemi et. al. 2018 Wang et. al. 2021 Zhu et. al.
2

---

Published in Transactions on Machine Learning Research (07 /204)
Constraining contest around foution points
Deploying G-Oof action
Action Denoising
Image briceer
Score Fandoon
Soone
Fandorr
anton
Fouiton
point
Figure 2: Constrained- Conteext Conditional Diffusion Model (C3DM) for visuomotor policy learning. Here we illustrate the iterative action-refieement procedure (4 timesteps) using our seation-while-denossin diffusion process (/DDP), wherein we constrain our input context around a "fixation point" predicted by the model (A) at each refinement step. Subsequently, we refine the predicted action by fixating only on the useful part of the context, hence removing distractions and making use of higher levels of detail in the input.
2022 have incorporated visual attention Mnih al. (2014) as inductive biases in the policy. For example, VIOLA (Zhu et al. 2022 uses object bounding priors to select salient regions as policy input. Our constrained context formulation can be similarly viewed a visual attention mechanism that iteratively refines with the diffusion process. Another line of work exploits the spatial invariance of fully convolutional models and discretized action space to improve learning sample efficiency (Zeng et al. 2021 Shridhar et al., 2022a b Notably, PerAct (Shridhar et al. 2022b) show strong performance on 6DoF manipulation tasks. However, to implement discretized 3D action space, PerAct requires a large voxel-wise transformer model that is expensive to train and evaluate, and it also requires 3D) point cloud input data. Our method instead adopts an implicit. model that can model arbitrary action space and only requires 2D image input.
Diffusion Policy Models. Diffusion models have shown remarkable performance in modeling complex data distributions such as high-resolution images (Song et al. 2020 Ho et al. 2020). More recently, diffusion models
have been applied to decision making (Chi et al.) 2023 Ajay et al. 2022 Zhong et al.
2023 Mishra et al.
2023 and show promising results in learning complex human actions (Chi et al. 2023
Pearce et al. 2023
Reuss et al. 2023) and conditionally generating new behaviors (Ajay et al.) 2022 Zhong et al. 2023). Closely related our work is DiffusionPolicy (Chi ct al. (023)) that uses diffusion model to learn visuomotor policios. However, despite the demonstrated capability in modeling multimodal actions, DiffusionPolicy still learns an
end-to-end mapping between the input and the score function and is thus prone to spurious correlation, we will demonstrate empirically. Our method introduces framework that enables the denoising diffusion process to fixate on relevant parts of the input and iteratively refine its predictions.
Implicit Policy Models. Closely related to diffusion policy models are implicit-moole model policies (Florence et al. 2021 Jarrett et al.] 2020), which represent distributions over actions using Energy-Based Models (EBMs) (LeCun et al.] 2006). Finding optimal actions with an energy function-basee policies can be done through sampling gradient-based procedures such as Stochastic gradient Langevin dynamics. Alternatively, actions prediction can be implicitly represented a distance field to the optimal action (Weng et al., 2023 Similar to diffusion policies, implicit models can represent complex action distributions but are also prone to spurious correlations in imitation, as we will demonstrate empirically.
3 Background
Denoising Diffusion Probabilistic Models (DDPMs) are class of generative models that introduce a hierarchy of latent variables and are trained to maximize the variational lower-bound on the log-likelihood of observations (ELBO). To compute the ELBO, this class of models fixes the inference distribution on the introduced latent variables to diffussion process, that can be aggregated to directly infer each latent variable from the observation. Say Xy represents the random variable that we want model, we introduce T latent variables {x,) each of which can be inferred from the observation Xq using
3

---

Published in Transactions on Machine Learning Research (07/2024)
((x1 == Va(t) xo
-xo + /1 ((t) c2.
(1)
where Et -N(0,)) I) is the standard normal Gaussian noise, and (tt) is a fixed "noise schedule" that determines the variance of the noising process, and satisfies (tt) 1 Vt and a(t) 0 as - T. We call the index "time," " and in principle the cardinality of the set [x+]== should tend to infinity. The choice of (xx+(xo) can play a significant role in the learning process of the generative distribution, and we will see in this paper how it affects the performance of our action prediction model during evaluation. Additionally, the denoising process can be obtained using the Bayes' rule as g(xe-1 xo) q(xx/xx-1, Xo) g [x))
Each latent variable Xi....,Xy XY can be interpreted over Xo is modeled using conditional distributions p(x:-1 (xx) as p(xo) = p(x)) II 1= p(xx-1|xe), where p(xr) is fixed to be a standard Gaussian prior to comply with the noising process in (. The process of learning p(xo) comes down to minimizing divergences (L(g(x,-1))) Xx, (xx/
noisy variant of xo, and the generative distribution
to learn p(x:-1|x:). As shown by Ho et al.] 2220), we can learn p by training an auxiliary "score function" fo. parameterized using a neural network, that is (generally) trained to predict the standard normal noise et that was used to infer x; (e (e-prediction), that is, = argming (S at). Other design choices include training fe to predict the denoised sample Xy directly (z (z-prediction), or combination of x and c (u-prediction) (Salimans & Ho 2022
To generate samples of Xo, we begin with samples of Xr, which for the chosen noising process in should result in a sample from the standard normal distribution. Then, we "denoise" Xy using the ep noise predicted by the learned score function fo(XT, T) to obtain the next latent Xy-1 ~ p(x -1 |XT).. This is one step of the "iterative refinement" process. We continue to refine the sample for T steps after which we return the last denoised sample, which is a sample from the distribution p(xa).
Problem Setup (Imitation Learning). In this work, we focus on modeling a conditional generative distribution over 12-dimensional action variables (Xo E ! conditioned on image observations O, i.e. O(xo/O). We obtain observations from a downward-facing camera over a table-top, and output two 6-DoF gripper poses as actions for robot, which comprise 3D position and Euler rotation angles around z-r-y axes (in that order) in the camera frame. The output actions then parameterize primitive pick-and-place motions that can be deployed on a robot using any off-the-shelf motion planner to generate a trajectory leading to the 6-DoF gripper poses output from the generative model.
4 Method
We present a Constrained-Connerr Conditional Diffusion Model (C3DM) that predicts robot actions Diffusion Process (fDDP) infers a
conditionod on image observations. Our novel Tixation-whil--eenniit "fixation point" the input at each action-refinement step, which it iteratively uses to 1) ignore distractions by constraining its input context around that point, and 2) improve its precision by querying for higher levels of detail in the image input around the predicted fixation point. fDDP exploits the observation-action coupling which allows it to determine fixation points in the context and constrain its observation throughout the action refinement process.
4.1 Conditional Diffusion for Action Prediction
Diffusion models have shown great success in image and video generation (Ho et al. 2020 2022 Song et al., 20200. Recently their conditional counterpart (Diffusion Policy (Chi et al. 22023) ) has been shown learn stable image-conditioned robot policies with little hyperparameter tuning. A major contributing factor for the success of these models is the iterative refinement procedure that reuses model capacity to ileratively denoise latent samples in the action space into useful ones that match the data distribution. More precisely, these models learn a denoiser function eg that learns to denoise a random action ar into a larget action an given a static observation O in an iterative fashion. This iterative refinement process can be depicted as, aT <ol 0) ar-1 sa(
O) r6( O), an. However, given limited offline demonstrations, these models are not immunc from lcarning spurious correlations between image input (O) and action output (a), which generally leads to imprecise and fragile exccution on robots. We argue that while is hard to control what correlations model learns from data, it is possible to control what the model sees and use it to our ad advantage.
4

---

Published in Transactions on Machine Learning Research (07 (07/2024) /024)
0
X
X
X
4
Pp1
(a) Diffusion Policy
(b) C3DM
(a) Action inference and generation in Diffusion Policy, compared with (b) inference and generation of observation-action tuples in C3DM. Filled circles represent observed variables. In (b), solid lines represent the generative distributions, po O0(O,1|X, for generating latent (fixated) observations and po po(a:-1 for the next latent. (noisy) actions. Solid and dashed lines logether show the inference (de-noising) distributions, g(O,11 O, Xo) and q(a,- X,, an) for inferring latent observations and actions respectively.
Figure 3:
In this paper, we build a diffusion model that learns itcratively constrain its input context around inferrod aclion-relevant parts of the scene in process we call "fixation." As will discuss later, this process can also be viewed as form of data augmentation that further boosts the learning sample efficiency. In summary, our model 1) learns to infer action-rclevant parts of the scene as "fixation-points" while performing action denoising, and 2) appends the iterative action refinement with observation refinement that we implement as zooming or masking around the inferred fixation-point, that helps improve robustness and sample efficiency in action prediction.
4.2 Constrained-Conter Conditional Diffusion Model (C3DM)
We propose a fixation-while-DDnee Seaiilll-Dniiinngggggg Diffusion Process (fDDP) that utilizes a joint iterative refinement of intermediale observations and actions (deemed fialeed and noisy respectively). In the following subsections, we formulate the different components that make up the JDDP, essentially the generative and inference distributions, and talk about our specific implementations of the same.
Generative distribution. We build theoretical framework for generating target action ap, given single global observation Or of the scene, by introducing both latent observations and actions for timesteps 1 to T-1 1. We define the joint distribution for such generation as,
po(au Or) po (au:r, Our-1 day dOuT-1.
where {a,)== are noisy actions and {0,)T=) T-1 are fixated observations. We define a generated tuple fixated obsorvation and action as X, = (a,, O,). We factorize the joint as reverse de-noising process given by,
T
(a (pT) = pa (ar (T) II pa (X. Xx).
(3)
t=1
We further brcak down the generation (X, (X) into,
Po (x X,) = po (p X, ) a
= -O (X,
pa (a. X.)
(4)
fixated abs next nolay action
where po(ai-1(X;) generates the "next noisy action" and po (O,-1/XX) the next "fixated observation." Given a global observation of the scene, we can use these generative distributions to iteralively generale denoised actions and fixated observations where the latter act as input to the next action denoising. This framework hence enables action denoising with a different observation input at each timestep, which this work we leverage for action generation with robustness to distractors. We illustrate these generative distributions solid lines in Figure
5

---

Published in Transactions on Machine Learning Research (07/2024)
Inference distribution. We formalize the inference distribution as reverse de-noising process ((X,-1Xx Xo); which we break down as, g(X,,1 x,))) = q(a 0-11 X Xo) g(0 X,, Xo)
q ( X an)
(5)
fixated ohs
next noisy action
We compute the fixated observation g(O,-1l O,, Xo) using either a zooming or masking process centered around a fixation point that we obtain from the given action ag by exploiting the observation-action alignment in the setup, The latent actions q(a--1 O--1. ag) are sampled using a fixed noising process, which we discuss in detail in [Section 4.3.1] The role of these inference distributions is to help train the generative distributions obtained in (4) by providing a way generate fixated observations and noisy actions that are privy of the ground truth action, which our training procedure then attempls to match with the parameterized generalive distributions in (4) (see Figures 3 and 12).
Note that we directly define the denoising process in whereas DDPM first defines the noising process as a Gaussian and uses Bayes' rule to obtain the denoising process in closed form. We do this because our noising process get fixated observations would be implemented as an unmask or zoom-out process whose inverse cannot be written in closed form, leading us resort directly defining the denoising distributions in q.
Training objective. The likelihood po(ao]Org can be maximized by minimizing the KI. divergence between the fixed inference distributions XX, Xo) and learned po (X,-1XX)) Vt. Please see proof in Appendix A.2
4.3 Implementing C3DM
4.3.1 Action Noising (and the choice of noising process)
During training, for each target action (1) where is the sample index in the training dataset, we sample
K "noisy" actions (a (a() fk=1 by adding noise vectors using a fixed noising process, conditioned on timesteps [ 44 ~ Unif(O, (we use T == our setup). Note that this implements the noisy action inference q(a--1 X,, ao) in
Fixation, we will see later, not only changes the observation, but also the coordinate frame of the action at each refinement iteration. This introduces an additional challenge to training the denoising model which has to infer the noise level for different denoising steps in addition to generalizing to different observation fixation levels. To address this challenge, we introduce a simple yet powerful modification to the standard noising process we remove the drifting of the target action and rather diffuse the action only when obtaining noisy (latent) actions. This lets us control the noisy action to always stay within bounds that are observable for the model in the fixated observation, making action denoising significantly easier to learn for different fixation levels. We formalize both these processes below.
Noising process with drift. The standard noising process in DDPM can be written as, (1) Va(l=) a+) + /1 (a(()))))--- c
where 6/ ~ NO 7) with N being a normal distribution and the identity matrix. a(4x) is a tunable noise schedule that satisfies the conditions specified in [Section 3
Noising process without drift (ours). Our noising process diffuses but does not drift the noisy actions, and be written as,
a ( = ()) + V1 - -att)))(c)) (1)
(7)
As we show in Section 5.1.2 we find the noising process in 7 (no drift) to perform better empirically than the one with drift, and hence is the default setting for all our main results. We also point out that using (7 would result in a being sampled from a uniform random distribution when t = T rather than a standard normal distribution, is the case when using 0 See detailed proof in Appendix A.3.2
4.3.2 Fixation and Context Constraining
We imbue our diffusion model with the ability to implicitly identify action-relevant regions and ignore distractions in the input it iteratively refines actions. We do this by determining "fixation point." in
6

---

Published in Transactions on Machine Learning Research (07 (224))
the observation at each step of the action refinement procedure. We exploit the alignment between the ion and action space and set the fixation point to the target actions during training. This echos with many prior works (e.g.. Zeng et al. (2021): 1 Shridhar et al. (002bb)) } that exploit this observation-action alignment for improving sample efficiency.
Fixation. The fixation point for each observation image is a function of the target action, given by pO : imggreal pos(a!"),, where img7rroal is a matrix that transforms points in the camera frame to image frame, and po)( extracts the 2-D position of actuation on the x-y plane from the input 6-DoF action. The constrained context can then be formulated as 0 == C(O(9; p (f,,t))) t&), where C either masks parts of the observation far away from p( (C3DM-Mask) or zooms into the context fixated at p() (C3DM). Note that context constraining implements the fixated observation inference q(O,-1l X,, Xo) formulated in (5) We implement pe (0 (X)) (in (4) during testing by using the intermediate denoised action for the fixation point.
Constraining context by masking (CaDM-Mask). For masking an image of size H x W, we first create a window of size -i'- x Wafe centered at the fixation point p()) add random jitter to the window location while still keeping (() inside the window to minimize the train-test gap where the next fixation point might not necessarily be at the center of the constrained context. Then, we mask all pixels outside this window with the background value. We illustrate this in Figure 4 top).
T
Constraining context by zooming (C3DM). Similar to fixation by masking, we first create a window around the fixation point p with random jitter to the window location. Then, we resolution image of the same scene to obtain zoomed-in image in the constrained window. Our new context is still of the same input size (H x W) but now has higher levels of detail. We note that while we do assume access to high-res images, our model does not utilize the entire image but rather only task-relevant parts for action inference. Not using the entire high-res image keeps the model activation size small and inference time as low possible for this method (given the additional computational overload introduced due to observation encoding at each denoising timestep). While we utilize cached high-res images during training, one can use this model to physically move the camera closer to the fixation point to obtain and process higher levols of detail in the input during inference. We illustrate this process in Figure 4 4(bottom).
cached high-
Figure 4: Illustrating masking (top) and zooming (bottom) for constraining context around pre- dicted fixation point (A).
After zooming in to obtain the constrained context, we ro-normalize the noisy action a to the new camera frame for imput to the denoiser network during training. This ensures that the noisy action and constrained observation input to the denoiser are consistent. The denoiser output is still expected in the unconstrained context to keep the scale of the model output constant.
Baked-in data augmentation. C3DM trains an action denoiser given a variety of constrained contexts O l over and above the training set of images. Additionally, with our zooming-in approach, we also transform the ground-truth action into new camera frames. This essentially increases the coverage of both the observation and action spaces seen during training and deems our method very sample efficient.
4.3.3 Action Denoising
We learn robot policies in a table-top setup that has single downward-facing camera to obtain top- down observations of the table. For each observation (()) in the dataset we obtain latent encodings ( (Oc)( where enc, is parameterized using a ResNet-18 (He et al., 2015). We build model ep parameterized by fully-connected MLP that is supervised to predict the standard normal noise vector (e-prediction) given the embeddings of the constrained input observation of and noisy action a That is, train our model using the MSE loss
7

---

Published in Transactions on Machine Learning Research (07 (07/2024)
N K L(D) = N
lIca(ok 2/ 11
(8)
K
t
We summarize the training procedure in Algorithm
4.3.4 fixation-while-Denoiiiin Denoosing Diffusion Process for Action Generation (fDDP)
The context constrainer C, observation encoder encs, and the action denoiser Eg in conjunction with the reversible noising process in together facilitate the fixaion-while-Denooie
Diffusion Process (/DDP) that we use for generating action given any observation input. We summarize our iterative observation and action respectively (in 4 in Algorithm Precisely, we implement context-cons constraiinn in pe (o.,-((X.. around intermediate denoised actions for action-relovant fixation, obtaín sequence of latent actions and constrained contexts (On, ao); and return the last denoised action ag as predicted action. We provide illustration in Figure 2 and example showing fixation points and fixated observations in fDDP
refinement process that implements the generative distributions po I and
(Or, (0r-1,
in Fig Figure 6
Algorithm 1 C3DM Training /DDP components
Algorithm C3DM Iterative refinement with fDDP for action prediction during testing
Require: D {O(i),a ( Rip K, max iters, img Team
enc,, co T for all n iter E {l....,, max max__ters) do
Require: O7, act bounds, team d, cuce, 0, co. T Unif (act_ bounds)
---0 0
for all (T.....1) do
all E (l,..., K] do for all i (I...., N) do
(O,)
encoding constrained obs
af
v a1() ar)
action denoising
14 ~ Tni
b sampled timestep
(0.1)
sampled noise
4 1)
b sampled noise
Pe - img pos(a.)
b fixation point
a
a(t) + (44)
noisy action
a - ap
noising using 7
p imggical pos(a (1) fixation point
0 C(O,, pt)
b. fixated obs for next iter
o ppo0):p (0,, (e) constrained context
end for
o (O encoding constrained obs
return ag
L L (o a()) 4 11)
end
end
-
NR L
b update encoder params
- 0 NK VoL
update denoiser params
end for
Cony. MI.P
NGDF
Diffussio Policy
CSDM-MMssk (oun))
C3DM (ours)
100
Success Rate (9)
75
50
25
sweeping-piles place-red-in-green
kitting-part
hang-cup
two-part-assembly
Figure 5: (Top) Success rates for manipulation tasks in simulation (average across 100 rollouts, peak performance in 500 epochs of training, (Bottom) Illustration of the simulated evaluation tasks.
8

---

Published in Transactions on Machine Learning Research (07 (07/2024)
5 Experiments
We evaluated C3DM extensively on 5 tasks in simulation and demonstrated our model on various tasks with real robot arms. For the simulated tasks, we used Implicit Behavioral Cloning (TBC, Florence et al. (2021),, Neural Grasp Distance Fields (NGDF, Weng et al. (2023)); ), Diffusion Policy (Chi et al. 2023), and an explicit behavior cloning model (Conv. MLP) with the same backbone our method for the baselines. Additionally, ablated against the masking version of C3DM, called C3DM-Mask. Fach task assumed access to 1000 demonstrations from oracle demonstrator. For the real robot evaluation, used Diffusion Policy as the baseline. We used 20 human demonstrations in one set of real robol experiments and just 5 simulated demonstrations for the to-ato-real experiments. We used RGB inputs for all simulation and real robot experiments except for when performing sim-to-real transfer where we used depth maps to reduce the domain gap between sim and real. Hence, we also highlight here that our method can be used to solve tasks irrespective of the input modality.
5.1
Simulation Experiments
5.1.1 Tasks
We evaluated our method on 5 different tasks in simulation (see Table and Appendix B for a detailed description of each lask). Since IBC experimented predorinantly on lasks with "push" " primitives, we included the sweeping-pin kitting-part, and two-part-assembly are based on boick/grasp" and "place" primitives. Place-red-in-green and sweeping-piles were made available by Zeng et al. (020)) part of the Ravens simulation benchmark, and we built the remaining tasks on the robotics research platform provided by Koga et al. (2022). An oracle demonstrator provided 1000 demonstrations for all tasks, where each demonstration is unique reset of the task containing a single top-down observation the scene and the pick-place action (no intermediate waypoint observations). The success rates for the various methods are shown in Figure
task for fair comparison. The remaining tasks, place-red- in in-green, hang-cup,
5.1.2 Main results
C3DM is sample efficient and learns policies from minimal data. We tested the sam- ple efficiency of C3DM against the Diffusion Policy baseline and found that our method was able
Picking
Placing
0.7
Diffusion Process (FDDP). This figure shows score fields when inferring the pick (top) and place (bottom) actions for the place-red-in prred-in-greeed tnegreen task using C3DM. The score field (output by <æ, depicted here a 2D projection on the input image) at each timestep induces fixation point denoted by x (that can be obtained by probing the field at random point, translating towards the predicted direction, and transforming into the image space). The input image al each timestep (other than t = is obtained by zooming around the fixation point (x) predicted at the previous timestep. For both picking and placing, we obscrve that the model is distracted at first with the entire table-top in view. As it zooms (with decreasing timestep), our model fizates closer and closer to the objects of interest (red block for picking and green bowl for placing) showcasing less distracted prediction owing to the removal of distractors and higher spatial resolution utilized during the denoising process.
Figure 6: Fixation-while-Denoiae
9

---

Published in Transactions on Machine Learning Research (07/202) /024)
C3DM
C3 CCDM-Mask
Final
Itentive Fixarion action refinement
Figure 7: C3DM vs C3DM-Mask. Context refinement by fixation in conjunction with action denoising using fDDP on the kitting-part task (picking) (C3DM (top) and C3DM-Mask (bottom)). The red region depicts the standard deviation of the predicted latent action, A represents the latent action (orientated with grasp yaw), and x is the fixation point. The observation at each refinement iteration is fixated around the fixation point (x) predicted the previous timestep, using zooming mechanism for C3DM and masking for C3DM-Mask. As refinement progresses, observe how the model fixates the target part (yellow) it refinos its action for grasping.
to achieve good performance using just 30 demos in simulation, reaching 90% success on the place-red-in-green sorting task while the baseline achieved very little success, shown in Figure 8 We attribute this to the high observation and action space coverage of our method that clearly helps with sample efficiency.
102 Tisic museess with incrensinn datrect si2
CBUM
C3DM can ignore distractions in table-top manipula- tion. Both C3DM-Mask and C3DM showed good performance on slace-red--in-geeee innegeenn due to their ability to ignore distractor objects leading to precise locations for picking. On the contrary, bascline methods struggled to fixate on the target object that needed to be picked. In killing-parl,, while baseline methods were able to predict actions approximately around the target action, they were not precise enough to succeed given the low tolerance of this task. C3DM ignored unnecessary objects on the table, as shown in Figure 7 leading to precise pick and place predictions. We observed failures when the distractor objects were too close to the object and fixation in C3DM was unable to minimize their effect.
Eiffesiin Polky
02
30
601
100
Number of traisinu derzco
Figure 8: Sample efficiency of C3DM (peak success rates in 500 epochs of training).
C3DM is invariant to unseen distractions. To test the generalization ability of C3DM against unseen distract we added unseen objects usually found on table-tops such as a pen, remote, mouse, ctc. (shown in Figure 13,, all made available in the SAPIEN dataset (Xiang al.] 2020 Mo et al., 2019 Chang et al. 2015). We report success rates comparing C3DM
Table 1: Model performance, with and with- out drift in the diffusion process, after train- ing for 200 epochs on heduee-ed--n-greee task.
Method
C3DM
C3DM-Mask
and Diffusion Policy in Table 2 We found <10% drop in success rate of C3DM when replacing seen distractors with unseen ones suggesting very good generalization to ignoring unseen distractors compared to Diffusion Policy.
Drift 6
60%
40%
No drift
79%
47%
Action refinement with fixated gaze can help predict G-DoF gripper poses with high precision. C3DM is the only method that could succeed iall the hang-cup and tuo-part-assembly tasks due to its ability to prociscly predict the full 6-DoF action. C3DM beat all other basclines and the C3DM-Mask ablation showing the importance of action refinement with a fixated gaze. We illustrate how our learned score field creates this fixation in Figure 6, The cases where our model did fail were in which the model fixated on spurious locations carly in the refinement process leading to the correct pick location being outside the region of view, as well as when small errors the action prediction led to irrecoverable scenarios.
10

---

Published in Transactions on Machine Learning Research (07/202) 2024)
Figure 10: Real-world experiment setup and tasks, from left to right: place- blcce-bocckinn-bow (big) place-block- in-bowl (small), screw-insert, kitting- aitting-partt part and hang-cup. First three were setup on a Franka Emika Panda robot and trained on human demos, the last two on a UR10 robot and deployed sim-to-real.
Iterative refinement with more levels of input detail can solve visual challenges in tasks. The sweeping-piles task poses the challenge of inferring the location of small particles and the target zone from an image, for which C3DM outper- forms all considered baselines. We note that IBC's performance on this task is lower than that reported their work for planar sweeping, because our version of this task more visually com- plex with the target zone marked by thin square opposed to being marked by color-filled region. The billing-part task also presents complex-looking part to infer the pose of in order to predict a stable grasp. C3DM can iteratively refine its pose by obtaining higher levels of detail in the input leading substantially low grasp error of 0.72 em and 11.28 and eventually higher success rate compared to all baseline methods, including the C3DM-Mask ablation, which can only ignore dist.actions in the input..
Table 2: tractors (showing average over 3 random seeds and standard deviation, across 20 rollouts, afler 500 train epochs) on place-red- in-green task. Models saw blocks as distrac- tors during training; replacod them with daily-life objects (see Figures 13 and to test generalization.
Task success rates with unseen dis-
Method
50% unseen 100% unseen
Diffusion Policy
41.7% + 5%
33.3%
10% 4%
C3DM (ours)
85% 4%
90%
Additional observations. Table 1 shows comparison between two diffusion processes for training our model, one that drifts the lalent action to the origin (Equation (6)), and the other a pure diffusion (Equation (7)). We observed the latter to perform better in practice and is the default choice for all our main results. We also show the success rates when varying the number of refinement steps during action refinement in Figure 9 and expected we observe rising trend in success number of refinement steps increase.
Tack success zate with increasing refinencct steps
Cabai
F3DM Mark
Rate (56)
50-
25-
Nunabet of refinescent timest ope
5.2 Real-robot Experiments
Figure 9: Success rate on the place-red-ine green lask with increasing number of refine- ment timesteps.
We demonstrated our model's ability ignore distractions and precise action prediction using two real robots. On Franka Emika robot, we deployed policies trained on human- driven demonstration, and used an Intel RealSense D435 camera mounted to view the workspace top-down. On a UR10 robot,
we deployed sim-to-real policies, with a MechEye Pro M depth camera that produced depth maps comparable those in the simulator. Testing both setups proved that our method can work reliably when trained real images well as be robust to the sim-to-real gap, making it extremely practical for real-world deployment.
5.2.1 Model Performance with Real-world Demonstrations
For the setup that trains on real-world demonstrations, we collected 20 human demonstrations using a space-mouse teleoperator for the blace-blockkinn-one and screw-insert tasks which required picking target objects and placing them into goal location. In the sim-to-real setup, we collected up to 100 demonstrations for the kilting-pant part and hang-cup tasks (as described in Appendix B) in simulation using scripted policy.
11

---

Published in Transactions on Machine Learning Research (07/2024)
Table 3: Success rates for tasks on a real Franka Emika robot.
Method
place-block-in-bowl
screw-insert
big blocks
small blocks
pick
pick+
Diffusion Policy
45%
0%
5%
0%
C3DM-Mask (ours)
60%
40%
0%
0%
C3DM (ours)
55%
65%
80%
60%
Each demo consisted of an initial image observation and the locations for pick and place actions in the robot's coordinate frame. We trained the baseline Diffusion Policy model, C3DM-Mask, and C3DM for 200 epochs. During testing, we evaluated all models using linear schedule of 10 timesteps. All success rates averaged over 20 trials are summarized in Table 3 and Table 4
C3DM can ignore distractions in real images. We per- formed experiments on two variants of the place-block-in broock-in-boult task, one with big blocks with edge size 4.5 em and other with small blocks of edge size 2.5 cm. Example image observations are shown in Figure 10 We observed that C3DM was able to ef- fectively identify and ignore distractions in the input leading to precisc actions for both variants. While C3DM-Mask was able to ignore distractions well, it lacked precision when evaluated with smaller blocks.
kitting part
hang-com cup
Simulated
Real
Iterative context constraining leads to precisc actions and high sample efficiency. We performed experiments on screw-insert task where the robot picks up hex-head screw of edge size cm, base diameter 1 cm, and places it in receptacle with a hole of diameter 1.5 cm. Successfully completing this task required high precision as a pick prediction that is slightly off from the center the screw would result in the screw falling on the lable instead of being clamped by the parallel-jaw gripper of the robot. We observed that C3DM was significantly more successful in completing this task. While other methods failed due to both imprecise action prediction as well as the lack of generalization capability given a small number of demonstrations, C3DM was able to generalize to all test. locations as well as be precise within the 1 cm tolerance needed for this task.
Figure 11: Sim-to-real experiment. Both real setups mimic the tasks we designed in sim. For hang-cup we evaluated C3DM with unseen distractors to further prove the bustness of our method even with the sim- to-real domain gap.
5.2.2 Model Performance in Sim-to-Rea
We conducted sim-to-real experiments using depth maps converted to height map observations of the kitting- part and hang-cup tasks. We used depth maps since the MechEye Pro M depth camera can produce depth map observations comparable to the simulator.
C3DM can predict precise actions even with the sim- to-real domain gap. Table shows the success rate results over 20 trials of C3DM and the baseline Diffusion Policy for the tasks. Both tasks required high precision for success with roughly 1-2 mm of positional and 1-2 degree rotational tolerance. The precision constraint for the cup hanging task is for the cup pick sub-task, where the clearance between the open fingers and the cup for grasping was roughly mm. We found demonstrations was sufficient to train C3DM in simulation and deploy on the real robot sctup with 100% success with random arrangements of the items. With 100 demonstrations, we could not get the deployed model to work for the kitting-pan partt task with Diffusion Policy. The model worked well in simulation, so attribute the slight noise difference in the real depth maps of the
Table Success rates for sim-to-real tasks on UR10 robot.
Mothod
#demos
kitting-part
hang-cup
Diffusion Policy
5
0%
01%
50
0%
85%
0%
60%
C3DM (ours)
5
100%
100%
12

---

Published in Transactions on Machine Learning Research (07/2024)
distractor objects for the failure. It took 50 demonstrations to get the hang-cup task to work reasonably well with Diffusion Policy. Unexpectedly, the simulated success results for these tasks using RGB input (Figure 5) are lower than these results. We speculate that height maps are a better observation signal for certain tasks and hope to investigate this further in future work.
C3DM can handle unseen distractors in real. We evaluated C3DM and Diffusion Policy on the hang-cup task where the training data did not contain any distractors while the evaluation rollouts did. We found that C3DM showed reasonable performance with unseen distractor objects in the real deployment as shown in however Diffusion Policy showed no task success.
F Figure
11
6 Limitations and Conclusion
We presented a Constrained-Conter Conditional Diffusion Model (C3DM) for visuomotor imitation learning that learns to fixate on action-relevant parts of the input context while denoising actions. We demonstrated that the fixation-based context constraining allows our diffusion model to remove distractors from the input, achieving high success rate in a wide range of tasks requiring varying levels of precision and robustness against distraction. We also showed that our method can be deployed on real robot for pick-and-place tasks either directly (sim-to-real) or by training a handful of human demonstrations.
A limitation of our method is the task specific labelling of the fixation point in the demonstrations and selection of the final cropping dimension. For our set of examples, we set the fixation point at the finger location for the task (action). However, in other tasks, the region of interest may be elsewhere, for example, inserting a portion of a shape that is far from the grasp. The final cropping dimension was chosen to give an "ideal" view of our tasks (e.g., for picking tasks, the final cropping had the target object roughly fitting the view). We also want to point out that method trades off precise action prediction and robustness against distractors for computational time. Since Dilfusion Policy conditions can cache visual encodings for reuse across all denoising timesteps, however our method needs to encode observations at each refinement timestep. Another limitation, was the overhead camera, whose view of the target parts was easily occluded by the robot arms. This prevented seamless closed-loop error recovery. We plan to solve this by incorporating multiple camera views, such as an eye-in-hand camera configuration. We believe that future work with goal-conditioning policies and training on action trajectories (rather than sub-task goals) will render our model applicable for longer horizon manipulation.
single observation input, it
References
Pooya Abolghasemi, Amir Mazaheri, Mubarak Shah, and Ladislau Bölöni. Pay attention! - robustifying a deep visuomotor policy through task-focused attention. CoRR, abs/1809.10093, 2018. URL http:
//arxiv. aaciiv.org/abbe wwgg/bs/////////////ee////////ee 18009100993
Anurag Ajay, Yilun Du, Abhi Gupla, Joshua Tenenbaum, Tommi Jaakkola, and Pulkit Agrawal. Is conditional
generative modeling all you need for decision-making? arXiv preprint Xrxiv:221 151657, 2022.
Aude Billard, Sylvain Calinon, Rüdiger Dillmann, and Stefan Schaal. Robot programming by demonstration. In Springer Handbook of Robolies, 2008
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Ri-1: Robotics transformer for real-world control at scale. arXiv preprint ar Xin: 2. 2.06817, 2022.
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song. Hao Su, et al. Shapenet: An information-rich 3d model repository. Xiviv preprint
2015.
Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion, 2023.
Pim De Haan, Dinesh Jayaraman, and Sergey Levine. Causal confusion in imitation learning. Advances in Neural Information Processing Systems, 32, 2019.
13

---

Published in Transactions on Machine Learning Research (07/2024)
Peter Englert and Marc Toussaint. Learning manipulation skills from a single demonstration. The International Journal of Robolics Research, 37(1):137-15) 2018.
Chelsea Finn, Tianhe Yu, Tianhao Zhang, Pieter Abbeel, and Sergey Levine. One-shot visual imitation learning via meta-learning. In Conference in Robot Learning, volume abs abs/1709.04905, 2017.
Pete Florence, Corey Lyuch, Andy Zeng, Oscar Ramirez, Ayzaan Wahid, Laura Downs, Adrian Wong, Johnny Lee, Igor Mordatch, and Jonathan Tompson. Implicit behavioral cloning, 2021. URL https; 00137
7/arxiv.org
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CoRR, abs/1512.03385, 2015. URL htttp::/aaaxiv.oggabbbb///////// 03385
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. CoRR, abs 2006.11239, 2020. URL https://arxiv. wwwp://rggaaa//eeeeee 1110006.11200 11239
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video diffusion models, 2022.
Daniel Jarrett, Ioana Bica, and Mihacla van der Schaar. Strictly batch imitation learning by energy-based distribution matching. Advances in Neurul Information Processing Systems, 33:7354 7365, 2020.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. CoRR, abs hbss/141..690 2014. URL http:://axiii http://wxivv.aa/aaa////// abs//412..1112.. (980
Yotto Koga, Heather Kerrick, and Sachin Chitta, On cad informed adaptive robotic assembly. In 2022 SEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 10207-10214. IEEE, 2022.
Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and Fujie Huang, A tutorial on energy-based learning. Predicting structured data, 1(0), 2006.
Ajay Mandlekar, Danfei Xu, Roberto Martín-Martín, Silvio Savareso, and Li Fei-Fei. Learning to generalize across long-horizon tasks from human demonstrations. arXiv preprint arXiv:2003.0610 06085, 2020.
Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang, Rohun Kulkarni, Li Fei-Fei, Silvio Savarese, Yuke Zhu, and Roberto Martin-Martín. What matters in learning from offline human demonstrations for robot manipulation. arXiv preprint or Xiv::21008. 08.9929, 2021.
Utkarsh Aashu Mishra, Shangjie Xue, Yongxin Chen, and Danfei Xu. Generative skill chaining: Long-horizon skill planning with diffusion models. In 7th Anrrual Conference on Robol Learning, 2023.
Volodymyr Mnih, Nicolas Heess, Alex Graves, and Koray Kavukeuoglu. Recurrent models of visual attention. CoRR, abs/1 00s//4066.2210 6247, 2014. URL sttt://arxis: wwwpe/vvv.rg//b///eeeeeee 14466
Kaichun Mo, Shilin Zhu, Angel X. Chang, Li Yi, Subarna. Tripathi, Leonidas J. Guibas, and Hao Su. PartNet: A large-scale benchmark for fine-grat ane-grained and hierarchical part-level 3D object understanding. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Tim Pearce, Tabish Rashid, Anssi Kanervisto, Dave Bignell, Mingfei Sun, Raluca Georgescu, Sergio Valcarcel Macua, Shan Zheng Tan, Ida Momennejad, Katja Hofmann, et al. Imitating human behaviour with diffusion models. arXio preprinl urXiv:2301.106% 10677, 2023.
Moritz Reuss, Maximilian Li, Xiaogang Jia, and Rudolf Lioutikov. Coal-conditioned imitation learning using score-based diffusion policies. arXiv preprint arXiv:2304.0255 2023.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. CoRR, abs/2202.00512, 2022. URL http:/:///axvvvo///ab//
Stefan Schaal. Is imitation learning the route humanoid robots? Trends in Cognitive Sciences, 3:233-242, 1999.
14

---

Published in Transactions on Machine Learning Research (07/2024)
Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Cliport: What and where pathways for robotic manipulation. In Conference Robot Learning, pp. 394-906, PMIR, 2022a.
Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Perceiver-actor: A multi-task transformer for robotic manipulation, 2022b.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. CoRR, abs/2011.13446, 2020.
URL https://arxiv.org// http://...g//a///2..../. 13456
Chen Wang, Rui Wang, Danfei Xu, Ajay Mandlekar, Li Fei-Fei, and Silvio Savarese. Generalization through hand-eye coordination: An action space for learning spat spatially-invariant visuomolor control. CoRR, abs/2110.00035,, 2021. URL http:://axxiv.or///////
00375
Thomas Weng, David Held, Franziska Meier, and Mustafa Mukadam. Neural grasp distance fields for robot manipulation, 2023.
Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, Li Yi, Angel X. Chang, Lconidas J. Guibas, and Hao Su. SAPIEN: A simulated part-based interactive environment. In The IEEE Conference Compuler Vision and Pallern Recognition (CVPR), June 2020.
Andy Zeng, Pete Florence, Jonathan Tompson, Stefan Welker, Jonathan Chien, Maria Attarian, Travis Armstrong, Ivan Krasin, Dan Duong, Ayzaan Wahid, Vikas Sindhwani, and Johnny Lee. Transporter networks: Rearranging the visual world for robotic manipulation, 2020. URL https://arxiv org/abs/ 2010. 14406
Andy Zeng, Pete Florence, Jonathan Tompson, Stefan Welker, Jonathan Chien, Maria Attarian, Travis Armstrong, Ivan Krasin, Dan Duong, Vikas Sindhwani, et al. Transporter networks: Rearranging the visual world for robotic manipulation. In Conference Robot Learning, pp. 726 747. PMLR, 2021.
Zhang, Z. McCarthy, O. Jow, D. Lee, X. Chen, K. Goldberg, and P. Abbeel. Deep imitation learning for complex manipulation tasks from virtual reality telcoperation. In 2018 IEEE International Conference on Robolics and Aulomation (ICRA), pp. 5628 5635, 2018. doi: 10. 111110////.01......
Ziyuan Zhong, Davis Rempe, Danfei Xu, Yuxiao Chen, Sushant Veer, Tong Che, Baishakhi Ray, and Marco Pavone. Guided conditional diffusion for controllable traflic simulation. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 3560-3566. IEEE, 2023.
Yifeng Zhu, Abhishek Joshi, Peter Stone, and Yuke Zhu. Viola: Imitation learning for rision-based manipulation with object proposal priors. arXiv preprint orxi2:2200. 1 11339, 2022.
15

---

Published in Transactions on Machine Learning Research (07 (07/))
A Method Details
A.1 Pictographic representation of the denoising pathway in C3DM
Intermediate fixated obs.
Given obs.
Most fixated obs.
T=T
Fixating observation
0
Denaising action *
Target action
Intermediate noisy actions
Figure 12: Whustrating action denoising without drift (bottom) and corresponding fixation in the observation (top). Fixation in the observation is implemented as a zooming process that directs camera observations towards zoomed-in view around the target action.
A.2 Proof of method
Given global view of the scene Or, train generative model pa (formulated in (4) that aims to generate target action ag given the global observation. We do this by minimizing the cross-entropy loss which is given by,
We
LCR = =o (ag
((a log
pa po(an:r, Oa: da
( q(ai:r Oor Or)
Po(an OuT-1 day:rdoar-)
= -F log
q(a1.T 00:T ao, Or)
Po ( 1
= -Eg(ng(0 log
1m0.
(a O |a0 Or)
Po(up:r - (r)
< -Egloorr 100r) log
q(ai.r, Oaiii,,pp..... Op.r-
ag. Or) Po Po(Xp:T1,,
- log
(X Oulag, Or)
log X 1,,0 + log q(ar Or)
( - (Or)
-oggo(arlOOr) (Og an. Or)
Pol X1 x(a))
Po Xx)
log ((X,|X--1,00 Or)
x=(Xx-1|X() + g(ar XT 0r)
Ex ar (0r)
loggo logg(Orlan. Or)
Pa(X7-1\X7)
+ log 9(x, Xo. OT)) no x))
T-1
log)
logg(Onlann Or) +
Σ log ps 1
/X, 0r,Or)
g(X+|Xn, Or)
g(X, Xo, Or)
+ log ((x lar, Xo. Or) Po Xxr)
Xa. Or)
log 9(X1 No. Or)
(X Xo, Or)
Po (X X1)
16

---

Published in Transactions on Machine Learning Research (07 (27/))
T-1
2 (0r) po loga(Oolann Orl +
Σ log
(XX,,1Xx,,X)) Xo, Or)
po Po(Xi-1(X4,)
+ log 9(XT-1 Xx. Or)
log g(xr- lay Xo. Or)
log glar Xo. Or)
+ log g(X, (XX, Or) Po (X(
q(x1 Xo, Or)
Po(XT-1(Xr)
g(xT-1 Xo. Or)
=
poggo(arl00)) + logg(Oolann Or) + B 2 - 1 (r
Σ 1-2
log g(X1 1X,, Xr)
Po (
1: log g(Xx- 1 lar, Xo. Or)
log g(ay Xo, Or)
9(X1 [oo. Or)
log
Po (x XT)
((xi Xv. Or)
(X)
(x T or (0r)
log g(ar| Xo, Or)
log (00la0 Or)
Σ log
g(x Xx)
+ log
g(xx- |T Xa) Po (XT)
po(ar
Po(X
lo (o X1)
garrl Xo, Or) po (arr (Or
g(X:-1 (x,, Xo)
-
log
log la Or)
log
log po (Xo (11)
Po (Xt (X.)
Lg
Li:T
Here Lr constant and hence can be ignored. We minimize (T)) by factorizing po (X(-i 1x( (X,), v fI,...,y) TY,, and subsequently ps(a; (X,) is formulated
where, when using the noising process without drift (as in 7 Ot), 0 (
14-1 a,, O2) ar (ac, O2,t)
(9)
01-1 = / 11--a0 -
and when using the noising process with drift (as in G)),
(4-1 O,)
(a. vI aveo O,,t))
Var
(10)
0t-1 at-1.
Additionally, po (O.-1(X))
formulated as a non-differentiable masking or zooming process.
A.3
Noising processes
A.3.1
Noising process with drift
Let ag be random variable over which we wish to define a generative distribution. We define inference distributions over latents {a),... {aj....,ayy ar} as
(a,/a;,1) -N(a;: Vara,-1 a,l),
(11)
where {(1 - a-) is a fixed noise schedule.
We can unroll this recursion to obtain the distribution over a, for any & given the sample ag, as follows:
a, Vara,1 + vI a
Varre-1a1-2 v1 11a-118-2
(12)
Varan VI- Tre,
where (00 l).
17

---

Published in Transactions on Machine Learning Research (07 (07/2024)
Proof of recursion:
Va,ar-1 + v1 a14-1
Varaa-iaa-- + vai(1-a-a - a1-1)e-2 vI 0141-1
Here 6--1 and 6-- are (o Gaussian distributions. Their weighted sum will result into Gaussian distribution whose variances are added up. That is,
v a. - 01 v1 014
(aa(1-aa-11 +1 AGE1-
= =V1 -1,;0-01111-
where 822 ~ IO, We observe here that as + oc, a ~ 100.1).
A.3.2 Noising process without drift
We fix the inference distribution over the latent variables {a....., ar) as diffusion process without the scaling term. That is, we define
(a.aar (oao-)))=((a;;a))) vI acl),
(13)
where {(1 a=)) is fixed noise schedule.
We can unroll this recursion to obtain the inference distribution over any directly given the sample dg, follows:
a: a V1 0444-1
= a:-2 + / - (at-1 +a) EL-2
(14)
au +
Σ ar e,
where E 1). We observe here that as t + 00, a, ~ Uni 00).
T, since a(42) - have a 0 This (i) around t2 + T the model is trained to model an identity function (Salimans & Ho. 2022). This hampers training and eventual model performance. We point out that the noising process without drift does not suffer from this issue. Additionally, we also found it useful directly tune æ (as opposed to tuning the underlying 3-schedule) to prevent identity-training issues.
Note: If we analyze the noising process in 6
that if we train our denoising network eo to predict 6
Tasks and their Desiderata
Sweeping-pilee piles. This task requires the model to output the parameters of a push primitive, that is the starting location and orientation of pusher and ending location, push piles of small objects into a target zone.
Place-red-in-green. This is slightly more precision-requiring task where the robot is supposed to pick up red 4x4x4 em block using a suction cup and place it in green bowl. The table is also laid with distractor blocks that can hinder model precision leading to imprecise picks. We also test our model for robustness against unseen distractor objects in this task by replacing blocks with daily-life objects such as pen, remole, stapler, etc.
Kitting-part. In this task, parts of skateboard truck assembly are laid out on tabletop, and the robot is tasked to grasp the truck base its slotted end and place correctly in kit. To succeed in the task,
18

---

Published in Transactions on Machine Learning Research (07/2024)
Table Simulation (+) and real-robot (f) tasks and their desiderata.
Task
Ignore distractions
Precision
Position tolerance
sweeping-piles
x
n/a
place-red-in-green'
/
4.0 cm
kitting-parde
/
2.0 cm
hang-cup"
/
1.0 em
two-part-assembly
0.5 cm
fhcc-block-innbowll
big blocks
X
4.5 cm
small blocks
/
2.5 em
screw-insert!
X
/
1.0 cm
hypothesis that the robot needs to fixate on and observe the part in detail, while ignoring distractions that can hinder accuracy.
Hang-cup. This task requires the model predict the pose of the gripper for picking and placing that should result in the cup hanging by its handle the hook of a Y-shaped hanger. In the best possible scenario where the cup is oriented such that the handle's plane is perpendicular to the hanger's hook, the tolerance for error in position prediction is only 1 cm.
Two-part-assena are-paar--ssembly.. This a high-precision assembly task where the robot is tasked to pick up a skateboard truck hanger and insert the hole of the truck base. This task a subroutine assembling a full skateboard truck.
C Model Architecture and Training Details
We deep convolutional neural network with residual connections (ResNet-18 He al. (2015) without any pretraining, process images, the output of which then flatten and process using MLP with two hidden layers (enca). We concatenate the encoder embeddings with query actions, which are then further processed by 4 fully-connected feedforward layers with skip connections (eg). We use ReLU activations for all intermediate layers. We implemented all baselines with the same backbone architecture, tune learning rate in the range [10-4, 10-00 and train using the Adam optimizer (Kingiaa & Ba 2014) with batch size of 100 demonst.rations on a single Nvidia 2080 Ti GPU. We use rotation augmentation of the demonstrations.
D Hyperparameters
Table 6: Hyperparameters used for simulation experiments. In case of multiple entries, the one in bold
worked best.
Hyperparameter
Diffusion Policy
C3DM-Mask (ours)
C3DM (ours)
Num noisy action samples
1, 10
1, 10
1, 10
Noise schedule
linear
linear
linear, linear
Timestep encoding size
64
64
64
Downsample ratio
N/A
N/A
0.2
Learning rate
le-4
1e-4
le-4
19

---

Published in Transactions on Machine Learning Research (07/2024)
Table 7: Hyperparameters used for real robot experiments. In case of multiple entries, the one in bold worked best.
Hyperparameter
Diffusion Policy C3DM-Mask (ours)
C3DM (ours)
Num noisy action samples
1, 10, 50
1, 10, 50
1, 10, 50
Noise schedule
linear
linear
linear, cos?
Timestep encoding size
64
64
64
Downsample ratio
N/A
N/A
0.4
Learning rate
le-4
1e-4
le-4
Table 8: Hyperparameters used for sim-to-real robot experiments.
Hyperparameter
Diffusion Policy
C3DM (ours)
Num noisy action samples
1
Noise schedule
linear
linear
Timestep encoding size
256
256
Downsample ratio
N/A
0.28
Learning rate
lc-4
lc-4
E
Unseen objects used in simulation experiments
Figure 13: Table-top objects, unseen during training for any experiment, that we used to evaluate C3DM on ignoring novel distractor objects.
Figure 14: Table setup showing the place-red-inn-reent task with novel distractor objects.
F
Details on motion planner used for tasks in simulation
For all tasks considered in [Section 5.1 we implemented motion planner that caleulates "hover" pose the pick and place poses, and made the robot reach the hover pose both before and after attempting to
20

---

Published in Transactions on Machine Learning Research (07/2024)
reach the corresponding predicted pose during action rollout. This hover pose computed using relative translation along the +2 axis on the predicted pick or place pose. To generate fine-grained motion, we did not obtain the collision model of the table-top, rather used linear interpolation to obtain the trajectories between target poses.
21


Reviews:
[{"review_id": "JRFLLgXneD", "reviewer": "Reviewer_A1im", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n* The proposed idea of observation fixation during action diffusion is interesting and a very good contribution\n* In the Appendix, the authors provide more detailed, thorough derivations for their approach.\n* Thorough evaluation of the proposed method and comparison with multiple baselines \n\nWeaknesses:\n\n* While most parts of the paper are very well-written and easy to follow, I personally feel that the paper would benefit if the authors would mention more clearly in Section 4.3, and in particular in Section 4.3.2 that the context is a function of the action. While this ultimately gets clear in the algorithm boxes and is also somewhat mentioned in 4.3.2 already, I think an additional and more explicit statement would help. Moreover, the paper would be easier to understand if the authors would also make clear that their approach, therefore, only necessitates diffusion on the action level.\n* In my opinion, the evaluations and, in particular, the comparison with diffusion policy is somewhat suboptimal. The output of the proposed approach in the paper are two 6D poses, which represent the picking and the placing pose. Contrarily, diffusion policy typically outputs more fine-grained, longer action trajectories. While the presented approach pre-computes the 2 waypoints before starting execution, diffusion policy is typically run online, and the action trajectories are constantly updated. This difference should, in my opinion, be mentioned more clearly. This aspect is crucial as the results suggest that the \"zooming in\" aspect of the work does have a significant effect (as shown in Fig. 5). From my point of view, part of the performance gains of C3DM vs Diffusion policy are clearly attributed to the increased spatial resolution which is available to C3DM (during the zooming process). This impression is underlined by the fact that the C3DM-masking baseline performs substantially worse than C3DM and only slightly better than diffusion policy. In this regard, I want to point out that if a wrist camera is added to the setup and if the action inference is also performed online (i.e., during trajectory execution), due to the movement of the arm and the wrist camera getting closer to the object of interest, C3DM-masking as well as Diffusion Policy could also benefit from a higher spatial resolution when being closer to the object that is to be manipulated or the placing location, which potentially could significantly close the performance difference between these methods and C3DM (zooming). While I still believe that the proposed method and formulation should result in better generalization, the previously mentioned points should be discussed in the paper. Ideally, the authors should provide additional ablation experiments, including wrist-mounted cameras and doing trajectory generation online, i.e., during policy execution.\n* Still related to the previous point, regarding the training of the proposed method and the baselines, I was wondering whether only the initial views have been used as data points, i.e., do 5 demonstrations only result in 5 training data points, or did the authors also train using intermediate waypoints during trajectory execution? This should be clarified.\n* The authors mention in the Problem Setup at the end of Section 3 that the policies can be deployed on a robot using \"any off-the-shelf motion planner\". While for most of the top-down pick and place tasks, I agree with the statement, for the more delicate tasks of \"hang-cup\" and \"two-part-assembly,\" the choice of motion planner and, in particular, how the placing pose is approached might have a significant impact on the performance. Therefore, it is essential that the authors provide further information about which motion planner has been used and how it has been set up.\n* In line with the previous comment, I want to remark that the work would greatly benefit from the authors' open-sourcing of their code.\n* In Section 4.3.2, under the point \"C3DM\", the authors mention that \"not using the entire high-res image keeps the model activation size short and inference time low, which is crucial for robotics.\" I personally found this statement problematic as the proposed C3DM approach does include the additional overhead of zooming in and having to re-render. Due to the fact that the vision input, therefore, changes on every iteration during denoising, also the vision-encoder has to be evaluated in every iteration. Since, in my personal experience, evaluating the vision-encoder often consumes most of the time, I would be curious about the inference times of C3DM compared to the other approaches for which the vision input is not changing and for which, therefore, the encoding of the visual input only has to be performed once. Moreover, the authors do not provide any ablations with respect to the resolution of the images, and as far as I understood, the activation sizes are the same for all of the approaches.\n* The experiment section 5 is missing some details and lacks clarity. In particular, in the introductory paragraph of Section 5, the authors mention the number of demonstrations used for the real robot experiments and for the sim-to-real-experiments. However, they do not mention the number of demonstrations for the simulation experiments. I, therefore, recommend moving this information about the number of demonstrations in the respective sub-sections to have everything clear and in one place.\n* I personally was a bit confused that, suddenly, for some of the experiments, the authors leveraged a depth camera. I personally think that the paper would benefit if the authors mentioned already during the method that the context can either be an RGB image or a depth map. Moreover, I would like to understand how the authors decided for which task to use the RGB images and for which ones to exploit the depth camera. Potentially, to improve clarity, the authors could add in brackets whether the respective experiment was conducted using RGB or depth input.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "* The authors present a new formulation for performing action diffusion while at the same time dealing with fixated, changing input observations (achieved through zooming in or masking)\n* The authors evaluate their method on a wide range of simulated and real-world robotic pick-and-place tasks", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* In Algorithm 2 the notation is different from Algorithm 1. In particular, the conditioning on $^{(i)}$ is missing\n* Mention more clearly in Section 4.3.2 that the context is a function of the action\n* A more thorough discussion regarding the comparison with diffusion policy in terms of offline trajectory generation before the movement vs. online trajectory generation during the movement. This is especially important since a wrist camera, in the latter case, can naturally provide more zoomed-in views of the scene and, therefore, more spatial resolution, which might be crucial to performing some of the tasks.\n* Ideally, the authors provide additional results when running Diffusion Policy and their method in an online fashion.\n* More details w.r.t. the employed Motion Planner should be provided\n* The claims on computational efficiency should be backed up with additional results or toned down.\n* The authors should mention more consistently during the paper that both, RGB and depth are considered as input modality. Moreover, the paper would benefit if the authors could comment on why they used the specific modality for the respective task.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "o66838U5WL", "reviewer": "Reviewer_ACEe", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "+ This paper is well written and clearly presented. I really like the simple and direct writing style.\n+ The real-world manipulation experiment is a plus.\n\nPlease see below for weaknesses.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents C3DM, a diffusion-based model that predicts 2 6 DoF gripper positions for robotic pick-and-place tasks. During each diffusion iterative refinement step, the presented approach either zooms in or crops the model inputs (RGBD images) based on the current denoised outputs.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- While the paper is well motivated, the presented approach is underwhelming. Image or video generation tasks leverage the power of diffusion models as their output space is high dimensional. Here, the output space is only 12 dimensions, which counters the very motivation of using a diffusion model which sacrifices training and inference throughput. Can the authors comment on this? While one might argue that the iterative process enables opportunity for the proposed “fixation” approach to avoid “spurious correlation”, this claim is unjustified in the paper. Please see below for more details.\n- Experiments\n   - Can we have confidence intervals in table 2, 3 and 4?\n   - In order to support the claim made in introduction, please include the following two baselines:\n       1. behavior cloning without diffusion, same architecture.\n       2. same as 1, but with the same amount of data augmentation (zooming and cropping) proposed in the approach. \n   - I would be surprised if the proposed approach significantly outperforms baseline 2 above. But if it does and the author provide clear explanation with experiments/ablations, I would be happy to recommend acceptance.", "broader_impact_concerns": "No particular ethical or societal concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "u0NjnSmGCV", "reviewer": "Reviewer_Hcxb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "STRENGTHS\n\nThe authors propose a novel extension of diffusion model.\n\nThis extension is well-justified under the assumption of observation-action alignment.\n\nThe experimental results both in simulation and on a real robot  system validate the approach.\n\nWEAKNESSES\n\nThe presentation and writing could be improved (see below).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents an extension of diffusion models for imitation learning. To enhance its robustness, the authors propose to augment the model such that denoised actions are generated by conditioning on a restricted image (masked or zoomed in), which is itself generated based on the previous noisy action. By focusing on a restricted image, the method can be less impacted by spurious features.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "To make the presentation clearer, I would suggest the authors to explain in more details the following points: \n\t- How is the integral in (2) computed?\n\t- How are the derivations in equation (14) obtained?\n\nThe paper contains a few minor issues:\n\t- Citation of authors should be either \"author et al. (year)\" or \"(author et al., year)\" depending if it's part of a sentence or not\n\t- Missing space before/after parentheses\n\t- Why does Figure 2 appear so early, while it is only referenced in the text on page 7?\n\t- Page 4: \\theta = argmin", "broader_impact_concerns": "No applicable", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Ib0STEDkD3", "reviewer": "Reviewer_nRSU", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- The idea of inference through observations augmented with fixation points is an interesting one.\n- The method seems to work well with its success rate often being substantially higher than Diffusion Policy.\n- Real robot experiments are presented. Sim2Real is considered, and it seems that even in this setting, the robustness carries over.\n- The motivation for solving this problem is clear.\n- I like the formatting of Figure 5 with the tasks shown beneath their bar charts.\n\nWeaknesses: \n\n- It took me quite a long time to understand how fixation points are determined. I had to go to the algorithm box and read that several times and then go back to section 4.3.2 and read that several times. None of this was really spelled out clearly in the introduction or earlier parts of the paper, which spent a lot of energy talking about how to infer actions from observations. While this type of inference is interesting, and these graphical models are fine, I spent the entire time while reading these sections wanting to understand what in the world a fixation point is and where it comes from. I think section 4.3.2 does not make it particularly clear that a fixation point depends on the action. Although I can see now clearly that it is a function of the action, I think there’s something about the  img-T-real notation that is very off-putting. In particular, the text on both sides of the superscript made it difficult for me to understand what was going on. I would prefer this is changed in future drafts.  \n\n- It seems like ResNet is used for image encoding. While I realize this is a standard choice and perhaps beyond the scope of this paper, a lot of recent evidence suggests there are much better choices for the vision backbone. In fact, I would say ResNet is rather old-fashioned. See Masked Visual Pre-Training for Robotics for some more discussion and some possible models, but at the very least one would hope to see something like DINO as a potential vision backbone. Perhaps I’m missing something that makes this not possible, in which case I apologize.\n\n- Figures 6 and 7 are extremely difficult for me to understand. Though I spent quite a long time thinking about what they’re showing and even asked a colleague if he understood what these figures were trying to demonstrate, I was ultimately unable to come to any solid conclusions. I can somewhat understand the density of arrows in figure 6, however that is about as far as I got. \n \n- It seems that several of the experiments used a depth camera, but I wasn’t sure when the depth camera was being deployed and how important it was. The authors themselves seem to admit that height maps may be an important signal for many tasks. While this is not a major complication, I left the paper a little unsure how important depth cameras are to facilitating this work. If other people were to build on this work in the future, I think it is important that they understand if depth cameras are required to get solid results.\n\n- As far as baselines, I think Diffusion Policy is fine. Why was Diffuser not included? I also am curious about more simplistic policies. For example, what if you collected five trajectories and next augment this data set by translating and zooming in on specific parts of the image? After that, train an MAE on the augmented data set. MAE, of course, works by masking certain parts of the image space, similar to your approach. Then you could use the MAE features to try and learn a diffusion policy directly. While I suspect this would still learn some spurious correlations, it would be good to verify that this sort of simplistic baseline does not work.  \n\n- Overall, I was left wondering how important is the alignment provided by fixation points between the image space and the observation space? Is that important, or is the diffusion pipeline for inferring actions from fixated observations more important? Put another way, can equation 4 still work out if we replace fixated obs with any other noisy observation. For example, directly adding Gaussian noise to the input image. I would like to see a more clear distinction between the work that was done in section 4.2 and the decisions that were made in section 4.3.2, as right now it is difficult for me to tell which part of this process is responsible for performance gains. While I do realize the paper has been written is such a way that these processes are inexorably linked, I do not think that this needs to be the case, and you should be able to ablate these components. Doing so would likely improve the paper significantly, because it could increase the number of baselines that are considered.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper considers the problem of distractor objects during imitation learning. In particular, it notes that many modern methods are fragile in real-world deployment because of ever-changing objects in the environment that are irrelevant to the task at hand. Because extant models are powerful, they often learn spurious correlations. This paper considers a new method for learning representations that are robust to such spurious correlations, which they name Constrained Context Conditional Diffusion models. The main contribution of the C3DM pipeline is a so-called “fixation step”. This step acts somewhat as a data augmentation strategy and somewhat as an alignment step between image-space and the robot’s action space. Images are cropped around the central actuator in different ways. The cropped images are subsequently fed into a ResNet and then a type of diffusion policy tries to infer the correct actions from these augmented features. The method seems to work fairly well on a variety of settings, including real robot experiments and Sim2Real transfer. In particular, it seems to outperform Diffusion Policy in these settings.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Please clarify the importance of depth cameras in obtaining the final results.\n- Clarifying figures 6 and 7 would also go a long way towards improving the paper.\n- Fix the notation in section 4.3.2 for fixation points.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper considers the problem that arises when policies trained via imitation learning become sensitive to spurious correlations that result from changes in objects that are not relevant to the tasks. To address this, the paper proposes Constrained Context Conditional Diffusion Models (C3DM) the primary component of which is a fixation step that serves as both an alignment and augmentation approach using either a zoomed in or masked version of the input. The paper demonstrates the effectiveness of C3DM through experiments on both simulated and real-world domains.

Considering both the initial submission as well as the discussion between the authors and reviewers, there is general agreement that the paper not only considers an important problem that arrises when employing imitation learning, but that the means by which the proposed diffusion model-based method mitigates the effects of spurious correlations via its "fixation step" is both novel and interesting. Notably, the reviewers particularly appreciate the fact that the paper demonstrates the effectiveness of C3DM not only in simulation, but on real robots as well. While there were initially some questions regarding the significance of the contributions of C3DM over contemporary methods, the subsequent discussion between the authors and reviewers largely resolved these concerns.
Decision: Accept (poster)