Policy Contrastive Decoding for Robotic Foundation Models
Authors: Shihan Wu
Xu Luo
Ji Zhang
Junlin Xie
Jingkuan Song
Heng Tao Shen
Lianli Gao
Venue: ICLR 2026 Poster (2026)
Keywords: Robotic Foundation Models
Contrastive Decoding
Primary area: applications to robotics, autonomy, planning

Abstract: Generalist robot policies, or robotic foundation models, hold immense potential to enable flexible, general-purpose and dexterous robotic systems. Despite their advancements, our empirical experiments reveal that existing robot policies are prone to learning spurious correlations from pre-training trajectories, adversely affecting their generalization capabilities during inference. To tackle this, we propose a novel Policy Contrastive Decoding (PCD) approach, which redirects the robot policy’s focus toward object-relevant visual clues by contrasting action probability distributions derived from original and object-masked visual inputs. As a training-free method, our PCD can be used as a plugin to improve different types of robot policies without needing to finetune or access model weights. We conduct extensive experiments on top of three open-source robot policies, including the autoregressive policy OpenVLA and the diffusion-based policies Octo and Pi-0. The obtained results in both simulation and real-world environments prove PCD’s flexibility and effectiveness, e.g., PCD enhances the state-of-the-art policy $\pi_0$ by 8.9% in the simulation environment and by 108% in the real-world environment. Our code is publicly available at: https://github.com/pcd-robot/PCD.

Published as a conference paper at ICLR 2026
POLICY CONTRASTIVE DECODING FOR ROBOTIC FOUNDATION MODELS
Shihan Wu' Xu Luo 1 Ji Zhanget Junlin Xie
Jingkuan Song Heng Tao Shen Lianli Gao
l University of Electronic Science and Technology of China
Southwest Jiaotong University Tongji University wbihan.w...korryeeotllooo. com, jizhang gmail com
https://kooryegitthun io/PCD
ABSTRACT
Robotic foundation models, or generalist robot policies, hold immense potential to enable flexible, general-purposs and dexterous robotic systems. Despite their advancements, our empirical experiments reveal that existing robot policies are prone to learning spurious correlations from pre-training trajectories, adversely affecting their generalization capabilities beyond the training data. To tackle this, propose novel Policy Contrastive Decoding (PCD) approach, which redirects the robot policy's s focus toward object-relevant visual clues by contrasting action probability distributions derived from original and object- masked visual inputs. As a training-free free method, our PCD can be used as a plugin to improve different types of robot policies without needing to finetune access model weights. We conduct extensive experiments on top of three open-source robot policies, including the autoregressive policy OpenVI.A and the lifffusionbbase policies Octo and mo- The obtained results in both simulation and real-world enviromments prove PCD's s flexibility and effectiveness, c.g., PCD enhances the state-of-the-art policy xo by 8.9% in the simulation environment and by 108% in the real-world environment.
INTRODUCTION
Recent efforts in developing flexible, general-purpose, and dexterous robotic systems have largely focused on robotic foundation models, or generalist robot policies. The goal of such policies is to empower users instruct robots perform arbitrary tasks, enabling autonomous task execution with minimal human supervision. Specifically, the policy takes as input a visual observation of the robot's state combined with a language instruction that defines the task, and outputs robot action, such as end-effector displacement. Ongoing advancements in scaling real-world robotic data corpora have facilitated the development of numerous robot policies (Brohan al. 2022, Kim et al.) 2024 M et al. 2024 Pertsch et al. 2025), which have demonstrated exceptional effectiveness in controlling various robots across diverse environments and acquiring wide range of manipulation skills.
Despite their notable achievements, existing robot policies are prone to learning spurious correlations (Ye et al., 2024] Geirhos al., 2020 Liu et al.) (023a) from pre-training trajectories, adversely affecting their ability to generalize outside their training data. As illustrated in Fig. 1 the robot policy predominantly relies on spurious features (e.g.. background or textures) rather than object features within observations to predict actions (see (a)(d)). Consequently, slight alterations to the visual observation background, like adjustments to the panning light region (i.c., (a)-(b)) and the handle position (i.e., (a)-(c)), lead to 36% and 32% drops in the policy's 's action prediction performance, respectively. Prior work in other fields has demonstrated that models relying on spurious correlations often suffer significant performance degradation test data when distribution shift occurs between
the training and test environments (Lu et al. 2025 Varma et al. 2024
Izmailov et al. 2022). This
"qqual contribution
Corresponning author

---

Published as a conference paper at ICLR 2026
underscores the importance of redirecting robot policies' focus from spurious features object- relevant ones to enhance their generalization across different scenarios during inference. Therefore, we provide first study on the following question:
Can we propose a training-free, plug-and aun-and- approach to mitigate the adverse effects of spurious correlations, thereby rectifying predicted actions for existing robot policies?
In this work, we present a Policy Contrastive Decoding (PCD) approach to answer the question. The core idea of our PCD approach is shift the robot policy's attention from spurious features to object- relevant ones by contrasting action probability distributions derived from original and object-masked observation images during inference. To ensure both effectiveness and flexibility of the approach, Tracking-to-Mausk (aaaaskkTTrrar (Truck2Mask) strategy is introduced, which first annotates target-specific objects in the initial visual observation using human expertise (e.g.. Point and Box prompts) or aff-the-shef object detection models (e.g., Grounding DINO (Liu et al. (024b)), then employs the SAM2 (Kirillov) al. 2023) model to track these objects across subsequent observations the trajectory. This enables precise object masking with minimal or no human intervention. Moreover, unlike autoregressive policies, diffusion-based policies cannot directly generate action probability distributions to feed into our PCD. To cope with this limitation, we introduce KDE-based Probabilistic Modeling KDE-PMM scheme, which calculates action probability distributions for diffusion- based policies using the kernel density estimation (KDE) /Wkeglarczyki (2018) technique. This enables our PCD approach to be compatible with both autoregressive and diffusion-bassee policies.
To the best of our knowledge, this is the first work to introduce a training-free approach for resolving the spurious correlation issue in robot policies. We conduct extensive evaluations in both simulation and real-world environments, across a total of 15 diverse tasks, and on top of three open-source robot policies-including -including the autoregressive policy OpenVIA (Kim et al., 2024) and the diffusion-
Success rate:
Succees rate: (38%
+PCD (ours): 0.40
1PCD (ours): 0.31
(a)
(b)
policies Octo (M) et al. (024) and To (Black et al.) 2024). The achieved results demonstrate the strong flexibil- ity and effectiveness of our PCD approach. On simulation benchmarks, PCD enhances the per- formance of Octo, OpenVLA and To by 29.7%, 50.6% and 8.9%, respectively. On real-world manipulation tasks, PCD outperforms the state- of-the-art robot policy To by 108%.
Success rate: 0.17 (32% +PCD (ours): 0.41
(c)
(d)
Figure 1: Robot policies tend to spuriously cor- relate task-irrelevant features with actions, com- promising their ability to generalize to unseen scenarios. As observed, changing the light posi- tion from (a) to (b) and the drawer handle position from (a) to (c) results in 36% and 32% drops in the performance of the baseline policy OpenVLA (Kim et al.) (0244), respectively. (d) Attention map. More results are in Section 4.4 and Appendix A
Contributions. Our contributions in this work are threefold. 1) We propose PCD, a simple, training-free, and easy-to-implement scheme to tackle the spurious correlation issue in robot policies. 2) PCD can be used as a plugin to en- hance both autoregressive and diffusion- based policies, without requiring fine-tuning or access to pre-trained model weights. 3) Extensive ex- periments demonstrate PCD's 's effectiveness and flexibility, consistently improving three state-of- the-art policies across 15 diverse tasks.
2 RELATED WORK
Robotic Foundation Models. The rapid development of robotic foundation models or generalist robot policies has been significantly inspired by the success of large visual and language models (Karamcheti et al., 2024a Zhai et al.) 2023a Touvron et al., 2023a). Robot policies can be catego- rized based action prediction strategies into autoregressive models and riffusion-baseed policies. Autoregressive policies sequentially generate action tokens based on previous outputs (Brohan et al..
2022 2023
/awalski al. 2024 Belkhale et al. 2024 Zhang et al. 2024). OpenVIA (Kim et al.
2

---

Published as a conference paper at ICLR 2026
2024) equips with a Llama 2 (Touvron et al.
2023a) language model alongside visual encoder
that merges pretrained features from SigLIP (Zhai et al.
2023a) and DINO-v2 (Oquab et al. (024))
Diffusion-basse bbaedd policies simultancously generate entire action dimensions by performing diffusion processes on noise (M. et al., 2024 Liu et al. 2024a). Octo (Team et al. 2024) learns a lightweight policy on the OXE dataset Collaboration et al. 2024), enabling rapid adaptation to unknown tasks massumer-grade GPUs. (Black et al., (024) exhibits flexible control over different robot types by leveraging pre-trained general knowledge from vision-language models. Although most robot policies are trained on extensive robotic behavior datasets, recent studies have shown that slight variations in deployment scenarios can significantly affect their generalization performance on downstream tasks in real-world environments (Firoozi et al. 2023 Gao et al. 2024).
and scenarios on standard
Contrastive Decoding. Recent studies large rision-language models (VI.Ms) have highlighted
the challenge of hallucination (Gunjal et al., 2024 Rawte et al. 2023 Liu et al. 2023b; (Zhang al. ).25). where models generate inaccurate or misleading outputs not grounded in the input data. Contrastive Decoding (CD) (Favero et al. 2024 Chen al.
2024; Leng et al. 2024) suppresses hallucinations by comparing output distributions from original and distorted input without altering the model's weights. Instruction Contrastive Decoding (ICD) (Wang et al. 2024) is used to combat hallucinations in multimodal tasks by modulating the confidence in multimodal alignment of the model's i visual and textual inputs, enabling Visual Contrastive Decoding (VCD) (Par et al.) [025] Favero et al. (022)) aims to reduce object hallucinations by comparing outputs from original and distorted visual inputs. This approach provides a computationally efficient solution as it requires neither additional training nor external pre-trainn trained models. Our proposed approach in this work is mainly inspired by the success of VCD schemes. However, unlike those schemes that contrast output distributions from original and (Park et al.) 2025) or image-removed (Favero et al., 2024) inputs to counteract LVLMs' overreliance on language priors, our PCD contrasts action probability distributions derived from original and object- inputs, thereby redirecting the robot policy's focus object-relevant visual features and enhancing their generalization across different scenarios.
distinguish between hallucinated and relevant tokens.
PROPOSED APPROACH
In this section, we present PCD, our training-free, plug-and-play scheme designed to tackle the spurious correlation issue for existing generalist robot policies. Notably, PCD does not require access to the pre-trained parameters of these robot policies; i.e., it treats each robot policy as a black-box.
3.1 PRELIMINARIES
This work focuses on robot manipulation tasks where the objective is to leverage a learned language- conditioned policy (a o;, {) to predict an M-dimensional action a; [a1,..., aml RT conditioned on the current visual observation o and the language instruction & of the task. In this part, we formalize two mainstream paradigms for action prediction: autoregressive action prediction (e.g., OpenVLA (Kim et al., 22024) and diffusion-basse action prediction (c.g., Octo (M. et al. 2024 and To (Black et al. (022))..
Autoregressive Robot Policies. The policy decomposes the action prediction task into sequential token generation. Each action dimension a, of a; sampled autoregressively from the probability distribution, conditioned on the visual observation o,, the language instruction é, as well as the previously predicted dimensions a.. = {a.)
at ~ To (a+ | llo, Oct).
For example, OpenVLA first encodes o; through hybrid vision backbone (DINOv2 (Caron et al.. 2023) w/ SigLIP (Zhai et al. 2023b) and tokenizes using pre-trained language model (Llama 2 (Touvron et al. 2023b)). These multimodal features are fused in the LLM's embedding space, which then autoregressively emits M discrete tokens. Each token is finally mapped to action dimension a via a de-tokenizer (Kim et al. 2024).
iterative denoising process (Ho et al. 2020). Denote e; the multimodal embeddings fusing o, and f. Starting from Gaussian noise ag (O, I), the reverse diffusion process refines all the M dimensions of the
Diffusion-based Robot Policies. The policy frames action generation
3

---

Published as a conference paper at ICLR 2026
nota, o.)
p
TT0
Ga Action probability distributions
Orignal observation
Right
-= xo(a,!!, â.)
Ay
PCD: Policy
Contrastive Decoding
Object-maske mojkkt-masked
Action probability distributions
Action: Move Left
Figure 2: Overview of our proposed Policy Contrastive Decoding (PCD) approach. PCD serves as plugin to redirect the robot policy's focus toward object-relevant visual cues by contrasting action probability distributions derived from original observations and object-masked observations p. For illustrative purposes, we visualize the predictions only in the Ar and Ay dimensions of the robot action space [Ax, Ay, Az, rotz, roty. rot,, gripper].
action a; in parallel over K steps through:
-ーー a (a TEo (a. k)) NN
(1)
where at the noisy action at diffusion step eg (a e,, k) is the denoising network, and a, 7, are parameters of a cosine noise schedule (Nichol and Dhariwal (02I). By jointly optimizing the entire M action dimensions of a, diffusion- based robot policies enable fine- ine-grained sampling from the joint probability no ro(a;l) (a:]l,00 o,) of all action dimensions.
3.2 POLICY CONTRASTIVE DECODING (PCD)
The core idea of our proposed PCD method to eliminate the adverse effect of spurious correlations by redirecting the robot policy's focus toward object-relevant features during inference. To facilitate the introduction of our PCD, we define spurious correlations as follows.
Definition 3.1 (Spurious Correlations): Let observations o and language instructions be generated by a set of L underlying latent factors {cz)/=>- These factors are decomposed into task-relevant factors, denoted by the random variable u, which exclusively determine the ground-truth expert policy such that p(alo, 1) () and by the random variable v. which have no causal effect on the expert's actions a. Spurious correlation is then defined the statistical dependence between the action a and these task- irrelevant factors u observed under the training distribution Pirain (a, v), quantified by the mutual information Luain v).
factors denoted
Spurious correlations, indicated by mutual information 4rrain (a, v) > O in the training set, allow robot policy нe infer action from task-irrelevant factors v. These factors act as shortcuts. However, such correlations are unreliable. Under disto--distribution (OOD) conditions-stemne from environmental shifts or compounding robotic errors may change independently of task- relevant factors u and, consequently, a, leading to inaccurate action predictions.
direct approach to mitigate learning from spurious correlations involves filtering out e and utilizing only u. Nevertheless, identifying and neutralizing spurious features is challenging due to their significant variability across tasks and scenarios, coupled with interdependencies within the feature space. Drawing inspiration from Vision Contrastive Decoding (VCD) (Park et al.) 2025 Favero ctt al. 2024). which effectively reduces hallucinations vision-language models, we introduce Policy Contrastive Decoding (PCD). PCD aims shift the robot policy's focus from spurious features towards task-relevant features u associated with the target object. This is achieved by contrasting model outputs derived from original visual inputs with those from object-me masked visual inputs.
Method Overview. Fig. presents an overview our PCD approach. Suppose we have a pretrained policy ny- Given the current visual observation o, and the language command f, we first generate two
4

---

Published as a conference paper at ICLR 2026
distinct action probability distributions of a; one is To(a;)l, o,) conditioned on the original visual input o,, and the other is xa (a,[(, o;) conditioned on object-masked counterpart o; that removes all task-relevant factors related to target objects. Then, new contrastive action probability distribution TØ [alll o:) is obtained by contrasting To (a.al o.) and Tp(a; o:):
T 1, 0:) ( 1, 6.)
(a, = C To To( o,)
where C' is a normalization constant and larger value of a 2 0 indicates stronger amplification of differences between the two distributions. Particularly, a 0 recovers the baseline policy. In such manner, the obtained T (a:l1, 1, 0:) amplifies model predictions object-relevant features in u, thus exhibiting insensitivity to spurious features in e, as illustrated in Fig.
To guarantee the flexibility and practical applicability of our PCD method, we need to address the following two questions:
1) How can object-masked observations ô, be automatically generated for each trajectory?
2) How can we approximate the policy distribution for diffusion- policies?
tracking-to-Mask We overcome the first question by devising a Trucking-to-Mutt (Track2Mask) strategy, which enables precise object masking for sequential visual observations along each trajectory, requiring minimal or human intervention. Concretely, the first step of Track2Mask is to annotate the task -ses-spe objects in the initial visual observation. Inspired by visual prompting techniques can annotate the target objects in the initial observation using Point
(Wan et al., 2024), Box prompts. Besides, we provide the option of leveraging off-the-shelf (open vocabulary) object detection models, such Grounding DINO (Liu al. (024b) for automatic object annotation. Next, the SAM2 (Kirillov el al., 2023) model is used to track and segment the target object across subsequent observations in the trajectory, and the segmented objects are then inpainted to obtain object-masked observations. For more details of Track2Mask, please refer to Appendix B
KDE-based Probabilistic Modeling. We address the second question using KDE- Probabilistic Modeling (KDE-PM), which approximates action probability distributions for diffusion- bassd policies through kernel density estimation (KDE) (Wkeglarczyk 2018). Specifically, we first use the pretrained diffusion-based policy to sample N candidate action predictions: f=) where a,()) =
(a1 (j), .... am (i)] ~ Ta (a:ll, 0z). With the assumption that all action dimensions are independent, i... (a. o.) 1I
M
na (aell, o.), we calculate approximated probability distribution for each
1=1
action dimension of the action a; via KDE:
1 N
a (a=ali)) t I,
Ta(ael(,0)) ~
Σx
K
(3)
C
where C' is a normalization constant and KC(-) indicates Gaussian kernel K(u)) e 4
the bandwidth parameter controlling the smoothness of the distribution, and {a)( are the N candidate actions for the 1-th dimension of the action a,, generated from the diffusion process. Similarly, we can derive an action probability distribution for a conditioned on object-masked o i.e. (a [1. 6.). With the independence assumption, from Te(a, V1, o.) and To (a,ll 6;) we can compute the joint action distributions, which are then used to compute the contrastive action probability rr (a:ll, o:) using Eq. 2 In this way, our PCD is compatible with diffusion-based robot policies.
Inference with PCD. As a training-free method, our proposed PCD can be employed as plugin to improve the inference stage of different types of robot policies (i.e., autoregressive policies and diffusion-based policies). Specifically, for pre-trained robot policy Te o,, (). given the current observation o; we can obtain the object-masked o, (i.e., ó;) leveraging the devised Track2Mask strategy. Based on o, Ô; and a language command f, the autoregressive policy directly computes contrastive action probability distributions for deriving a, (w/ Eq. (2), whereas the diffusion- policy requires performing the KDE-PM process on its output predictions to achieve the same goal. Pseudocode for our PCD method is provided in Algorithm
5

---

Published as a conference paper at ICLR 2026
Algorithm 1 PCD: Policy Contrastive Decoding
Require: A pre-trained robot policy: To(a; Os. e); initial observation Ou; language command maximum time step S.
1:
+- 0
2: while 8 S do
3:
Obtain object-masked 0,: O,
w/ Track2Mask > w/ KDE-PM for diffusion-based policies
4:
Obtain To (0,a, ( o,) and Ty a. ,,
5: Compute T6 (a 0₂) using Eq. 2
6:
Sample a, from x =a, ([a,|1,000 o,) and execute
7:
O++1 6 New observation
b Obtain a new observation
8:
<- -++1
9:
end while
Table 1: SIMPLER Performance. Task-specific objects in the initial observation are annotated by artificial Point and Box prompts or the automatic detection results of GDINO (Liu et al., 2024b).
The results are the success rate and the 95% confidence interval of 300 trials. As a plug-and-play approach, PCD consistently enhances the three policies by large margins over the 9 tasks.
OpenVIA Kim
2024
(M. al. 2024
BBlack et al. 2024
Tasks
Base
+PCD
Base
+PCD
Base
+PCD
GDINO
GDINO
GDINO
16.8214
13.82..3
63.9.1..68.
Average
+45.2%
+36.3% +50.6%
+27.5%
+26.1%
+29.7%
+6.6%
+7.4% +8.9%
Close Drawer
11. 31.0
21.025.26.022...222
26.025.02.04550222
75.724.9
75.02g
Move Near
58.725.e 64.0:...62.0sa..59900
68.7
3.7
6.7.12.
6.0.2.7
9.0
67.3
72.3
Google
Robot
Open Drawer
0.3+0.7
0.3+0.7
0.3+0.7
1.0=11
Pick Coke Can
18.00...3..02........
51.
84.0a..1
84.3 37.
88.0
Apple Drawer
0.02zaa
0.3.1.7
0.710.9
0.0
0.0
17.0g4.:26.01.
Carrot Plate
0.0g0
7.312a
0.022..
4.7194
12.023.7
22.01.17
19.014
28.72..
59.715.
WidX.
Eggpl. Basket
0.3go.r
5.3+2.
1.7514
5,3
38.3
33.0
38.7 a5.
87.0
Spoon Towel
0.0=0.0
2.341.7
1.0s11
4.3+23
9,3+3.8
15.0+4.0
18.74..1..744.4
16.0+4.1
16.014..80...4.....
84.00444....
Stack Cube
3.7
2.7.18
0.020.
0.020.
0.0g0.0
4.3
4.0ga.
3.7
68.725. 26.
77.24.
EXPERIMENTS
In this section, we seek answer the following questions:
I) Can PCD improve robot policies in both simulation and real-world environments?
2) How does the performance vary with different design choices?
3) What kinds of spurious correlations can our PCD tackle?
We answer the first question in Sections 4.1 and 4.2 the second question Section 4.3 and the third question in Section 4.4 Detailed descriptions of the baseline robot policies and the evaluation tasks used simulation and real-world environments are presented in Appendix
4.1 SIMULATION EXPERIMENTS
We perform simulation experiments in SIMPLER Li et al. 2024). real-to-sim evaluation en- vironment designed specifically for real-robot policies. SIMPLER accurately reflects real-world performance, enabling reliable assessment of robotic policies. We evaluate 9 tasks across two robot platforms: tasks using the Google Robot and 4 tasks using the WidowX arm.
Experimental Setup. We conduct experiments on top of three diverse robot policies, including the autoregressive policy OpenVLA (Kim et al., 2024). and the diffusion- based policies Octo (base) (M) et al., 2024) and (Black et al.) 2024). We treat these policies as black-boxes, applying PCD solely based on the output action probability distributions for OpenVLA and the output actions for Octo and To- For our PCD, we annotate the target object in the initial observation using three manners: artificial Point and Box prompts, and automatic detection results from Grounding DINO (Liu et al. 2024b). The number of the sampled noise vectors N in KDE-PM is set to 24 for Octo and mio We perform ablation studies on the hyperparameter a in Eq. ( as well as on the object detection strategies and object inpainting paradigms of the devised Track2Mask module in Section 4.3

---

Published as a conference paper at ICLR 2026
Success
Rate(%)
20
40
25
40
15
Baseline
10
25
10
(+108)
35
20
35
Pick Ball
Pick Plug
Move Near
Cookies Towel
Banana Plate
Stack Cube
Time Cost(s)
20
12
16
19
18
24
20
25
22
25
19
Baseline
+PCD (+24%)
Figure 3: Real-world Performance. The target objects in the initial observation are automatically annotated by Grounding DINO (Liu et al. 20224b. PCD delivers remarkable 108% performance improvement on the baseline, though it incurs a 24% increase in time cost.
Results and Analysis. Table I reports the performance three robot policies (i.e., OpenVLA (Kim al.) 2024), Octo (M. et al. 2024) and To (Black et al.] (2244) integrated w/ or w/o our PCD across a total of tasks from Google Robot and WidowX. From the results in the table, we have the following key observations.
Our PCD consistently improves the three baseline policies over the 9 tasks from both the Google Robot and WidowX robotic platforms. On average, PCD boosts the success rates of the autoregressive policy OpenVLA, and the diffusion--asse policies Octo and To by to 50.6%. 29.7% % and 8.9%, respectively. 2) Our PCD exhibits stable and outstanding performance across the three object annotation strategies, including artificial Point and Box prompts and automatic object detection with Grounding DINO, underscoring its effectiveness and adaptability. In particular, on the Octo and OpenVLA baseline policies, leveraging the off- the- shelf model Grounding DINO demonstrates better performance than using human Point and Box prompts, while showing comparable results on mo, highlighting the practical applicability the PCD method. 3) Compared to OpenVLA and Octo, the baseline model xo demonstrates significantly superior performance across all tasks. For instance, OpenVLA and Oct o achieve near-zero success rates on "Apple Drawer" task whereas To achieves success rates of 17.0% and 68.7% these two tasks, respectively. Despite this, from the average results over the 9 tasks, our PCD still enhances the strong baseline To by 8.9 %. In summary, the obtained results in the table demonstrate that our PCD approach can be used as a plugin to improve different types of robot policies.
4.2 REAL-WORLD EXPERIMENTS
In this section, we conduct experiments real-world tasks to assess PCD's practical effectiveness.
Experimental Setup. The simulation experiments show that To significantly outperforms Octo and OpenVLA. Therefore, employ To the baseline policy to validate the effectiveness of our PCD approach, as shown in Fig. 3 Considering the generalization limitations of existing robot policies in real-world environments, evaluating PCD's performance. We design 6 manipulation tasks that focus on evaluating the policy's s performance along multiple dimensions: spatial reasoning and interacting with various objects and scenes. The тo policy is jointly fine-tuned on the 6 real-world tasks, with each task consisting of 10 training demonstrations. During inference, randomize the configurations and orientations of task-specific objects for each trial. We use an AGILEX PIPER 6DOF robot arm in our experiments. For our PCD, we leverage the off off-the-shelf model Grounding DINO to detect task-specific objects in the initial observation, and use LaMa (Suvorov et al. 2022) to inpaint those objects segmented by the SAM2 (Kirillov et al.) (023) model. All other hyperparameters are kept consistent with the settings applied in the simulation experiments.
first finetune To on downstream real-world tasks before
conduct 20 trials for each real-world task and
Results and Analysis. Fig. reports the success rates as well as time costs of the state-of-the-art robot policy To (Black et al. 2024) integrated w/ or w/o our PCD on 6 real-world tasks. For each task, we record the time cost (in seconds) from the start of the inference process to the successful completion of the task. From the obtained results in the figure, have several key observations. 1) PCD consistently improves the success rates of the strong baseline policy across six real-world tasks, achieving an average enhancement of 108%. While PCD significantly boosts success rates, it also
7

---

Published as a conference paper at ICLR 2026
success Rate(%) (%)
70
Octo OpenVLA
7g
60
YOLO World
GDINO
60
Telea
LaMa
SED
Navier-Stokes
65
40
40
25
20
20
20
15
0.0
0.2
0.4
0.6
0.8
1.0
Octo
OpenVLA
0
Octo
OpenVLA
Ta
(a)
(b)
(c)
Figure 4: Ablation studies on (a) the hyperparameter in Eq. (2.. (b) the object detection schemes and (c) object inpainting strategies in Track2Mask. = 0 in (a) and the black dotted lines in (b)(c) represent the performance of the baseline policies. The results are averaged over the 9 simulation
tasks. PCD consistently improves the three policies when a > 0 and exhibits low sensitivity to changes off-the-shelf object detection and inpainting strategies.
increases the time cost by 24% on average compared to the baseline policy. However, considering the notable improvement in task completion success rates, this trade-off is acceptable in wide range of real-world robotic applications. 3) PCD demonstrates consistent performance improvements across tasks of varying complexity. For example, in the challenging "Stack Cube" task, PCD improves the success rate from 0.05 to 0. 9.10, showing its strong adaptability and robustness across diverse scenarios. Additionally, the backgrounds of these six real-world tasks are far more complex than those of the nine simulation tasks, containing numerous distracting elements such as plastic bottles, baskets, and trash cans. Nevertheless, our PCD shows remarkable advantages. It is worth mentioning that consistent PCD hyperparameters across tasks in both the simulation and real-world environments, without carefully tuning task-specific PCD hyperparameters (e.g., o:) for each individual task, leaving room for potential performance improvements through task-specific parameter tuning.
4.3 ABLATION STUDIES
In this section, we conduct ablation studies to explore how the performance of our PCD varies with different design decisions. We perform experiments in the SIMPLER simulation environment, and report the average results across the nine tasks for our PCD applied to the three baseline policies.
Effect of As formulated in Eq. 2 PCD leverages the hyperparameter amplification between output distributions from original and object- masked visual inputs. A larger value indicates stronger amplification of differences between the two distributions, while a = reduces to regular prediction. We adjust a by setting it to [0, 0.2, 0.4, 0.6, 0.8. 1.0), and report the performance of our PCD method over the tasks in Fig. (a). a = 0 represents the baseline policies. As can be observed, our PCD consistently improves Octo, OpenVLA and To when a > 0, demonstrating its effectiveness and stability. Particularly, PCD yields the best results on Octo, OpenVLA and To when the values of are set 1.0, 0.8. and 0.2, respectively.
control the level of
Effect of Off-the-shelf Object Detection Models. Beyond leveraging artificial Point and Box prompts, the devised Track2Mask module can employ off-the-shelf open vocabulary object detection models to annotate task-specific objects in the initial observation of the trajectory during inference. Fig. (b) presents ablation study on various automatic object detection models, including the Grounding DINO (Liu et al. 2024b) model discussed in the main paper, as well as the open vocabulary object detection model YOLO World (Cheng al. 2024) and semantic segmentation model SED (Xie et al., 2024). The output of these models is fed into the SAM2 (Kirillov et al., 2023) model tracking and segmenting the detected objects across subsequent observations in the trajectory. As can be seen, our PCD consistently improves the baseline policies when integrated with each of the three off-the-shelf models, showeasing its effectiveness and stability. Generally, the Grounding DINO model yields the best results among the three models.
Effect of Object Inpainting Strategies. Once the objects specified in the language instruction of the target task are segmented by the SAM2 model, they are inpainted to create object-masked observations. In this experiment, we investigate the effect of different object inpainting strategies on PCD's performance. Fig. 4 (c) shows the results of PCD integrated with three inpainting strategies: Telea (Telea) 2004) and Navier-Stokes (Bertalmio et al., (001)) and LaMa (Suvorov et al. (022). As indicated in the figure, our PCD approach consistently improves the baseline policies across all
8

---

Published as a conference paper at ICLR 2026
Success Rate (%)
Baseline
+PCD
47
40
41
25
20
16
19
19
Original
Spatial Relation
Spatial Relation
Brightness
Table Texture
Table Texture
(a) Simulation task: "pick coke can"
Success Rate (%6)
Baseline
+PCD
40
40
30
25
20
20
15
5
5
Original
Distractors
Spatial Relation
Brightness
Table Texture
Table Texture
(b) Real-world task: "put the banana in the plate"
Figure 5: Performance of baseline policies integrated w/ w/o our proposed PCD approach in unseen testing scenarios. Dotted lines represent the original performance. OpenVLA (Kim et al. 2024) and To (Black et al. 2024) used baselines in simulation and real- wolld environments, respectively. PCD effectively mitigates the side effects of various types of spurious correlations, boosting the robot policy's generalization to novel scenarios.
strategies, showcasing its effectiveness and stability. Furthermore, PCD demonstrates low sensitivity to variations in the three object inpainting strategies, further validating its applicability and robustness. Notably, LaMa delivers the best results across all three baseline policies.
4.4 ROBUSTNESS TO VARIOUS KINDS OF SPURIOUS CORRELATIONS
The central concept of our proposed PCD) approach this work is to overcome the adverse effects of spurious features on the decision-making of robot policies. So, what types of spurious correlations can PCD effectively address? To answer the question, we evaluate the robustness of the OpenVLA (Kim et al. 2024) and To (Black al. 2024) policies task scene variations in both simulation and real-world environments. As illustrated in Fig. 5 we report the performance of the two baseline policies integrated
w/o our proposed PCD approach in different testing scenarios with unseen spatial relationships, brightness, distractors and table textures within visual observations. The results shown in the figure lead to the following observations. results, our PCD approach significantly improves the performance of the bascline policies on both simulation and real-world tasks (Orange line vs. Blue line). 2) Both bascline policies experience modified. For example, altering the
Consistent with the previous experimental
substantial drops in performance when testing scenarios brightness of visual observations results in 48% and 75% decreases in success rates for the two policies, respectively. 3) PCD consistently alleviates the performance degradation of the baseline policies across all unseen testing scenarios. Remarkably, after integrating PCD, the two baseline policies even performs better 4/10 of unseen scenarios than the original training scenarios. The results in the figure reveal that our proposed PCD approach can serve as a plugin to mitigate the adverse effects of various types of spurious correlations the generalization of robot policies.
5 CONCLUSION, LIMITATIONS AND FUTURE WORK
In this work, propose Policy Contrastive Decoding (PCD) to tackle the adverse effects of spurious correlations learned by generalist robot policies. PCD redirects the policy's attention from spurious features relevatte obtained from original and object-mart masked inputs. As a pplug-and-pla approach, PCD can enhance both autoregressive and diffusion-based policies, without requiring fine-tuning access to pre-
during inference by contrasting action probability distributions

---

Published as a conference paper at ICLR 2026
trained model weights. Comprehensive experiments demonstrate PCD's flexibility and effectiveness, achieving consistent improvements over three state-of-the-ar robot policies across a total of 15 tasks in simulation and real-world environments.
While our PCD demonstrates effectiveness and flexibility, there are certain limitations. 1) PCD performs object detection and segmentation for each visual observation in the trajectory to produce object-masked observations, which increases the computational overhead of the baseline police. Recent progress in fast LLM inference techniques (Zhou al. 2024 Leviathan al. 2023 Liu et al.) 2023c) could potentially enhance the computational efficiency of our PCD approach. 2) This work concentrates exclusively on addressing the spurious correlation issue in robot policies during the testing stage, without exploring methods to prevent the learning of such correlations during training. Hence, future work will focus on investigating effective strategies tackle spurious correlations at both the training and inference levels.
ACKNOWLEDGEMENT
This work is supported by grants from the National Natural Science Foundation of China NN.662503110, No.62425208, No. U2222292, No.U23A20315, No.82441006, the China Post- doctoral Science Foundation (No.2025M781517), and the Sichuan Science and Technology Program (No.2026NSFSC1479)
REFERENCES
Suneel Belkhale and Dorsa Sadigh. Minivla: A better vla with a smaller footprint. 7/ai stanforr.......... www/blo///bleeeeeee hatllooommiiiia////a/./.. 2024.
https
Suncel Belkhale, Tianli Ding, Ted Xiao, Pierre Sermanet, Quon Vuong, Jonathan Tompson, Yevgen Chebotar, Debidatta Dwibedi, and Dorsa Sadigh. Rt-h: Action hierarchies using language. arXiv preprint arXiv:2403.01823 2024.
Marcelo Bertalmio, Andrea L Bertozzi, and Guillermo Sapiro. Navier-stokes, fluid dynamics, and image and video inpainting. In CVPR, volume 1. IEEE, 2001.
K. Black, N. Brown, D. Driess, al. Pi-O: A vision-language-action flow model for general robot control. arXiv preprint 210.i::2110.24144, 2024.
A. Brohan, N. Brown, J. Carbajal, et al. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint /22.0222/2.068/7, 2022.
A. Brohan, N. Brown, J. Carbajal, et al. Rt-2: to robotic control. arXiv preprint arXhv:2307.15818 2023.
models transfer web knowledge
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Anmand Joulin. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023.
Zhaorun Chen, Zhuokai Zhao, Hongyin Luo, Huaxiu Yao, Bo Li, and Jiawei Zhou, Hale: Object hallucination reduction via adaptive focal-contrast decoding. arXiv preprint arXiv:2403.00425, 2024.
Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xinggang Wang, and Ying Shan. Yolo-world: Real-time open-vocabulary object detection. In CVPR, 2024.
Embodiment Collaboration, Abby O'Neill, Abdul Rehman, Abhinav Gupta, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acor Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, Albert Tung, Alex Bewley, Alex Herzog, Alex Irpan, Alexander Khazatsky, Anant Rai, Anchit Gupta, Andrew Wang, Andrey Kolobov, Anikait Singh, Animesh Garg, Aniruddha Kembhavi, Annie Xie, Anthony Brohan, Antonin Raffin, Archit Sharma, Arefeh Yavary, Arhan Jain, Ashwin Balakrishna, Ayzaan Wahid, Ben Burgess-Limerick, Beomjoon Kim, Bernhard Schölkopf, Blake Wulfe, Brian Ichter, Cewu Lu, Charles Xu, Charlotte Le. Chelsea Finn, Chen Wang, Chenfeng Xu, Cheng Chi, Chenguang Huang, Christine Chan, Christopher Agia, Chuer
10

---

Published as a conference paper at ICLR 2026
Pan, Chuyuan Fu, Coline Devin, Danfei Xu, Daniel Morton, Danny Driess, Daphne Chen, Deepak Pathak, Dhruv Shah, Dieter Büchler, Dinesh Jayaraman, Dmitry Kalashnikov, Dorsa Sadigh, Edward Johns, Ethan Foster, Fangchen Liu, Federico Ccola, Fei Xia, Feiyu Zhao, Felipe Vieira Frujeri, Freek Stulp, Gaoyue Zhou, Gaurav S. Sukhatme, Gautam Salhotra, Ge Yan, Gilbert Feng, Giulio Schiavi, Glen Berseth, Gregory Kahn, Guangwen Yang, Guanzhi Wang, Hao Su, Hao-Shu Fang, Haochen Shi, Henghui Bao, Heni Ben Amor, Henrik Christensen, Hiroki Furuta, Homanga Bharadhwaj.. Homer Walke, Hongjee Fang, Huy Ha, Igor Mordatch, Ilija Radosavovic, Isabel Leal, Jacky Liang, Jad Abou-Chakra, Jachyung Kim, Jaimyn Drake, Jan Peters, Jan Schneider, Jasmine Hsu. Jay Vakil, Jeannette Bohg, Jeffrey Bingham, Jeffrey Wu, Jensen Gao, Jiaheng Hu, Jiajun Wu, Jialin Wu, Jiankai Sun, Jianlan Luo, Jiayuan Gu, Jie Tan, Jihoon Oh, Jimmy Wu, Jingpei Lu, Jingyun Yang, Jitendra Malik, João Silvério, Joey Hejna, Jonathan Booher, Jonathan Tompson, Jonathan Yang, Jordi Sal Joseph J. Lim, Junhyek Han, Kaiyuan Wang, Kanishka Rao, Karl Pertsch. Karol Hausman, Keegan Go, Keerthana Gopalakrishnan, Ken Goldberg, Kendra Byrne, Kenneth Oslund, Kento Kawaharazuka, Kevin Black, Kevin Lin, Kevin Zhang, Kiana Ehsani, Kiran Lekkala, Kirsty Ellis, Krishan Rana, Krishnan Srinivasan, Kuan Fang. Kunal Pratap Singh, Kuo-Hao Zeng, Kyle Hatch, Kyle Hsu, Laurent Itti, Lawrence Yunliang Chen, Lerrel Pinto, Li Fei-Fei. Liam Tan, Linxi "Jim" Fan, Lionel Ou,, Lisa Lee, Luca Weihs, Magnum Chen, Marion Lepert, Marius Memmel, Masayoshi Tomizuka. Masha Itkina, Mateo Guaman Castro, Max Spero, Maximilian Du, Michael Ahn, Michael C. Yip. Mingtong Zhang, Mingyu Ding, Minho Heo, Mohan Kumar Srirama, Mohit Sharma, Moo Jin Kim, Naoaki Kanazawa, Nicklas Hansen, Nicolas Heess. Nikhil Joshi, Niko Suenderhauf, Ning Liu, Norman Di Palo, Nur Muhammad Mahi Shafiullah, Oier Mees, Oliver Kroemer, Oshert Bastani, Pannag Sanketi, Patrick "Tree" Miller, Patrick Yin, Paul Wohlhart, Peng Xu, Peter David Fagan, Peter Mitrano, Pierre Sermanet, Pieter Abbeel, Priya Sundaresan, Qiuyu Chen, Quan Vuong, Rafael Rafailov, Ran Tian, Ria Doshi, Roberto Mart inr''in-Matttii in, Rohan Baijal, Rosario Scalise, Rose Hendrix, Roy Lin, Runjia Qian, Ruohan Zhang, Russell Mendonca, Rutav Shah, Ryan Hoque, Ryan Julian, Samuel Bustamante, Sean Kirmani, Sergey Levine, Shan Lin, Sherry Moore, Shikhar Bahl, Shivin Dass, Shubham Sonawani, Shubham Tulsiani, Shuran Song, Sichun Xu, Siddhant Haldar, Siddharth Karamcheti, Simeon Adebola, Simon Guist, Soroush Nasiriany, Stefan Schaal, Stefan Welker, Stephen Tian, Subramanian Ramamoorthy, Sudeep Dasari, Suneel Belkhale, Sungjae Park, Suraj Nair, Suvir Mirchandani, Takayuki Osa, Tanmay Gupta, Tatsuya Harada, Tatsuya Matsushima, Ted Xiao, Thomas Kollar, Tianhe Yu, Tianli Ding, Todor Davchev, Tony Z. Zhao. Travis Armstrong, Trevor Darrell, Trinity Chung. Vidhi Jain, Vikash Kumar, Vincent Vanhoucke, Wei Zhan, Wenxuan Zhou, Wolfram Burgard, Xi Chen, Xiangyu Chen, Xiaolong Wang. Xinghao Zhu, Xinyang Geng, Xiyuan Liu, Xu Liangwei, Xuanlin Li, Yansong Pang, Yao Lu, Yecheng Jason Ma, Yejin Kim, Yevgen Chebotar, Yifan Zhou, Yifeng Zhu, Yilin Wu, Ying Xu, Yixuan Wang. Yonatan Bisk, Yongqiang Dou, Yoonyoung Cho, Youngwoon Lee, Yuchen Cui, Yue Cao, Yueh-Hua Wu, Yujin Tang. Yuke Zhu, Yunchu Zhang, Yunfan Jiang, Yunshuang Li, Yunzhu Li, Yusuke Iwasawa, Yutaka Matsuo, Zehan Ma. Zhuo Xu, Zichen Jeff Cui, Zichen Zhang, Zipeng Fu, and Zipeng Lin. Open x-embodiment: Robotic learning datasets and rt-x models. arXiv preprint 23..:::33000880000. 2024.
Alessandro Favero, Luca Zancato, Matthew Trager, Siddharth Choudhary, Pramuditha Perera, Alessandro Achille, Ashwin Swaminathan, and Stefano Soatto. Multi-modal hallucination control by visual information grounding. In CVPR, pages 14303-14312, 2024.
Roya Firoozi, Johnathan Tucker, Stephen Tian, Anirudha Majumdar. Jiankai Sun, Weiyu Liu, Yuke Zhu, Shuran Song, Ashish Kapoor, Karol Hausman, et al. Foundation models in robotics: Ap- plications, challenges, and the future. The International Journal of Robotics Research, page 02783649244281 2023.
Jensen Gao, Bidipta Sarkar, Fei Xia, Ted Xiao, Jiajun Wu, Brian Ichter, Anirudha Majumdar, and Dorsa Sadigh. Physically grounded vision-language models for robotic manipulation. In ICRA, pages 12462-12469. IEEE, 2024.
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(1 (1:6)))66667777 2020.
Anisha Gunjal, Jihan Yin, and Erhan Bas. Detecting and preventing hallucinations in large vision language models. In AAAI, volume 38, pages 18 18135-18143, 2024.
11

---

Published as a conference paper at ICLR 2026
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurIPS, 33: 6840-6851, 2020.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. ICLR, 2022. URL
https://openrevvil wotp:
Pavel Izmailov, Polina Kirichenko, Nate Gruver, and Andrew G Wilson. On feature learning in the presence of spurious correlations. NeurlPS, 35:38516-38553 2022.
S. Karamcheti, S. Nair, A. Balakrishna, al. Prismatic vims: Investigating the design space of cisuaaly-conditioned language model. In ICML, page 235, 2024a.
S. Karamcheti, S. Nair, A. Balakrishna, al. Prismatic vlms: Investigating the design space of visually-conditioned language models. In ICML, page 235, 2024b.
M. J. Kim, K. Pertsch, S. Karamcheti, et al. Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.0922 09246, 2024.
Alexander Kirillov, Xinyu Liu, and Kaiming He. Segment anything model 2 (sam2). arXiv preprint arXiv:2304.00 0142,, 2023.
Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chumyan Miao, and Lidong Bing. Mitigating object hallucinations in large vision-language models through visual contrastive decod- ing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13872-13882, 2024.
Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In ICML, pages 19274-19286 PMLR, 2023.
Xuanlin Li, Kyle Hsu, Jiayuan Gu, Karl Pertsch, Oier Mees, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, Isabel Sich, Sean Kirmani, al. Evaluating real-world robot manipulation policies in simulation. arXiv preprint arXiv:2405.059941 2024.
Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. NeurIPS, 36:44776-4479) 2023a.
Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. Mitigating halluci- nation in large multi-modal models via robust instruction tuning. arXiv preprint 4rX15:2306.14565, 2023b.
S. Liu, L. Wu, B. Li, et al. Rdt-1b: A diffusion foundation model for bimanual manipulation. arXiv preprint arXiv:2410.07864 2024a.
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang. Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In ECCV, pages 38-55. Springer, 2024b.
Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher Re, et al. Deja vu: Contextual sparsity for efficient Ilms at inference time. ICML, pages 22137-22176. PMLR, 2023c.
Shenyu Lu, Junyi Chai, and Xiaoqian Wang. Mitigating spurious correlations in zero-shot multimodal models. In The Thirteenth International Conference on Learning Representations, 2025.
Team O. M., D. Ghosh, H. Walke, et al. Octo: An open-source generalist robot policy. arXiv preprint arXiv:2405.12213. 2024.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In ICML, pages 8162-8171. PMLR, 2021.
12

---

Published as a conference paper at ICLR 2026
M. Oquab, T. Darcet, T. Moutakanni, et al. Dinov2: Learning robust visual features without supervision. Transactions on Machine Learning Research Journal, 11:1-31, 2024.
Yeji Park, Deokyeong Lee, Junsuk Choe, and Buru Chang. Convis: Contrastive decoding with hallucination visualization for mitigating hallucinations in multimodal large language models. In AAAI, volume 39, pages 1434-6442, 2025.
K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, and S. Levine. Fast: Efficient action tokenization for vision aationlanguage-actti models. arXiv preprint arXiv:2501.09747, 2025.
Vipula Rawte, Amit Sheth, and Amitava Das. A survey of hallucination in large foundation models. arXiv preprint arXiv:230.05592,, 2023.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localiza- tion. In ICCV, pages 618-626, 2017.
Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong. Harshith Goka, Kiwoong Park, and Victor Lempitsky. Resolution- robust large mask inpainting with fourier convolutions. In WACV, pages 2149-21522 2022.
O. M. Team, D. Ghosh, H. Walke, et al. Octo: An open-source generalist robot policy, arXiv preprint 12Xi::2405.12213, 2024.
Alexandru Telea, An image inpainting technique based on the fast marching method. Journal of graphics tools, 9(1):23-34 34, 2004.
H. Touvron, T. Lavril, G. Izacard, X. Martinet, et al. Llama: Open and efficient foundation language models. arXiv preprint urXiv:2302.3397/ 2023a.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307,09288 2023b.
Maya Varma, Discovering and mitigating spurious correlations fine-tuned vision-language models. NeurIPS, 37:82235 82264, 2024.
Delbrouck, Zhihong Chen, Akshay Chaudhari, and Curtis Langlotz. Ravl:
David Wan, Jaemin Cho, Elias Stengel- and Mohit Bansal. Contrastive region guidance: Improving grounding in vision-language models without training. In ECCV, pages 198 215. Springer, 2024.
Xintong Wang, Jingheng Pan, Liang Ding, and Chris Biemann. Mitigating hallucinations in large vision-language models with instruction contrastive decoding. arXiv preprint arXiv.2403.18715, 2024.
Stanislaw Wkeglarezyk. Kernel density estimation and its application. ITM web of conferences, volume 23, page 00037, EDP Sciences, 2018.
Bin Xie, Jiale Cao, Jin Xie, Fahad Shahbaz Khan, and Yanwei Pang. Sed: A simple encoder-decoder for opea-vocabulary semantic segmentation. In CVPR, pages 3426-3436, 2024.
Wenqian Ye, Guangtao Zheng, Xu Cao, Yunsheng Ma. and Aidong Zhang, Spurious correlations in machine learning: A survey, arXiv preprint arXiv:2402.12715, 2024.
Michal Zawalski, William Chen, Karl Pertsch, Oier Mees, Chelsea Finn, and Sergey Levine. Robotic control via embodied chain-of-thought reasoning. arXiv preprint arXiv:2407.0866 2024.
X. Zhai, B. Mustafa, A. Kolesnikov. et al. Sigmoid loss for language image pre-training. In ICCV, pages 11975-11986. 2023a.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Siglip: Sigmoid loss for language image pre-training. arXiv preprint arXiv:2303.1334 2023b.
13

---

Published as a conference paper at ICLR 2026
Ji Zhang, Shihan Wu, Xu Luo, Hao Wu, Lianli Gao, Heng Tao Shen, and Jingkuan Song. Inspire lasion-laguuge-actinn models with intrinsic spatial reasoning. arXiv preprint arXiv:2505.33888, 2025.
Yuhang Zhang, Chen Li, Xiaolong Wang, Yi Yang, Abhinav Gupta, and Trevor Darrell. SpatialVLA: Exploring spatial representations for visual-language-action models. In CVPR, pages 14567-14578, 2024.
Zixuan Zhou, Xuefei Ning, Ke Hong, Tianyu Fu, Jiaming Xu, Shiyao Li, Yuming Lou, Luning Wang. Zhihang Yuan, Xiuhong Li, et al. A survey efficient inference for large language models. arXiv preprint arXiv:2404. 14294, 2024.
14

---

Published as a conference paper at ICLR 2026
A CAM VISUALIZATION OF FAILURE CASES
We present the CAM (Selvaraju et al. 2017) visualization of failure cases for the OpenVL model across the nine tasks in Fig. For each task, the averaged CAM results over the seven action dimensions ([Ar, Ay, Az, rot,, ro ro gripper]) are plotted. As can be observed from the figure, the model tends to predict actions based on spurious or background features in visual observations. For instance, the "move the blue plastic bottle sponge" (denoted as "Move Near") task, the model model habitually focuses on the "coke can" object that it frequently encountered during the training phase. These observations demonstrate that the learned policy utilizes perceptual data for decision-making in way that significantly diverges from human strategies. Robot policies often erroneously associate task-irrelevant features with actions, major cause of overfitting and poor generalization across tasks.
Close Drawer
Move Near
Open Drawer
Pick Coke Can
Apple Drawer
Overemphasized Inrelevant Regions
Overlooked Object Regions
Carrot Plate
Eggplant Basket
Spoon Towel
Stack Cube
Figure 6: CAM visualization of failure cases.
B DETAILS OF THE TRACK2MASK MODULE
In our devised Track2Mask module, the target object specified by the language instruction the initial observation can be annotated using Point and Box prompts, along with off-the-shelf open-vocabulary object detection models, as illustrated in Fig. The SAM2 (Kirillov et al., (023) model is then applied automatically track and segment the target object across subsequent observations in the trajectory, which enables precise object masking with minimal or no human intervention during inference. Note that, once the task -asc-spec objects are segmented by the SAM2 model, they are inpainted to create object-masked observations.
t=1
Original Observation
Human Point
Or
1=0
Human
Original Observation
SAM 2
SAM
Box
Or
Detection Results
Inpainting
t= 0
Object-masked Observation
Figure 7: Illustration of the Track2Mask module.
We only visualize the CAM results of OpenVIA A, the action prediction mechanism of diffusion-based models (c.g., Octoo and тo) makes difficult produce their CAM results.
15

---

Published as a conference paper at ICLR 2026
C DETAILS OF THE EXPERIMENTAL SETUP
C.I BASELINE POLICIES
Simulation Experiments. We conduct simulation experiments using three diverse robot policies in the SIMPLER environment, including the autoregressive policy OpenVLA (Kim al. 2024). and diffusion-baan based policies Octo (base) (M. et al. 2024) and To (Black et al. 2024).
OpenVLA: a vision-language-actioon model with 7 billion parameters, trained on 970,000 episodes of robotic demonstrations from the Open X-Embodiment dataset, This policy is fine-tuned using the pre-trained Prismatic (Karamcheti et al. 2024b) model
Octo (base): an open-source generalist policy with 93 million parameters, is pre-trained a blend of 25 different datasets from the Open X-Embodiment dataset. It utilizes transformer backbone derived from ViT-B, accompanied by diffusion action head to model expressive action distributions Fl
Tn. flow matching architecture built top of pre-trained vision-language model to leverage Internet-scale semantic knowledge. To facilitate evaluation in the SIMPLER environment, we leverage the reproduced version of no, pretrained on tasks from this environment
Real-world Experiments. Since the simulation experiments show that To significantly outperforms Octo and OpenVLA, conduct real-world experiments using the To policy [ The policy is finetuned using on six real-world tasks, with each task consisting of 10 trajectories. We employ the parameter-effiiient finetuning strategy LoRA (Hu al. (022) to adapt mo to these real world tasks. The training hyperparameters are listed in Table
Table 2: Hyperparametes for finetuning To on real-world tasks.
Hyperparametes
Setting
Batch Size
128
Optimizer
AdamW
Learning Rate
2.5e-5
T.R Schedule
Cosine Decay
Weight Decay
le-10
Training Step
25000
Action Chunk
10
LoRA Rank
16
C.2 EVALUATION TASKS
We conduct extensive evaluations both simulation and real-world environments using a total of 15 diverse tasks. The details of these tasks presented in Table 3] We conduct 20 trials for each real-world task, randomizing the task configurations in every trial, as illustrated in Fig. g
D PERFORMANCE AT THE MAXIMUM STEP
In Table tasks completed before the predefined maximum step are included in the success rate calculation, and the task terminated upon completion. Table Sableprreeents X presents success rates computed at the maximum step, irrespective of whether tasks were completed earlier. As seen, PCD demonstrates its advantages by improving the success rates of OpenVLA, Oct and To by up to 45.0%. 19.3% and 7.9%. respectively. We also observe that the vast majority of models underperform on these tasks relative to their results in Table I] which indicates that even though they complete the task successfully ahead of the maximum steps, their later predicted actions lead task failure. The
ittpesr/opear/la.iiiibbbbb
httess//ecto-nccelll....bb...
httea///ttubblcco///////////////
16

---

Published as a conference paper at ICLR 2026
Figure 8: We conduct 20 trials for each real-world task, randomizing the task configurations in every trial. The language command for this task is "stack the green cube on the yellow cube."
Table Details of simulation and real-world tasks.
Env.
Tasks
Language Instruction
Task-specific Objects
# Trials
Close Drawer
close top/middle/hootom drawer move X near
top/middil/hottom drawer
300
Move Near
apple, orange, pepsi can,
300
Simulation
Open Drawer
open top/middlc/hottom drawer pick coke can
top/middle/hottom drawer
300
Pick Coke Can
coke can top drawer, apple carrot, plate eggplant, yellow basket
300
Apple Drawer
open top drawer/place apple into top drawer put carrot on plate put eggplant into yellow basket the spoon the towel
300
Carrot Plate
300
Eggplant Basket
300
Spoon Towel
spoon, towel green cube, yellow cube
300
Stack Cube
stack the green block the yellow block
300
Pick Ball
pick the ball pick the plug
ball
20
Real-world
Pick Plug
plug
Move Near
move the can near the apple/waternelon
can, apple. watermelon
20
Cookies Towel
put the cookies on the towel
cookies, towel
20
Banana Plate
the banana in the plate
banana, plate
20
Stack Cube
stack the green eube the yellow cube
green cube, yellow eube
20
Table 4: Success rates (%) and 95% confidence interval of three baseline policies integrated w/ or w/o our PCD over tasks from Google Robot and WidowX calculated at the maximum step.
Open Lad Kim et al. 2024]
Octo
(M. et al. (0024]
To (Black et al. 2022
Model
Base
Point
Box
DINO
Base
Point
Box
DINO
Base
Point
Box
DINO
13.121.211
21.9
119
13.41a
14.0 14.0 2
62.751.
63.7 63.9
Average
+39.7%
+12.6%
+17.6%
+19.3%
+7.6%
Close Drawer
31333.66..22.363377.33
28.0
24.324.
28.3
20.32
75.02
Move Near
47.3=3.048.0e5.
50.7=5.741.3
2.7+1.8
5.0=2.5
4.7+2.4
8.0
60.0
36.0+a.65.0=-5.
62.00...........
Google
Robot
Open Drawer
0.33..77
0.3g0.7
0.0.000
1.0
34.7
42.
Pick Cake
10..3
81..
Apple Drawer
0.010.0
0.31g.7
1.0g1.1
0.7.ma
0.0...o
0.0.0.0
0.0.0.0
0.0
26.
Carroc
0.020.0
3.011.0
0.0g0.0
2.721.8
9.0aa.2
12.7aa. 11.7.3.
12.0
62.
Eggplant Basket
0.340.7
5.0+2.
1.7414
3.7=2.1
37.0=5.:2..34..3
33.34
80.
WidX.
Spoon Towel
0.020.0
1.74144
0.320.7
3.0
7.0an.9
9.0ga.a
13.7 9.0
79.3
Stack Cube
0.0ano
0.010.
0.010.0
2.7.1.a
2.3
49.0
60.3 4
superiority of the proposed PCD method compellingly evidenced by the consistent performance gains over baseline policies presented in Table and Table
COMPUTATIONAL OVERHEAD
Table reports the computational overhead three baseline policies integrated w/ or w/o our PCD method in the SIMPLER environment. 1) Inference Latency. PCD approximately doubles the
17

---

Published as a conference paper at ICLR 2026
Table Computational overhead of three baseline policies integrated w/ w/o our PCD method in the SIMPLER environment.
Model
OpenV
T
Base
+PCD
Base
+PCI)
Base
+PCD
Average time cost for each infer. step (s)
0.86
1.77
0.21
0.39
0.66
1.09
Memory cost (MB)
16357
16869
2884
3528
8535
11699
inference latency per step across all three policies. This is an expected trade-off, as PCD requires a second forward pass the object-masked input compute the contrastive signal. For Octo and To- although the KDE-PM process introduces additional inference time, the overall time overhead is still kept to roughly twofold increase due to their parallel processing of the original and object- masked images. Crucially, this increased inference time does not significantly affect the total task completion time in real-world scenarios, given the much longer duration of the robot's physical execution. As proven by Figure 3] PCD only brings 24% extra execution time on VLe 2) Memory Cost. The impact on memory cost varies based on the policy's 's architecture. For OpenVLA, which processes the original and masked inputs serially, the memory overhead is negligible, with the primary cost being the increased sequential processing time. contrast, the memory costs of Octo and mo are 1. 1.22x and 1.37x x that of the baseline respectively, due to their parallel data processing mechanisms.
VLA.
F PCD vs. CLASSIFIER-FREB GUIDANCE (CFG)
The proposed PCD algorithm shares similarity with the training- freee approach classifier-ff free guidance (CFG) (Ho and Salimans 2022), where the masked image serves as the unconditional inputs. However, the essential distinction lies in how and when the guidance is applied: CFG provides implicit guidance during the iterative decoding process. At each step of diffusion, it steers the generation away from the unconditional prediction. This is implicit probabilistic modeling process. In contrast, PCD an explicit post-hoc correction. The quantitative comparison shown in Table that our PCD method significantly outperforms CFG across the nine SIMPLE tasks. However, the time cost of PCD is 1 1.29 times that CFG. Therefore, harnessing the complementary strengths of PCD and CFG to further address spurious correlations remains promising avenue for future research.
Table 6: Comparison of PCD and Mlassffir-FFre Guidance (CFG)
(Ho and Salimans
2022
Model
To
+CFG
+PCD
Average time cost for each inference step (seconds)
0.66
0.84
1.09
Average success rate nine tasks (%)
63.9
62.7
68.1
Close Drawer
75.7
75.7
74.7
Move Near
67.3
62.7
66.7
Open Drawer
38.0
46.3
47.7
Pick Coke
84.0
81.3
84.3
Apple Drawer
17.0
13.0
26.0
Carrot Plate
58.0
59.0
67.7
Egg. Basket
86.0
87.3
83.7
Spoon Towel
80.7
78.3
86.3
Stack Cube
68.7
60.3
76.3
To further investigate CFG, we conduct step-by-step analysis in Table 7 applying CFG after certain step and running 100 trials for each setting. The results show clear trend: the earlier the CFG intervention, the more severe the performance degradation. The primary reason appears to be that CFG's influence on the noise space disrupts the semantic integrity of the model's reasoning process, ultimately degrading performance.
Table 7: Performance of CFG under different starting steps.
Starting step
Success Rate (%)
0
62.7
2
63.8
4
64.8
8
64.5
18

---

Published as a conference paper at ICLR 2026
a) Object Detection Failure
b) Incomplete Object Masking
Input
Result
move the sponge near up can
place the spoon on the towel
close top drawer
put the carrot on the plate
Figure 9: Failure cases of the Track2Mask module.
G FAILURE CASES OF TRACK2MASK
From the Track2Mask pipeline presented Appendix Fig the failure cases fall into two categories: a) Object Detection Failure-the the off-the-shelf open-vocabulary detector (i.e., GDINO) fails to localize objects in the initial observation; b) Incomplete Object Masking- the target objects along the trajectory are partially masked, as shown in Fig. ?
According to Equation 2, when the first failure case occurs, PCD's prediction becomes equivalent the original baseline result.
We conduct a ablation study to assess the impact of the incomplete masking failure cases in Table 8 where 3 indicates the ratio of masked pixels manually excluded. As can be seen, PCD's performance progressively declines as 3 increases, until it approaches the baseline performance when / reaches 60%. Nevertheless, both kinds of failure cases are exceptionally rare in our experiments, owing to the stability and effectiveness of the employed off-the-shell models, i... GDINO for object detection and SAM v2 for object tracking and segmentation.
Table 8: Performance of PCD under different ratios (3) of masked pixels manually excluded. The task is "pick coke can".
Model
Baseline
+PCD (3
8
0.2
0.4
0.6
VL.
25
40
32
28
27
84
88
88
85
84
H APPLICABILITY TO MORE COMPLEX TASKS
In our experiments on multi-objects tasks, PCD masks all task-relevant objects simultaneously. Here, we perform new ablation study using the "Move Near" task to validate this strategy. Specifically, for task "Move A Near B", we compare the results of i) masking only object A, ii) masking only object B. iii) masking A first, followed by masking upon a successful grasp of A; and iv) Masking both and B. The results shown in Table g
where the baseline model is VLe As can be observed, masking all task specific objects simultancously yields the best performance.
Table Ablation study on masking strategies for multi-object tasks.
Model
Task
Mask A
Mask B
Mask A then B
Mask A and B
OLe
Move Near
60
50
37
62
No
Carrot Plate
67
Our PCD method has been evaluated on the long-horizon task "Apple Drawer" in the SIMPLER environment. As illustrated Appendix Table 3, the official language instruction the task is composed of two sequential sub-tasks: "open the top drawer" and "place the apple into the top drawer". Our strategy is to mask objects based on the instruction of the current sub-task. For sub-task mask only the "top drawer"; for sub-task 2, mask both the "apple" and the "top drawer". In other words, PCD holds potential for long-horizon horizon tasks by leveraging an LLM- planner with CoT reasoning decompose abstract commands into concrete sub-tasks.
19

---

Published as a conference paper at ICLR 2026
I PERFORMANCE GAPS AMONG DIFFERENT OBJECT ANNOTATION STRATEGIES
The observed performance gaps among different object annotation strategies in Table arises from the distinct "preferences" inherent in each approach, For instance, prompts from humans versus those from object detection models exhibit different understandings of object presence. As illustrated in Fig 10,, in the case of a "drawer", a human annotator tends to point to or box a specific drawer within cabinet, whereas an object detection model typically bounds the entire cabinet containing all drawers. This leads to significant performance disparities on "drawer" dtrawer--related tasks. We argue that no single object annotation strategy is universally optimal; rather, the effective approach contingent upon the specific task.
Human Point
Human Box
GDINO
Object
Annotaton
Strategies
Object
Masking
Results
Figure 10: Object masking results of Track2Mask using different object annotation strageties.
J ROBUSTNESS UNDER VARIOUS MAGNITUDES OF DISTRACTORS
Here, we investigate the robustness incrementally increasing the number of distractors in the image background of the "pick coke can" task. The obtained results confirm that PCD maintains superior performance and exhibits greater robustness against these disturbances compared to the baseline.
PCD method under various magnitudes of distractors by
Model
Original
Few distractors
Many distractors
OpenVLA
25
14
+PCD
40
39
36
Table 10: Robustness PCD to various magnitudes of distractors presented in Fig
pick coke can
Original Observation
Few Distractors
Many Distractors
Figure Various magnitudes of distractors.
K PERFORMANCE IN MULTI-PERPPECIVE SCENARIOS
In this part, we conduct an experiment to investigate PCD's effectiveness in multi pulti-perspeectve scenarios. Specifically, we leverage the miniVLA+VQ h8itrist (Belkhale and Sadigh 2024)
20

---

Published as a conference paper at ICLR 2026
Balicy-whichh incorporates both third-person perspective and wrist images input our baseline.
The achieved results on five randomly sampled tasks from the LIBERO-90 (Liu et al. (023a)
benchmark are shown in Table II demonstrating that PCD consistently improves the baseline policy across all five tasks.
LIBER-90 Task
mini miniVLA+ VQ WriWtist
+PCD
put the butter at the
the top drawer cabinet and close
put the black bowl on top of the cabinet
98
pick up the ketchup and put it in the basket
72
84
the white mug the plate
86
88
pick up the book and place in the front compartment of the caddy
46
54
Average
71.2
79.6
Table 11: Performance of PCD on five LIBER-90 Tasks.
L LLMS USAGE STATEMENT
In the preparation of this paper, employed Large Language Models (LLMs) solely writing assistance tool for limited text polishing and language refinement. LLMs were not involved in any aspects of research ideation, conceptual development, technical analysis, algorithm design, experi- mental execution, or result interpretation. All scientific contributions, methodological innovations, and intellectual content remain entirely our own.
21


Reviews:
[{"review_id": "B9Dwt9gDw9", "reviewer": "Reviewer_ReGE", "summary": "The paper proposes Policy Contrastive Decoding (PCD), a training-free method that improves generalist robot policies at inference time by comparing the action distributions from the original observation with those from an object-masked observation. The object mask is obtained using a Track2Mask pipeline. The authors also introduce a KDE-based method to estimate action probabilities for diffusion policies. Experiments show that PCD improves OpenVLA, Octo, and Pi0 on SIMPLER simulation tasks, and Pi0 on a real-world pick-and-place task.", "questions": "- At a high level, the method seems to be trying a similar idea as classifier-free guidance (CFG) in the diffusion model literature. Can the authors discuss more about this relation? In addition, can the authors try evaluating the diffusion policy with CFG instead of PCD + KDE-PM?  Since there is already the original observation $o$ and the object-masked observation $\\hat{o}$, we can simply try a CFG-based action sampling by predicting the noise with a weighted combination of the two predictions\n```\neps_uncond = model(x_t, t, \\hat{o})\neps_cond   = model(x_t, t, o)\neps = eps_uncond + w * (eps_cond - eps_uncond)\n```\n- The paper mainly studies the use case for generalist policies, but will this method also work on smaller single-task policies trained on limited data? It might be interesting to train a smaller diffusion policy on the real-world dataset used in the experiments and test it out.\n- Why does the performance differ so much between each object annotation strategy? What are the failure cases of Track2Mask?\n\n\nI’d be happy to reconsider my score once these points are addressed.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The method is a training-free approach that can be applied to various generalist policies in a plug-and-play manner.\n- The evaluation on both SIMPLER and real-world tasks shows improvement over the base policy.", "weaknesses": "- The method is mainly limited to simple pick-and-place tasks with a clear target object to mask, and it is hard to extend to more complex tasks that need long-horizon planning or involve multiple objects.\n\n- The method introduces additional computation and latency, both from running Track2Mask and from the KDE-PM process. This is discussed in Appendix A.5. It may not be a major issue for the pick-and-place tasks studied in this paper, but it could be a limitation for tasks that require fast inference.\n\n- In Table 1, the performance seems to vary depending on the object annotation strategy, and it is a sensitive design choice for each task and base policy. This requires the user to perform additional tuning to find the best setting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zjghUZgCCC", "reviewer": "Reviewer_DdfD", "summary": "This paper proposes \"Policy Contrastive Decoding\" (PCD), a novel method to address the issue of \"spurious correlations\" in robotic foundation models, where policies incorrectly learn from task-irrelevant features like background or lighting instead of the target objects. PCD is a \"training-free\" and \"plug-and-play\" approach that works by contrasting the action probability distribution from the original visual input with a distribution derived from an \"object-masked\" version of the same input. This process, which requires no model retraining, amplifies the policy's focus on object-relevant features. The method is designed to be flexible, using a \"Tracking-to-Mask\" (Track2Mask) strategy to identify and mask objects automatically, and a \"KDE-based Probabilistic Modeling\" (KDE-PM) scheme to make it compatible with both autoregressive (like OpenVLA) and diffusion-based (like Octo and $\\pi_0$) policies. Experiments in simulation and the real world show that PCD significantly improves the success rates of these state-of-the-art policies, enhancing the $\\pi_0$ policy by 8.9% in simulation and 108% in the real world.", "questions": "The proposed Policy Contrastive Decoding (PCD) method, which contrasts outputs from original ($o_i$) and object-masked ($\\hat{o}_i$) observations, bears a strong conceptual resemblance to Classifier-Free Guidance (CFG) in diffusion models. In CFG, guidance is achieved by steering the generation process away from a null or unconditional prompt and toward a conditional prompt.\n\nGiven this parallel, have the authors considered implementing a more direct CFG-based approach for the diffusion-based policies (Octo and $\\pi_0$)? Specifically, one could treat the object-masked input ${\\hat{o_i}}$ as the \"negative condition\" (analogous to the unconditional prompt in standard CFG) and the original observation $o_i$ as the positive condition. The denoising network $\\epsilon_{\\theta}$ could then be guided using the standard CFG formulation, such as $\\hat{\\epsilon_{\\theta}} = \\epsilon_{\\theta}(a_i^k, e_i, k) + w(\\epsilon_{\\theta}(a_i^k, e_i, k) - \\epsilon_{\\theta}(a_i^k, \\hat{e}_i, k))$, where $\\hat{e}_i$ is the embedding from the object-masked input and $w$ is a guidance scale.\n\nThis CFG-based approach would be a more native and direct way to apply guidance within the diffusion framework, potentially obviating the need for the proposed KDE-based Probabilistic Modeling (KDE-PM) scheme. Could the authors comment on whether they explored this alternative? If so, how did its performance and computational efficiency compare to the proposed PCD method? If not, do they foresee any challenges with such an implementation?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- The paper identifies and addresses a critical, known weakness in robotic foundation models: their tendency to learn \"spurious correlations\" (e.g., relying on background or lighting) rather than the actual task-relevant objects, which hurts generalization.\n\n- The proposed \"Policy Contrastive Decoding\" (PCD) is a major strength because it is a \"plug-and-play\" solution. It can be applied to existing, pre-trained policies without requiring any costly retraining, fine-tuning, or access to the model's internal weights.\n\n- The method is not designed for just one type of model. It introduces a \"KDE-based Probabilistic Modeling\" (KDE-PM) scheme that allows it to work with *both* autoregressive policies (like OpenVLA) and diffusion-based policies (like Octo and $\\pi_0$).\n\n- PCD demonstrates significant performance gains on top of three different state-of-the-art policies. It achieves impressive improvements, boosting the strong $\\pi_0$ baseline by 8.9% in simulation and, most notably, by 108% in real-world tasks.\n\n- The paper includes extensive experiments to prove its claims, and it is open-sourced.", "weaknesses": "- The method adds significant computational overhead and latency at inference time. The paper notes it \"approximately doubles the inference latency per step\" and adds a 24% total time cost to real-world tasks. This may influence the performance on highly dynamic tasks,\n\n- The method introduces a new, highly sensitive hyperparameter, $\\alpha$, that controls the strength of the correction. Ablation studies show that the best $\\alpha$ is different for each model (e.g., 0.2 for $\\pi_0$ but 1.0 for Octo), meaning it requires careful, model-specific tuning, which undermines the \"plug-and-play\" claim.\n\n- The authors admit the method is a patch at inference time, not a fix. It only addresses the symptoms of spurious correlations during testing and does nothing to prevent the model from learning them during training in the first place.\n\n- The real-world experiments, while showing a large relative gain (108%), were conducted on a $\\pi_0$ model that was already fine-tuned on 10 demonstrations for each specific task. This makes it unclear how PCD would perform on a general-purpose model in a true zero-shot, real-world setting. Also, the performance gain of 108% in real-world compared to the 8.9% in the simulation is very large, which seems unusual.\n\n- Even with PCD, the absolute success rates in many challenging real-world tasks remain low (e.g., \"Stack Cube\" improves from only 5% to 10%), suggesting it is not a complete solution for robust, real-world generalization.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hVS5ocglRR", "reviewer": "Reviewer_gdDA", "summary": "This paper proposes Policy Contrastive Decoding (PCD), a training-free plugin for pretrained robotic foundation models. It aims to redirect pre-trained robotic foundation models away from spurious visual correlations toward object-relevant cues. The key idea is: given a visual observation and language instruction, the robot policy is run twice on the original image and on a version where task-relevant objects are masked. It can be applied to both autoregressive and diffusion-based policies, showing its versatility.", "questions": "1. What is the value of $\\alpha$ in the main simulation experiment? Is it fixed or adapted to the VLAs?\n\n2. For diffusion-based policies with KDE-PM, why do the authors choose 24 for experiments? Do authors have any time/memory trade-offs for this method compared to standard inference?\n\n3. When the PCD is applied to VLAs, the success rate is improved but can not reach 100%, nor even 80% in most tasks. Why does it fail? Is it because PCD can not fully solve the spurious correlations problem? \n\n4. How does the performance of PCD degrade when object detection or masking fails (e.g., false negatives, multiple objects, occlusion)? Can authors provide results showing robustness to mask quality or missing objects?\n\n5. It seems that all the experiments were completed from a single perspective. How about multiple perspectives?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The method does not need retraining or fine-tuning of a large robot policy, showing practicality in robotics, as VLA training is expensive.\n\n2. The problem that the paper tries to solve is important: reliance on spurious visual correlations leading to poor generalization when distribution shifts happen.\n\n3. The authors evaluate across multiple policies (i.e., autoregressive and diffusion-based) and in both simulation and real-world robot settings. They also include ablations, which help to analyze design choices.", "weaknesses": "1. It lacks some analysis for hyperparameters.\n\n2. It does not include training and implementation details.\n\n3. The paper defines “spurious correlations” formally (Definition 3.1) but then moves to a generic contrastive decoding mechanism without carefully clarifying which types of spurious correlations are mitigated and which are not. For instance, masking the object may remove task-relevant cues if the object is part of the visual context, yet the method tacitly assumes that the object is the only relevant cue and everything else is spurious. This assumption is too strong for many robotics settings.\n\n4. The ablations, while present, focus mainly on hyperparameter $\\alpha$, object detection model choice, and inpainting method. There is little ablation on the magnitude of spurious correlation removal, or how performance degrades when masking fails.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)