Contrastive Imitation Learning for Language-guided Multi-Task Robotic Manipulation
Authors: Teli Ma
Jiaming Zhou
Zifan Wang
Ronghe Qiu
Junwei Liang
Venue: CoRL 2024 (2024)
Keywords: Contrastive imitation learning
Multi-task learning
Robotic manipulation

Abstract: Developing robots capable of executing various manipulation tasks, guided by natural language instructions and visual observations of intricate real-world environments, remains a significant challenge in robotics. Such robot agents need to understand linguistic commands and  distinguish between the requirements of different tasks. In this work, we present $\mathtt{\Sigma\mbox{-}agent}$, an end-to-end imitation learning agent for multi-task robotic manipulation. $\mathtt{\Sigma\mbox{-}agent}$ incorporates contrastive Imitation Learning (contrastive IL) modules to strengthen vision-language and current-future representations. An effective and efficient multi-view querying Transformer (MVQ-Former) for aggregating representative semantic information is introduced. $\mathtt{\Sigma\mbox{-}agent}$ shows substantial improvement over state-of-the-art methods under diverse settings in 18 RLBench tasks, surpassing RVT by an average of 5.2% and 5.9% in 10 and 100 demonstration training, respectively. $\mathtt{\Sigma\mbox{-}agent}$ also achieves 62% success rate with a single policy in 5 real-world manipulation tasks. The code will be released upon acceptance.

Contrastive Imitation Learning for Language-guided Multi-Task Robotic Manipulation
Teli Ma
Jiaming Zhou' Zifan Wang! Ronghe Qiu' Junwei Liangh.
Thrust, The Hong Kong University of Science and Technology (Guangzhou) CSE Department, The Hong Kong University of Science and Technology
E-agent.com com
Abstract: Developing robots capable of executing various manipulation tasks, guided by natural language instructions and visual observations of intricate real- world environments, remains a significant challenge in robotics. Such robot agents need to understand linguistic commands and distinguish between the requirements of different tasks. In this work, we present E-agent, end-to-end imitation learning agent for multi-task robotic manipulation. E-agent incorporates con- trastive Imitation Learning (contrastive IL) modules to strengthen lision-language and current-future representations. An effective and efficient multi-view query- ing Transformer (MVQ- for aggregating representative semantic informa- tion is introduced. E-agent shows substantial improvement over state-of-the-art methods under diverse settings 18 RLBench tasks, surpassing RVT [1] by an average of 5.2% and 5.9% in 10 and 100 demonstration training, respectively. E-agent also achieves 62% success rate with single policy in 5 real-world ma- nipulation tasks.
Keywords: Contrastive Imitation Learning, Multi-task learning, Robotic Manip- ulation
Introduction
One of the ultimate goals of robotic manipulation learning is enable robots to perform variety of tasks based instructions given by humans in natural language. This requires robots to understand and distinguish minor variations in linguistic commands and visual cues. However, training robots is difficult due to the limited rewards available in simulated environments and the lack of extensive real-world data. Imitation learning is an effective off-policy method that avoids complex reward design and low-efficient agent-environment interactions [2, 3, 4, 1]. In this paper, we focus on imitation learning for 3D object manipulation.
Previous works have mainly concentrated improving the perception ability of robotic agents, but ignoring the development of discriminating different instructions and related tasks. A portion of these studies has been directed towards enhancing the transferability from 2D pre-trained visual representation to the real-world [2. 7, 8, 9]. Nonetheless, to maintain geometric details for both simulated and real-world environments, 3D vision learning dominates
custruction-guidee manip- ulation [4, 3, 10, 1, 11, 12, 13, 14]. For instance, C2FARM [4] leveraged 3D ConvNets to aggregate visual representations based pre-constructed voxel space. PerAct [3] constructed voxelized obser- vations and discrete action space RGB-D images, utilizing Percevier [15] Transformer to encode features. Moreover, PolarNet [11] directly encoded the point cloud features reconstructed from RGB-D to predict actions. However, these works do not address how to train visual representations to align with linguistic features and differentiate between multiple tasks.
Previous methods [2, 3, 16, 11. 1, 12, 13, 14] can be summarized training the agent to learn parameterized policy me that imitates the target policy n based data collected using the target
Corresponding author.
8th Conference Robot Learning (CoRL. 2024), Munich, Germany.

---

wO
Dontrnatte
Yellow Mug
Lemon
Tennia
Barrel
Figure 1: Left: t-SNE [5] visualization of multi-task representation learning without/withh con- trastive II.,, and learning with contrastive II shows much more obvious separation of features belonging to different tasks. Right: Visualize the interested regions with Grad-CAM [6]. which shows accurate object-level understanding.
policy via behavior cloning (BC) loss. These approaches typically involve mapping visual rep- resentations o, linguistic representations ei and vision-language interactions d to low-level end- effector actions. To tackle the aforementioned misalignment problem, we introduce an end-to-end contrastive Imitation Learning (contrastive IL) strategy to the original language-conddtionee conditioned policy learning, inspired by contrastive Reinforcement Learning (contrastive RL) methods [17, 18, 19, 20). The contrastive framework strengthens the training process by establishing: 1) relationship be- tween language instructions and corresponding visual inputs, which helps differentiate multi-task representations, and 2) link between current and future states, which enhances the recognition of movement trajectories across different tasks. This contrastive loss provides additional supervision to the feature extraction e and interaction 8 components (as shown in Fig. 2 (b)), complementing the standard BC loss.
Based the contrastive II, present end-to-end trained language-conditioned multi-task agent to complete 6-DoF manipulation, dubbed contraStive Imitation learninG for Multi-iAsk ma- nipulation agent (SIGMA-agent, abbreviated as E-agent ). E-agent follows the state-of-the-art baseline model, RVT [1], and leverages the re-rendered virtual images from RGB-D reconstruc- tion to explicitly represent visual information. We propose a Multi-Vien Querying Transformer (MVQ-Former) [21, 22, 23] to minimize the number of tokens for efficient contrastive IL. The E-agent framework offers guidance on how to incorporate contrastive learning into existing imita- tion learning methods, while the inference process stays the same.
Experiments on RLBench [24] and real-world tasks demonstrate the effectiveness of the E-agent. The results on RLBench show E-agent significantly outperforms previous agents in both the 10 demonstrations (by +5.2% on average) and the 100 demonstrations (by +5.9% on average) training under the setting one policy for 18 tasks with 249 variations. Also, we integrate
contrastive IL module into existing methods (PolarNet [11], RVT [1]) and experiment E-agent on another simulation environment (Ravens [25]). The significant improvements show the general applicability of our approach across various models and environments. E-agent also achieves 62% multi-task success rate on average with single policy over tasks in real-world robot experiments.
Related Work
2.1 Language-conditioned Robotic Manipulation
Language-conditionn robotic manipulation has emerged as pivotal research branch the robotics
domain due to extensive applicability in human-robot interaction [26, 2, 3, 1, 11, 16, 12, 13, 14, 27, 28, 29, 30). Many previous studies have delved into vision-based representations and strategies for vision-language interaction in policy learning. For example, RT-1 [31] encodes multi-modal tokens via pre-trained FiLM EfficientNet model and feeds them into the Transformer for multi-

---

modal information aggregation. The later version RT-2 [32] leverages the auto-regressive generative capacity of L.MSs to project visual tokens into linguistic space, and uses L.Mss to generate the actions directly. Diverse benchmarks have been curated to benchmark the language-conditinee manipulation [24, 25, 33, 34, 35, 36). In this paper, we mainly focus on RLBench [24], which provides hundreds of challenging tasks and diverse variants covering object poses, shapes, colors, sizes, and categories to evaluate agents based on RGB-D cameras.
Numerous efforts have been made in this challenging benchmark. C2FARM [4], PerAct [3] and GN- Factor [14] harness the 3D voxel representation for policy learning. C2FARM [4] detects actions at two levels of voxelization in a coarsc-to-fing to-fine manner. PerAct utilizes the Perceiver network [15] to encode 3D voxel features to predict the next keyframe positions with lower voxel resolution than C2FARM [4]. To refine the 3D scene geometry understanding. GNFactor [14] incorporates a gen- eralized neural feature fields module to distill 2D) semantic features into NeRFs [37] based Per- Act [3]. Besides the voxelized features, policy learning based on 3D point cloud representation has gained significant attention such as PolarNet [11], Act3D [12] and ChainedDiffuser [13]. PolarNet trains agents on 3D point clouds constructed from RGB-D and adopts pre-trained PointNext [38] to extract point-wise features. Both Act3D [12] and ChainedDiffuser [13] employ a coarse-to-fine sampling strategy to select 3D points in space and feature them with relative spatial attention, while ChainedDiffuser [13] synthesize end-effector trajectories with a diffusion model. Our work follows RVT [1], which re-renders virtual view images from reconstructed 3D point clouds and processes the images using Transformer network.
2.2 Contrastive Learning in RL
A large volume of prior work combines representation learning objective with RL objective [39, 40, 41, 42, 43, 44]. Contrastive learning has gained significant attention among these representation learning methods [40, 41, 45, 46, 43, 47]. Recently the paradigm of unifying the representation learning and reinforcement learning objective has emerged a research hotspot in the field of RL [19, 45, 18, 48, 49, 50, 51]. For instance, C-learning [19] regards coal-conditioned RL as timating the probability density over future states, learning a classifier to distinguish the positive future state from the random states. Eysenbach et al. [17] demonstrate that contrastive representa- tion learning can be used as value function estimation, connecting the learned representations with reward maximization. Zheng et al. [18] propose to stabilize contrastive RL in offline goal-reachin tasks, analyzing the intrinsic mechanism of contrastive RL deeply to explore ingredients for stabiliz- ing offline policy learning. Note that the contrastive RL methods above mainly focus on reinforce- ment learning with reward updating. One of the most similar works to ours is GRIF [52], which learns linguistic representations that are aligned with the collected transitions in the trajectory via contrastive learning. However, our contrastive IL is different from GRIF [52] in three aspects. First, contrastive IL is an end-to-end training paradigm, while the GRIF [52] decouples the contrastive representation pre-training and policy learning into two phases. Second, we target the 3D multi-task setting in this paper, whereas GRIF [52] utilizes RGB images and single policy training. Lastly, GRIF [52] performs contrastive learning on (state, goal) pairs with language instructions. For con- trastive II., we perform contrastive learning to refine both the feature extraction and vision-language feature interaction. More related work is present in Appendix D.
3 Method
Fig. 2 provides an overview of the Contrastive IL design in E-agent (Fig. 2 (a)) and how to apply to the previous eagguage-guided imitation learning paradigm (Fig. 2 (b)). In this section, we introduce the components of E-agent. Additional details about E-agent are provided in Appendix C.
3.1 Preliminaries
We assume Language-conditions Markov decision process (MDP) defined by states st S, actions de A and language instructions
L. S,A are the state and action spaces, and L
3

---

language features
Langungh Encoder
drgg the cube onto the teall larger
Proj
Contractive Learning
Langunge
Current Stane
Re-Render
Visual Eneader
MyQ-foomer
Festure
w(
8(s.
Futura State
query
Contrastive Leariing
xydai ss,
Re-Render
Visual
Pool
Visual features
Figure 2: (a) The pipeline Contrastive IL in E-agent. (b) The overview of imitation learning for ingguage-conditione multi-tase manipulation, where representation o, v 8 and policy network are learned for policy тe to mimic target policy x². Red Dotted Line: The contrastive IL aim to refine visual representation (visual encoder) and joint vision language representation 6 (MVQ- Former and Feature Fusion).
represents the of language instructions. The goal is to learn policy S L A maximize the expected rewards of predicted actions. Following previous work [3], utilize behavior cloning to maximize the Q-function without specific rewards. Therefore, the objective of the policy learning can be formulated as:
0 arg max (l),,.))> log mo(aelse, !),
(1)
where 0 means parameters of the policy network and D represents the transitions from demonstra- tions collected for behavior cloning. Note that D is sampled from the expert policy n+, and we train the 0 to drive to mimic the + In this paper, the state includes aligned RGB and depth images from the front, left shoulder, right shoulder, and wrist position. We follow RVT [1] by adopting re-rendered virtual images from the RGB-D inputs to feed into the model. The action space A is composed of translation in Cartesian coordinates at atrans E IR3, rotation quaternion of IR4
gripper open of E (0, 1} and collision state af {0, 1}.
3.2 Visual and Language Encoder
We aequire re-rendered virtual images following RYT [1] from cubic viewpoints: the front, left, right, behind and top. Each view image comprises RGB, depth, and (a,, v) coordinates channels. The visual encoder comprises patch embedding layer and two-layer self-attention Transformer. We split the images into 20 x 20 patches and leverage MLP layer to project the embeddings of patch tokens for self-attention. For the self-attention Transformer, each patch token only attends to other tokens within the same virtual view image, which aims to aggregate the information from the same view. The visual encoder is trained from scratch with normalized initialization.
For the language encoder, follow previous works [3, 16, 11, 1, 12] and use pre-trained language encoder from CLIP [53] for fair comparison. The language encoder remains frozen during training. The language token embeddings from the encoder are then projected by trainable ML. for the cross-attention with visual tokens.
3.3 Multi-Vies Query Transformer (MVQ-Former)
With the extracted visual features from the visual encoder, we follow the query Transformer [23, 53, 21, 22] to pre-define set of learnable queries. These queries are utilized by the contrastive IL. module, minimizing the computational complexity by reducing the number of original visual tokens. The number of learnable queries set 5, one for each virtual view aggregate the intra-view information. The MVQ-Former in Fig. 2 consists of cross-attention layers, where the query tokens co-attend to the extracted visual features. We denote the queries produced by MVQ-Former as 4.

---

The query tokens, visual tokens and language tokens are then concatenated together and fed into self-attention layers for feature fusion. During this process, the queries interact sufficiently with both the visual and language features, resulting in queries at this stage named 4.2-- The q, and q are utilized for contrastive IL as described in the following section. Finally, the context features (denote as v) after the self-attention layers are imput to the decoder for action prediction.
3.4 Contrastive Imitation Learning
For multi-task agents conditioned by language instructions, we propose the key question: What are effective task representations for accurate, fine-grained controls? Two factors matter. First, aligning task and language representations is essential for the agent to comprehend correlations and differentiate between tasks. Second, the agent needs discriminative features to accurately perceive the current state.
Learning Objective. Inspired by contrastive RL [19, 17, 20, 18), we propose to integrate the con- trastive representation learning into current imitation learning framework to train the policy in an end-to-end manner. The contrastive II. comprises state-language (s <> I) and (state, language)- future (s, ++ g) contrastive learning to refine both the feature extraction and interaction as shown Fig. 2.
For the 1, we define the similarity function as fa.v (xps) (r: temperature parameter), where á and & are the state and language instruction representations, respectively. The contrastive IL objective between states and language instructions is:
(st
(st))($111)
Σ
log N
So.v(s 1t) + Ei-er- So
(2)
(uolllt,ot)
Σ log (1 sx))+ Enjes- fe,o(t,,87)
where N is the batch size in training, and 5, 1+ denotes corresponding positive states and instruc- tions. L and 5 stand for all negative language and state samples in the current batch. Essentially this enforces the network to learn more discriminative representations by aligning state and language pairs in the joint embedding space.
For the (s, 1 ++ g), we randomly sample future states y from D, and applying network & encode. 6 encodes the joint representation of (st,)) The negative samples from the other tasks in the same batch. Similar to Eqn. 2, we formulate the contrastive training loss between the future and state, language pairs as:
(st
((a.)))n (s:: (g)
N Σ log
[6 (t),tgt)) + Eg foyp(st ,1),gT)
(3)
fe.lg (st,,tttttt))
+ N
Σ N
log
Soslgt (6t,11) E fo (5r
The E(s.))eg aims to refine the representation of vision-langues interaction 8, which is crucial for the vision-language understanding of agents. G and f signify negative space of future states and (st, 1) in the current batch. Based on the Leest and ((a.))egg we re-formulate the learning objective in Eqn. as:
max 8
Lal [} (Alog To (ccl(ss))) 8t up (1lee.))
+ (1 ([=e)(C))((sss + (5
(4)
Ler.
where À is the coefficient. The positive future state g+ conforms to the transition probability P..* of the target policy . More discussions can found in Appendix C.2.
Module Details. For the contrastive training between language and state observations, choose the last token (i.c., [EOS] the feature representation of the whole text following CLIP [53], which
5

---

Table 1: Multi-task performance trained with 100 episodes and evaluated using 25 episodes per task on RLBench (5 times average). The input resolution 128 x 128. Train Time, the represents days and h represents hours. All other results are success rates, measured in percentage (%).
Avg Succ.
open drewer
alide block
sueep dustpan
put
drag stick
Time
grill
tap
Graver
jar
I-BCICNN) [2]
1.3
4,0
0
0
8.0
8.0
0
0
1-BCVVII) [2]
C2FARM [4]
16.0
20.0
16.0
2000
68.0
24.0
24.0
Hiveformer [16]
45.3
PolasNet
53.3
92.0
37.3
PerAct [3]
49.4
~16d
88.0+-5.7
74.0+13.0
$2.0+0.0
70.4+200
88.0-4444
51.2-4477
55.2+44.7
89.6+4-.11
RVT [1]
62.9
~23h
71.2:669
81.61:5
72.0+0.0
88 08.0
93.6=t4,1
88.0=5.7
52.0+2.55
99.2++16
P-agent
76.8113.
74.114555
97.62:199
95.2
78.412.9
100,010.0
stack
screv
put in
place
put in cupboard
sert shape
push buttona
insert
stack
place
blocks
safe
pog
cupa
cupa
1-CCCNN) [2]
0
4.0
0
0
LBC(VVT)) [2]
0
0
0
0
0
0
C2FARM
12.0
8.0
8.0
HiveFormer [16]
PolarNet [1]]
1.3
41.3
84.0
41.3
12.0
8.0
96.0
1.3
5.3
0
PerAct
26.4+3.2
17.6+2.0
84.0+-3.6
44.8+7.8
28.0+4.4
16.8+4.77
92.8+3.0
5.6+4,1
2.4-2.0
4.++2.5
RVT [1]
28.8-33.9
48.0=5.7
91,2433.0
91.01:522
49.6+13.2
36.0#2255
100=00.0
11.2=330
26.448.2
40::2.5
T-agent 51.2-554
732-222
98.4+199
90.4++3.5
66.414.3
3.2
100.0-0.0
15.2-29
33.6+6.77
0.+113
is then linearly projected into the mudti--oddl embedding space. The queries q, are projected along the token dimension to aggregate the representative visual features as shown in Fig. 2 (a). Then, the visual token and text token are trained align in the joint embedding space based on Eqn. 2.
We utilize the next state in the trajectory as the future state. The visual encoder of the future state shares the same backbone parameters with that of the current state shown in Fig. 2. Hence, the in Eqn. 3 equals to e in Eqn. 2. The queries 4r, which include both the visual features of the current state and the language features, are projected to perform contrastive training with average- pooled suture-state features. This process conforms to Eqn. Note that the contrastive module designed to enhance representations during training but disabled during the inference process.
Experiments
4.1 Simulation Experiments
Simulation Setup. RLBench is robot manipulation benchmark built CoppelaSim [54] and PyRep [55]. We follow the protocols of PerAct [3] and RVT [1] to test the model 18 tasks in RLBench [24]. These tasks, which include picking and placing, bulbs screwing, and drawer opening etc., are all performed by controlling a Franka Panda robot with parallel gripper. Details of the 18 tasks and their variations are provided in Appendix A.1. The input RGB-D observations are obtained from four RGB-D cameras at the front, left shoulder, right shoulder, and wrist positions. The input resolution 128 128 for experiments on RLBench unless otherwise specified.
Implementation Details. We follow RVT [1] to perform training on cube-viewed re-rendered images from 3D point clouds. We evaluate E-agent with and 100 demonstrations per task for training. Following previous work [24, 56, 3, 1], we perform behavior cloning on the replay buffer of extracted keyframes rather than all frames from episodes. Similar to PerAct [3]. we adopt translation and rotation data augmentations during training, perturbing the point clouds randomly in the range of +0.125m for translation and rotating the point clouds along the z-axis within +45º. For the training scheme, we train E-agent for 25K steps with a batch size of 96 and an initial learning rate 9.6 10 The LAMB [57] optimizer is applied, with cosine learning rate decay and 2K warming steps. The training is conducted on XXNVIDIA A6000 GPUs for around 22 hours. E-agent is evaluated on all 18 tasks with variants. The initial observations given to the agent, and the agent explores to reach the final state by the observation action loop. The agent scores
RLBench results evaluated on 100 episodes per task with 256 x 256 input resolution.
Table 2:
Avg Succ.
Train
Method
Time
Act3D [12] ChainedDiffuser [13]
65.1
~5.5d
66.1
~4.5
E-agent
68.4
~2hh

---

Table 3: Multi-task performance of integrating contrastive IL into baselines, E-PolarNet and E-RVT represents PolarNet [11] and RVT [1] model trained with proposed contrastive IL module.
Methods
Avg Succ.
Model Paren. (N)
open Grewer
slide block
sveep dustpan
neat off grill
put in draser
close jar
drag stick
tap
PolarNet (11)
44.7
14.1
81.3
53.3
$2.0
92.0
78.7
28.0
37.3
92.0
P-PolarNet
47.5
15.3
74.7
59.3
54.7
94.7
81.3
40.0
37.3
93.3
improvenent
+2.8
-6.6
+6.0
+2.7
-2.7
+2.6
+12.0
+0
+1.3
71.2
81.6
72.0
93.6
88.0
52.0
99.2
E-RVT
64.7
38.2
82.6
81.3
53.3
96.0
94.7
70.7
65.3
100.0
improvement
+11.4
-0.3
-18.7
+8.0
-17.3
+13.3
40.3
stack blocks
put in
place wine
put in cupboard
push buttons
insert
stack
place
bulb
safe
shape
peg
cups
cups
PolarNet [11]
84.0
41.3
12.0
96.0
1.3
0.0
E-PolaiNet
9,3
37,3
85,3
54.6
16.0
6.7
100.0
4,0
4,0
4.0
improvement
+1.3
+13.3
+4.0
-1.3
+4.0
+2.7
-1.3
+4.0
RVT
28.8
48.0
91.2
91.0
49.6
36.0
100.0
11.2
26.4
4,0
EVVV
92.0
93.3
69.3
100.0
21.3
12.0
inprovesent
+13.9
+1.0
+0.8
+2.3
+19.7
-2.7
+0
+10.1
-14.4
4.0
100 for reaching the final state and 0 for failures without partial credits. Following PerAct [3], we report the average success rates on 25 episodes for each task and the average success rates on all 18 tasks.
Comparison with the State-of-the-art Methods. We evaluate the E-agent five times on the same 25 episodes for each task and report the mean results due the randomness of the sampling-band -aaeed motion planner. The evaluation comprises two settings, training with 10 (Table 8) or 100 (Table 1) demonstrations per task. We re-implement the results of PolarNet (100 demos) and RVT (10 demos) since they are missing from the original paper. Other results are taken from the related literature. From Table and 8, E-agent outperforms previous methods on both the 10 and 100 demonstrations by large margin, up to 5.2% and 5.9% in average success rate over 18 tasks, respectively. In specific tasks, our E-agent achieves state-of-the-ee performance on 13 out of 18 tasks for both the 10 and 100 demonstration settings.
Moreover, we compare E-agent with two state-of-the-ard models, Act3D [12] and ChainedDif- fuser [13], which are trained on 256 x 256 input resolution and tested on 100 episodes for each task. As shown in Table 2, our E-agent surpasses the two methods by 3.3% and 2.3% respectively, with 5x less training time. Results of E-agent in other simulated environments are present in Appendix A.2.
Contrastive Imitation Learning for Baselines. To validate the effectiveness of contrastive II, we integrate the contrastive IL module into other baseline models. We choose PolarNet [11] and RVT [1] as the baselines to demonstrate improvements in representations of point clouds and 3D re-rendered images. The main network and inference pipeline are kept unchanged, with only a con- trastive II. module incorporated, adding a negligible increase in parameter count during the training process. As shown in Table 3, the contrastive IL module improves the performance of both Po- larNet [11] and RVT [1] by +2.8% and +1.8% in average success rate over 18 tasks, respectively. Specifically, the performance of most tasks is improved (13 out of 18 and 11 out of 18), with largest margin of 13.9% improvement. These improvements demonstrate two points: first, our proposed contrastive imitation learning can transfer across multiple models. Second, the learning method is effective for both 3D re-rendered images (RVT [1]) and point cloud representations (PolarNet [1 I]).
Future State and Language Ablations. We ablate the influence of future state and language con- trastive IL shown in Fig. 3 (a). We summarize three key points from the results. (1) Both language and future-state contrastive learning with current observations enhance performance. (2) Contrastive learning between current observations and language instructions accelerates the convergence speed of agent training. achieving higher performance at the early stage of training. (3) When future-state contrastive learning is added to a model that already includes language-star contrastive learning, the performance achieves a more substantial gain. We believe that the limited improvement only utilizing future-state contrastive learning can be attributed to the insufficient demonstrations as the trajectory is the unit for constructing contrastive learning categories. Consequently, without lever- aging language features, it is challenging to learn discriminative features.
7

---

b--4
Immb
MDo language language
05:00
bpm3
Immbsa003
ts=10
lmmbdaw0.7
bom12
lambdam0.
2500
10020
15000
20000
25000
10203
15000
20000
25000
Taining Stops
Training Steps
Training Stops
Figure 3: Ablation experiments. (a). The success rate of E-agent ablating language & future-state contrastive learning with current observations. (b). The success rate of E-agent ablating batch size of contrastive IL. (c). The success rate of E-agent ablating different X.
Batch-size Influence in Contrastive IL. Contrastive training is sensitive to the scale of batch size [58, 53]. We vary the batch size as shown in Fig. 3 (b). It can be concluded that scaling up batch size plays a crucial role boosting the agent's performance as can include more negative samples.
Coefficient À Ablations. In Eqn. 4, A is the hyper-parameter that conditions the relation between contrastive IL and behavior cloning. We vary the À ranging from [0. 1] to find the optimal value. From Fig. 3 (c), we train the agent with five different À values and observe makes slightly different results. We choose = 0.5 for agent training as it is the relatively optimal value.
4.2 Real-world Experiments
We conduct experiments on a real robot, 6-DoF UR5 robotic arm. The E-agent is validated on 5 real- world tasks, including a total of variants. For each task, we collect 10 human demonstrations and train the E-agent with a single policy from scratch using all task demonstrations. Details of the real-world setup and data- collecting are provided in Appendix B. Table 4 presents the real-wor world results. We test the E-agent in 10 episodes for each task, and it achieves an average success rate of 62% across all tasks. We show the failure cases in Fig. 8. To analyze the reasons for failure: first, the limitation a single front-view camera cannot pro- vide precise visual information for tasks that require aiming, such as "Put tennis in barrel" and "Stack cups". Second, during the grasping process, imperfect grasp poses result in the translation orientation of objects, worsening the collision problems. In the future, plan to add extra RGB-D camera at the wrist position to provide first-person view. Additionally, integrating a pose estimation model for objects into E-agent would improve grasping poses and avoid collisions.
Table 4: Performance of E-agent on real- worl-world tasks.
Task
Succ. %
Stack cups
60
Put fruit in plate
90
Hang mug
Put item in barrel
50
Put tennis mug
50
Average
62
5 Conclusions and Limitations
In this work, we propose contrastive IL, a plug pllug-and-pl ald- imitation learning strategy for language- guided multi-task 3D object manipulation. The contrastive IL optimizes the original imitation learn- ing framework by integrating the contrastive IL module refine both the feature extracting and in- teracting. Based on the contrastive II., we design an end-to-end imitation learning agent E-agent uti- lizing the re-rendered virtual images from RGB-D input. E-agent is effective and efficient in both simulated environments and real-world experiments. However, we identify some limitations that exist. First, the scale and the number of camera views of our real-world experiments are limited. Second, the contrastive II. module in our method only demonstrates the effectiveness of closed-set scenarios like RLBench, but does not show the applicability in open-vocabulary scenarios. Lastly, like previous work, the generalization capacity of E-agent is limited, as shown in Table 7. We leave these issues for future work.
8

---

Acknowledgments
This work was supported by the Meituan Academy of Robotics Shenzhen and Guangzhou- HKUST(GZZ) Joint Funding Program (Grant No.2023A03300088, Education Bureau of Guangzhou Municipalitty.
References
[1] A. Goyal, J. Xu, Y. Guo, V. Blukis, Y.-.. Chao, and D. Fox. Rvt: Robotic view transformer for 3d object manipulation. arXiv preprint arXiv:2306. 10830066649666 /4896, 2023.
[2] E. Jang, A. Irpan, M. Khansari, D. Kappler, Ebert, C. Lynch, S. Levine, and C. Finn. BC-z: Zero-shot task generalization with robotic imitation learning. In 5th Annual Conference on Robot Learning, 2021. URL https:
[3] M. Shridhar, L. Manuelli, and D. Fox. Perceiver-Actor: multi-task transformer for robotic manipulation. In CoRL, 2022.
[4] James, Wada, T. Laidlow, and A. Davison. Coarse-to-fine q-attention: Efficient learning for visual robotic manipulation via discretisation. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13739-13748, 2022.
[5] L. Van der Maaten and G. Hinton. Visualizing data using t-sne. Journal of machine learning research, 911)). 2008.
[6] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pages 1188-26,111 2017.
[7] S. Nair, A. Rajeswaran, V. Kumar, C. Finn, and A. Gupta. R3m: A universal visual represen- tation for robot manipulation. arXiv preprint arXiv:2203. 2022.
[8] Y. J. Ma. W. Liang, V. Som, V. Kumar, A. Zhang, O. Bastani, and D. Jayaraman. Liv: Language-image representations and rewards for robotic control. arXiv preprint arXiv:2306.00958, 2023.
[9] Y. Ma. S. Sodhani, D. Jayaraman, O. Bastani, V. Kumar, and A. Zhang. Vip: Towards universal visual reward and representation via value- implicit pre-training. arXiv preprint 2022.
[10] T. Ma, M. Wang, J. Xiao, H. Wu, and Y. Liu. Synchronize feature extracting and matching: A single branch framework for 3d object tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9953-9963, 2023.
[11] S. Chen, R. Garcia, C. Schmid, and Laptev. Polamet: 3d point clouds for language-guided robotic manipulation. arXiv preprint arXiv:2309.15596 2023.
[12] T. Gervet, Z. Xian, N. Gkanatsios, and K. Fragkiadaki. Act3d: Infinite resolution action detec- tion transformer for robotic manipulation. arXiv preprint arXiv:2306. 17817,777,,,.. 2023.
[13] Z. Xian, Gkanatsios, Gervet, T.W. Ke, and K. Fragkiadaki. Chaineddiffuser: Unifying trajectory diffusion and keypose prediction for robotic manipulation. In Conference on Robot Learning, pages 2323-2339. PMLR, 2023.
[14] Y. Ze, G. Yan, Y.H.. Wu, A. Macaluso, Ge. J. Ye, N. Hansen, L. E. Li, and X. Wang. Gnfactor: Multi-task real robot learning with generalizable neural feature fields. In Conference on Robot Learning, pages 284-301. PMLR, 2023,

---

[15] A. Jaegle, F. Gimeno, A. Brock, O. Vinyals, A. Zisserman, and J. Carreira. Perceiver: General perception with iterative attention. In International conference on machine learning, pages 4651-4664. PMLR, 2021.
[16] P.-L. Guhur, Chen, R. G. Pinel, M. Tapaswi, I. Laptev, and C. Schmid. Instruction-driven history-aware policies for robotic manipulations. In 6th Annual Conference on Robot Learning. 2022.
[17] B. Eysenbach, T. Zhang, S. Levine, and R. R. Salakhutdinov. Contrastive learning as goal- conditioned reinforcement learning. Advances in Neural Information Processing Systems, 35: 35603-35620, 2022.
[18] C. Zheng, B. Eysenbach, H. Walke, P. Yin, K. Fang, R. Salakhutdinov, and S. Levine. Stabi- lizing contrastive rl: Techniques for offline goal reaching. arXiv preprint arXiv:2306.03344 2023.
[19] B. Eysenbach, R. Salakhutdinov, and S. Levine. C-learning: Learning to achieve goals via recursive classification. In International Conference on Learning Representations, 2020.
[20] R. Yang, Y. Lu, W. Li, H. Sun, M. Fang, Y. Du, X. Li, L. Han, and C. Zhang. Rethinking goal- conditioned supervised learning and its connection to offline rl. In International Conference on Learning Representations, 2021.
[21] J. Li, D. Li, C. Xiong, and S. Hoi. Blip: Bootstrapping language-image pre-training for uni- fied vision- languae understanding and generation. In International Conference on Machine Learning, pages 12888-12900. PMI.,, 2022.
[22] J. Li, D. Li, S. Savarese, and S. Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.1255/ 2023.
[23] N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision, pages 213- 229. Springer, 2020.
[24] S. James, 7. Ma, D. R. Arrojo, and A. Davison. Rlbench: The robot learning benchmark & learning environment. IEEE Robotics and Automation Letters, 5(2):3019-3026, 2020.
[25] A. Zeng, P. Florence, J. Tompson, S. Welker, J. Chien, M. Attarian, T. Armstrong, I. Krasin,
D. Duong, V. Sindhwani, et al. Transporter networks: Rearranging the visual world for robotic manipulation. In Conference on Robot Learning, pages 726-747. PMLR, 2021.
[26] C. Lynch and P. Sermanet. Language conditioned imitation learning over unstructured data. arXiv preprint arXiv:2005.07648, 2020.
[27] M. Shridhar, L. Manuelli, and D. Fox. Cliport: What and where pathways for robotic manipu- lation. In Proceedings of the 5th Conference on Robot Learning (CoRL), 2021.
[28] L. Shao, T. Migimatsu, Q. Zhang, K. Yang, and J. Bohg. Concept2robot: Learning manip- ulation concepts from instructions and human demonstrations. The International Journal of Robotics Research,
2021.
[29] M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakr- ishman, K. Hausman, A. Herzog, D. Ho, J. Hsu, J. Ibarz, B. Ichter, A. Irpan, E. Jang, R. J. Ruano, K. Jeffrey, S. Jesmonth, N. Joshi, R. Julian, D. Kalashnikov, Y. Kuang, K.-H. Lee, S. Levine, Lu, L. Luu, C. Parada, P. Pastor, J. Quiambao, Rao, Rettinghouse, Reyes, Sermanet, N. Sievers,
Tan, A. Toshev, V. Vanhoucke, F. Xia, T. Xiao, Xu, S. Xu. M. Yan, and A. Zeng. Do as can and not say: Grounding language in robotic affordances. In Xiv preprint arXiv:2204.01666 2022.
10

---

[30] S. Stepputtis, J. Campbell, M. Phielipp. S. Lee, C. Baral, and H. Ben Amor. Language- conditioned imitation learning for robot manipulation tasks. Advances in Neural Information 2020.
Processing Systems,
[31] A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabís, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog. J. Hsu, J. Ibarz, B. Ichter, A. Irpan, T. Jackson, S. Jesmonth, N. Joshi, R. Ju- lian, D. Kalashnikov, Y. Kuang, I. Leal, K.-H. Lee, S. Levine, Y. Lu, U. Malla, D. Manjunath, Parada, Peralta, E. Perez, K. Pertsch, Quiambao, K. Rao, M. Ryoo, G. Salazar, P. Sanketi, K. Sayed, J. Singh, S. Sontakke,
I. Mordatch, O. Nachum,
Stone, Tan, H. Tran, V. Vanhoucke, S. Vega, Q. Vuong, F. Xia, T. Xiao, P. Xu, S. Xu, T. Yu, and B. Zitkovich. Ri-
1: Robotics transformer for real-world control at scale. In arXiv preprint arXiv:2212.06817 2022.
[32] A. Brohan, N. Brown, Carbajal, Chebotar, X. Chen, K. Choromanski, T. Ding, D. Driess,
A. Dubey, C. Finn, al. Rt-2: Vision-language-e models transfer web knowledge to robotic control. arXiv preprint arXhv:2307.15816 2023.
[33] K. Zheng, X. Chen, O. C. Jenkins, and X. Wang. VImbench: compositional benchmark for ahion-and-languae manipulation. Advances in Neural Information Processing Systems, 35:
665-678, 2022.
[34] C. Li, R. Zhang, Wong, Gokmen, S. Srivastava, R. Martín-Martín, C. Wang, G. Levine, M. Lingelbach, J. Sun, al. Behavior-Ik: A benchmark for embodied ai with 1,000 everyday activities and realistic simulation. In Conference on Robot Learning, pages 80-93. PMLR, 2023.
[35] Yu, D. Quillen, Z. He, R. Julian, K. Hausman, C. Finn, and S. Levine. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference on robot learning, pages 1094-1100. PMLR, 2020.
[36] O. Mees, Hermann, F. Rosete-Beas, and Burgard. Calvin: benchmark for language- conditioned policy learning for long-horizon robot manipulation tasks. IEEE Roboties and Automation Letters, 7(3):7327-334,, 2022.
[37] B. Mildenhall, P. Srinivasan, M. Tancik, T. Barron, R. Ramamoorthi, and Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
[38] G. Qian, Y. Li, H. Peng, J. Mai, H. Hammoud, M. Elhoseiny, and B. Ghanem. Pointnext: Revisiting pointnet++ with improved training and scaling strategies. Advances Neural In- formation Processing Systems, 35:23192-23204, 2022.
[39] C. Finn, X. Y. Tan, Y. Duan, T. Darrell, S. Levine, and P. Abbeel. Deep spatial autoencoders for visuomotor learning. In 2016 JEEE International Conference on Robotics and Automation (ICRA), pages 512-519. IEEE, 2016.
[40] M. Laskin, A. Srinivas, and P. Abbeel. Curl: Contrastive unsupervised representations for reinforcement learning. In International Conference on Machine Learning, pages 5639-5650. PMLR, 2020.
[41] O. Nachum, S. Gu, H. Lee, and S. Levine. Near-optimal representation learning for hierarchical reinforcement learning, arXiv preprint arXiv:1810.01257, 2018.
[42] K. Rakelly, A. Gupta, C. Florensa, and S. Levine. Which mutual-information representation learning objectives are sufficient for control? Advances in Neural Information Processing Systems, 34:26345-266 2021.
[43] A. Stooke, K. Lee, Abbeel, and M. Laskin. Decoupling representation learning from re- inforcement learning. In International Conference Machine Learning, pages 9870-9979. PMLR, 2021.

---

[44] T. Zhang, T. Ren, M. Yang, J. Gonzalez, D. Schuurmans, and B. Dai. Making linear mdps prac- tical via contrastive representation learning. In International Conference on Machine Learning, pages 26447-26466 PMLR, 2022.
[45] S. Qiu, L. Wang, C. Bai, Z. Yang, and Z. Wang- Contrastive ucb: Provably efficient con- trastive self-supervised learning in online reinforcement learning. In International Conference on Machine Learning, pages 18168-18210. PMLR, 2022.
[46] A. v. d. Oord, Y. Li, and O. Vinyals. Representation learning with contrastive predictive coding. arXiv preprini arXiv: 1807.03748, 2018.
[47] T. Ma, R. Li, and J. Liang. An examination of the compositionality of large generative vision- language models. arXiv preprint arXiv:2308. 10509, 2023.
[48] D. Kalashnikov, J. Varley, Y. Chebotar, B. Swanson, R. Jonschkowski, C. Finn, S. Levine, and
K. Hausman. Mt-opt: Continuous multi-task robotic reinforcement learning at scale. arXiv preprint arXiv:2104. 08212, 2021.
[49] K. Konyushkova, K. Zolna, Y. Aytar, A. Novikov, Reed, S. Cabi, and N. de Fre- itas. Semi-supervised reward learning for offline reinforcement learning. arXiv preprint arXiv:2012.06899, 2020.
[50] D. Xu and M. Denil. Positive-unlabeled reward learning. In Conference on Robot Learning. pages 205-219, PMLR, 2021.
[51] A. Touati and Y. Ollivier. Learning one representation to optimize all rewards. Advances in Neural Information Processing Systems, 34:13-23, 2021.
[52] V. Myers, A. W. He, K. Fang, H. R. Walke, P. Hstruch C.-A. Cheng, M. Jaloheanu, A. Kolobov, A. Dragan, and S. Levine. Goal representations for instruction following: A semi- supervised language interface to control. In Conference on Robot Learning, pages 09894-9908... PMLR, 2023.
[53] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, pages 8748-8763, PMLR, 2021.
[54] E. Rohmer, S. P. Singh, and M. Freese. V-rep: A versatile and scalable robot simulation framework. In 2013 IEEE/RSJ international conference intelligent robots and systems, pages 1321-1326. IEEE, 2013.
[55] S. James, M. Freese, and A. J. Davison. Pyrep: Bringing v-rep deep robot learning. arXiv preprint 1rXiv:1966.11176 2019.
[56] James and A. J. Davison. Q-attention: Enabling efficient learning for vision-based robotic manipulation. IEEE Roboties and Automation Letters, 762):1112-169,, 2022.
[57] Y. You, J. Li, S. Reddi, J. Hseu, S. Kumar, S. Bhojanapalli, X. Song, J. Demmel, K. Keutzer, and C.-J. Hsich. Large batch optimization for deep learning: Training bert 76 minutes. arXiv 2019.
preprint
[58] K. He. H. Fan, Y. Wu, S. Xie, and R. Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7729-9738, 2020.
[59] E. Coumans and Y. Bai. Pybullet, python module for physics simulation for games, robotics and machine learning. 2016.
[60] E. Parisotto, J. L. Ba, and R. Salakhutdinov. Actor-mimic: Deep multitask and transfer rein- forcement learning, arXiv preprint
2015.
12

---

[61] Z. Xu, K. Wu, Z. Che, J. Tang, and J. Ye. Knowledge transfer in multi-task deep reinforcement learning for continuous control. Advances in Neural Information Processing Systems, 33: 15146-15155, 2020.
[62] R. Yang, H. Xu, Y. Wu, and X. Wang. Multi-task reinforcement learning with soft modular- ization. Advances in Neural Information Processing Systems, 33:4767-4777, 2020.
[63] A. Kumar, R. Agarwal, X. Geng, G. Tucker, and Levine. Offline q-learning on diverse multi-task data both scales and generalizes. arXiv preprimi arXiv:2211.15144. 2022.
[64] C. Rosenbaum, I. Cases, M. Riemer, and T. Klinger. Routing networks and the challenges of modular and compositional computation. arXiv preprint arXiv: 1904. 2019.
[65] T. Yu, S. Kumar, A. Gupta, S. Levine, K. Hausman, and C. Finn. Gradient surgery for multi-
task learning. Advances in Neural Information Processing Systems, 33:5824-5836, 2020.
[66] H. Liu, C. Li, Q. Wu, and Y. J. Lee. Visual instruction tuingg. Advances in neural information processing systems, 36, 2024.
[67] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 13971, 2023.
[68] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04803: 2018.
[69] Y. Liu, M. Out, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis. L. Zettlemoyer, and V. Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv: / 1692, 2019.
13

---

Task
Language Template
# of Variations
Avg. Keyframes
open drawer
"open the drawer"
3.0
slide block
"slide the block to target"
4.7
sweep to dustpan
"sweep dirt to the dustpan"
4.6
meat off grill
"take the off the grill"
5.0
turn tap
"turn tap"
2.0
put drawer
"put the item in the drawer"
12.0
close jar
"close the jar""
20
6,0
drag stick
"use the stick to drag the cube onto the target"
20
6.0
stack blocks
"stack blacks"
60
14.6
screw bulb
"screw the light bulb"
7.0
put safe
"put the money away in the safe on the shelf"
5.0
place wine
"stack the wine bottle to the of the rack"
3
5.0
put in cupboard
"put the in the cupboard"
5.0
sort shape
"put the the shape sorter"
5.0
push buttons
"push the button, [then the button]"
3.8
insert peg
"put the peg in the spoke"
20
5.0
stack cups
"stack the other cups on top of the cup"
20
10.0
place cups
"place cups on the cup holder"
3
11.5
Table Tasks in RLBench. We evaluate agents 18 RLBench tasks, which include 249 variations like PerAct [3] does.
Table 6: Multi-task performance in the Ravens [25] environment. The results show the effectiveness of both E-agent and contrastive IL (Coz in table).
Methods
Put. Blocks in Boel
Pack Box Paira
Packing Google Object Seq
Packing Google Objecta Group
Stack Block Pyramid
A11.gn Rope
CLiPort [27]
92.3
88.8
80.1
85.4
75.0
$1.2
Elagent Ler
87.2
E.agent w/ Cee
96.7
97.3
$2.1
87.6
83.5
65.1
A Simulation Experiments
A.1 RLBench Tasks
The RLBench setting in our paper is elaborated in Table 5. We follow the PerAct [3] to use the 18 tasks with 249 variations in the R..Bench. The examples of the 18 tasks and corresponding human instructions of specific variants these tasks are shown in Fig.
A.2 Effectiveness in Other Simulated Environments.
Ravens [25] is a simulated benchmark environment built with PyBullet [59] for robotic rearrange- ment based on a Universal Robot URSe. There are simulated 640x480 RGB-D cameras covering the 0.5xl m tabletop workspace from the front, left shoulder and right shoulder. We train the model on six tasks in the Ravens environment [25] as shown in Table 6, selecting from the seen split of CLIPort [27]. The models are trained 100 demonstrations and evaluated on 100 evaluation in- stances for each task. The Ravens benchmark scores 0 for task failure and 100% for success. The
Table 7: Evaluation on the generalization capacity of models. We test the models trained on 18 seen tasks directly on the 14 unseen tasks selected from the tasks of RLBench [24].
18 seen tasks
14 unseen tasks
Method
PerAct [3]
49.4
1.52
RVT [1]
62.9
2.95
E-agent
68.8
3.81
14

---

put item in drawer
reach and drag
turn tap
slide block to color target
open drawer
put groceries in cupboard
put the item middle drawer
use the stick drag the cube onto the yellow target
turn tap
slide block to green Larget
open bottom drawer
put the coffee in the cupboard
place shape shape sorter
put money safe
push buttons
close jar
stack blocks
place cups
pul the cube in the shape sorter
pul the money away in the safe the middle shelf
push the maroon button, then push the green button, then push the purple button sweep dustpan of size
close the silver
stack 3 white blocks
place cups cup holder
place wine rack location
light hulb
insert onto square peg
grill
cups
stack the wine bottle the middle of the rack
Screw in the azure light bulb
sweep dirt to the short dustpan
put ring on the lime spoke
take steak off the
stack the other cups top of black cup
Figure 4: Examples of the 18 RLBench tasks (front view) with corresponding human instructions.
Table Multi-Task performance rained with 10 episodes and evaluated using 25 episodes per task on RLBench (3 times average). The input resolution is 128 x 128. In Train Time, the d represents days and represents hours. All other results are success rates, measured percentage (%).
stack blocks
screw bulb
put in safe
place wine
put in sort shape
push buttons
insert peg
stack cups
place
cupboard
cups
I-BC(CNN) [2]
4.0
0
20.0
0
0
I-BC(VIT) [2]
4.4
16.0
8.0
8.0
0
24.0
0
0
0
C2FARM [4]
22.7
28.0
12.0
4.0
40.0
60.0
12.0
28.0
72.0
PerAct [3]
~166
68.0
72.0
68.0
72.0
16.0
32.0
36.0
RVT
42.1
~23h
77.3
52.0
65.3
68.0
93.3
45.3
34.7
100.0
E-agent
47.3
~22hh
74.6
56.0
46.7
80.0
94.7
90.7
64.0
100.0
put.
place wine
put in cupboard
sort shape
push buttons
insert
stack cups
place
blocks
bulb
safe
peg
cups
1-BCICNN) [2]
0
4.0
I-BC(VIT)
0
4.0
0
16.0
0
C2FARM [4]
4.0
12.0
0
36.0
4,0
8.0
88.0
0
0
PerAct [3]
12.0
28.0
16.0
20.0
0
16,0
$6.0
4.0
0
0
(VT
13.3
25.3
40.0
57.3
14.7
6.7
61.3
4.0
0
0
E-agent
14.6
45.3
14.7
72.0
4.0
evaluation also assigns partial credit for different tasks, such as 40% for packing out of objects required by the instruction. The results in Table show improvements in both our E-agent and contrastive IL, surpassing the CLIPort [27] by large margin in all six tasks.
15

---

0.75
0.50
0.25
CL
Sort
Tur tap
Open drewer
Close jar
CL
Screw bulb
shape
Open drawer
Figure 5: We visualize the similarity of key-point trajectory of close jar task with multiple tasks' language instructions. Training with contrastive IL module maximizes the similarity between the visual observations and related language instructions (deeper color), reducing the similarity with negative instructions (lighter color).
A.3 Visualization.
To qualitatively illustrate the effectiveness of contrastive IL, we visualize the multi-modal similarity between visual observations and language instructions. As shown in Fig. 5, we compute the co- sine similarity between the embeddings of observations trajectory close jar and different language instructions. For E-agent without contrastive module, we utilize the original text pro- jection layer of CLIP [53] to project the textual embeddings for similarity computation. It is evident that contrastive IL amplifies the alignment between language instructions and local visual observa- tions. For example, in the third keyframe, the agent without CL tends to confuse the close jar task with screw bulb and sort shape, as the three task instructions have close similarity with the visual observation.
B Real-world Experiments
B.1 Hardware Setup
We conduct real-world experiments on a table-top setup with 6-DoF UR5 robotic arm. The vi- sual input is provided by third-person perspective Orbbec Femto Bolt (RGB-D) camera mounted directly above and facing forward. The camera streaming the RGB-D images of 1280 x 960 (hard- ware D2C align) at 30 Hz. The images will be resized to 640 480 for feeding into the model. The extrinsics of the RGB-D camera and robotic arm are calibrated via hand-eye calibration. We mount the ARUCO AR marker in the base of UR5 arm. Additionally, we mount a DJI Osmo Action for recording inference demos. The hardware setup is shown in Fig. 6.
B.2 Data Collection
We collect the human demonstrations by manual teaching, dragging the robotic arm pre-defined keypoints and collecting the visual observations and gripper poses. These poses executed with a motion-planner using ROS 5 and Movelt We define 5 tasks to experiment, including Stack cups, Put fruit in plate, Hang mug, Put item in barrel, and Put tennis in mug. We
http://wwdd....comm///ooooaaaaae 'nttpa://ros.org/ http:////o........ooo.../ata//aa
16

---

6-DoF UR5 Robotic Arm
DJI Osmo Action 4 video recorder
Hand-eye Calibration
Orbbec Femto RGB-D Camera
Figure 6: Real robot setup with Orbbec Femto Bolt and UR5.
Task
Language Template
# of Variations
Variants Category
stack cups
"stack the cup on top of the cup"
color
put fruit in plate
"put the in the plate"
2
object
hang mug put item in barrel
"hang mug on the rack"
color
"put the in the barrel"
object
put tennis mug
"put the tennis the mug"
color
Table 9: Tasks in real-world. There are 5 tasks and 9 variants totally.
collect 10 demonstrations for each task. The details of the collected data samples are shown in Table 9 and Fig 7.
fruit plate
cups
item barrel
Hang mug
Put tennis in mug
Figure 7: Illustration of the 5 real-world tasks.
B.3 Training and Evaluating
The E-agent is trained on 50 demonstrations for 5 tasks. The training samples augmented with 125m translation perturbations and +45 yaw rotation perturbations following PerAct [3]. The

---

training is from scratch, not fine-tuning based on checkpoints trained in simulated environments. For evaluation, Z-agent validated on 10 episodes for each task.
Imprecise aiming
Collision
Figure 8: Failure cases.
Additional Model Details
C.1 Action Prediction
Based on the context features after feature fusion, the decoder outputs the 6-DoF end-effector pose (3-DoF for translation and 3-DoF for rotation), the gripper state (open close) and binary value for whether allow collision for the motion planner. We simply utilize a 2D convolution layer and bi-linear upsampling to decode and upsample the encoded context features to the original rendered image size (220 x 220). Following RVT [1], E-agent predicts the heatmaps for the 5 virtual views, and the heatmaps will be projected back to the 3D space to predict the point-wise scores for the robot workspace. Then, the translation of the end-effector determined by the 3D point with the highest score. The rotations, gripper state and collision indicator are predicted based on the max- pooled image features and the sum of image features weighted by the heatmaps. Suppose the h is the view-wise heatmaps predicted, the features for predicting rotations, gripper state and collision indicator are formulated as:
[sum(v (v h), maxpool (v)]
where sum and maxpool denote the sum and max-pooling operation across the spatial dimension of tokens. @ represents the element-wise multiplication between the context features and heatmaps.
Then, following the PerAct [3] and RVT [1], utilize the Euler angles representation for the rotation, and each angle is discretized into bins of 5 for dæ, dy, dz. In that case, the rotation prediction is converted to a classification problem, where the agent is trained to classify the angles into 216 categories (3 x 360 /59). Hence, we project the features f onto a 220-dimensional space using a linear layer. Within this space, 216 dimensions are allocated for rotation prediction, while dimensions each are dedicated to binary gripper state prediction and binary collision state prediction.
For the training loss for action prediction, we the cross-entropy loss for the translation and rotation. Binary classification loss is utilized for the gripper state and collision state. Contrastive loss is utilized besides the aforementioned losses to supervise representation learning in the contrastive IL module.
C.2 Q-Function Analysis
Same PerAct [3], we decode the features to estimate the Q-function of action-values, as ((a 1). The Q(ay|st,) is equivalent to the transition probability Pa of state and to the next state 8141 under the discounted state occupancy measure [19, 17, 20]:
(aclse) 1) A Pa (5
(6)
18

---

When the g is the next state of 8 in Eqn. 3, the fa.e maximizes the similarity between (s,,)) and 8141- other words, training fo.p aims to maximize the transition probability from 8, to $141 with the language instruction by minimizing the distance between corresponding pairs [[$z, 1), $t-1), while simultaneously maximizing the distance of negative pairs. Therefore, training of Jö with +=g beneficial for maximizing the pa (S $t, !), thereby maximizing the 2x (ar St. !). Já can be extra critic function to facilitate the policy mimicking target policy a+ in the level of representation learning.
D Related Multi- Learning in RL
Learning single agent for multiple tasks is of vital significance to robotíc learning. One of the main challenges in multi-task learning is the conflicting representations and gradients among differ- ent tasks. Previous works address multi-task learning via strategies like knowledge transfer [60, 61], representation sharing [62, 63, 64], and gradient surgery [65]. With the advent of large vision- language models [53, 21, 22, 66] and I.LMs [67, 68, 69), language instructions serve as comple- mentary hints to differentiate task representations policy learning [31, 32, 16, 11, 3, 1, 12, 13, 14]. In this paper, leverage contrastive learning to amplify the distinction function of linguistic hints.
19


Reviews:
[{"review_id": "6DXc1paPIr", "reviewer": "Reviewer_9vfD", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 4, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This study presents Σ-agent, an end-to-end imitation learning agent designed to address language-guided multi-task robotic manipulation.", "summary_of_recommendation": "Overall, this article proposes a novel learning approach in the field of robotic manipulation, effectively improving model performance through contrastive imitation learning, while still facing challenges in real-world applications.", "technical_quality": 4, "clarity_of_presentation": 3, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "See weakness.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FskEqN6qj4", "reviewer": "Reviewer_6fNC", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper proposes $\\Sigma$-agent, a novel multi-task robotic manipulation learning agent. $\\Sigma$-agent introduces new contrastive imitation learning modules that help to learn discriminative features to accurately perceive the current state and also to align task and language representations better. Then, a multi-view querying transformer module (MVQ-Former) is proposed to aggregate visual information into a set of 5 learnable queries (one per available input camera) that are fused later on with language features to produce the context features from which the final robot action is decoded. $\\Sigma$-agent outperforms other baselines on 18 RLBench tasks. It shows how the proposed contrastive imitation learning modules can be plugged during training to previous methods (RVT and PolarNet) to improve its final performance. Finally, the paper demonstrates that the method can be deployed on a real-robot performing 5 different manipulation tasks.", "summary_of_recommendation": "My recommendation is Weak Accept. I believe that $\\Sigma$-agent and specifically the proposed contrastive imitation learning modules will benefit the robotics community. While contrastive learning already exists and has been well studied, the proposed modules are novel and can be easily integrated with other language-guided robotic manipulation agents. However, when integrated with RVT and PolarNet, the performance only improves slightly. Further discussion with the authors and other reviewers' points of view can help towards the final decision.                                                             ||||||    Update: I want to thank the authors for the effort and work put into the rebuttal to respond to the 7 reviews. I will keep my final recommendation as Weak Accept. The authors addressed most of my concerns. However, I still believe that the improvements of this work when plugged into other methods remain very limited.", "technical_quality": 4, "clarity_of_presentation": 4, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "Please, take a look at the weaknesses in the review. Questions should be addressed to make the paper stronger.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bkBnGgzV2o", "reviewer": "Reviewer_H222", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper introduces a contrastive imitation learning approach for language-guided multi-task robotic manipulation. The paper builds upon the RVT [1] with the key additions being (i) the integration of contrastive learning into the imitation learning framework to refine vision feature extraction and vision-language feature interaction, and (ii) introducing a multi-view querying transformer for efficient contrastive RL. The authors evaluate their method on the RLBench benchmark demonstrating some improvements over state-of-the-art methods. The authors also test their method on a few real-world tasks.   [1] RVT: Robotic View Transformer for 3D Object Manipulation", "summary_of_recommendation": "The main contribution of the paper is the idea of using contrastive learning in the RVT framework, and I think the results from RLBench don't necessarily show yet that this led to significant improvement in performance.", "technical_quality": 3, "clarity_of_presentation": 4, "potential_impact": 2, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "In addition to the questions in the previous sections, I have the following questions:\n\n1. Do you have any qualitative examples of why the proposed approaches performs better than say RVT in some of the RLBench tasks?\n\n2. In limitations, you state that the large discrepancy between RLBench and real-world environments leads to failure of sim-to-real transfer. Could you clarify this statement? Do you mean that “conclusions made in sim possibly may not hold in real-world” or “training in sim and deploying zero-shot in real-world wouldn’t work”? I usually associate sim-to-real with the latter, but that doesn’t make a lot of sense in this paper’s setting.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SloKuS9l09", "reviewer": "Reviewer_h5Zm", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper introduces an end-to-end imitation learning framework for language-conditioned multiple 6-DoF manipulation tasks for agents. It presents a novel method called contrastive imitation learning to refine the visual representation and also the vision-language representation. It also proposes the Multi-View Querying Transformer to improve efficiency by minimizing the token numbers. Experiments are performed on several benchmarks and various manipulation tasks. The contrastive IL module is integrated with baseline models to demonstrate its effectiveness in different visual representations.", "summary_of_recommendation": "Overall it is a good work with comprehensive results in the simulated experiments. The paper is easy to read and the proposed method is easy to understand. I will be more confident in recommending the paper for acceptance if the questions are well answered or addressed.", "technical_quality": 3, "clarity_of_presentation": 4, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "- Efficiency of MVQ-Former: Can you please provide a more detailed analysis and concrete comparisons to substantiate the claim that the proposed MVQ-Former efficiently reduces the number of visual tokens?\n- Real-World Experiment Improvement: The success rates in real-world pick-and-place tasks are not very satisfying. Could these rates be improved by relatively simple means, such as using multi-view cameras to provide more precise visual information or adding an extra RGB-D camera at the wrist position as proposed? If these improvements are feasible, please provide several results.\n- Task-Specific Improvements: In simulated experiments, the method shows a significant improvement on tasks like \"put in drawer\" and \"close jar.\" Can you please provide a more detailed analysis explaining why the proposed method excels in these specific tasks?\n- Baseline Reimplementation and Demonstration Variability: The baselines are reimplemented using different numbers of demonstrations for training. Can you include more detailed comparisons and analyses on the impact of varying demonstration numbers on the performance of the proposed method?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LUpEWfNGAF", "reviewer": "Reviewer_wN6C", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 2, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper titled “Contrastive Imitation Learning for Language-guided Multi-Task Robotic Manipulation” presents a language-conditioned imitation learning framework for robotic manipulation. The contribution consists of augmenting the standard imitation learning objective with two auxiliary contrastive learning objectives to promote alignment between 1) the language instruction and the visual representation and 2) the language instruction + visual representation and the goal. This objective is used to train a multi-task, multi-view policy architecture, which is evaluated against a comprehensive set of baselines. The simulated results indicate that 1) the overall proposed system (new objective + policy architecture) outperforms existing methods and 2) contrastive objectives marginally improve the performance of baselines. The proposed method is validated on a real robot.", "summary_of_recommendation": "The paper is in relatively good condition in that it is well organized, quite clear, and experiments are extensive. The contribution contains some originality, and while it does improve policy performance (in aggregate) and learning efficiency, the significance of the results (and hence the contribution) is questionable. On these bases, my recommendation is weak reject.", "technical_quality": 3, "clarity_of_presentation": 3, "potential_impact": 2, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "- A more compelling and intuitive motivation should be established for each of the contrastive learning terms. At the present moment, the motivation for these terms in the form of a need for feature “alignment” is vague to me. The language-state term is more intuitive than the goal term. \n- Can the authors please explain the discrepancy of results across tasks? Why does contrastive learning benefit some but harm others? Can this be improved?\n- Quality / Clarity: I encourage the authors to simplify Figure 2 and address the above comments pertaining to quality and clarity.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rVw9LsI2k7", "reviewer": "Reviewer_HHmD", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper introduces an imitation learning approach that leverages contrastive learning to align the image + language embeddings and the <image, language> + goal image embeddings. The vision encoder is additionally trained on re-rendered virtual images (following RVT). The evaluations are conducted on RLBench and a set of real-world manipulation tasks, and demonstrates improvements over RVT in the multi-task setting.", "summary_of_recommendation": "This paper studies an important problem that is relevant to the robot learning community, and presents an approach that is simple to integrate into existing approaches. However, from the present experiments, it's not entirely clear to me that there are strong advantages, and stronger analysis may highlight where this approach would be most beneficial.", "technical_quality": 3, "clarity_of_presentation": 4, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "Clarification questions:\n- In Table 2, is $\\Sigma$-agent using RVT as the base model? If so, what is the performance of RVT in the 100-episode per task setting?\n- How much training time does contrastive learning add to RVT and to PolarNet? In Table 1, is it correct that $\\Sigma$-agent applies the contrastive learning module to RVT and is faster to train compared to RVT?\n- For real robot experiments, how does the performance compare to the base model without the contrastive representation?\n\nOther questions:\n- What would happen if this is to scaled to a much larger number of tasks? How does dataset size and number of training tasks affect performance?\n- Does this alignment of vision and language modalities enable better generalization to new tasks and task variations?\n- Currently, the variations in tasks are reference variations in the color and object instance. Have the authors tried testing other variations such as position/direction (e.g., \"stack the cup located in the top-right corner on top of the cup in the bottom-left corner\"), size (e.g. \"stack the smaller cup on top of the bigger cup\"), and shape? It would also be interesting to organize the results in terms of different types of variations to see which variations are harder for the model to handle.\n- Have the authors tried varying the camera pose? Some of the aforementioned variations (position/direction) depend heavily on the camera perspective. It'd be interesting to see whether aligning the vision-language modalities allow generalization to new camera poses and references to position/direction.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UPj8NZHjUL", "reviewer": "Reviewer_Ejqe", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 2, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The authors introduce an imitation learning method that incorporates contrastive learning in the training objective to better align the underlying vision and language representations and improve downstream policy performance. The contrastive learning objective is two fold: (1) aligning states and language representations for the same task, and (2) aligning joint state-language and goal representations for the same task. The authors also propose a Multi-View Querying Transformer to improve the training efficiency of the model. On the simulated RLBench suite, the authors show that the model generally outperforms existing approaches, including RVT.", "summary_of_recommendation": "I think at least one of the following is needed: (1) more technical novelty; (2) larger performance gains over the current state-of-the-art methods.", "technical_quality": 3, "clarity_of_presentation": 4, "potential_impact": 2, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "Please see the questions I raised in my review above.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes an end-to-end imitation learning method which introduces auxiliary contrastive learning objectives to align visual representations with language instructions as well as the language-visual representation with the goal. Additionally, a multi-view querying transformer is proposed as an important component for efficiently incorporating contrastive imitation learning into the RVT framework. 

Overall, reviewers appreciated the motivation of the proposed method, the diverse simulated benchmark and real-world experiments, and the high writing quality. However, reviewers also raised significant questions and concerns. Of particular note:

- Most reviewers note that the empirical gains are marginal. Furthermore, the ablations suggest that the performance improvements can not necessarily be attributed to the novel contributions of the work.
- Numerous reviewers request clarity on comparing the work against RVT and RVT2, both in the form of discussions as well as analysis.
- Some reviewers raise questions about the novelty of the work, since prior robot learning works have also incorporated auxiliary contrastive losses.
- Some reviewers note that it’s unclear whether the method can scale to more complex or unseen tasks.

The reviewers raise other questions and also provide helpful suggestions for improvement. It would be helpful for authors to address these during the rebuttal phase.

Update: The authors provided thorough and helpful responses to the numerous concerns raised by 7 reviewers. After discussions in the post-rebuttal phase, most reviewers are positive about the work and appreciate the statistically significant (if marginal) performance improvements over a variety of strong methods. *This work will be a valuable contribution to CoRL, so I am recommending an acceptance*. However, I recommend that the authors address the remaining reviewer feedback, especially those regarding performance and generalization. One reviewer provides the following helpful suggestions for the camera-ready:
- calling this goal-state contrastive learning is somewhat misleading. one would not normally associate the next state as a "goal". the semantics here need to be rephrased and we need to see an updated manuscript with a better interpretation of what the role of this "goal-state" contrastive learning is.
- we need to better understand why both language and goal contrastive learning are needed, and that why without language contrastive learning we do not see many benefits to goal contrastive learning.
Decision: Accept