CLIP ORT : What and Where Pathways for Robotic Manipulation
Anonymous Author(s)
AfﬁliationAddress email
Abstract: How can we imbue robots with the ability to manipulate objects precisely but also to reason about them in terms of abstract concepts?
Recent worksin manipulation have shown that end-to-end networks can learn dexterous skillsthat require precise spatial reasoning, but these methods often fail to generalize tonew goals or quickly learn transferable concepts.
In parallel, there has been greatprogress in learning generalizable semantic representations for vision and language by training on large-scale internet data, however these representations lackthe spatial understanding necessary for ﬁne-grained manipulation.
To this end, wepropose a framework that combines the best of both worlds: a two-stream architecture with semantic ( what ) and spatial ( where ) pathways for vision-based manipulation.
Speciﬁcally, we present CLIP ORT , a language-conditioned imitationlearning agent that combines the broad semantic understanding of CLIP [1] withthe spatial precision of Transporter [2].
Our end-to-end framework is capable ofsolving a variety of language-speciﬁed tabletop tasks from packing unseen objectsto folding cloths, all without any explicit representations of object poses, instancesegmentations, history, symbolic states, or syntactic structures.
Experiments insimulation and hardware show that our approach is data efﬁcient in few-shot settings and generalizes effectively to seen and unseen semantic concepts.
We evenlearn one multi-task policy for 10 simulated and 9 real-world tasks that showsbetter or comparable performance to single-task models.
Introduction
Ask a person to “get a scoop of coffee beans” or “fold the cloth in half” and they can naturally takeabstract concepts like scoop or fold and ground them in concrete physical actions within an accuracyof a few centimeters.
We humans do this intuitively, without explicit geometric or kinematic modelsof coffee beans or cloths.
Moreover, we can generalize to a broad range of tasks and concepts froma minimal set of examples on what needs to be achieved.
How can we imbue robots with this abilityto efﬁciently ground abstract semantic concepts in precise spatial reasoning?
Recently, a number of end-to-end frameworks have been proposed for vision-based manipulation[2, 3, 4, 5].
While these methods do not use any explicit representations of object poses, instance segmentations, or symbolic states, they can only replicate demonstrations with a narrow range of variability and have no notion of the semantics underlying the tasks.
Switching from packing red pensto blue pens involves collecting a new training set [2], or if using goal-conditioned policies, involvesthe user providing a goal-image from the scene [5, 6].
In realistic human-robot interaction settings,collecting additional demonstrations or providing goal-images is often infeasible and unscalable.
Anatural solution to both these problems is to condition policies with natural language.
Language provides an intuitive interface for specifying goals and also for implicitly transferring concepts acrosstasks.
While language-grounding for manipulation has been explored in the past [7, 8, 9, 10], thesepipelines are limited by object-centric representations that cannot handle granular or deformable objects and often do not reason about perception and action in an integrated manner.
In parallel, therehas been great progress in learning models for semantic understanding [11, 12] and aligning representations of vision and language [13, 14, 15] by training on large-scale internet data.
However,these representations lack the spatial understanding necessary for ﬁne-grained manipulation.
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
To this end, we propose the ﬁrst framework that combines the best of both worlds: end-to-endlearning for ﬁne-grained manipulation with the broad generalization capability of vision-languagegrounding systems.
We introduce a two-stream architecture for manipulation with semantic ( what )and spatial ( where ) pathways broadly inspired by (or vaguely analogous to) the two-stream hypothesis is cognitive psychology [16, 17, 18].
Speciﬁcally, we present CLIP ORT , a language-conditionedimitation-learning agent that integrates the semantic understanding of CLIP [1] with the spatial precision of Transporter [2].
Transporter has been applied to a wide range of rearragement tasks fromindustrial packing [2] to manipulating deformable objects [6].
The key insight of the approach isformulating tabletop manipulation as a series of pick-and-place affordance predictions, where theobjective is to directly detect actions rather than to detect objects and then learn a policy.
Thisaction-centric approach [19] is data efﬁcient and effective at overcoming the limitations of explicitobject representations.
However, Transporter is a tabula rasa system that learns all visual representations from scratch and so every new task or goal requires collecting a new set of demonstrations.
Toaddress this problem, we bake in a strong semantic prior while learning policies.
We condition oursemantic stream with visual and language-goal features from a pre-trained CLIP model [1].
Since
CLIP was trained to align image and language features from large-amounts of image-caption pairsfrom the internet, it provides a powerful prior for grounding semantic concepts that are commonacross tasks like categories, parts, shapes, colors, texts, and other visual attributes, all without atop-down pipeline that requires bounding boxes or segmentations [13, 14, 15, 20].
This architecture effectively allows us to formulate tabletop rearrangement as a series of language-conditionedaffordance predictions, a predominantly vision-based inference problem, and thus beneﬁt from thestrengths of data-driven paradigms like scale and robustness.
To study these beneﬁts, we conduct large-scale experiments in the Ravens [2] framework with asimulated suction-gripper robot.
We propose 10 language-conditioned tasks with 1000s of uniqueinstances per task that require both semantic and spatial reasoning (see Figure 1 a-j).
CLIP ORT isnot only effective at solving these tasks, but surprisingly, it can even learn a multi-task model forall 10 tasks that achieves better or comparable performance to single-task models.
Further, ourevaluations indicate that our multi-task model can effectively transfer attributes like “pink block”across tasks, having never seen pink blocks or the word ‘pink’ in the context of the evaluation task.
We also demonstrate our approach on a Franka Panda manipulator with one multi-task model for 9real-world tasks trained with just 179 image-action pairs (see Figure 1 k-o).
In summary, our contributions are as follows:
• An extended benchmark of language-grounding tasks for manipulation in Ravens [2].
• Two-stream architecture for using internet pre-trained vision-language models for condi-
tioning precise manipulation policies with language goals.
• Empirical results on a broad range of manipulation tasks, including multi-task models,validated with real-robot experiments.
We will release our code, models, and language-grounding benchmark upon publication.
Related Work
Vision-based Manipulation.
Traditionally, perception for manipulation has centered around objectdetectors, segmentors, and pose estimators [21, 22, 23, 24, 25, 26].
These methods cannot handledeformable objects, granular media, or generalize to unseen objects without object-speciﬁc trainingdata.
Alternatively, dense descriptors [27, 28, 29] and keypoint representations [30, 31, 32] forgosegmentation and pose representations, but do not reason about sequential actions and struggle torepresent scenes with variable numbers of objects.
On the other hand, end-to-end perception-toaction models can learn precise sequential policies [2, 4, 6, 33, 34, 35], however, these methodshave limited understanding of semantic concepts and rely on goal-images to condition the policies.
Recently, Yen-Chen et.
al [36] showed that pre-training on semantic tasks like classiﬁcation andsegmentation helps in improving efﬁciency and generalization of grasping predictions.
Semantic Models.
With the advent of large-scale models [37, 38, 39], a number of methods forlearning joint vision and language representations have been proposed [13, 14, 15, 20, 40].
However, these methods are restricted to bounding box or segmentation detections, which make theminapplicable for detecting things like piles of coffee beans or squares on a chessboard.
Alternatively,recent works in contrastive learning forgo top-down object-detection and learn continuous representations by pre-training on unlabeled data [11, 12].
Recently, CLIP [1] applied a similar approach toalign vision and language representations by training on image-caption pairs from the internet.
Language Grounding for Robotics.
Several works have proposed systems for instructing robotswith natural language [7, 8, 9, 10, 41, 42, 43, 44, 45, 46, 47].
However, these methods use disentangled pipelines for perception and action with the language primarily being used to guide theperception.
As such, these pipelines lack the spatial precision necessary for tasks like folding cloths.
Recently, Lynch et.
al [48] proposed an end-to-end system for grounding language in continuouscontrol, but it requires several hours of human teleoperation data for a single simulated desk setting.
Two-Stream Architectures are prevalent in action-recognition networks [49, 50, 51] and audiorecognition systems [52, 53].
In robotics, Zeng et. al [54] and Jang et.
al [55] have proposed twostream pipelines for affordance predictions of novel objects.
The former requires goal-images andthe latter is restricted to one-step grasps with single-category goals.
In contrast, our frameworkprovides a rich and intuitive interface with composable language commands for sequential tasks.
CLIP ORT
CLIP ORT is an imitation-learning agent based on four key principles: (1) Manipulation through atwo-step primitive where each action involves a start and ﬁnal end-effector pose.
(2) Spatially equivariant visual representations [56, 57] that are robust to translations and rotations.
(3) Two separatepathways for semantic and spatial information.
(4) Language-conditioned policies for specifyinggoals and also transferring concepts across tasks.
Combining (1) and (2) from Transporter with (3)and (4) allows us to achieve generalizable policies that go beyond just imitating demonstrations.
Section 3.1 describes the problem formulation, gives an overview of Transporter [2], and presentsour language-conditioned model.
Section 3.2 provides details on the training approach.
3.1 Language-Conditioned Manipulation
We consider the problem of learning a goal-conditioned policy π that generates actions a t giveninput γ t =
( o t , l t ) consisting of a visual observation o t and an English language instruction l t :
[Equation]
[Figure] [Figure]
RGB-D
S e m a n t
i c
V e n t r a l
S p a t i a l
RGB HxWx
[Figure]
“pack all
[Figure]
D o r s a l
[Figure]
The actions a =
( T pick , T place ) specify the end-effector pose for picking and placing, respectively.
We consider tabletop tasks where T pick , T place ∈ SE (2) .
The visual observation o t is a top-downorthographic RGB-D reconstruction of the scene where each pixel corresponds to a point in 3Dspace.
The language instruction l t either speciﬁes step-by-step instructions e.g. “pack the scissors”→ “pack the purple tape”
→ etc., or a single goal description for the whole task e.g “pack all theblue and yellow boxes in the brown box” .
See Figure 4 for speciﬁc examples.
We assume access to a dataset D = { ζ 1 , ζ 2 , . . .
, ζ n } of n expert demonstrations with associateddiscrete-time input-action pairs ζ i = { ( o 1 , l 1 , a 1 ) , ( o 2 , l 2 , a 2 ) , . . .
} where a t =
( T pick , T place ) corresponds to expert pick-and-place coordinates at timestep t .
These expert demonstrations are used tosupervise the policy π .
Transporter for Pick-and-Place.
The policy π is trained with Transporter [2] to perform spatialmanipulation.
The model ﬁrst (i) attends to a local region to decide where to pick, then (ii) computesa placement location by ﬁnding the best match through cross-correlation of deep visual features.
Following Transporter [2, 6], the policy π is composed of two action-value modules (Q-functions):
The pick module Q pick decides where to pick, and conditioned on this pick action the place module
Q place decides where to place.
These modules are implemented as Fully-Convolutional-Networks(FCNs) that are translationally equivariant by design.
As we will describe in more detail below,we extend these networks to two-stream architectures that can handle language input.
The pick
FCN f pick takes input γ t =
( o t , l t ) and outputs a dense pixelwise prediction Q pick ∈ R H
× W ofaction-values, where are used to predict a pick action T pick :
[Equation]
Since o t is an orthographic heightmap, each pixel location ( u, v ) can be mapped to a 3D pickinglocation using the known camera calibration.
f pick is trained in a supervised manner to predict thepick action T pick that imitates the expert demonstration with the speciﬁed language instruction attimestep t .
The second FCN Φ query takes in γ t [ T pick ] , which is a c × c crop of o t centered at T pick along withthe language instruction l t , and outputs a query feature embedding of shape R c × c × d .
The third FCNΦ key consumes the full input γ t and outputs a key feature embedding of shape R H × W × d .
The placeaction-values Q place are then computed by cross-correlating the query and key features:
[Equation]
where ∆ τ ∈ SE (2) represents a potential placement pose.
Since o t is an orthographic heightmap,rotations in the placement pose ∆ τ can be captured by rotating the crop of o t for k discrete anglesbefore passing it through the query network Φ query .
Then T place = argmax ∆ τ Q place (∆ τ | γ t , T pick ) ,where the place module is trained to imitate the placements in the expert demonstrations.
For allmodels, we use c = 64 , k = 36 and d = 3 .
As in Transporter [2, 6], our framework can beextended to handle any motion primitive like pushing, sliding, etc.
that can be parameterized by twoend-effector poses at each timestep.
For more details, we refer the reader to the original paper [2].
Two-Stream Architecture.
In CLIP ORT , we extend the network architecture of all three FCNsf pick , Φ query and Φ key from Transporter [2] to allow for language input and reasoning about high-levelsemantic concepts.
We extend the FCNs to two-pathways: semantic (ventral) and spatial (dorsal).
The semantic stream is conditioned with language features at the bottleneck and fused with intermediate features from the spatial stream.
See Figure 2 for an overview of the architecture.
The spatial stream is identical to the ResNet architecture in Transporter – a tabula rasa network thattakes in RGB-D input o t and outputs dense features through an hourglass encoder-decoder model.
The semantic stream uses a frozen pre-trained CLIP ResNet50 [1] to encode the RGB input 1 ˜ o tup until the penultimate layer ˜ o t → v (0) t : R 7 × 7 × 2048 , and then introduces decoding layers thatupsample the feature tensors to mimic the spatial stream v ( l − 1) t → v ( l ) t : R h × w × C at each layer l .
The language instruction l t is encoded with CLIP’s Transformer-based sentence encoder to producea goal encoding l t → g t : R 1024 .
This goal encoding g t is downsampled with fully-connected layersto match the channel dimension C and tiled to match the spatial dimensions of the decoder featuressuch that g t → g ( l ) t : R h × w × C .
The decoder features are then conditioned with the tiled goalfeatures through an element-wise product v ( l ) t (cid:12) g ( l ) t (Hadamard product).
Since CLIP was trainedwith contrastive loss on the dot-product alignment between pooled image features and languageencodings, the element-wise product allows us to use this alignment while the tiling preserves thespatial dimensions of the visual features.
This language conditioning is repeated for three subsequentlayers after the bottleneck inspired by LingUNet [58].
We also add skip connections to these layersfrom the CLIP ResNet50 encoder to utilize different levels of semantic information from shapes toparts to object-level concepts [59].
Finally, following existing two-stream architectures in videoaction recognition [51], we add lateral connections from the spatial stream to the semantic stream.
These connections involve concatenating two feature tensors and applying 1 × 1 conv to reducethe channel dimension [ v ( l ) t (cid:12) g ( l ) t ; d ( l ) t ] : R h × w × C v + C d → R h × w × C v , where v ( l ) t and d ( l ) t arethe semantic and spatial tensors at layer l , respectively.
For the ﬁnal fusion of dense features,addition for f pick and 1 × 1 conv fusion for Φ query and Φ key worked the best empirically.
See
Appendix B for details on the exact architecture.
3.2 Implementation Details
Training from demonstrations.
Similar to Transporter [2] we train CLIP ORT using imitation learning on a set of expert demonstrations D = { ζ 1 , ζ 2 , . . .
, ζ n } consisting of discrete-timeinput-action pairs ζ i
= { ( o 1 , l 1 , a 1 ) , ( o 2 , l 2 , a 2 ) , . . . } .
During training, we randomly sample aninput-action pair from the dataset and supervise the model end-to-end with one-hot pixel encodings of demonstration actions Y pick : R H × W × k and Y place : R H ×
W × k with k discrete rotations.
In simulated experiments with the suction-gripper, we use k = 1 for pick actions andk = 36 for place actions.
Following Transporter [2], the model is trained with cross-entropyloss: L = − E Y pick [ log V pick ] − E Y place [ log V place ] where V pick = softmax ( Q pick (( u, v ) | γ t )) and
V place = softmax ( Q place (( u (cid:48) , v (cid:48) , ω (cid:48) )
| γ t , T pick )) .
Compared to the original Transporter models thatwere trained for 40K iterations, we train our models for 200K iterations (with data augmentation;see Appendix D) to account for additional semantic variation in tasks – randomized colors, shapes,objects.
All models are trained on a single commodity GPU for 2 days with a batch size of 1.
Training multi-task models.
Multi-task training is nearly identical to single-task training exceptfor the sampling of training data.
First, we randomly sample a task, and then select a random inputaction pair from that task in the dataset.
Using this strategy, all tasks are equally likely to be sampledbut longer horizon tasks are less likely to reach full coverage of input-action pairs available in thedataset.
To compensate for this, we train all multi-task models 3 × longer for 600K iterations or
GPU days.
Results
We perform experiments both in simulation and hardware aimed at answering the following questions: 1) How effective is the language-conditioned two-stream architecture for ﬁne-grained manipulation compared to one-stream alternatives and other simpler baselines?
Is it possible to train amulti-task model for all tasks, and how well does it perform and generalize? 3)
How well do thesemodels generalize to seen and unseen semantic attributes like colors, shapes, and object categories?
4.1 Simulation Setup
Environment.
All simulated experiments are based on a Universal Robot UR5e with a suctiongripper.
The setup provides a systematic and reproducible environment for evaluation, especially forbenchmarking the ability to ground semantic concepts like colors and object categories.
The inputobservation is a top-down RGB-D reconstruction from 3 cameras positioned around a rectangulartable: one in the front, one on the left shoulder, and one on the right shoulder, all pointing towardsthe center.
Each camera has a resolution of 640 × 480 and is noiseless.
Language-Conditioned Manipulation Tasks.
We extend the Ravens benchmark [2] set in PyBullet [60] with 10 language-conditioned manipulation tasks.
See Figure 1 for examples and Table 3 for challenges associated with each task.
Each task instance is constructed by sampling aset of objects and attributes: poses, colors, sizes, and object categories.
8 of the 10 tasks havetwo variants, denoted by seen and unseen , depending on whether the task has unseen attributes (e.g.color) at test time.
For colors: T seen colors =
{ yellow, brown, gray, cyan } and T unseen colors =
{ orange, purple, pink, white } with 3 overlapping colors T all = { red, green, blue }used in both the seen and unseen spilts.
For packing objects, we use 56 tabletop objects from the
Google Scanned Objects dataset [61] and split them into 37 seen and 19 unseen objects.
The language instructions are constructed from templates for simulated experiments, and human-annotatedfor real-world experiments.
For more details about individual tasks, see Appendix A.
Evaluation Metric.
We adopt the 0 (fail) to 100 (success) scores proposed in the Ravens benchmark [2].
The score assigns partial credit based on the task, e.g. 3 / 5 ⇒ 60 . 0 for packing 3 out of5 objects speciﬁed in the instructions, or 30 / 56 ⇒ 53 . 6 for pushing 30 out of 56 particles into thecorrect zone.
See Appendix A for the speciﬁc evaluation metric used in each task.
During an evaluation episode, an agent keeps interacting with the scene until an oracle indicates task-completion.
Wereport scores on 100 evaluation runs for agents trained with n = 1 , 10 , 100 , 1000 demonstrations.
4.2 Simulation Results
[Figure]
Table 1 presents results fromour large-scale experiments in
Ravens [2] and Figure 3 summarizes these results with average scores across seen and unseensplits.
Baseline Methods.
To studythe effectiveness of our two-stream architecture, we broadly compare against two baselines:
Transporter-only and CLIP-only.
Transporter-only is the original Transporter [2], or equivalently,the spatial stream of CLIP ORT with RGB-D input.
Although Transporter-only does not receive anylanguage goals, it shows what can be achieved through chance by exploiting the most likely actionsseen during training.
On the other hand, CLIP-only is just the semantic stream of CLIP ORT with
RGB and language input.
CLIP-only shows what can be achieved by ﬁne-tuning a pre-trained semantic model for manipulation without spatial information, particularly depth.
Two-Stream Performance.
Figure 3 (seen) captures the essence of our main claims.
The performance of Transporter-only saturates at 50% since it doesn’t use the language instruction to groundthe desired goal.
CLIP-only does have a goal, but lacks the spatial precision to go the last mile andthus saturates at 75% .
Only CLIP ORT (single) achieves more than 90% , which indicates that boththe semantic and spatial streams are crucial for ﬁne-grained manipulation.
Further, CLIP ORT (single) achieves 90% on most tasks with just 100 demonstrations, showcasing its efﬁciency.
In addition to these baselines, we present various ablations and alternative one-stream and twostream models in Appendix E.
To brieﬂy summarize these results, CLIP is essential for few-shot
[Table]
learning (i.e. n ≥ 10 ) in lieu of semantic stream alternatives like ImageNet-trained ResNet50 [62]with BERT [38].
Image-goal models outperform CLIP ORT (single) in packing Google objects, butthis is only because they do not have to solve the language-grounding problem.
Multi-Task Performance.
In realistic scenarios, we want the robot to be capable of any task, notjust one task.
We investigate this through CLIP ORT (multi) in Table 1 with one multi-task modeltrained on all 10 tasks.
CLIP ORT (multi) models are trained only on seen-splits of tasks, so an unseen attribute like ‘pink’ is consistent throughout single and multi-task settings.
Surprisingly, CLIP ORT (multi) outperforms single-task CLIP ORT (single) models in 44 / 72 = 61% of the evaluationsin Table 1.
This trend is also evident in Figure 3, especially in instances with 100 demonstrationsor less.
Although CLIP ORT (multi) is trained on more diverse data from other tasks, both CLIP ORT (multi) and CLIP ORT (single) have access to the same amount of data per task.
This supportsour premise that language is a strong conditioning mechanism for reusing concepts from other taskswithout learning them from scratch.
It also validates a trait of data-driven approaches where trainingon lots of diverse data leads to more robust and generalizable representations [1, 63].
However,
CLIP ORT (multi) performs worse on longer-horizon tasks like align-rope .
We hypothesize that thisis because longer-horizon tasks get less coverage of input-action pairs in the dataset.
Future workscould use better sampling methods that balance tasks according to their average time horizon.
Generalizing to Unseen Attributes.
Tasks that require generalizing to novel colors, shapes, andobjects are more difﬁcult and all our agents achieve relatively lower performance on these tasks,as shown in Figure 3 ( unseen ).
However, CLIP ORT (single) models do substantially better thanchance, i.e., Transporter-only.
The lower performances are due to the difﬁculty of grounding unseenattributes such as ‘pink’ and ‘orange’ in the language instruction “put the pink block on the orangebowl”, when the agent has never encountered words ‘orange’, ‘pink’ or their corresponding visualcharacteristics in the context of the physical environment.
Although pre-trained CLIP has beenexposed to the attribute ‘pink’, it could correspond to different concepts in the physical settingdepending on factors like lighting condition, and thus requires at least few examples to conditionthe trainable semantic decoder layers.
Additionally, we notice that CLIP ORT (single) is also lessprone to overﬁtting compared to Transporter-only.
As evidenced in towers-of-hanoi-seq-unseen-colorstask in Table 1, Transporter-only suffers from a performance drop because of rings with unseencolors despite the fact that Tower of Hanoi can be solved without attending to the colors and simplyfocusing on the ring size.
We hypothesize that since CLIP was trained on diverse internet data, itenables our agent to focus on task-relevant concepts while ignoring irrelevant aspects of the task.
[Figure]
I n p u t
P i c k
P l a c e t= t= t= t= t=
Transferring Attributes across Tasks.
One solution for dealing with unseen attributes is to explicitly learn these attributes from other tasks.
We study this with CLIP ORT (multi-attr) in Table 1 and
Figure 3 ( unseen ).
For these models, CLIP ORT (multi) is trained on both seen -and- unseen splits fromall tasks except for the task being evaluated on, for which it was only trained on the seen split.
Assuch, this evaluation measures whether having seen pink blocks in put-blocks-in-bowl-unseen-colorshelps solve “pack all the pink and cyan boxes” in packing-box-pairs-unseen-colors .
Results indicatethat such explicit transfers result in signiﬁcant improvements.
For instance, on the put-blocks-inbowls-unseen-colors task for n = 1000 , CLIP ORT (multi)’s performance increases from 43 .
3 to 73 .
4.3 Real-Robot Experiments
[Figure]
[Table]
We validated our results in hardware with a Franka
Panda manipulator.
See Appendix C for setup details.
Table 2 reports success rates for a multi-task modeltrained and evaluated on 9 real-world tasks.
Due to
COVID restrictions, we could not conduct large-scaleuser-studies, so we report on small train (5-10 demos) and test sets (5-10 runs) per task.
Overall, CLIP ORT (multi) is effective at few-shot learning with just179 samples, and the performances roughly correspondto those in simulated experiments, with simple block manipulation tasks achieving ∼ 70% .
We estimate that for more robust real-world performance at least50 to 100 training demonstrations are necessary, as evident in Figure 3.
Interestingly, we observedthat the model sometimes exploits biases in the training data instead of learning to ground instructions.
For instance, in Put Blocks in Bowl , the training set consisted of only one datapoint on “yellowblocks” being placed inside a “blue bowl”.
This made it difﬁcult to condition the model to place“yellow blocks” in non-blue bowls.
But instances with just one or two examples where a coloredblock went to different colored bowls was sufﬁcient to make the model pay attention to the language.
In summary, unbiased datasets containing both a good coverage of expected skills and invariances,and a decent number of training demonstrations, are crucial for good real-world performance.
Conclusion
We introduced CLIP ORT , an end-to-end framework for language-conditioned ﬁne-grained manipulation.
Our experiments, speciﬁcally with multi-task models, indicate that data-driven approachesto generalization have yet to be fully-exploited in robotics.
Coupled with the right action abstraction and spatio-semantic priors, end-to-end methods can quickly learn new skills without requiringtop-down pipelines that need task-speciﬁc engineering.
While CLIP ORT can solve a range of tabletop tasks, extending it to dexterous 6-DOF manipulation that goes beyond the two-step primitive remains a challenge.
As such, it cannot handle complex partially-observable scenes, or output continuous control for multi-ﬁngered hands, or predicttask-completion (see Appendix H for an extended discussion).
But overall, we are excited by theconﬂuence of data and structural priors for building scalable and generalizable robotic systems.
A Task
[Table]
We extend the Ravens benchmark [2] to 10 language-conditioned.
8 out of 10 tasks have two evaluation variants, denoted by seen and unseen in their names.
See Table A for an overview of thechallenges associated with each task and split.
Figure 5 presents the full list of attributes, shapes,and objects across seen and unseen splits.
All tasks use hand-coded experts to generate expert demonstrations.
These experts use privileged state information from the simulator along with pre-speciﬁedheuristics to complete the tasks.
We refer the reader to the original Transporter paper [2] for detailsregarding these experts.
The following is a description of each language-conditioned task:
A.1 Align Rope
Example: Figure 1(a).
Task: Manipulate a deformable rope to connect its end-points between two corners of a 3-sidedsquare.
There are four possible combinations for aligning the rope: “front left tip to front right
tip”.
“front right tip to back right corner”, “front left tip to back left corner”, and “back right corner toback left corner”.
Here ‘front’ and ‘back’ refer to canonical positions on the 3-sided square.
Theposes of both the rope and 3-sided square are randomized for each task instance.
Objects: All align-rope instances contain a rope with 20 articulated beads and a 3-sided square.
Success Metric : The poses of all beads match the line segments between the two correct sides.
A.2 Packing Unseen Shapes
Example: Figure 1(b).
Task: Place a speciﬁed shape in the brown box.
Each task instance contains 1 shape to be pickedalong with 4 distractor shapes.
The shape colors are randomized but have no relevance to the task.
This task does not require precise placements and is mostly a test of the agent’s semantic understanding of arbitrary shapes.
Objects: packing-unseen-shapes is trained with seen shapes but evaluated on unseen shapes from
Figure 5.
Success Metric :
The correct shape is inside the bounds of the brown box.
A.3 Assembling Kits Seq
Example: Figure 1(c).
Colors seen
Shapes
Google
[Figure]
Task: Precisely place each speciﬁed shape in the speciﬁed hole following the order prescribed in thelanguage instruction at each timestep.
This is one of the hardest tasks in the benchmark requiringprecise placements of unseen shapes of unseen colors and grounding spatial relationships like “themiddle square hole” or “the bottom letter R hole”.
Each task instance contains 5 shapes and a kitwith randomized poses.
Objects: Both assembling-kits-seq-seen-colors and assembling-kits-seq-unseen-colors are trained on seenshapes but evaluated on unseen shapes from Figure 5.
However for color randomization, assemblingkits-seq-seen-colors is trained and evaluated on seen colors, and assembling-kits-seq-unseen-colors istrained with seen colors but evaluated on unseen colors from Figure 5.
Success Metric:
The pose of each shape matches the speciﬁed hole at the correct timestep.
Theﬁnal score is the total number of shapes that were placed in the correct pose at the correct timestep,divided by the total number of shapes in the scene (always 5).
A.4 Put Blocks in Bowl
Example: Figure 1(d).
Task: Place all blocks of a speciﬁed color in a bowl of speciﬁed color.
Each bowl ﬁts just one blockand all scenes contain enough bowls achieve the goal.
Each task instance contains several distractorblocks and bowls with randomized colors.
The solutions to this task are multi-modal in that therecould be several ways to place the blocks speciﬁed in the language goal.
This task does not requireprecise placements and mostly tests an agent’s ability to ground color attributes.
Objects: put-blocks-in-bowl-seen-colors is trained and evaluated on seen colors from Figure 5 for bothblocks and bowls.
put-blocks-in-bowl-unseen-colors is trained on seen colors but evaluated on unseencolors from Figure 5 for both blocks and bowls.
Success Metric: All blocks of the speciﬁed color are within the bounds a bowl of the speciﬁedcolor.
The ﬁnal score is the total number of correct blocks in the correct bowls, divided by the totalnumber of relevant color blocks in the scene.
A.5 Packing Box Pairs
Example: Figure 1(e).
Task: Tightly pack all the boxes of two speciﬁed colors inside the brown box.
All scenes containthe exact number of relevant color blocks to ﬁll the box completely, but also contain some distractorboxes of irrelevant colors.
The sizes of the boxes and the brown box are randomized.
The distractorobjects have equivalent sizes to the relevant objects to make the task more difﬁcult.
Sometimes thescene only contains one of the two speciﬁed speciﬁed colors and the agent has to actively ignore themissing color.
Overall, this task requires both semantic understanding of colors and precise spatialreasoning for tightly packing boxes of unknown sizes.
Objects: Boxes with randomized widths and lengths and a brown box.
packing-box-pairs-seen-colorsis trained and evaluated on seen color boxes from Figure 5.
packing-box-pairs-unseen-colors is trainedon seen color boxes but evaluated on unseen color boxes from Figure 5.
Success Metric: All blocks of the two speciﬁed colors are tightly packed inside the bounds of thebrown box.
The ﬁnal score is the total volume of the correct color blocks inside the box, divided bythe total volume of the relevant color blocks in the scene.
A.6 Packing Google Objects Seq
Example: Figure 1(f).
Task:
Place the speciﬁed objects in the brown box following the order prescribed in the languageinstruction at each timestep.
This task does not require precise placements and mostly evaluates anagent’s ability to ground semantic object descriptions.
All objects in a scene are unique without anyduplicates.
The poses of the objects and the box are randomized for each scene.
Objects: packing-seen-google-objects-seq is trained and evaluated on all 56 objects in Figure 5.packing-unseen-google-objects-seq is trained on 37 seen objects but evaluated on 19 unseen objectsin Figure 5.
Success Metric: Each speciﬁed object is within the bounds of the brown box at the correct timestep.
The ﬁnal score is the total volume of the correct objects placed inside the box at the correct timestep,divided by the total volume of the relevant objects.
A.7 Packing Google Objects Group
Example: Figure 1(g).
Task: Place all objects of the speciﬁed category in the brown box.
This task does not requireprecise placements or following a speciﬁc action sequence.
Each scene contains objects of multiplecategories with each category containing at least 2 duplicates.
The task cannot be solved by countingthe number of objects since there are distractor objects, each with 2 or more duplicates.
Objects: packing-seen-google-objects-group is trained and evaluated on all 56 objects in Figure 5.packing-unseen-google-objects-group is trained on 37 seen objects but evaluated on 19 unseen objectsin Figure 5.
Success Metric: All speciﬁed objects of a category are within the bounds of the brown box.
Theﬁnal score is the total volume of the correct objects in the box, divided by the total volume of therelevant objects of the speciﬁed category in the scene.
A.8 Stack Block Pyramid
Example: Figure 1(h).
Task:
Build a pyramid of colored blocks in a color sequence speciﬁed through the step-by-steplanguage instructions.
Each task contains 6 blocks with randomized colors and 1 rectangular base,all initially placed at random poses.
Objects: 6 blocks and 1 rectangular base.
stack-block-pyramid-seq-seen-colors is trained and evaluatedon seen color blocks from Figure 5.
stack-block-pyramid-seq-unseen-colors is trained on seen colorblocks but evaluated on unseen color blocks from Figure 5.
Success Metric:
The pose of each block at the corresponding timestep matches the speciﬁed location.
The ﬁnal score is the total number of blocks in the correct pose at the correct timestep, dividedby the total number of blocks (always 6).
A.9 Separating Piles
Example: Figure 1(i).
Task:
Sweep the pile of blocks into the speciﬁed zone.
Each scene contains two square zones: onerelevant to the task, another as a distractor.
The pile and zones are placed at random poses on thetable.
Objects: A pile of colored blocks and two squares.
separating-piles-seen-colors is trained and evaluated on seen colors from Figure 5 for all blocks and squares.
separating-piles-unseen-colors is trainedon seen colors but evaluated on unseen colors from Figure 5 for all blocks and squares.
Success Metric: All blocks are inside the bounds of the speciﬁed zone.
The ﬁnal score is the totalnumber of blocks inside the correct zone, divided by the total number of blocks in the scene.
A.10 Towers of Hanoi Seq
Example: Figure 1(j).
Task: Move the ring to the speciﬁed peg in the language instruction at each timestep.
The sequenceof ring placements is always the same, i.e. the perfect solution to three-ring Towers of Hanoi.
Thistask can be solved without using colors by just observing the ring sizes.
However, it tests the agent’sability to ignore irrelevant concepts to the task (color in this case).
The task involves precise pickand place actions for moving the rings from peg to peg.
Objects: 1 peg base and 3 rings (small, medium, and big).
towers-of-hanoi-seen-colors is trained andevaluated on seen ring colors from Figure 5.
towers-of-hanoi-unseen-colors is trained on seen ringcolors but evaluated on unseen ring colors from Figure 5.
Success Metric:
The pose of each ring at the corresponding timestep matches the speciﬁed peglocation.
The ﬁnal score is the total number of correct ring placements, divided by total steps in theperfect solution (7 for three-ring Towers of Hanoi).
Conv Block, 3x3, 256, /
Indentity Block, 3x3, 256, /
[Table]
Spatial Dense Features
[Table]
Semantic
Language 6x320x dx320x 3x3 conv, 64, /
Conv & Identity Block
Up Block 3x3 conv, 1024, /
Up Block,
Conv Block, 3x3, 64, /
Identity
[Table]
3x3, 64, /
[Table]
1x1 conv, 128, / 3x3 conv, 128, / 1x1 conv, 128, / 1x1 conv, 128, / 3x3 conv, 128, / 1x1 conv, 128, /
Conv Block, 3x3, 128, /
Identity Block, 3x3, 128, /
Conv Block, 3x3, 256, /
Identity Block, 3x3, 256, /
Conv Block, 3x3, 256, /
Identity Block, 3x3, 256, /
Conv Block, 3x3, 512, /
Identity Block, 3x3, 512, /
Conv Block, 3x3, 1024, /
Identity Block, 3x3, 1024, /
Conv Block, 3x3, 1024, / 2x
Bilinear Upsample
Identity Block, 3x3, 1024, /
Conv Block, 3x3, 512, / 2x
Bilinear Upsample
Identity Block, 3x3, 512, /
Conv Block, 3x3, 256, / 2x
Bilinear Upsample
Identity Block, 3x3, 256, /
Conv Block, 3x3, 128, / 2x
Bilinear Upsample
Identity Block, 3x3, 128, / 2x
[Table]
Upsample
Concat 3x3 conv, 128, / 3x3 conv, 128, /
Conv Block, 3x3, 64, / 2x
Bilinear Upsample
Identity Block, 3x3, 64, /
Conv Block, 3x3, 32, / 2x
Bilinear Upsample
Identity Block, 3x3, 32, /
Conv Block, 3x3, 16, / 2x
Bilinear Downsample
Identity Block, 3x3, d, /
Semantic Dense Features dx320x
Conv Block, 3x3, 128, / 2x
Bilinear Upsample
Identity Block, 3x3, 128, /
Conv Block, 3x3, 64, / 2x
Bilinear Upsample
Identity Block, 3x3, 64, /
Conv Block, 3x3, 32, / 2x
Bilinear Upsample
Identity Block, 3x3, 32, /
Conv Block, 3x3, 16, / 2x
Bilinear Downsample
Identity Block, 3x3, d, /
Stem
Layer Layer
Layer Layer
Up Block,
Up Block,1024x 1024x7x 512x14x 256x28x
Transformer
FC & Tile (7x7)
FC & Tile (14x14)
FC & Tile (28x28)
CLIP Sentence Encoder (Frozen) d
B Two Stream Architecture Details
[Figure]
Figure 6 provides a detailed architecture diagram of CLIP ORT ’s two-stream design.
We use ReLUactivations after each conv and identity blocks without any Batch Normalization.
Note thatwe repeat the depth input to match the dimensions of the RGB image
R H × W × 1 → R H × W ×
3following
Transporter [2].
All models were implemented in PyTorch [64].
For CLIP, we use theimplementation and pre-trained checkpoint released by the authors 2 .
C Robot Setup
[Figure]
Hardware Setup.
All real-robot experiments wereconducted on a Franka Panda robot with a parallelgripper.
For perception, we use a Kinect-2 RGB-D camera mounted on a tripod, tilted down looking at the table.
Although the Kinect-2 provides images at a resolution of1280 × 720 , we use downsampled 960 × 540 images fora faster user-interface.
The extrinsic calibration betweenthe camera and the robot base-frame is computed with an
AR Marker through ARUCO ROS 3 .
See Figure 7 for anoverview of the setup.
Demonstrations and Execution.
For collecting demonstrations with the Franka Panda, we developed a 2D interactive tool that uses the top-down RGB view from the
Kinect-2 to specify pick-and-place locations.
The userﬁrst selects a 2D bounding box on the live RGB feed, andthen picks a discrete rotation angle by clicking around thebounding box.
For grasping, we use a simple heuristic todetermine the height at which to close the ﬁngers.
Firstwe segment the pointcloud encapsulated by the bounding box, then we vertically crop the pointcloud up to theheight of the gripper ﬁngers, and then compute a 3D centroid of the selected points by taking an average.
This 3D centroid is used to plan a path for theend-effector with an RRT* motion-planner to execute a predeﬁned sequence – go down, open/closethe gripper, raise up.
For executing a trained CLIP ORT model, a similar grasping approach is used,but instead of the user-speciﬁed bounding box, we take 32 × 32 crops centered around the pick andplace predictions (i.e. affordance argmax) to compute 3D centroids from the pointcloud.
Only thesweeping and folding actions are different in that the end-effector does not raise up after grasping.
Pick Rotations for Parallel Grippers.
The suction gripper used in simulation does not require apick rotation since the grasps are speciﬁed as pin-point locations.
However, with the Franka Panda,the parallel gripper requires a speciﬁc yaw rotation at which to grasp an object.
To handle this, weseparate the pick module Q pick into two components: locator and rotator.
The locator predicts a pixellocation ( u, v ) given the full observation and language input.
The rotator takes a 64 × 64 crop of theobservation at ( u, v ) along with the language input and predicts a discrete rotation angle by selectingfrom one of the k rotated crops.
We use k = 36 in all our hardware experiments.
While it’s possibleto predict both the location and rotation with a single module, this decoupled approach allows us toﬁt the model on a single GPU (NVIDIA P100) with reduced memory usage from cropped rotations.
D Data Augmentation
Following common practice and the original Transporter implementation [2], we augment the training samples by applying random SE (2) transformations.
Augmentations where T pick or T place are outof frame after the transformation are discarded.
These augmentations are particular important forlearning spatially-equivariant representations with FCNs without overﬁtting to images from limitedtraining demonstrations.
2 https://github.com/openai/CLIPhttps://github.com/pal-robotics/aruco
_ros
E
Ablations and Baselines
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
Table 4 presents various baselines and ablations from our simulated experiments.
The following isa description of each model:
One-Stream Transporter-only is the original Transporter [2] with RGB-D input, or equivalently, thespatial stream of CLIP ORT .
For all experiments, we implemented our own version of Transporter in
PyTorch and did not use the modeling code provided with the original paper.
Our Transporter modelsare also trained for 200K iterations instead of 40k iterations.
One-Stream CLIP-only is the semantic stream of CLIP ORT with RGB and language input.
One-Stream Language Transporter is Transporter [2], but the bottleneck features are conditionedwith CLIP language features in a similar fashion to the semantic stream in CLIP ORT .
This modelperforms very poorly because the high-level language features corrupt the low-level spatial featuresnecessary for precise pick-and-place actions.
One-Stream Image-Goal Transporter is a goal-conditioned version of Transporter [6] which receives a goal-image as input.
For sequential tasks with a speciﬁc order (indicated with seq in theirname), we provide the goal-image from the next timestep, and for non-sequential tasks we providethe goal-image from the ﬁnal timestep.
The implementation follows the goal-conditioned Transporter proposed in [6], except we found that element-wise addition worked better than element-wiseproduct for combining goal-image features with Q place features.
Two-Stream CLIP-Transporter w/o skips is a variant of the CLIP ORT model without skip connections from the CLIP-ResNet encoder to the decoder layers.
The results in Table 4 show that theseskip connections are particularly important for good performance.
We hypothesize that utilizingdifferent levels of semantic information from the visual encoder – patterns, shapes, parts, objects,and high-level concepts, is crucial for conditioning the semantic stream decoders.
Two-Stream RN50-BERT-Transporter is the same two-stream architecture as CLIP ORT , except instead of the CLIP ResNet50, we use a standard ResNet50 [62] pre-trained on ImageNet classiﬁcation.
And instead of the CLIP sentence encoder, we use a pretrained DistilBERT model [65] toextract language embeddings.
CLIP offers the beneﬁt of multi-modal alignment between vision andlanguage features while not being restricted to instance segmentation or bounding box detectionpipelines.
Two-Stream Untrained-Sem-Transporter uses an untrained ResNet50 and Transformer language encoder for the semantic stream.
Even without any pre-training, the random features from the semantic stream somewhat help in conditioning policies.
However, the performances are substantiallyworse than models with pre-trained multimodal features, which enable better generalization.
F Performance on Demo-Conditioned Tasks
[Table]
To investigate if our framework can be applied to demo-conditioned tasks that do not require language instructions, we run evaluations on the original Transporter tasks [2].
Table 5 compares ourtwo-stream architecture without language conditioning to Transporter.
Our method outperforms
Transporter in 30 / 40 =
75% of the evaluations in Table 5, especially in low-data regimes with100 demonstrations or less.
Particularly for the assembling-kits and manipulating-rope tasks, the twostream architecture shows signiﬁcant performance gains.
We hypothesize that this is because the
CLIP-ResNet model provides a strong visual prior on object representations for learning generalizable policies.
G Affordance Prediction Examples
Figure 9 showcases more examples of affordance predictions from trained CLIP ORT (multi) models.
Traditional object-centric representations like pose and instance segmentation generally struggle torepresent piles of beans or squares on a chessboard.
In such cases, a single detector would have tobe trained (with supervision data) to detect every bean and square on the chessboard, which is ofteninfeasible, especially in multi-task settings.
[Figure]
I n p u t
P i c k
P l a c e
H Limitations and Risks
While CLIP ORT is highly capable, it is not without issues.
In the following sections we discussvarious limitations and risks of using CLIP ORT for real-world manipulation.
Balanced Datasets.
CLIP ORT can learn generalizable policies from very few demonstrations, but itrelies heavily on a balanced training dataset with a good converge of expected skills and invariances.
As discussed in Section 4.3, the model will exploit any bias, e.g. always place “yellow blocks” inside‘blue bowls” if that is the only example of yellow blocks that it’s provided with.
Sometimes thesebiases can be hard to spot since everything (from perception to action) is trained end-to-end throughdemonstrations.
During our real-world experiments we ended up iteratively reﬁning some datasetsafter ﬁnding such biases during execution.
Hand-Eye Calibration and Closed-Loop Control.
The execution of policies is sensitive to theaccuracy of the hand-eye calibration.
The action-space of CLIP ORT is 2D pixels with yaw-rotations.
Translating these pixel coordinates to end-effector poses relies on carefully calibrated extrinsicsbetween the robot’s base frame and the RGB-D camera.
Further, while the framework takes closedloop actions across discrete pick-and-place timesteps, the execution of each pick and place primitiveitself is open-loop.
This restricts usage to mostly quasi-static tasks and leads to issues if objectsmove while the robot is executing a pick or place primitive.
Future works could incorporate aseparate visuo-servoing mechanism for more robust grasping.
Dexterous Manipulation.
Extending CLIP ORT ’s action-space to 6-DOF or N-DOF control fordexterous non-quasi-static manipulation is non-trivial.
The SE (2) action-space is one of the keyfactors that make Transporter and CLIP ORT highly data efﬁcient.
Since the actual end-effectorcontrol is abstracted away, the model can easily reason about high-level affordances at discretetimesteps, but at the price of loosing dexterity.
Similarly, extending SE (2) equivariance to SE (3)equivariance is also non-trivial.
Cross-correlating in voxelized 3D spaces might be expensive andslow.
Grasping Novel Objects.
CLIP ORT has some limited capacity in grasping unseen instances ofobjects in one-shot or few-shot settings.
While CLIP is a pure vision-language model with nounderstanding of affordances, actions, or physical properties, in CLIP ORT we ﬁne-tune CLIP’svisual representations in the semantic decoder layers to produce visual affordance predictions – likegrasping pliers by the handle.
We illustrate this in Figure 10 with an example of one-shot learning.
Despite having seen just a single training example with pliers, CLIP ORT is able to correctly graspthe handles of 2/3 unseen pliers of different shapes, sizes, and colors.
The model fails in Test 3where the instance is signiﬁcantly outside the training distribution.
But even so, the model is ableto correctly localize the pliers among the distractor objects, and with a few more training examplesit might be able to correctly grasp the instance.
In contrast, RN50-BERT struggles to identify plierswith just a single example since pliers are not part of the 1000 ImageNet classes [66].
Further,without the appropriate language goal to condition the policy, e.g. when provided with a nonsensicalobject name like “dax”, the model falls back to the most familiar object seen during training.
Grounding Complex Object Relationships.
In general, CLIP ORT struggles with complex objectrelationships that require reasoning about several objects.
The model performs poorly on assemblingkits-seq tasks that involve grounding spatial relationships like “middle” with unseen shapes and language.
The model’s capacity to infer these relationships purely from dense global features might belimited.
Also, CLIP ORT cannot count objects since it does not maintain a history or belief acrosstimesteps, thus limiting instructions to ‘any’ or ‘all’ quantiﬁers.
Future works could explore neurosymbolic [67] or attention-based [68] methods for better generalization to novel object-relationships.
Scope of Language Grounding.
CLIP ORT ’s understanding of verb-noun phrases is tightlygrounded in the demonstrations and tasks seen during training.
For instance, an user could haveused “sort out all the Mars bars from the pile and put them in the yellow bin” while demonstratinga task.
Here the model only understands ‘sort’ in the context of separating something from the pileand putting it in a bin, and not in the most generic sense that is applicable in any context, like sortingnumbered blocks in descending order.
Task Completion.
CLIP ORT relies on an expert to indicate task-completion.
For real-world tasks,this means the model keeps taking actions until an user stops the execution.
Future works can addressthis issue by training a success classiﬁer [2] to predict task completion from RGB-D observations.
Risks from Pre-Trained Models.
CLIP was trained with massive amounts of “in-the-wild” imagecaption pairs from the internet.
This makes it prone to unchecked biases and associations [59, 69]that can be harmful to certain individuals and communities.
These issues are further exacerbated bythe fact that we use CLIP’s representations to take actions with a physical robot.
The end-to-endframework is also vulnerable to adversarial attacks [59] that try to maliciously affect the model’sbehavior.
For safe deployment in the real-world, keeping humans in the loop – both during thetraining phase and while instructing the robot, might help in mitigating some of these issues andpotential risks.
One
[Figure]
“put the pliers in the brown box” “put the pliers in the brown box” “put the pliers in the brown box” “put the pliers in the brown box” “put the pliers in the brown box”
[Figure]
t
[Figure]
“put the dax in the brown box” “put the dax in the brown box” “put the dax in the brown box”
[Figure]
“put
[Figure]