A Visually-Grounded Library of Behaviors for Learning to Manipulate Diverse Objects across Diverse Conﬁgurations and Views
Anonymous Author(s)
AfﬁliationAddress email
Abstract: We propose a visually-grounded library of behaviors approach forlearning to manipulate diverse objects across varying initial and goal conﬁgurations and camera placements.
Our key innovation is to disentangle the standard image-to-action mapping into two separate modules that use different typesof perceptual input: (1) a behavior selector which conditions on intrinsic andsemantically-rich object appearance features to select the behaviors that can successfully perform the desired tasks on the object in hand, and (2) a library ofbehaviors each of which conditions on extrinsic and abstract object properties,such as object location and pose, to predict actions to execute over time.
Theselector uses a semantically-rich 3D object feature representation extracted fromimages in a differential end-to-end manner.
This representation is trained to beview-invariant and affordance-aware using self-supervision, by predicting varying views and successful object manipulations.
We test our framework on pushingand grasping diverse objects in simulation as well as transporting rigid, granular,and liquid food ingredients in a real robot setup.
Our model outperforms imageto-action mappings that do not factorize static and dynamic object properties.
Wefurther ablate the contribution of the selector’s input and show the beneﬁts of theproposed view-predictive, affordance-aware 3D visual object representations.
Introduction
Object manipulation in unstructured environments is challenging since methods to manipulate objects largely depend on the object’s visual appearance.
One approach to capture the dependencebetween actions and visual features is to learn a direct mapping from image to actions with deepneural networks [1, 2, 3].
Despite their ﬂexibility, such end-to-end image-to-action mappings havebeen shown to be very data aggressive [4] and cannot easily generalize across objects, cameraviewpoints or scene conﬁgurations [5]
Approaches that abstract away object details and encode only a subset of their properties, e.g., their3D locations and velocities [6] or 3D keypoints [7, 8] make the state-to-action mapping easier tolearn with less data.
However, this abstraction may substantially limit the range of objects thata policy can handle, since useful information (object shapes, softness, weight, and material, forexample) for the downstream task may be ignored.
The challenging question is: how can we designa framework for object manipulation that uses abstract representations for sample efﬁcient behaviorlearning, but at the same time is capable of utilizing semantically-rich representations for handlingdiverse objects and views.
We propose Visually-grounded library of BEhaviors (V-BEs), a hierarchical framework for visionbased object manipulation.
Our main contribution is that the two levels of our policy hierarchy usedifferent visual representations.
At the lower level of the hierarchy, a behavior library contains a setof distinct behaviors each of which operates on an abstract object state representation that capturesdynamic properties of objects in the environment, such as object and gripper positions over time.
Generalize across different objects Generalize across different camera viewpoints
Generalize across different initial configurations
Generalize across different objects Generalize across different camera viewpoints Generalize across different initial configurations Generalize across different objects
Generalize across different camera viewpoints
Generalize across different initial configurations
Generalize across different objects Generalize across different camera viewpoints
Generalize across different initial configurations
Generalize across different objects Generalize across different camera viewpoints Generalize across different initial configurations Generalize across different objects Generalize across different camera viewpoints Generalize across different initial configurations Generalize across di ﬀ erent objects
Generalize across di ﬀ erent camera viewpoints
Generalize across di ﬀ erent initial conﬁgurations
Generalize across di ﬀ erent objects
Generalize across di ﬀ erent camera viewpoints
Generalize across di ﬀ erent initial conﬁgurations Generalize across different objects
Generalize across different camera viewpoints
Generalize across different initial configurations Generalize across different objects
Generalize across different camera viewpoints
Generalize across different initial configurations Generalize across different objects
Generalize across different camera viewpoints
Generalize across different initial configurations
Generalize across different objects
Generalize across different camera viewpoints
Generalize across different initial configurations Generalize across different objects
Generalize across different camera viewpoints
Generalize across different initial configurations Generalize across different objects
Generalize across different camera viewpoints
Generalize across different initial configurations
Behaviors can be redundant, use different state abstractions, be closed loop or open loop, and belearned or manually engineered with different algorithms [9, 10, 11].
At the higher level of thehierarchy, a selector is trained using environment interactions to associate objects to behaviors thatcan successfully manipulate them.
This selector takes as input RGB-D images of the scene, mapsthem to semantically-rich and view-invariant 3D object feature representations [12], and outputsthe behavior that best suits the presented object and task.
The decomposition of visual informationinto a perceptually rich selector network and behaviors with state abstraction makes the behaviourseasier to learn with reinforcement learning or be designed by an engineer, and the selector easier tolearn by trial-and-error learning as it does not bear the burden of producing low-level actions.
Attest time, we use 3D object detectors to localize the object, and present its location and size to thelow-level controllers when necessary.
Our second contribution is proposing a visual representation for our behaviour selector that is robust to changing camera viewpoint (with respect to a non-changing robot torso), as well as objectplacements.
We use the differentiable 2D-to-3D neural networks of Tung et al.
[12] to map an input
RGB-D image alongside camera intrinsics and extrinsics to transform 3D visual scene feature mapsfrom the current camera viewpoint to the world frame, ensuring that visual features are bound to astatic world frame—that coincides with the frame of the agent’s body—and do not change dramatically while the camera moves.
The 3D visual feature representations learn based on environmentinteractions to embed objects close to the behaviours that can manipulate them.
They are furthertrained from the auxiliary task of view prediction using a multi-view camera setup, which encourages the model to complete missing information from the current view so the 3D visual feature mapsare consistent across views and through occlusions.
We test the proposed model in two manipulation tasks: pushing and grasping.
We show that ourproposed framework can successfully manipulate diverse objects and generalize the learned skillsto unseen objects with varying object starting positions, initial poses, goal locations, and cameraviewpoints in simulation.
We empirically show the proposed model outperforms existing end-to-endimage-to-action mappings [4] and state abstracted object-to-action mappings that use only 3D objectlocations [6] in simulation.
We ablate the contribution of our higher-level 3D feature representationused in our selector and then compare against 2D pretrained baselines.
Lastly, we test our model ina real robot setup where the robot transports rigid, granular, and liquid objects onto a plate.
Related Work
Libraries of Behaviors: Libraries of behaviors have been considered in many previous works, andlibrary elements have been called action, motion, behavioral, or motor primitives, parameterizedpolicies, options, chunks, macros, subroutines, behavioral units, and skills, as well as many otherterms.
However, most previous work either assumes that the environment state is known and doesnot use perception, or has a different approach to perception than ours.
Work of Neumann et al.on modular policies [13] deﬁnes a motor primitive as a mapping from a robot’s joint angles andvelocities and not the locations of objects.
Rather, object locations are used as context for selectinga motor primitive.
This means i) motor primitives are “blind” to the object 3D location throughoutthe episode, and ii) object appearance is discarded which means the library of primitives cannothandle object variability.
Work of de Silva [14] on parameterized policies similarly considers asthe contextual vector the desired goal location of a dart, and learns a mapping from this locationto policy parameters, where each policy operates only over the robots’ joints and is representedas a Dynamic Motion Primitive (DMP) [15, 16, 17].
To extend DMPs to tasks involving physicalinteractions, Kroemer et al.
[18] parameterize DMPs based on the distance of an object part fromthe gripper, but only consider known object 3D shapes.
Strudel et al.
[19] use a depth frame as
[Figure]
BehaviorLibrary
[Figure]
input to a meta-policy that selects skills to execute.
However, the use of modularity over skills isfor temporal sequencing: there is no mixture of behaviors per manipulation task, rather an image toaction mapping is learnt with behavior cloning from demonstrations.
reinforcement learning typically considers only one monolithic policy learned via reinforcementlearning [9, 20, 21], imitation learning [22, 23, 1], or a combination of the two [24, 25].
Mostapproaches take as input 3D object and gripper centroids, orientations, and velocities, and ignorecontextual visual information [26].
Existing methods are commonly trained in a ﬁxed environmentand evaluated in terms of their ability to discover a policy rather than their ability to generalize topreviously unseen circumstances, e.g., novel objects and views, which is the goal of this work.
Methods that do aim to generalize across objects learn a mapping from images-to-actions [4], depthto-actions [27] or pointcloud-to-actions [28, 29].
They have been successful in object grasping[27, 30] and object pushing from a ﬁxed camera view [31, 4].
Seminal works of [4, 32] train animage-to-action deep neural network by imitating trajectories obtained from planners that operate ina low-dimensional state space.
Nevertheless, such monlithic image-to-action deep mappings don’tgeneralize well across different camera viewpoints [5, 33, 34] and objects.
Works that attempt totransfer visuomotor policies learned in simulation to the real-world often require identical placementof the camera in the real world [35, 6].
Florence et al.
[36] train manipulation policies parameterizedby the 2D or 3D locations of a designated set of visual descriptors obtained from RGBD images,and show their policies generalize across objects of the same object category, e.g., shoes of different shapes.
The proposed work further allows generalization across object categories, and noexplicit descriptor selection or optimization is necessary.
Furthermore, our framework permits eachbehaviour to use a different state representation.
Hierarchical Reinforcement Learning (HRL): Most existing HRL methods [37, 38] focus ondiscovering behaviors and temporally sequencing them for long-horizon tasks.
This paper insteaduses a hierarchical vision-based policy architecture to improve model generalization across objects,conﬁgurations, and viewpoints.
There is additional discussion of related work in the supplementary materials.
Methods
Problem Setup We consider the problem of manipulating (e.g. grasping, pushing, etc.)
diverse setsof objects from various initial conﬁgurations and camera viewpoints.
At training time, the agenthas access to training objects O train , RGB-D images from J viewpoints v 1 , . .
. , v J ∈ V , and it caninteract with the training object from randomly selected viewpoints.
At test time, the agent has toperform the same manipulation task as at training time on test objects O test from selected viewpoints.
Performance is measured by the success rate of manipulating all test objects with random initialconﬁgurations and camera viewpoints.
Our framework is comprised of two major components — (1) a library of behaviors
Π = { π i | i =1 , 2 , . . .
, K } and (2) a selector function G .
Given an RGBD image I of the scene captured fromcamera view v , denoted by I v =
{ I, v } , and an object 3D bounding box o , the selector G obtains theprobability of successfully manipulating the object when applying behavior
π i on the object by com-
[Figure]
Crop to Object
[Figure]
(Camera
[Figure]
) (Robot Frame) 3D Transform
Retrieval Key
Retrieval Key Retrieval Key
Behavior Behavior
Behavior
Execute Selected Behavior
GRNN
Behaviors
The robot interacts with training objects using behaviors in the library to collect binary success or failure outcomes as interaction labels.
The collected interaction labels are then used to train the behavior selector.
[Figure]
puting a query object feature representation f
( I v , o ) and compares it with learned key embeddingsκ
i = κ ( π i ) , i = 1 , ..., K associated with each behavior π i ,
[Equation]
where φ is the learnable neural network parameters for function f, (cid:104)· , ·(cid:105) is the inner product operationand σ is the sigmoid function.
We will detail the feature dimension of f ( I v , o ; φ ) and κ in thenext section.
At test time, we select the behavior π i that has the highest probability of leading tosuccessful manipulation.
Each behavior π i uses its own abstract state representation for the objects to manipulate, for example, 3D object locations and 3D poses, and does not necessarily take into account other aspects ofthe input image.
Each behaviour is designed or learns to handle only a subset of object shapes andorientations.
To handle a diverse set of objects and orientations, our selector learns to pick differentbehaviors for different objects or orientations.
One advantage of the modularity of our policy architecture is that each behavior can use its own state abstraction, such as object 6D-poses, 3D boundingboxes, part-based 3D boxes, or 2D or 3D object keypoint locations [7].
Our framework supportsintegrating a wide range of existing models, representations, and controllers as selectable behaviors.
In the rest of this section, we detail the architecture and training details of the behavior selector in
Section 3.1, and describe how we acquire a library of behaviors for grasping and pushing—the tworobot manipulation tasks we evaluate our framework on—in Section 3.2.
3.1 Visually-Grounded Behavior Selector ( G )
The behavior selector G is a classiﬁer that learns object-centric visual feature representationf ( I v , o; φ ) for the object box o in image view I v , and behavioural key embeddings κ i , i = 1 ..
Kfor the behaviours in the library to retrieve behaviours compatible with a particular object.
Training the selector with interaction labels.
We learn the object feature representation and behavioural keys through trial-and-error.
In each trial, our agent applies a randomly sampled behaviourπ i on an object o in the workspace which results in binary success or failure outcome (cid:96) ∈ { 0 , 1 } ,which we call interaction labels .
Agent interactions are organized as tuples of the form ( I v , o ,
π i , (cid:96) ) .
With the interaction experience as training data T , we train the feature encoder f and behaviouralkeys κ i = κ
( π i ) , i = 1 ...K with the loss:
[Equation]
afford (
[Equation]
(2) where BCE ( y, p ) = − y · log( p ) − (1 − y ) log(1 − p ) is the binary cross entropy loss.3D object feature representation.
Our object feature representation f ( I v , o; φ ) is computed using
Geometry-aware Recurrent Neural Networks (GRNNs) of Tung et al.
[12], which are end-to-enddifferentiable architectures that map a single RGB-D image or a set of multi-view images to 3Dfeature representation of the scene the image(s) depict.
Given a posed image I v , GRNNs obtainscene-centric 3D feature representation M = GRNN φ ( I v ) ∈
R H
× W × D × C through differentiable 2D-to-3D operations, 3D convolution-based reﬁnement, and 3D rotation operations that align the3D feature representations with the robot’s coordinate frame (as opposed to camera frame), with
W, H, D denoting the width, height and depth of the scene map and C denotes the feature dimensionof the 3D scene feature map.
From the scene map M , we obtain object-centric feature representation f
( I v , o; φ ) =
crop ( M , o ) ∈ 163 R 64 × 64 × 64 × 32 by cropping the scene map using a ﬁxed-size axis-aligned box o , centered around theobject we wish to manipulate.
The feature cropping operation is similar to the one used in MaskRCNN [39].
The retrieval policies keys are also learned in the same representation space.
Thus,both f ( I v , o; φ ) and κ i , i = 1 ..K have length 64 × 64 × 64 × 32 in the experiments.3D Object detector We learn a detector with 3D Mask-RCNN built on top of the GRNNs featureencoder [12].
We use groundtruth 3D object boxes at training time, and predicted 3D object boxesat test time, where we train our representation to detect objects in 3D.
View prediction and occupancy prediction as an auxiliary task.
We use view prediction andoccupancy prediction as an auxiliary task to help our image encoder generalize better in its ability toselect behaviours.
These two self-supervised prediction tasks have been shown to provide a usefulpretraining or co-training objective for 3D object detection in [40].
Given an input posed image I nvand a query view q n , the overall self-supervised prediction loss reads:
[Equation]
[Table]
[Table]
) where P θ ( M , q ) is a projection function that projects a 3D feature map M from the query viewpointq to a 2D feature map and decodes it to a target image ¯ I nq using an image decoder with neuralnetwork weights θ ,
Occ η ( M ) ∈ R 64 × 64 × 64 is a voxel occupancy prediction function that predictsa 3D occupancy map from an input 3D feature map M using a single 3D convolution layer withweights η , and occ n is the estimated occupancy map computed from all available input views inthe n th data point by voxelizing the unprojected point clouds from all available depth images.
Wetrain the model from unlabelled multi-view images captured around the table by simply moving thecameras, capturing the images, and recording the corresponding camera locations.
The ﬁnal objective for training the affordance-based visual features is
[Equation]
where λ a is a hyperparameter for balancing the two losses.
3.2 Building a Library of State Abstracted Behaviors
Any existing behaviors, whether engineered or learned using reinforcement or imitation learning,can be included in our library.
This ﬂexibility is a contribution of our modular architecture.
In thispaper, we consider two common manipulation tasks: pushing and grasping, and we build appropriatebehavioral libraries for each.
For pushing, the behaviors are deterministic goal conditioned policies a t =
π ( s t , g ) that map a stateof the environment and the robot s t =
[ s et , s rt ] and a goal state g to an action a t at time step t .
Theenvironment state s et is the 3D object centroid and the robot state s rt is the gripper 3D location, pose,and whether it is opened or closed.
Actions include 3D translation, opening (position control), andclosing (force control) of the gripper.
A goal state g is a target centroid location for the object.
Weuse a total of 25 goal conditioned policies – one is trained from the whole set of objects, while theothers are trained on disjoint subsets of object conﬁgurations organized based on object categoryand initial poses.
We train all policies using deterministic policy gradients (DDPG) [25] with goalrelabelling (HER) [9] while randomizing initial and goal object 3D locations.
For grasping, we design controllers π ( a t | g ; p grasp , q grasp ) which given a 3D grasping pointp grasp ∈ R 3 relative to the center of the object and a grasping 3D angle q grasp ∈ R 2 , movethe gripper (open loop) to the grasping 3D point location, close it, and move it to the desired goallocation.
The grasping angle q grasp consists of two numbers describing the yaw of the gripper andthe elevation angles between the gripper and the table surface.
When the elevation angle is smallerthan 90 degrees (not top-down grasps), we constrain the gripper to point toward the center of the object on the x-y plane.
We manually select 30 different controllers including top-down grasps withdifferent yaw orientations (top-grasps) and grasps from the side with different elevation angles ofthe gripper (side-grasps).
We empirically found that these parameterized controllers are quite stableand can be shared across multiple objects.
Experiments
Our experiments aim to answer the following questions: (1) Does the proposed library-based approach outperform existing methods that use a single combined perception and policy module, either using 2D images, 3D object locations, or 3D scene feature maps as input?
(2) Is the proposedview-invariant and affordance-aware 3D feature representation a necessary choice for the selector?(3) Does the method work on a real robot?
We test our model on grasping and pushing a wide varietyof objects in the MuJoCo simulator [41] and further test a transporting task on a real-world Franka
Panda robot arm.
4.1 Simulation Experiments
Our simulated environment consists of a Fetch Robot equipped with a parallel-jaw gripper.
Therobot is positioned in front of a table of height 0 .
4 m .
To obtain the visual observations, on eachepisode we choose 3 random cameras from cameras placed at 30 nominal different views including10 different azimuths ranging from 0 ◦ to 360 ◦ combined with 3 different elevation angles from 20 ◦ ,40 ◦ , 60 ◦ .
All cameras are looking at the center of the table top, and are 0.5 meter away from thatpoint.
All images have size 128 × 128 .
Task Descriptions:
In the grasping task, the agent has to grasp an object and move it to a speciﬁedtarget location above the table.
We use 274 distinct object meshes from 6 categories in ShapeNet[42] including toy buses, toy cars, cans, bowls, plates, and bottles.
The materials and densities of allobjects are identical.
We randomly split the dataset into 207 training objects, and 67 testing objects.
After augmenting the meshes with random scaling from 0.8 to 1.5 and random rotations aroundthe vertical z-axis, we get a total of 800 distinct object conﬁgurations (object instance and pose),600 for training and 200 for testing.
At the start of each episode, an object is placed in an area of30 cm × 16 cm around the center of the table, and a goal is sampled uniformly 10 ∼ 30 cm awayfrom the gripper’s initial position.
An episode is successful if the object centroid is within 5 cm ofthe target at the ﬁnal timestep.
In the pushing task, the agent has to push an object placed on the table to a speciﬁed target location.
We use 100 objects from 12 categories in ShapeNet [42]: baskets, bowls, bottles, toy buses, cameras,cans, caps, toy cars, earphones, keyboards, knives, and mugs.
After augmentation and splitting totrain and test sets, we obtain 615 training object conﬁgurations and 200 for testing.
The initial andthe goal position of the object are both uniformly sampled to be within 15 cm of the center of thetable along both x-axis and y-axis, although we resample if that location is already in the goal area.
An episode is successful if the object centroid is within 5 cm of the goal within 50 timesteps.
Baselines: We compare our method with various learning and non-learning based methods for objectmanipulation:
(a) Single Behavior w/
Abstract 3D State (Abstract 3D)
[9, 25]: a policy takes as input ground truth3D bounding box of the object and gripper and outputs actions.
(b) Single Behavior w/ Abstract 3D State and 2D Images (Abstract 3D + Image): a policy takes
both RGB-D images and the ground truth 3D bounding box as inputs and outputs actions.
Our
architecture resembles that of [43], but we further include ground truth object position as extra
inputs to the model.
For fair comparisons to other methods, the model only takes as input the
current state as opposed to the states in 5 past steps, as in [43].
(c) Single Behavior w/
3D Feature Tensor (Contextual 3D): a policy that takes as an input
RGB-
D images and the ground truth 3D bounding box and outputs actions.
Different from (b), the
model ﬁrst transforms the image into a -invariant 3D feature tensor
[12],then converts vector though three convolutional and a fully connected
, and concatenates
Ablation
visual feature representation Abstract 3D Abstract 3D Contextual
V-BEs
V-BEs w/
V-BEs w/o Fine-tuning [9, 25] + Image 3D (Ours) 2D features on Interaction Labels [45, 12] grasping 0.
0.31 pushing 0.
We train the baselines with different learning methods including behavior cloning [4], DDPG-HER[11, 9] and DAGGER [44].
We report the best performance we got by training the model with thesedifferent methods.
We also attempt to make all the models have similar number of parameters sothe comparison is fair.
However, larger networks are empirically harder to train and do not convergewell, so we instead increase the number of parameters in smaller networks until their performancesaturates.
For pushing, we found that using DDPG-HER is enough to lean a good Abstract 3D policyfrom scratch.
For abstract 3D + Image , we found it is critical to use behavior cloning from expertdemonstrations to obtain good policies.
The expert demonstrations are obtained from trained expertpolicies on single objects.
For Contextual 3D , we include DAGGER to enforce behavior cloningduring execution.
To train the grasping policies, we further include human demonstrations in thereplay buffer when training it with DDPG-HER.
Both abstract 3D + Image and Contextual 3D aretrained with DAGGER since ofﬂine behavior cloning is insufﬁcient.
4.2 Single Behavior versus a Library of Behaviors
We compare the proposed model with models that do not use a library-based approach, i.e., singlebehavior approaches.
As shown in Table 1, our method outperforms all the single behavior baselines.
Abstract 3D performs well, but since it does not use any visual information, its performancesaturates at around 0 . 8 for pushing and 0 .
3 for grasping.
Abstract 3D performs poorly for grasping.
The learned behaviors do not transfer well to new objects.
Adding a 2D image helps, but not dramatically (see Abstract 3D + Image in Table 1).
Although 3D feature maps obtained from GRNNs aresemantically rich and can handle varying viewpoints, the mapping to actions is harder to learn due tothe higher dimensionality of the 3D scene map, resulting in under-ﬁtting models.
Our model takesadvantage of both abstract and semantically rich representation and thus can handle better objectvariability and transferability.
The combinatorial nature of the proposed method allows the modelto capture the multi-modality in trajectory generation.
4.3 The Necessity of Building the Selector with the Proposed 3D Representations
Next, we show the importance of using view-invariant 3D visual feature representations and ﬁnetuning the selector with interaction labels.
We compare our method with two baselines: (a) a modelwith a selector that learns the visual affordance features using 2D visual features extracted from 2D
CNNs, and (a) a model with a selector that operates over 3D visual feature representation learnedonly with the view and occupancy prediction loss, as suggested in [45, 12], without ﬁne tuningwith interaction labels.
See Table 1 for the results.
Our method signiﬁcantly outperforms these twobaselines, which shows the importance of both proposed components.
To fully test the power ofexisting 2D CNNs, we also tested 2D feature selector with existing VGG network [46] pretrainedon ImageNet and ﬁne-tuned on our interaction labels.
However, the performance does not differ toomuch with shallower 2D CNNs training from scratch (We obtain a success rate of 0.78 on pushingwith the pretrained VGG network).
4.4 Transporting Task on a Real Robot
We test our model on a 7-DOF Franka robot arm equipped with a parallel-jaw gripper for a transporting task, where the robot needs to transport various rigid, granular, or liquid food ingredientsfrom random initial positions and poses onto a plate (see Figure 4).
We set up 4 Intel RealSense
RGB-D cameras that have full view of the workspace around the the center of the table.
In eachtrial, an object is placed in a 50 cm × 30 cm region on the table, and the goal is transport all objectsto a plate 25 cm to the left of the starting region.
Granular objects and liquids are placed inside containers in the beginning of the trial.
An episode is considered successful if the object is successfullytransported into the plate.
For granular objects and liquid, an episode is considered successful if atleast half of the total quantity ends up in the plate.
[Figure]
We construct our library of behaviors with 26 controllers, in which 13 of them are pick-and-placecontrollers from various grasping angles and the other 13 are pick-and-pour controllers using thesame grasping angles as the pick-and-place ones.
At evaluation, we randomly select 3 views from4 possible camera views to obtain RGB-D images as inputs to the learned selector.
We use 20 rigidobjects, 20 granular objects, and 12 bottles of liquids for our experiment, and we split them into38 training object and 14 testing objects.
To train our model, we collect a total of 3510 interactionlabels by running the 26 behaviors on the training objects with random initial poses and using thelabels to ﬁnetune the visual selector with the objective speciﬁed in Equation (4).
Our model achieves a success rate of 88.6% on the test set.
We compare our model with two baselines: 1) an image-to-action model trained with behavior cloning ( Image ), and 2) a hierarchicalmodel that uses a library of behaviors and a selector with 2D representations ( V-BEs with 2D features ).
We use the data collected during the interaction label collecting process as the data used totrain both baselines. For the Image baseline, we are not able to get it to work at all, while we didmake it work in simulation where there is more data.
It may need more data to learn a general androbust policy in the real world [47].
For the V-BEs with 2D features baseline, we get a success rate of38.0% which is worse than the proposed model. This again shows the importance of operating theselector in the proposed view-invariant and object-centric 3D feature space. Sample executions andtransporting objects in clutter are visualized in Figure 5 and included in our supplementary video.
Conclusion
We have presented V-BEs, a hierarchical policy architecture where 3D object visual feature representations are used to select from a library of behaviours.
The proposed modular architecturesupports both low-level behaviours and the selector to be learnt in a data efﬁcient manner.
We haveshown results on pushing and grasping diverse objects in simulation and the real world, across diverse viewpoints.
Our method outperforms image-to-action monolithic policies of previous works,as well as policies that operate on 3D locations and velocities alone.
Since our framework is sampleefﬁcient and simple to run, we can easily deploy complex transporting skills on a real robot arm.