VAT-M ART : L EARNING V ISUAL A CTION T RAJECTORY P ROPOSALS FOR M ANIPULATING 3D ART ICULATED O BJECTS
A BSTRACT
Perceiving and manipulating 3D articulated objects ( e.g. , cabinets, doors) in human environments is an important yet challenging task for future home-assistant robots.
The space of 3D articulated objects is exceptionally rich in their myriad semantic categories, diverse shape geometry, and complicated part functionality.
Previous works mostly abstract kinematic structure with estimated joint parameters and part poses as the visual representations for manipulating 3D articulated objects.
In this paper, we propose object-centric actionable visual priors as a novel perception-interaction handshaking point that the perception system outputs more actionable guidance than kinematic structure estimation, by predicting dense geometry-aware , interaction-aware , and task-aware visual action affordance and trajectory proposals.
We design an interaction-for-perception framework VATM ART to learn such actionable visual representations by simultaneously training a curiosity-driven reinforcement learning policy exploring diverse interaction trajectories and a perception module summarizing and generalizing the explored knowledge for pointwise predictions among diverse shapes.
Experiments prove the effectiveness of the proposed approach using the large-scale PartNet-Mobility dataset in SAPIEN environment and show promising generalization capabilities to novel test shapes, unseen object categories, and real-world data.
I NTRODUCTION
We live in a 3D world composed of a plethora of 3D objects.
To help humans perform everyday tasks, future home-assistant robots need to gain the capabilities of perceiving and manipulating a wide range of 3D objects in human environments.
Articulated objects that contain functionally important and semantically interesting articulated parts ( e.g. , cabinets with drawers and doors) especially require signiﬁcantly more attention, as they are more often interacted with by humans and artiﬁcial intelligent agents.
Having much higher degree-of-freedom (DoF) state spaces, articulated objects are, however, generally more difﬁcult to understand and subsequently to interact with, compared to 3D rigid objects that have only 6-DoF for their global poses.
There has been a long line of research studying the perception and manipulation of 3D articulated objects in computer vision and robotics.
On the perception side, researchers have developed various successful visual systems for estimating kinematic structures (Abbatematteo et al., 2019; Staszak et al., 2020), articulated part poses (Li et al., 2020; Jain et al., 2020; Liu et al., 2020), and joint parameters (Wang et al., 2019b; Yan et al., 2019).
Then, with these estimated visual articulation models, robotic manipulation planners and controllers can be leveraged to produce action trajectories for robot executions (Klingbeil et al., 2010; Arduengo et al., 2019; Florence et al., 2019;
Urakami et al., 2019; Mittal et al., 2021).
While the commonly used two-stage solution underlying most of these systems reasonably breaks the whole system into two phases and thus allows bringing together
[Figure]
well-developed techniques from vision and robotics communities, handshaking
– the standardized visual articulation models ( i.e. kinematic structure, articulated part poses, and joint parameters), may not be the best choice, since essential geometric and semantic features for robotic manipulation tasks, such as interaction hotspots ( e.g. edges, holes, bars) and part functionality ( e.g. handles, doors), are inadvertently abstracted away in these canonical representations.
We propose a new type of actionable visual representations (Do et al., 2018; Nagarajan et al., 2019;
Mo et al., 2021) exploring a more geometry-aware , interaction-aware , and task-aware perceptioninteraction handshaking point for manipulating 3D articulated objects.
Concretely, we train the perception system to predict action possibility and visual action trajectory proposals at every point over parts of 3D articulated objects (See Figure 1).
In contrast to previous work that use standardized visual articulation models as visual representations, our framework VAT-M ART predicts per-point dense action trajectories that are adaptive to the change of geometric context ( e.g. , handles, door edges), interactions ( e.g. , pushing, pulling), and tasks ( e.g. , open a door for 30 ◦ , close up a drawer by 0 .
1-unit-length).
Abstracting away from concrete external manipulation environments, such as robot arm conﬁgurations, robot base locations, and scene contexts, we aim for learning uniﬁed objectcentric visual priors with a dense and diverse superset of visual proposals that can be potentially applied to different manipulation setups, avoiding learning separate manipulation representations under different circumstances.
The proposed actionable visual priors , as a ”preparation for future tasks” (Ramakrishnan et al., 2021) or ”visually-guided plans” (Wang et al., 2019a; Karamcheti et al., 2021), can provide informative guidance for downstream robotic planning and control.
Sharing a similar spirit with Nagarajan et al.
; Mo et al.
(2021), we formulate our visual action possibility predictions as per-point affordance maps, on which the downstream robotic planners may sample a position to interact according to the predicted likelihood of success.
Then, for a chosen point for interaction, the discrete task planner may search for applicable interaction modes ( e.g. , whether to attempt a grasp) within a much smaller space formed by the visual action trajectory distribution, instead of searching in the entire solution space.
Next, considering the robot kinematic constraints and physic collisions, the continuous motion planner can further select an open-loop trajectory from the set of proposed visual action trajectory candidates as an initial value for optimization, and ﬁnally pass to the robot controller for execution.
More recent reinforcement learning (RL) based planners and controllers can also beneﬁt from our proposed solution spaces for more efﬁcient exploration.
To obtain such desired actionable visual priors , we design an interaction-for-perception learning framework VAT-M ART , as shown in Figure 2.
By conducting trial-and-error manipulation with a set of diverse 3D articulated objects, we train an RL policy to learn successful interaction trajectories for accomplishing various manipulation tasks ( e.g. , open a door for 30 ◦ , close up a drawer by 0 .
1unit-length).
In the meantime, the perception networks are simultaneously trained to summarize the
RL discoveries and generalize the knowledge across points over the same shape and among various shapes.
For discovering diverse trajectories, we leverage curiosity feedback (Pathak et al., 2017) for enabling the learning of perception networks to reversely affect the learning of RL policy.
We conduct experiments using SAPIEN (Xiang et al., 2020) over the large-scale PartNetMobility (Chang et al., 2015; Mo et al., 2019) dataset of 3D articulated objects.
We useshapes in object categories to perform our experiments and show that our VAT-M ART framework can successfully learn the desired actionable visual priors .
We also observe reasonably good gen- eralization capabilities over unseen shapes, novel object categories, and real-world data, thanks to large-scale training over diverse textureless geometry.
In summary, we make the following contributions in this work:
• We formulate a novel kind of actionable visual priors making one more step towards bridging the perception-interaction gap for manipulating 3D articulated objects; • We propose an interaction-for-perception framework VAT-M ART to learn such priors with novel designs on the joint learning between exploratory RL and perception networks; • Experiments conducted over the PartNet-Mobility dataset in SAPIEN demonstrate that our system works at a large scale and learns representations that generalize over unseen test shapes, across object categories, and even real-world data.
R ELATED W ORK
Perceiving and Manipulating 3D Articulated Objects has been a long-lasting research topic in computer vision and robotics.
A vast literature (Yan & Pollefeys, 2006; Katz et al., 2008; Sturm et al., 2009; 2011; Huang et al., 2012; Katz et al., 2013; Martin & Brock, 2014; H¨ofer et al., 2014;
Katz et al., 2014; Schmidt et al., 2014; Hausman et al., 2015; Mart´ın-Mart´ın et al., 2016; Tzionas &
Gall, 2016; Paolillo et al., 2017; Martın-Martın & Brock, 2017; Paolillo et al., 2018; Mart´ın-Mart´ın &
Brock, 2019; Desingh et al., 2019; Nunes & Demiris, 2019) has demonstrated successful systems, powered by visual feature trackers, motion segmentation predictors, and probabilistic estimators, for obtaining accurate link poses, joint parameters, kinematic structures, and even system dynamics of 3D articulated objects.
Previous works (Peterson et al., 2000; Jain & Kemp, 2009; Chitta et al., 2010;
Burget et al., 2013) have also explored various robotic planning and control methods for manipulating 3D articulated objects.
More recent works further leveraged learning techniques for better predicting articulated part conﬁgurations, parameters, and states (Wang et al., 2019b; Yan et al., 2019; Jain et al., 2020;
Zeng et al., 2020; Li et al., 2020; Liu et al., 2020; Mu et al., 2021), estimating kinematic structures (Abbatematteo et al., 2019; Staszak et al., 2020), as well as manipulating 3D articulated objects with the learned visual knowledge (Klingbeil et al., 2010; Arduengo et al., 2019; Florence et al., 2019; Urakami et al., 2019; Mittal et al., 2021).
While most of these works represented visual data with link poses, joint parameters, and kinematic structures, such standardized abstractions may be insufﬁcient if ﬁne-grained part geometry, such as drawer handles and faucet switches that exhibit rich geometric diversity among different shapes, matters for downstream robotic tasks.
Learning Actionable Visual Representations aims for learning visual representations that are strongly aware of downstream robotic manipulation tasks and directly indicative of action probabilities for robotic executions, in contrast to predicting standardized visual semantics, such as category labels (Russakovsky et al., 2015; Wu et al., 2015b), segmentation masks (Lin et al., 2014; Mo et al., 2019), and object poses (Hinterstoisser et al., 2011;
Xiang et al., 2016), which are usually deﬁned independently from any speciﬁc robotic manipulation task.
Grasping (Montesano & Lopes, 2009;
Lenz et al., 2015; Mahler et al., 2017; Fang et al., 2020; Mandikal & Grauman, 2021; Corona et al., 2020; Kokic et al., 2020; Yang et al., 2020; Jiang et al., 2021) or manipulation affordance (Kjellstr ¨ om et al., 2011; Do et al., 2018;
Fang et al., 2018; Goff et al., 2019; Nagarajan et al., 2019; Nagarajan & Grauman, 2020; Nagarajan et al., 2020;
Xu et al., 2021; Mo et al., 2021) is one major kind of actionable visual representations, while many other types have been also explored recently ( e.g. , spatial maps (Wu et al., 2020; 2021), keypoints (Wang et al., 2020;
Qin et al., 2020), contact points (You et al., 2021), etc).
Following the recent work Where2Act (Mo et al., 2021), we employ dense affordance maps as the actionable visual representations to suggest action possibility at every point on 3D articulated objects.
Extending beyond Where2Act which considers task-less short-term manipulation, we further augment the per-point action predictions with task-aware distributions of trajectory proposals, providing more actionable information for downstream executions.
Learning Perception from Interaction augments the tremendously successful learning paradigm using ofﬂine curated datasets (Russakovsky et al., 2015;
Lin et al., 2014; Chang et al., 2015; Mo et al., 2019) by allowing learning agents to collect online active data samples, which are more task-aware and learning-efﬁcient, during navigation (Anderson et al., 2018; Ramakrishnan et al., 2021), recognition (Wilkes & Tsotsos, 1992; Yang et al., 2019; Jayaraman & Grauman, 2018), segmentation (Pathak et al., 2018; Gadre et al., 2021), and manipulation (Pinto et al., 2016; Bohg et al., 2017).
Many works have also demonstrated the usefulness of simulated interactions for learning perception (Wu et al., 2015a;
Mahler et al., 2017; Xu et al., 2019; Ramakrishnan et al., 2021;
Nagarajan & Grauman, 2020; Lohmann et al., 2020; Mo et al., 2021) and promising generalizability to the real world (James et al., 2019; Chebotar et al., 2019; Hundt et al., 2019; Liang et al., 2020;
Kadian et al., 2020; Anderson et al., 2020;
Rao et al., 2020).
Our method follows the route of learning perception from interaction via using the action trajectories discovered by an RL interaction policy to supervise a jointly trained perception system, which reversely produces curiosity feedback (Pathak et al., 2017) to encourage the RL policy to explore diverse action proposals.
A CTIONABLE V ISUAL P RIORS : Action Affordance and Trajectory Proposals
We propose novel actionable visual representations for manipulating 3D articulated objects (see
Fig. 1).
For each articulated object, we learn object-centric actionable visual priors , which are comprised of: 1) an actionability map over articulated parts indicating where to interact; 2) per-point distributions of visual action trajectory proposals suggesting how to interact; and 3) estimated success likelihood scores rating the outcomes of the interaction.
All predictions are interaction-conditioned ( e.g. , pushing, pulling) and task-aware ( e.g. , open a door for 30 ◦ , close a drawer by 0 .
1-unit-length).
Concretely, given a 3D articulated object O with its articulated parts P = { P 1 , P 2 , ···} , an interaction type T , and a manipulation task θ , we train a perception system that makes dense predictions at each point p over each articulated part p ∈ ∪ P : 1) an actionability score a p | O , T , θ ∈ [ 0 , 1 ] indicating how likely there exists an action trajectory of interaction type T at point p that can successfully accomplish the task θ ; 2) a distribution of visual action trajectories P p ( ·| O , T , θ ) , from which we can sample diverse action trajectories τ p | O , T , θ ∼ P p ( ·| O , T , θ ) of interaction type T to accomplish the task θ at point p ; and 3) a per-trajectory success likelihood score r τ | O , p , T , θ ∈ [ 0 , 1 ] .
Inputs.
We represent the input 3D articulated object O as a partial point cloud S O .
We consider two typical interaction types: pushing and pulling .
A pushing trajectory maintains a closed gripper and has 6-DoF motion performing the pushing, whereas a pulling trajectory ﬁrst performs a grasping operation at the point of interaction by closing an initially opened gripper and then has the same 6-DoF motion during the pulling.
For articulated objects we use in this work, we only consider 1-DoF part articulation and thus restrict the task speciﬁcation θ ∈ R .
For example, a cabinet drawer has a 1-DoF prismatic translation-joint and a refrigerator door is modeled by a 1-DoF revolute hinge-joint.
We use the absolute angular degrees in radian ( i.e. θ ∈ [ − π , π ] ) for revolute joints and use the units of length ( i.e. θ ∈ [ − 1 , 1 ] ) relative to the global shape scale for prismatic joints.
Outputs.
Both the actionability score a p | O , T , θ and per-trajectory success likelihood score r τ | O , p , T , θ are scalars within [ 0 , 1 ] , where larger values indicate higher likelihood.
One can use a threshold of 0.5 to obtain binary decisions if needed.
Every action trajectory τ p | O , T , θ is a sequence of 6DoF end-effector waypoints ( wp 0 , wp 1 , ··· , wp k ) , with variable trajectory length ( k ≤ 5).
In our implementation, we adopt a residual representation ( wp 0 , wp 1 − wp 0 , ··· , wp k − wp k − 1 ) for the action trajectory, as it empirically yields better performance.
Each 6-DoF waypoint is comprised of a 3-DoF robot hand center x and 3-DoF orientation R .
We use the 6D-rotation representation (Zhou et al., 2019) for the orientation of wp 0 and predict 3-DoF euler angles for subsequent orientation changes.
VAT-M ART : AN Interaction-for-perception L EARNING F RAMEWORK
The VAT-M ART system (Fig. 2) consists of two parts: an RL policy exploring diverse action trajectories and a perception system learning the proposed actionable visual priors .
While the RL policy collects interaction trajectories for supervising the perception networks, the perception system provides curiosity feedback (Pathak et al., 2017) for encouraging the RL policy to further explore diverse solutions.
In our implementation, we ﬁrst pretrain the RL policy, then train the perception network with RL-collected data, and ﬁnally ﬁnetune the two parts jointly with curiosity-feedback enabled.
We describe key system designs below and will release code for our implementation.
4. T HE RL P OLICY FOR I NTERACTIVE T RAJECTORY E XPLORATION
For every interaction type T and part articulation type, we train a single conditional RL policy using
TD3 (Fujimoto et al., 2018) to collect trajectories τ that can accomplish the interaction of varying task speciﬁcations θ across all shapes O and contact points p .
In the RL training, since the RL policy is trained in the simulation for only collecting training data to supervise the perception networks, we can have access to the ground-truth state information of the simulation environment, such as the part
[Figure]
Trajectory Exploration
Data Collection
CuriosityGuidance
Perception Networks for Actionable Visual Priors 3D-PN++
O p e no
MLP MLP
MLP
Diverse Trajectory
[Figure]
TD3 RL
Curiosity Feedback
Task Environment
Intrinsic Rewards
Extrinsic
[Figure]
TrajectoryScoringModule TrajectoryProposalModule
ActionabilityPredictionModule
[Figure]
poses, joint axis, and gripper poses.
At the test time, we discard the RL network and only use the learned perception networks to predict the proposed actionable visual priors .
Fig.
2 (left) illustrates the RL training scheme and example explored diverse trajectory proposals for the task of pulling open the microwave door for 30 degrees. Below, we describe the RL speciﬁcations. Task Initialization. For the shape to interact with, we ﬁrst randomly sample a shape category with equal probability, alleviating the potential category imbalance issue, and then uniformly sample a shape from the selected category for training.
For the task speciﬁcation θ , we randomly sample within [ 10 ◦ , 70 ◦ ] for revolute parts and [ 0 . 1 , 0 . 7 ] for prismatic parts.
We also randomly sample a starting part pose θ 0 with the guarantee that the task θ can be accomplished.
For the gripper, we initialize it with ﬁngertip 0.02-unit-length away from the contact point p and pointing within a cone of 30 degrees along the negative direction of the surface normal at p .
State Space.
The RL state includes the 1-DoF part pose change ∆ θ i
= θ i − θ
at the current timestep, the target task θ , the difference θ − ∆ θ i , the gripper pose wp
= ( x 0 , R 0 ) ∈ SE ( 3 ) at the ﬁrst timestep of interaction, the current gripper pose
wp
i
=
( x i , R i ) ∈ SE ( 3 ) , the local positions for the gripper ﬁngers x f ∈ R 2 , the current contact point location p i ∈ R 3 , a normalized direction of the articulated part joint axis n j ∈ R 3 , the articulated part joint location x j ∈ R 3 (deﬁned as the closest point on the joint axis to the contact point p ), the closest distance from the contact point to the joint axis d cj ∈ R , and a directional vector n cj ∈ R 3 from the joint location to the contact point.
We concatenate all the information together as a 33-dimensional state vector for feeding to the RL networks.
Action Space.
At each timestep, we predict a residual gripper pose
wp
i − wp
i − 1 ∈ SE ( 3 ) to determine the next-step waypoint
wp i as the action output of the RL networks.
We estimate a center offset x i − x i − 1 ∈ R 3 and an euler angle difference R
i − R i − 1 ∈ SO ( 3 ) .
Reward Design.
There are two kinds of rewards: extrinsic task rewards and intrinsic curiosity feedbacks.
For the extrinsic task rewards, we use: 1) a ﬁnal-step success reward of 500 for a task completion when the current part pose reaches the target within 15% relative tolerance range, 2) a step-wise guidance reward of 300 ( | θ − ∆ θ i − 1 |−| θ − ∆ θ i | ) encouraging the current part pose to get closer to the target than previous part pose, and 3) a distance penalty of 300 · 1 [ d gc > 0 .
1 ]+ 150 d gc to discourage the gripper from ﬂying away from the intended contact point p , where d gc denotes the l 2 distance from the contact point p to the current ﬁngertip position and 1 [ q ] is a zero or one function indicating the boolean value of the predicate q .
We will describe the curiosity rewards in Sec. 4.3.
Stopping Criterion.
We stop an interaction trial until the task’s success or after ﬁve maximal waypoint steps.
and Training.
We implement the TD3 networks using MLPs and use a replay buffer of size 2048.
To improve the positive data rates for efﬁcient learning, we leverage Hindsight
Experience Replay (Andrychowicz et al., 2017): an interaction trial may fail to accomplish the desired task θ , but it ﬁnally achieves the task of θ k − θ 0 .
See Sec.
A in appendix for more details.
4. T HE P ERCEPTION N ETWORKS FOR A CTIONABLE V ISUAL P RIORS
The perception system learns from the interaction trajectories collected by the RL exploration policy and predicts the desired per-pixel actionable visual priors .
Besides several information encoding modules, there are three decoding heads: 1) an actionability prediction module that outputs the actionability score a p | O , T , θ ∈ [ 0 , 1 ] , 2) a trajectory proposal module that models per-point distribution of diverse visual action trajectories τ p | O , T , θ
= ( wp 0 , wp 1 − wp 0 , ··· , wp k − wp k − 1 ) , and 3) a trajectory scoring module that rates the per-trajectory success likelihood r τ | O , p , T , θ ∈ [ 0 , 1 ] .
Fig.
2 (right) presents an overview of the system.
Below we describe detailed designs and training strategies.
Input Encoders.
The perception networks require four input entities: a partial object point cloud
S O , a contact point p , a trajectory τ , and a task θ .
For the point cloud input S O , we use a segmentationversion PointNet++ (Qi et al., 2017) to extract a per-point feature f s ∈ R 128 .
We employ three MLP networks that respectively encode the inputs p , τ , and θ into f p ∈ R 32 , f τ ∈ R 128 , and f θ ∈ R 32 .
We serialize each trajectory as a 30-dimensional vector after ﬂattening all waypoint information.
We augment the trajectories that are shorter than ﬁve waypoint steps simply by zero paddings.
Actionability Prediction Module.
The actionability prediction network D a , similar to
Where2Act (Mo et al., 2021), is implemented as a simple MLP that consumes a feature concatenation of f p , f s , and f θ , and predicts a per-point actionability score a p | O , T , θ ∈ [ 0 , 1 ] .
Aggregating over all contact points p , one can obtain an actionability map A O , T , θ over the input partial scan S O , from which one can sample an interaction point at test time according to a normalized actionability distribution.
Trajectory Proposal Module.
The trajectory proposal module is implemented as a conditional variational autoencoder (cVAE, Sohn et al.
, composed of a trajectory encoder E τ that maps the input trajectory τ into a Gaussian noise z and a trajectory decoder D τ that reconstructs the trajectory input from the noise vector.
Both networks take additional input features of f p , f s , and f θ as conditions.
We use MLPs to realize the two networks.
We regularize the resultant noise vectors to get closer to a uniform Gaussian distribution so that one can sample diverse trajectory proposals by feeding random Gaussian noises to the decoder D τ with the conditional features as inputs.
Trajectory Scoring Module.
The trajectory scoring module D s , implemented as another MLP, takes as inputs features of f p , f s , and f θ , as well as the trajectory feature f τ , and predicts the success likelihood r τ | O , p , T , θ ∈ [ 0 , 1 ] .
One can use a success threshold of 0 .
5 to obtain a binary decision.
Data Collection for Training.
We collect interaction data from the RL exploration to supervise the training of the perception system.
We randomly pick shapes, tasks, and starting part poses similar to the RL task initialization.
For positive data, we sample 5000 successful interaction trajectories outputted by the RL.
We sample the same amount of negative data, which are produced by offsetting the desired task θ 0 of a successful trajectory by a random value with [ 0 . 1 θ 0 , 45 ◦ ] for revolute parts and [ 0 .
1 θ 0 , 0 . 45 ] for prismatic parts.
For the pulling experiments, we also consider another type of negative data that the ﬁrst grasping attempt fails.
Implementation and Training.
We train the trajectory scoring and the trajectory proposal modules before the actionability prediction module.
We use the standard binary cross-entropy loss to train the trajectory scoring module D s .
To train the trajectory proposal cVAE of E τ and D τ , besides the
KL divergence loss for regularizing Gaussian bottleneck noises, we use an L 1 loss to regress the trajectory waypoint positions and a 6D-rotation loss (Zhou et al., 2019) for training the waypoints orientations.
For training the actionability prediction module D a , we sample 100 random trajectories proposed by D τ , estimate their success likelihood scores using D s , and regress the prediction to the mean score of the top-5 rated trajectories with a L 1 loss.
See Sec.
B in appendix for more details.
4. C URIOSITY - DRIVEN E XPLORATION
We build a bidirectional supervisory mechanism between the RL policy and the perception system.
While the RL policy collects data to supervise the perception networks, we also add a curiosityfeedback (Pathak et al., 2017) from the perception networks to inversely affect the RL policy learning for exploring more diverse and novel interaction trajectories, which will eventually diversify the per-point trajectory distributions produced by the trajectory proposal decoder D τ .
The intuitive idea is to encourage the RL network to explore novel trajectories that the perception system currently gives low success scores.
In our implementation, during the joint training of the RL and perception networks, we generate an additional intrinsic curiosity reward of − 500 r τ | O , p , T , θ for a trajectory τ [Table]
t e s tca t
[Table]
ng t r a i nca t t e s t - s h a p e t
[Table]
s t -
[Figure]
t t r a i n
[Table]
[Figure]
t
[Table]
E XPERIMENTS
We perform our experiments using the SAPIEN simulator (Xiang et al., 2020) and PartNet-Mobility dataset (Chang et al., 2015; Mo et al., 2019).
We evaluate the prediction quality of the learned visual priors and compare them to several baselines for downstream manipulation tasks.
Qualitative results over novel shapes and real-world data show promising generalization capability of our approach.
Data and Settings.
In total, we use 562 shapes from 7 categories in the PartNet-Mobility dataset.
We conduct experiments over two commonly seen part articulation types: doors and drawers.
For each experiment, we randomly split the applicable object categories into training and test categories.
We further split the data from the training categories into training and test shapes.
We train all methods over training shapes from the training categories and report performance over test shapes from the training categories and shapes from the test categories to test the generalization capabilities over novel shapes and unseen categories.
We use the Panda ﬂying gripper as the robot actuator and employ a velocity-based PID-controller to realize the actuation torques between consecutive trajectory waypoints.
We use an RGB-D camera of resolution 448 × 448 and randomly sample viewpoints in front of the objects.
See Sec.
D E in appendix for more data statistics and implementation details.
5. A CTIONABLE V ISUAL P RIORS
We present qualitative and quantitative evaluations of our learned actionable visual priors.
Metrics.
For quantitatively evaluating the trajectory scoring module, we use standard metrics that are commonly used for evaluating binary classiﬁcation problems: accuracy, precision, recall, and Fscore.
Since there is no ground-truth annotation of successful and failed interaction trajectories, we run our learned RL policy over test shapes and collect test interaction trajectories.
In our implementation, we gather 350 positive and 350 negative trajectories for each experiment.
To evaluate if the learned trajectory proposal module can propose diverse trajectories to cover the collected ground-truth ones, door drawer pu s h
i ng t e s tca t pu lli ng t r a
i nca
t t e s t - s h a p e t
e s tca t t r a
i nca
t t e s t - s h a p
e
o -33.
o +20.
o
+30.
o -0.
+0. +0.
coverage
Results and Analysis.
Table 1 presents quantitative results which demonstrate that we successfully learn the desired representations and that our learned model also generalizes to shapes from totally unseen object categories.
Since we are the ﬁrst to propose such actionable visual priors, there is no baseline to compare against.
Fig.
3 presents qualitative results of the actionability prediction and trajectory proposal modules, from which we can observe that, to accomplish the desired tasks: 1) the actionability heatmap highlights where to interact ( e.g. , to pull open a drawer, the gripper either grasp and pull open the handle from outside, or push outwards from inside), and 2) the trajectory proposals suggest diverse solutions for how to interact ( e.g. , to push closed the door, various trajectories may succeed).
In Fig.
4, we additionally illustrate the trajectory scoring predictions using heatmap visualization, where we observe interesting learned patterns indicating which points to interact with when executing the input trajectories.
Fig.
5 (left) further visualizes the trajectory scoring predictions over the same input shape and observe different visual patterns given different trajectories and tasks.
5. D OWNSTREAM M ANIPULATION
We can easily use our learned actionable visual priors to accomplish the downstream manipulation of 3D articulated objects.
To this end, we ﬁrst sample a contact point according to the estimated actionability heatmap and then execute the top-rated trajectory among 100 random trajectory proposals.
Baselines.
We compare to three baselines: 1) a naive TD3 RL that takes the shape point cloud together with the desired task as input and directly outputs trajectory waypoints for accomplishing the task; 2) a heuristic approach, in which we hand-engineer a set of rules for different tasks ( e.g. , to pull open a drawer, we grasp the handle and pull straight backward).
Note that we use ground-truth handle masks and joint parameters for the heuristic baseline; 3) a multi-step Where2Act (Mo et al., 2021) baseline that concatenates multiple (up to 15 steps) Where2Act-generated short-term pushing or pulling interactions that gradually accumulates the part pose changes for accomplishing the task.
While the Where2Act work originally considers task-less short-term interactions, different from our long-term task-driven setting, we adapt it as a baseline by assuming an oracle part pose tracker is given.
See Sec.
H in appendix for more detailed descriptions of the baseline methods.
Metrics.
We run interaction trials in simulation and report success rates for quantitative evaluation.
Results and Analysis.
Table 2 presents the quantitative comparisons.
Our method outperforms baselines on most comparisons.
See below for detailed result analysis and more results in appendix.
For the naive RL baseline , it largely fails since we ﬁnd it extremely difﬁcult to train from scratch an end-to-end RL over highly diverse shapes for all tasks, which in fact implies the necessity of certain intermediate visual abstractions.
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table]
For the heuristic baseline , we ﬁnd that we win all comparisons except one task of drawer pushing.
Knowing the ground-truth front board and prismatic joint axis, it is quite easy for the rule-based method to push a drawer along a straight-line trajectory, while our method does not take the groundtruth information and achieves worse results since we have to predict such information.
However, our method achieves better performance for the other three experiments since we ﬁnd that the heuristic method fails (see Fig.
H.7) severely when 1) there are no handle parts for pulling, 2) some grasps over intricate handle geometry may slip and fail, or 3) the motion dynamics ( e.g. , inertia) for non-prehensile manipulation (pushing) affects the accurate task accomplishment ( e.g. , in case of an overshoot).
For the Where2Act baseline , it consistently performs worse than ours since it does not explicitly take the desired task as input and thus is more vulnerable of overshooting.
Besides, the Where2Act baseline has the following issues: 1) it assumes an accurate part pose tracker, 2) it has to execute multiple segmented interactions and thus is empirically found 30 times slower, and 3) the gripper needs to be driven to multiple independent positions across different trajectory segments and thus it is very time-consuming and fragile if cases of motion planning failures.
5. R EAL - WORLD E XPERIMENTS
Fig.
5 (middle) presents qualitative results directly testing our model on real-world data: a microwave from the RBO dataset (Mart´ın-Mart´ın et al., 2019), one cabinet from Google Scanned Object (OpenRobotics), and one real-world 3D cabinet scan we capture using a ZED MINI RGB-D camera.
We observe that our model trained on synthetic textureless data can generalize to real-world depth scans to some degree.
We also show real-robot experiment in Fig.
5 (right) and supplementary video.
C ONCLUSION
In this paper, we propose a novel perception-interaction handshaking point – object-centric actionable visual priors – for manipulating 3D articulated objects, which contains dense action affordance predictions and diverse visual trajectory proposals.
We formulate a novel interaction-for-perception framework VAT-M ART to learn such representations.
Experiments conducted on the large-scale
PartNet-Mobility dataset and real-world data have proved the effectiveness of our approach.
Limitations and Future Works.
First, our work is only a ﬁrst attempt at learning such representations and future works can further improve the performance.
Besides, the current open-loop trajectory prediction is based on a single-frame input.
One may obtain better results considering multiple frames during an interaction.
Lastly, future works may study more interaction and articulation types.
Acknowledgements.
National Natural Science Foundation of China —Youth Science Fund (No.62006006): Learning Visual Prediction of Interactive Physical Scenes using Unlabelled Videos.
Leonidas and Kaichun are supported by a grant from the Toyota Research Institute University 2.
program 1 , NSF grant IIS-1763268, and a Vannevar Bush faculty fellowship.
We thank Yourong
Zhang for setting up ROS environment and helping in real robot experiments, Jiaqi Ke for designing and doing heuristic experiments.
Ethics Statement.
Our project helps build home-assistant robots to aid people with disabilities, or senior.
Our proposed visual affordance and action trajectory predictions can be visualized and analyzed before use, which makes the AI system more explainable and thus less vulnerable to potential dangers.
Training our system requires training data, which may introduce data bias, but this is a general concern for learning-based methods.
We do not see any particular major harm or issue our work may raise up.
Reproducibility Statement.
We will release our code, data, and pre-trained model weights publicly to the community upon paper acceptance
A PPENDIX A M ORE D ETAILS FOR THE RL P OLICY (S EC . 4.1) A. N ETWORK A RCHITECTURE D ETAILS
We leverage TD3 (Fujimoto et al., 2018) to train the RL policy.
It consists of a policy network and a
Q-value network, both implemented as a Multi-layer Perception (MLP).
The policy network receives the state as input ( R 33 ) and predicts the residual gripper pose as action ( R 6 ).
Its network architecture is implemented as a 4-layer MLP (33 → 512 → 512 → 512 → 512), followed by two separate fully connected layers (512 → 6) that estimate the mean and variance of action probabilities respectively.
The Q-value network receives both the state and action as input ( R 39 ), and predicts a single Q value ( R 1 ).
Its network architecture is implemented as a 4-layer MLP (39 → 512 → 512 → 512 → 1).
A. M ORE I MPLEMENTATION AND T RAINING D ETAILS
We use the following hyper-parameters to train the RL policy: 2048 (buffer size); 512 (batch size); 0.0001 (learning rate of the Adam optimizer (Kingma & Ba, 2015) for both the policy and Q-value network); 0.1 (initial exploration range, decayed by 0.5 every 500 epochs.
B M ORE D ETAILS FOR THE P ERCEPTION N ETWORKS (S EC . 4.2) B. N ETWORK A RCHITECTURE D ETAILS
The perception networks consist of four input encoders that extract high-dimensional features from the input partial point cloud S O , contact point p , trajectory τ and task θ , and three parallel output modules that estimate the actionability a p | O , T , θ , trajectory proposal τ p | O , T , θ and scores r τ | O , p , T , θ from the extracted high-dimensional features of input encoders.
Regarding the four input encoders
• The input point cloud S O with point-wise coordinate ( R 3 ) is fed through the PointNet++ (Qi et al., 2017) with the segmentation head and single-scale grouping (SSG) to extract the point-wise feature f s ∈ R 128 .
• The input contact point p ∈ R 3 is fed through a fully connected layer (3 → 32) to extract a high-dimensional contact point feature f p ∈ R 32 .
• The input trajectory τ ∈ R 30 is composed of ﬁve waypoints, each of which contains 3D rotation and 3D translation information.
The input trajectory is further ﬂattened to be fed through a 3-layer MLP (30 → 128 → 128 → 128) to extract a high-dimensional trajectory feature f τ ∈ R 128 .
• The input task θ ∈ R 1 is fed through a fully connected layer (1 → 32) to extract a highdimensional task feature f θ ∈ R 32 .
Regarding the three output modules
• The actionability prediction module D a concatenates the extracted point cloud f s , contact point f p and task f
θ features to form a high-dimensional vector ( R 192 ), which is fed through a 5-layer MLP (192 → 128 → 128 → 128 → 128 → 1) to predict a per-point actionability score a p | O , T , θ .
• The trajectory proposal module is implemented as a conditional variational auto-encoder (cVAE), which learns to generate diverse trajectories from the sampled Gaussian value and given conditions (point cloud f s , contact point f p and task f θ features).
The encoder E τ is implemented as a 3-layer MLP (320 → 128 → 128 → 256) that takes the concatenation of the trajectory feature f τ and given conditions as input and estimates a Gaussian distribution ( µ ∈ R 128 , σ ∈ R 128 ).
The decoder D τ is also implemented as a 3-layer MLP (320 → 512 → 256 → 30) that takes the concatenation of the sampled Gaussian value z ∈ R 128 and given conditions as input to recover the trajectory τ .
• The trajectory scoring module D s concatenates the extracted point cloud f s , contact point f p , trajectory f τ and task f
θ features to form a high-dimensional vector ( R 320 ), which is fed through a 2-layer MLP (320 → 128 → 1) to predict the success likelihood r τ | O , p , T , θ .
B. M ORE I MPLEMENTATION AND T RAINING D ETAILS
The interaction data provided by the RL policy provides the ground truth label to train the perception networks.
We start by training the trajectory scoring module, followed by joint training of all the three modules.
To train the trajectory scoring module D s , we ﬁrstly collect 5000 successful and 5000 unsuccessful interaction trajectories produced by the RL policy, then supervise the estimated success likelihood of the trajectory with binary cross entropy loss.
To train the trajectory proposal module, we supervise the output rotation and translation with the 6D-rotation loss (Zhou et al., 2019) and L 1 loss respectively from the 5000 successful collected interaction trajectories.
We follow the common cVAE training process to add a KL divergence loss on the estimated Gaussian distribution for regularization.
To train the actionability prediction module D a , we estimate the scores from 100 sampled trajectories with both the trajectory proposal and scoring modules, then calculate the mean value of top-5 rated scores as the ground truth label to supervise the actionability predictions.
We adopt L 1 loss for supervision.
We use the following hyper-parameters to train the perception networks: 32 (batch size); 0.
(learning rate of the Adam optimizer (Kingma & Ba, 2015) for all three modules).
C M ORE D ETAILS FOR THE C URIOSITY - DRIVEN E XPLORATION (S EC . 4.3)
The curiosity-driven exploration alternates between training the RL policy and perception networks at odd and even epochs respectively.
It aims for exploring more diverse trajectories with the RL policy that the perception networks are not conﬁdent with and are further used to diversify the trajectory proposals generated by the perception networks.
When training the RL policy, besides the extrinsic task rewards, it also awards the policy with the intrinsic curiosity reward.
It is computed as − 500 r τ | O , p , T , θ , which is the weighted negative success likelihood estimated by the trajectory scoring module.
It penalizes the learned trajectories that the trajectory scoring module is conﬁdent with and encourage more diverse trajectories generated by the
RL policy.
When training the perception networks, we collect the equal number of successful and unsuccessful trajectories produced by the RL policy.
To encourage the perception networks to learn more diverse trajectories, we sample the successful trajectories ﬁfty-ﬁfty with both high (¿0.5) and low (¡0.5) success likelihood estimated by the trajectory scoring module.
Then we use these data to train the perception networks.
[Table]
[Table]
D M ORE D ETAILED D ATA S TATISTICS
Table D.3 summarizes the data statistics and splits.
Fig.
D.6 presents a visualization of some example shapes from our dataset.
E SAPIEN S IMULATION E NVIRONMENT S ETTINGS
We follow exactly the virtual environment settings as in (Mo et al., 2021), except that we randomly sample camera viewpoints only in front of the target shape since most articulated parts are present in the front side of the PartNet-Mobility shapes and it is very unlikely that robot agents need to manipulate object parts from the back of objects.
By default, the camera looks at the center of the target shape, and the location is sampled on a unit sphere centered at the origin.
For the spherical sampling of the camera location, we set the range of the azimuth angle to [-90 ◦ , 90 ◦ ], and the elevation angle to [30 ◦ , 60 ◦ ].
Please refer to Section A and Section B in the appendix of (Mo et al., 2021) for more in-depth details of the virtual environment.
F C OMPUTATIONAL C OSTS AND T IMING
Training of the proposed system took around 3 days per experiment on a single Nvidia V100 GPU.
More speciﬁcally, the initial RL policy training took around 18 hours, followed by a 15-hour training of the perception module using the RL-collected data.
The subsequent curiosity-driven exploration stage took around 10 hours to jointly train the RL and the perception networks.
At last, the three prediction branches in the perception module were further ﬁne-tuned for around 20 hours.
During the inference time, a forward pass takes around 15 ms, 8 ms, and 8 ms for the actionability map prediction, diverse trajectory proposals, and success rate predictions, respectively.
Our model only consumes 1,063 MB memory of the GPU in test time.
G M ORE M ETRIC D EFINITION D ETAILS G. E VALUATING A CTIONABLE V ISUAL P RIORS (S EC . 5.1)
We adopt the following evaluation metrics for quantitative evaluation: precision, recall and F-score for the positive data, and the average accuracy equally balancing the negative and positive recalls.
Formally, let f p , f n , t p , and t n denote the data counts for false positives, false negatives, true positives, and true negatives, respectively.
The F-score is the harmonic mean of precision and recall, and the other metrics are deﬁned as follow:
[Equation]
We also compute a coverage score for evaluation, which is calculated as the percentage of groundtruth trajectories that are similar enough to be matched in the predicted trajectories.
To this end, we need to measure the distance D ( l 1 , l 2 ) between two trajectories l 1 and l 2 , which takes into account both the position difference and the orientation difference at every waypoint.
Concretely, we calculate the L1 distance of the waypoint positions as the position distance D pos between the two trajectories.
Then, the orientation distance D ori is measured as the 6D-rotation distance of the waypoint orientations.
To balance the dimension of the quantities, D ( l 1 , l 2 ) is calculated as:
D ( l 1 , l 2 ) =
5 D pos ( l 1 , l 2 ) + D ori ( l 1 , l 2 ) .
We consider a ground truth trajectory to be covered if the distance between this ground truth trajectory and a predicted trajectory is lower than a threshold (10, in all our experiments), and then report the percentage of the ground truth trajectories that are covered by the predictions.
To compensate the stochastic error, all reported quantitative results are averaged over 10 test runs.
G. E VALUATING D OWNSTREAM M ANIPULATION (S EC . 5.2)
For each downstream manipulation task, we locate the contact point sample with the highest actionability score over the surface on each test shape, and then generate 100 diverse trajectories at this contact point, and pick the trajectory with the highest success rate for executing the downstream task.
This results in 350 trajectories for each of the door pushing, door pulling, drawer pushing, and drawer pulling tasks.
In practice, generating trajectories that can perfectly fulﬁll the given task speciﬁcation ( e.g. , push to open a door by 30 ◦ ) is very hard.
Hence, we consider a trajectory can successfully accomplish the task if the speciﬁcation is fulﬁlled within a tolerance threshold in percentage ε = 15%.
For example, if the task is to push a door open by 10 ◦ , a trajectory that can open the door between 8.5 ◦ and 11.5 ◦ is counted as a successful sample.
We then report the success rate in percentage of all the generated trajectories on each task.
H M ORE D ETAILS OF B ASELINES IN S EC . 5.
In the following, we present more details about the baseline methods that we compare against in
Section 5.2.
H. N AIVE RL B ASELINE
The naive RL baseline, which is comprised of two sub-RL modules denoted as sub-RL1 and sub-RL respectively.
Sub-RL1 module takes as input the initial state of the target shape ( i.e. , the partial scan of the initial shape, and the mask of the target part), and the task speciﬁcation, and then outputs the initial position and orientation of the gripper ( i.e. , the parameters of the ﬁrst waypoint).
Sub-RL module takes as input the state of the initial scene ( i.e. , the partial scan of the shape, the mask of the target part, and the position and orientation of the gripper at the initial state), the state of the current scene ( i.e. , the partial scan of the shape, the mask of the target part, and the position and orientation of the gripper at the current state), and the task speciﬁcation, and then predicts the parameters of the next waypoint using the residual representation.
The network architectures of sub-RL1 and sub-RL2 are based on our RL networks as described in
Section A.1, with minor adaptations for different input and output dimensionalities.
We use a buffer size of 2048 and a batch size of 64 for training both sub-RL1 and sub-RL2 modules.
We use the
Adam optimizer and a learning rate 0.0001 for the policy and Q-value networks in both modules.
We set the initial exploration range to 0.1, which is linearly decayed by a factor of 0.5 every 500 epochs during training.
H. R ULE - BASED H EURISTIC B ASELINE
For the rule-based heuristic baseline, we hand-craft a set of rules for different tasks.
We describe the details as follows.
• Door pushing: we randomly sample a point p on the door surface, then initialize the gripper such that the gripper ﬁngertips touch the point p and set the forward orientation along the negative direction of the surface normal at p .
Let d denotes the distance between p and the rotation shaft of the door, to push the door by a degree θ , we push the door by a degree θeach time and repeat for four times.
During each process, we simply move the gripper along the negative direction of the surface normal by a distance of d sin ( θ 4 ) .
• Door pulling: we initialize the gripper at a point p sampled on the handle of the door and set the forward orientation along the negative direction of the surface normal at p .
And then we pull the door by a degree θ by pulling θ 4 each time as in door pushing.
Different from door pushing, we also perform a grasping at contact point p for pulling.
• Drawer pushing: similarly, we randomly sample a point p on the front board of the drawer, and then initialize the gripper such that the gripper ﬁngertips touch the point p and set the forward orientation along the negative direction of the normal at p .
Let t denote the push distance in the task speciﬁcation, we simply move the gripper by a distance of t along the slide-to-close direction of the drawer.
• Drawer pulling: the middle point of the handle is particularly sampled as the contact point.
Note that we count the trial as a failure if the drawer does not have handles.
Then we initialize the gripper as in drawer pushing, and move the gripper by a distance of t along the slide-to-open direction.
In Fig.
H.7, we present some failure cases for the rule-based heuristic baseline to explain why such seemingly intuitive and easy heuristics often fail.
See the ﬁgure caption for detailed explanations.
Such rule-based heuristics require careful human hand-engineering given different task semantics.
One needs to hand-design rules for different tasks and sometimes will ﬁnd it difﬁcult to enumerate all possible rules.
Our system, instead, provides a uniﬁed system that automatically discovers useful knowledge for different tasks, without the need of spending human efforts, and learn a rich collection of data-driven priors from training over diverse shapes.
H. W HERE 2A CT BASELINE
For the Where2Act baseline, we use the pre-trained networks released by Mo et al.
(2021) to generate multiple short-term trajectory segments, each of which samples a contact point and an action direction from Where2Act.
We assume an oracle part pose tracker and choose the pushing or pulling
Where2Act network to query at each time by comparing the ground-truth pose and the goal pose of the target part.
Since Where2Act only executes short-term task-less interaction trajectories and does not explicitly consider task degree, we allow it to operate for up to 15 steps and record the closest distance it has ever reached to the desired target part pose.
Concretely, in each step, this method
[Table] 62.56 / 49.43 w 78.41 / 71. • uses Where2Act’s actionability module to predict the affordance of each point, randomly selects a contact point within the estimated top-5 actionable points, and initializes the gripper in front of the selected contact point; • initializes the gripper’s initial 6D pose by running Where2Act’s action proposal network and selecting the top-rated action; • if pushing, directly moves the gripper 0.05 unit-length forward; • if pulling, moves the gripper slightly forward, closes the gripper to grasp, and moves the gripper 0.05 unit-length backward when the gripper successfully grasps the target part; • checks if
the target part’s pose achieves the goal pose, using the target part’s ground-truth pose.
I A BLATION S TUDY ON C URIOSITY F EEDBACK
We compare our ﬁnal approach to an ablated version that removes the curiosity feedback (Sec. 5.3), in order to validate that the proposed curiosity-feedback mechanism is beneﬁcial.
In Table I.4, we evaluate the prediction quality of the proposed actionable visual priors.
From this table, it is clear to see that the introduced curiosity feedback mechanism (Sec. 5.3) is beneﬁcial as it improves the performance in most entries.
pu s h
i ng t e s t - ca t door drawer pu lli ng t r a
i nca
t t e s t - s h a p e t
e s t - ca t t r a
i nca
t t e s t - s h a p e -34.
o
-32.28 o +22.
o +17.
o -0.
+0. +0.
J M ORE R ESULTS AND A NALYSIS
In Fig.
J.8, we show additional qualitative results of the actionability prediction and trajectory proposal modules to augment Fig.
3 in the main paper.
In Fig.
J.9, we present an additional qualitative analysis of the learned trajectory scoring module to augment Fig.
4 in the main paper.
In addition, we provide another set of result analyses in Fig. J.10, where we show that to accomplish the same task, interacting at different contact points over the shape will give different trajectories.
K R EAL - ROBOT S ETTINGS AND E XPERIMENTS
For real-robot experiments, we use one real cabinet and set up one Franka Panda robot facing the front of the cabinet.
One ZED MINI RGB-D camera is mounted at the front right of the cabinet.
The camera captures 3D point cloud data as inputs to our learned models.
We control the robot using Robot Operating System (ROS)
(Quigley et al., 2009).
The robot is programmed to execute each waypoint in the predicted trajectory step by step.
We use MoveIt! (Chitta et al., 2012) for the motion planning between every adjacent waypoint pair.
We demonstrate various tasks on the cabinet, including pulling open the door at the edge or handle, pushing closed the door at different contact points, t r a i nca t t e s t - s h a p e t e s t - ca t t r a
i nca t t e s t - s h a p e t e s t - ca t pu s h
i ng pu lli ng door drawer -38.2 o +22.6 o
+0.
+0.
-29.7 o
+20.2 o
[Figure]