VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects

Perceiving and manipulating 3D articulated objects ( e.g. , cabinets , doors ) in human environments is an important yet challenging task for future home-assistant robots . The space of 3D articulated objects is exceptionally rich in their myriad semantic categories , diverse shape geometry , and complicated part functionality . Previous works mostly abstract kinematic structure with estimated joint parameters and part poses as the visual representations for manipulating 3D articulated objects . In this paper , we propose object-centric actionable visual priors as a novel perception-interaction handshaking point that the perception system outputs more actionable guidance than kinematic structure estimation , by predicting dense geometry-aware , interaction-aware , and task-aware visual action affordance and trajectory proposals . We design an interaction-for-perception framework VATMART to learn such actionable visual representations by simultaneously training a curiosity-driven reinforcement learning policy exploring diverse interaction trajectories and a perception module summarizing and generalizing the explored knowledge for pointwise predictions among diverse shapes . Experiments prove the effectiveness of the proposed approach using the large-scale PartNet-Mobility dataset in SAPIEN environment and show promising generalization capabilities to novel test shapes , unseen object categories , and real-world data . 1 INTRODUCTION . We live in a 3D world composed of a plethora of 3D objects . To help humans perform everyday tasks , future home-assistant robots need to gain the capabilities of perceiving and manipulating a wide range of 3D objects in human environments . Articulated objects that contain functionally important and semantically interesting articulated parts ( e.g. , cabinets with drawers and doors ) especially require significantly more attention , as they are more often interacted with by humans and artificial intelligent agents . Having much higher degree-of-freedom ( DoF ) state spaces , articulated objects are , however , generally more difficult to understand and subsequently to interact with , compared to 3D rigid objects that have only 6-DoF for their global poses . There has been a long line of research studying the perception and manipulation of 3D articulated objects in computer vision and robotics . On the perception side , researchers have developed various successful visual systems for estimating kinematic structures ( Abbatematteo et al. , 2019 ; Staszak et al. , 2020 ) , articulated part poses ( Li et al. , 2020 ; Jain et al. , 2020 ; Liu et al. , 2020 ) , and joint parameters ( Wang et al. , 2019b ; Yan et al. , 2019 ) . Then , with these estimated visual articulation models , robotic manipulation planners and controllers can be leveraged to produce action trajectories for robot executions ( Klingbeil et al. , 2010 ; Arduengo et al. , 2019 ; Florence et al. , 2019 ; Urakami et al. , 2019 ; Mittal et al. , 2021 ) . While the commonly used two-stage solution underlying most of these systems reasonably breaks the whole system into two phases and thus allows bringing together well-developed techniques from vision and robotics communities , the current handshaking point – the standardized visual articulation models ( i.e . kinematic structure , articulated part poses , and joint parameters ) , may not be the best choice , since essential geometric and semantic features for robotic manipulation tasks , such as interaction hotspots ( e.g . edges , holes , bars ) and part functionality ( e.g . handles , doors ) , are inadvertently abstracted away in these canonical representations . We propose a new type of actionable visual representations ( Do et al. , 2018 ; Nagarajan et al. , 2019 ; Mo et al. , 2021 ) exploring a more geometry-aware , interaction-aware , and task-aware perceptioninteraction handshaking point for manipulating 3D articulated objects . Concretely , we train the perception system to predict action possibility and visual action trajectory proposals at every point over parts of 3D articulated objects ( See Figure 1 ) . In contrast to previous work that use standardized visual articulation models as visual representations , our framework VAT-MART predicts per-point dense action trajectories that are adaptive to the change of geometric context ( e.g. , handles , door edges ) , interactions ( e.g. , pushing , pulling ) , and tasks ( e.g. , open a door for 30◦ , close up a drawer by 0.1-unit-length ) . Abstracting away from concrete external manipulation environments , such as robot arm configurations , robot base locations , and scene contexts , we aim for learning unified objectcentric visual priors with a dense and diverse superset of visual proposals that can be potentially applied to different manipulation setups , avoiding learning separate manipulation representations under different circumstances . The proposed actionable visual priors , as a ” preparation for future tasks ” ( Ramakrishnan et al. , 2021 ) or ” visually-guided plans ” ( Wang et al. , 2019a ; Karamcheti et al. , 2021 ) , can provide informative guidance for downstream robotic planning and control . Sharing a similar spirit with Nagarajan et al . ( 2019 ) ; Mo et al . ( 2021 ) , we formulate our visual action possibility predictions as per-point affordance maps , on which the downstream robotic planners may sample a position to interact according to the predicted likelihood of success . Then , for a chosen point for interaction , the discrete task planner may search for applicable interaction modes ( e.g. , whether to attempt a grasp ) within a much smaller space formed by the visual action trajectory distribution , instead of searching in the entire solution space . Next , considering the robot kinematic constraints and physic collisions , the continuous motion planner can further select an open-loop trajectory from the set of proposed visual action trajectory candidates as an initial value for optimization , and finally pass to the robot controller for execution . More recent reinforcement learning ( RL ) based planners and controllers can also benefit from our proposed solution spaces for more efficient exploration . To obtain such desired actionable visual priors , we design an interaction-for-perception learning framework VAT-MART , as shown in Figure 2 . By conducting trial-and-error manipulation with a set of diverse 3D articulated objects , we train an RL policy to learn successful interaction trajectories for accomplishing various manipulation tasks ( e.g. , open a door for 30◦ , close up a drawer by 0.1- unit-length ) . In the meantime , the perception networks are simultaneously trained to summarize the RL discoveries and generalize the knowledge across points over the same shape and among various shapes . For discovering diverse trajectories , we leverage curiosity feedback ( Pathak et al. , 2017 ) for enabling the learning of perception networks to reversely affect the learning of RL policy . We conduct experiments using SAPIEN ( Xiang et al. , 2020 ) over the large-scale PartNetMobility ( Chang et al. , 2015 ; Mo et al. , 2019 ) dataset of 3D articulated objects . We use 562 shapes in 7 object categories to perform our experiments and show that our VAT-MART framework can successfully learn the desired actionable visual priors . We also observe reasonably good generalization capabilities over unseen shapes , novel object categories , and real-world data , thanks to large-scale training over diverse textureless geometry . In summary , we make the following contributions in this work : • We formulate a novel kind of actionable visual priors making one more step towards bridging the perception-interaction gap for manipulating 3D articulated objects ; • We propose an interaction-for-perception framework VAT-MART to learn such priors with novel designs on the joint learning between exploratory RL and perception networks ; • Experiments conducted over the PartNet-Mobility dataset in SAPIEN demonstrate that our system works at a large scale and learns representations that generalize over unseen test shapes , across object categories , and even real-world data . 2 RELATED WORK . Perceiving and Manipulating 3D Articulated Objects has been a long-lasting research topic in computer vision and robotics . A vast literature ( Yan & Pollefeys , 2006 ; Katz et al. , 2008 ; Sturm et al. , 2009 ; 2011 ; Huang et al. , 2012 ; Katz et al. , 2013 ; Martin & Brock , 2014 ; Höfer et al. , 2014 ; Katz et al. , 2014 ; Schmidt et al. , 2014 ; Hausman et al. , 2015 ; Martı́n-Martı́n et al. , 2016 ; Tzionas & Gall , 2016 ; Paolillo et al. , 2017 ; Martın-Martın & Brock , 2017 ; Paolillo et al. , 2018 ; Martı́n-Martı́n & Brock , 2019 ; Desingh et al. , 2019 ; Nunes & Demiris , 2019 ) has demonstrated successful systems , powered by visual feature trackers , motion segmentation predictors , and probabilistic estimators , for obtaining accurate link poses , joint parameters , kinematic structures , and even system dynamics of 3D articulated objects . Previous works ( Peterson et al. , 2000 ; Jain & Kemp , 2009 ; Chitta et al. , 2010 ; Burget et al. , 2013 ) have also explored various robotic planning and control methods for manipulating 3D articulated objects . More recent works further leveraged learning techniques for better predicting articulated part configurations , parameters , and states ( Wang et al. , 2019b ; Yan et al. , 2019 ; Jain et al. , 2020 ; Zeng et al. , 2020 ; Li et al. , 2020 ; Liu et al. , 2020 ; Mu et al. , 2021 ) , estimating kinematic structures ( Abbatematteo et al. , 2019 ; Staszak et al. , 2020 ) , as well as manipulating 3D articulated objects with the learned visual knowledge ( Klingbeil et al. , 2010 ; Arduengo et al. , 2019 ; Florence et al. , 2019 ; Urakami et al. , 2019 ; Mittal et al. , 2021 ) . While most of these works represented visual data with link poses , joint parameters , and kinematic structures , such standardized abstractions may be insufficient if fine-grained part geometry , such as drawer handles and faucet switches that exhibit rich geometric diversity among different shapes , matters for downstream robotic tasks . Learning Actionable Visual Representations aims for learning visual representations that are strongly aware of downstream robotic manipulation tasks and directly indicative of action probabilities for robotic executions , in contrast to predicting standardized visual semantics , such as category labels ( Russakovsky et al. , 2015 ; Wu et al. , 2015b ) , segmentation masks ( Lin et al. , 2014 ; Mo et al. , 2019 ) , and object poses ( Hinterstoisser et al. , 2011 ; Xiang et al. , 2016 ) , which are usually defined independently from any specific robotic manipulation task . Grasping ( Montesano & Lopes , 2009 ; Lenz et al. , 2015 ; Mahler et al. , 2017 ; Fang et al. , 2020 ; Mandikal & Grauman , 2021 ; Corona et al. , 2020 ; Kokic et al. , 2020 ; Yang et al. , 2020 ; Jiang et al. , 2021 ) or manipulation affordance ( Kjellström et al. , 2011 ; Do et al. , 2018 ; Fang et al. , 2018 ; Goff et al. , 2019 ; Nagarajan et al. , 2019 ; Nagarajan & Grauman , 2020 ; Nagarajan et al. , 2020 ; Xu et al. , 2021 ; Mo et al. , 2021 ) is one major kind of actionable visual representations , while many other types have been also explored recently ( e.g. , spatial maps ( Wu et al. , 2020 ; 2021 ) , keypoints ( Wang et al. , 2020 ; Qin et al. , 2020 ) , contact points ( You et al. , 2021 ) , etc ) . Following the recent work Where2Act ( Mo et al. , 2021 ) , we employ dense affordance maps as the actionable visual representations to suggest action possibility at every point on 3D articulated objects . Extending beyond Where2Act which considers task-less short-term manipulation , we further augment the per-point action predictions with task-aware distributions of trajectory proposals , providing more actionable information for downstream executions . Learning Perception from Interaction augments the tremendously successful learning paradigm using offline curated datasets ( Russakovsky et al. , 2015 ; Lin et al. , 2014 ; Chang et al. , 2015 ; Mo et al. , 2019 ) by allowing learning agents to collect online active data samples , which are more task-aware and learning-efficient , during navigation ( Anderson et al. , 2018 ; Ramakrishnan et al. , 2021 ) , recognition ( Wilkes & Tsotsos , 1992 ; Yang et al. , 2019 ; Jayaraman & Grauman , 2018 ) , segmentation ( Pathak et al. , 2018 ; Gadre et al. , 2021 ) , and manipulation ( Pinto et al. , 2016 ; Bohg et al. , 2017 ) . Many works have also demonstrated the usefulness of simulated interactions for learning perception ( Wu et al. , 2015a ; Mahler et al. , 2017 ; Xu et al. , 2019 ; Ramakrishnan et al. , 2021 ; Nagarajan & Grauman , 2020 ; Lohmann et al. , 2020 ; Mo et al. , 2021 ) and promising generalizability to the real world ( James et al. , 2019 ; Chebotar et al. , 2019 ; Hundt et al. , 2019 ; Liang et al. , 2020 ; Kadian et al. , 2020 ; Anderson et al. , 2020 ; Rao et al. , 2020 ) . Our method follows the route of learning perception from interaction via using the action trajectories discovered by an RL interaction policy to supervise a jointly trained perception system , which reversely produces curiosity feedback ( Pathak et al. , 2017 ) to encourage the RL policy to explore diverse action proposals . 3 ACTIONABLE VISUAL PRIORS : Action Affordance and Trajectory Proposals We propose novel actionable visual representations for manipulating 3D articulated objects ( see Fig . 1 ) . For each articulated object , we learn object-centric actionable visual priors , which are comprised of : 1 ) an actionability map over articulated parts indicating where to interact ; 2 ) per-point distributions of visual action trajectory proposals suggesting how to interact ; and 3 ) estimated success likelihood scores rating the outcomes of the interaction . All predictions are interaction-conditioned ( e.g. , pushing , pulling ) and task-aware ( e.g. , open a door for 30◦ , close a drawer by 0.1-unit-length ) . Concretely , given a 3D articulated object O with its articulated parts P = { P1 , P2 , · · · } , an interaction type T , and a manipulation task θ , we train a perception system that makes dense predictions at each point p over each articulated part p ∈ ∪P : 1 ) an actionability score ap|O , T , θ ∈ [ 0,1 ] indicating how likely there exists an action trajectory of interaction type T at point p that can successfully accomplish the task θ ; 2 ) a distribution of visual action trajectories Pp ( ·|O , T , θ ) , from which we can sample diverse action trajectories τp|O , T , θ ∼ Pp ( ·|O , T , θ ) of interaction type T to accomplish the task θ at point p ; and 3 ) a per-trajectory success likelihood score rτ|O , p , T , θ ∈ [ 0,1 ] . Inputs . We represent the input 3D articulated object O as a partial point cloud SO . We consider two typical interaction types : pushing and pulling . A pushing trajectory maintains a closed gripper and has 6-DoF motion performing the pushing , whereas a pulling trajectory first performs a grasping operation at the point of interaction by closing an initially opened gripper and then has the same 6-DoF motion during the pulling . For articulated objects we use in this work , we only consider 1-DoF part articulation and thus restrict the task specification θ ∈ R. For example , a cabinet drawer has a 1-DoF prismatic translation-joint and a refrigerator door is modeled by a 1-DoF revolute hinge-joint . We use the absolute angular degrees in radian ( i.e . θ ∈ [ −π , π ] ) for revolute joints and use the units of length ( i.e . θ ∈ [ −1,1 ] ) relative to the global shape scale for prismatic joints . Outputs . Both the actionability score ap|O , T , θ and per-trajectory success likelihood score rτ|O , p , T , θ are scalars within [ 0,1 ] , where larger values indicate higher likelihood . One can use a threshold of 0.5 to obtain binary decisions if needed . Every action trajectory τp|O , T , θ is a sequence of 6- DoF end-effector waypoints ( wp0 , wp1 , · · · , wpk ) , with variable trajectory length ( k ≤ 5 ) . In our implementation , we adopt a residual representation ( wp0 , wp1−wp0 , · · · , wpk−wpk−1 ) for the action trajectory , as it empirically yields better performance . Each 6-DoF waypoint is comprised of a 3-DoF robot hand center x and 3-DoF orientation R. We use the 6D-rotation representation ( Zhou et al. , 2019 ) for the orientation of wp0 and predict 3-DoF euler angles for subsequent orientation changes . 4 VAT-MART : AN Interaction-for-perception LEARNING FRAMEWORK The VAT-MART system ( Fig . 2 ) consists of two parts : an RL policy exploring diverse action trajectories and a perception system learning the proposed actionable visual priors . While the RL policy collects interaction trajectories for supervising the perception networks , the perception system provides curiosity feedback ( Pathak et al. , 2017 ) for encouraging the RL policy to further explore diverse solutions . In our implementation , we first pretrain the RL policy , then train the perception network with RL-collected data , and finally finetune the two parts jointly with curiosity-feedback enabled . We describe key system designs below and will release code for our implementation .

Summary: This paper is solving the problem of pushing and pulling objects (mostly things like cabinets) by learning visual action trajectories proposals via a curiosity-driven RL / Perception joint training.  The system input point clouds the object and outputs the actionable score and the per-trajectory success likelihood score on the most likely approach to interacting with the object.  This approach is validated both in simulations but also in simulation and with real results.