We present a deep imitation learning framework for robotic bimanual manipulation in a continuous state-action space. Imitation learning has been effectively utilized in mimicking bimanual manipulation movements, but generalizing the movement to objects in different locations has not been explored . We hypothesize that to precisely generalize the learned behavior relative to an object's location requires modeling relational information in the environment  . To achieve this, we designed a method that (i) uses a multi-model framework to decomposes complex dynamics into elemental movement primitives, and (ii) parameterizes each primitive using a recurrent graph neural network to capture interactions . Our model is a deep, hierarchical, modular architecture with a high-level planner that learns to compose primitives sequentially and a low-level controller which integrates primitive dynamics modules and inverse kinematics control. We demonstrate the effectiveness using several simulated bimanual robotic manipulation tasks . Compared to models based on previous imitation learning studies , our model generalizes better and achieves higher success rates in the simulated tasks.
We present a deep imitation learning framework for robotic bimanual manipulation in a continuous state-action space. A core challenge is to generalize the manipulation skills to objects in different locations . We hypothesize that modeling the relational information in the environment can significantly improve generalization . To achieve this, we propose to (i) decompose the multi-modal dynamics into elemental movement primitives, (ii) parameterize each primitive using a recurrent graph neural network to capture interactions , and (iii) integrate a high-level planner that composes primitives sequentially and a low-level controller to combine primitive dynamics and inverse kinematics control. Our model is a deep, hierarchical, modular architecture . Compared to baselines , our model generalizes better and achieves higher success rates on several simulated bimanual robotic manipulation tasks. We open source the code for simulation, data, and models at: URL
R
Imitation learning has been effectively utilized in mimicking bimanual manipulation movements, but generalizing the movement
A core challenge is to generalize the manipulation skills
D
has not been explored
R
to precisely generalize the learned behavior relative to an object's location requires modeling
modeling the
A
can significantly improve generalization
R
designed a method that
propose to
R
uses a multi-model framework to decomposes complex
decompose the multi-modal
D
and
R
parameterizes
parameterize
R
. Our model is a deep, hierarchical, modular architecture with
, and (iii) integrate
R
learns to compose
composes
R
which integrates primitive dynamics modules
to combine primitive dynamics
R
We demonstrate the effectiveness using several simulated bimanual robotic manipulation tasks
Our model is a deep, hierarchical, modular architecture
R
models based on previous imitation learning studies
baselines
R
in the simulated tasks.
on several simulated bimanual robotic manipulation tasks. We open source the code for simulation, data, and models at: URL