Stabilize to Act: Learning to Coordinate for Bimanual Manipulation
Authors: Jennifer Grannen
Yilin Wu
Brandon Vu
Dorsa Sadigh
Venue: CoRL 2023 Oral (2023)
Keywords: Bimanual Manipulation
Learning from Demonstrations
Deformable Object Manipulation

Abstract: Key to rich, dexterous manipulation in the real world is the ability to coordinate control across two hands. However, while the promise afforded by bimanual robotic systems is immense, constructing control policies for dual arm autonomous systems brings inherent difficulties. One such difficulty is the high-dimensionality of the bimanual action space, which adds complexity to both model-based and data-driven methods. We counteract this challenge by drawing inspiration from humans to propose a novel role assignment framework: a stabilizing arm holds an object in place to simplify the environment while an acting arm executes the task. We instantiate this framework with BimanUal Dexterity from Stabilization (BUDS), which uses a learned restabilizing classifier to alternate between updating a learned stabilization position to keep the environment unchanged, and accomplishing the task with an acting policy learned from demonstrations.
We evaluate BUDS on four bimanual tasks of varying complexities on real-world robots, such as zipping jackets and cutting vegetables. 
Given only 20 demonstrations, BUDS achieves 76.9% task success across our task suite, and generalizes to out-of-distribution objects within a class with a 52.7% success rate. BUDS is 56.0% more successful than an unstructured baseline that instead learns a BC stabilizing policy due to the precision required of these complex tasks. Supplementary material and videos can be found at https://tinyurl.com/stabilizetoact.

Stabilize to Act: Learning to Coordinate for Bimanual Manipulation
Jennifer Grannen, Yilin Wu, Brandon Vu, Dorsa Sadigh
Stanford University, Stanford, CA
jgrannen@stanford.
Abstract: Key to rich, dexterous manipulation the real world is the ability coordinate control across two hands. However, while the promise afforded by bimanual robotic systems is immense, constructing control policies for dual arm autonomous systems brings inherent difficulties. One such difficulty is the high-dimensionality of the bimanual action space, which adds complexity to both model-based and data-driven methods. We counteract this challenge by drawing inspiration from humans to propose a novel role assignment framework: a stabi- lizing arm holds an object place to simplify the environment while an acting arm executes the task. We instantiate this framework with BimanUal Dexterity from Stabilization (BUDS), which uses learned restabilizing classifier to al- ternate between updating a learned stabilization position to keep the environment unchanged, and accomplishing the task with acting policy learned from demon- strations. We evaluate BUDS on four bimanual tasks of varying complexities on real-world robots, such as zipping jackets and cutting vegetables. Given only 20 demonstrations, BUDS achieves 76.9% task success across our task suite, and generalizes out-of-distribution objects within class with 52.7% success rate. BUDS 56.0% more successful than unstructured baseline that instead learns a BC stabilizing policy due to the precision required of these complex tasks. Sup- plementary material and videos can be found at https///iiyyrr..co//ttiiiiooooo
Keywords: Bimanual Manipulation, Learning from Demonstrations, Deformable Object Manipulation
Introduction
Bimanual coordination is pervasive, spanning household activities such cutting food, surgical skills such as suturing a wound, industrial tasks such as connecting two cables. In robotics, the addition of a second arm opens the door to higher level of task complexity, but comes with number control challenges. With a second arm, we have to reason about how to produce coordi- nated behavior a higher dimensional action space, resulting in more computationally challenging learning, planning, and optimization problems. The addition of second arm also complicates data collection-it requires teleoperating robot with more degrees of freedom-whit hinders our abil- ity to rely methods that require expert bimanual demonstrations. To combat these challenges, we can draw inspiration from how humans tackle bimanual tasks-specifice alternating between using one arm to stabilize parts of the environment, then using the other arm to act conditioned on the stabilized state of the world.
Alternating stabilizing and acting offers significant gain over both model-based and data-driven prior approaches for bimanual manipulation. Previous model-based techniques have proposed plan- ning algorithms for bimanual tasks such collaborative transport scooping [1, 2, 3]. but quire hand-designed specialized primitives follow predefined trajectories limiting their abilities to learn new skills or adapt. On another extreme, we turn to reinforcement learning (RL) tech- niques that do not need costly primitives. However, RL methods are notoriously data hungry and high-dimensional bimanual action space further exacerbates this problem. While simulation-to to-oonlttt--lll transfer techniques offer an appealing alternative [4, 5, 6, 7], key component of bimanual tasks
7th Conference Robot Learning (CoRL. 2023), Atlanta, USA.

---

closed-chain contacts and high-force interactions (consider cutting or connecting cables), which are hard to simulate and widen the gap with reality [8, 9, 10]. A more promising data-driven approach is learning from demonstration. However, collecting high-dimensional bimanual demonstrations difficult as simultaneously controlling two high-degree freedom arms often requires specialized hardware or multiple operators [11, 12, 8, 13, 14]. The increased dimensionality of the action space also necessitates significantly more data, especially for more precise dexterous tasks [15].
Our insight about how humans iterate between sta- bilizing and acting presents way to overcome
Acting
these challenges. In tasks such as plugging in
Stabilizing
phone or cutting a steak, a stabilizing arm holds an object (e.g. the phone or steak) stationary to
simplify the environment, making it easier for the
acting arm to complete the task with high preci-
sion. Factoring control across stabilizing and act- ing additionally offers benefits for data collection; the role-specific policy can be learned indepen- dently for each arm, bypassing the need for biman- ual demonstrations. Adjusting a stabilizing posi- tion iteratively as the acting arm progresses enables even more expressive and generalizable behavior. For example, fork should skewer steak differ- ent points depending on where the knife cuts.
(1) Stabilizing
(3) BC-RNN
Position Model
of ~ x9)
W fg00
ResNet
Slabilaning
Acting Action of
Kaypoint
(2) Noncompliant Stabilizing a' = x' w0
(4) Bimanual
Stakillaing
Policy (a a")
Action
Thus, the key insight driving this work is that to en- able sample-efficiens nipulation, we need two roles: a stabilizing arm stabilizes an object to simplify the environment for an acting arm to perform the task.
Observation 0.
generalizable bimanual ma-
Restabilize
Conticue Acting
fl
(5) Restabilizing
Ki00
Classifier f((0
ResNel
We propose BimanUal Dexterity from Stabilization (BUDS), a method that realizes this coordination paradigm by decomposing the bimanual problem into two single-arm problems: learning to stabilize and learning to act. The stabilizing policy decides where to stabilize in the scene and when to adjust, while the acting arm learns to perform the task in this simpler environment. For example when cut- ting a steak, our stabilizing policy learns where to hold steak and when adjust so the steak remains stationary while the acting policy makes the cut.
Figure 1: BUDS: Biman Dexterity from Stabilization: BUDS is a bimanual manipu- lation framework that uses novel stabilizing and acting role assignment to efficiently learn to coordinate. For the stabilizing role, BUDS learns a Stabilizing Position Model (I) to pre- dict point to hold stationary using noncom- pliant controller (2). In this simplified envi- ronment, BUDS learns to act from single-arm demonstrations (3). Combined, these two ac- tions comprise a bimanual policy (4). Finally, at every timestep, BUDS's Restabilizing Clas- sifier (5) predicts whether the stabilizing posi- tion is still effective or needs to be updated.
To learn where to stabilize. we use vision-base based system that takes an environment image as input and outputs a stabilization keypoint position. We then learn a restabilizing classifier that determines from images when the stabilizing keypoint no longer effective and needs to be updated. We deploy this stabilizing policy while collecting single-arm trajectory demonstrations for an acting policy to sidestep the need for precise and expensive bimanual demonstration collection interface. Using these demonstrations, the acting arm learns a policy via imitation learning to accomplish the task in this simplified, stationary environment. We demonstrate the efficacy of this paradigm on four di- verse, dexterous manipulation tasks on a physical UR 16e dual-arm platform. BUDS achieves 76.9% success, and outperforms an unstructured baseline fully learned from expert trajectory demonstra- tions by 56.0%. Additionally, BUDS achieves 52.7% when generalizing to unseen objects of similar morphology (e.g. transferring a cutting policy trained on jalapeños to cutting zucchini and celery).
Our contributions are: (1) A paradigm for learning bimanual policies with role assignments, where the stabilizing arm stabilizes the environment and reduces the non-stationarity present while an acting arm learns to perform task allowing in simpler learning setting, (2) A framework for

---

collecting bimanual demonstrations that bypasses the need for dual-arm interface by collecting demonstrations for the stabilizing and acting roles independently, and (3) A system, BUDS, that instantiates this paradigm to learn a centralized bimanual policy to perform a broad range of tasks.
2 Related Work
In this section, we describe the current data-driven and model-based methods available for bimanual manipulation tasks, along with prior work using ideas of stabilizing for manipulation.
Learning-based Bimanual Manipulation. A recurring challenge throughout bimanual manipu- lation is the high-ding dith-mmmeniiillttyy of the action space. This appears both in reinforcement learning (RL) and imitation learning (II.) works [16, 11, 17, 18, 19, 20, 21]. Prior multi-agent coordination works have considered shrinking the high-dimensionallty of the problem by using a second agent stabilizing a latent intent [22], however learning a stabilizing policy and latent intent mapping both require a significant amount of data that not realistic for physical robot manipulation tasks.
RL methods for learning high-frequents bimanual control policies can require a large number of samples and many hours of robot time, which makes simulation to real policy transfer an appealing approach [21, 7, 6]. However, sim-to-real approaches are limited to settings where the sim-to- real gap is small, which precludes many contact-rich bimanual tasks such as zipping zipper or cutting food [8, 9, 10]. Instead, works in both RL and IL settings have proposed using parame- terized movement primitives to reduce the action space, and have achieved reasonable success on tasks such as opening a bottle and lifting a ball [17, 19, 20, 23, 24, 25, 26, 27, 28, 29, 15, 30]. However, these movement primitives greatly limit the tasks achievable by the method as they often require costly demonstrations or labor-intensive hard-coded motions for each task-specifie primi- tive. Additionally, learning from demonstrations in bimanual settings difficult as teleoperating two high-degree-freedon robots collecting kinesthetic demonstrations on both arms simultane- ously is challenging and sometimes impossible for a single human and may require specialized hardware [16, 11, 31, 12, 8, 13). Recent works have demonstrated more effective interfaces for data collection in a bimanual setting, but these interfaces are limited to specific hardware instantiations and would still require large amounts of expert data to learn a high dimensional policy [14]. To avoid the need for expert bimanual demonstrations, we use a novel stabilizing paradigm to decouple the arms' policies and learn role-specific policy for each arm from single-arm demonstrations. This added structure also brings down the dimensionality of the large action space in task-agnostic way.
Model-based Bimanual Manipulation. The majority of model-based bimanual manipulation methods are limited to using planning and constraint solving methods to jointly move or hold a large object [32, 33, 34, 35, 12, 2, 1, 36). Bersch et al. [37] and Grannen et al. [3] present systems using a sequence of hard coded actions for folding a shirt and scooping food respectively. However, as tasks become more complex, the primitives required also become more unintuitive and costly to hand-design. We instead learn control policy from single-arm demonstrations, avoiding the need for labor-intensive hand-coded primitives while performing dexterous bimanual manipuation tasks.
Stabilizing for Manipulation. Stabilizing and fixturing can yield large benefits in a manipulation context by providing additional steadiness for high precision tasks and unwieldy object interactions. Early works in industrial robotics have proposed planners for autonomous fixture placement that reason about friction forces [38] or use CAD model designs [39] add structure to the environment. More recent works have used additional fixture arms vises to bootstrap sample efficiency [40] or avoid robot force and torque limits [41]. Similarly, Chen et al. [42] consider a collaborative setting-an an assistive robot arm reasons about forces to hold a board steady for human to cut or drill into. The addition of an assistive stabilizing role naturally points towards a bimanual setting, and indeed many bimanual manipulation works implicitly use a stabilizing role in their designs [23, 11, 3, 21]. Holding food place while cutting is, perhaps, an obvious application of stabilizing, and this assistance is critical for overcoming the highly variable geometries and dynamics of food [43, 44, 45]. While prior stabilizing works are limited as a task-specific systems, propose a general bimanual system that learns from demonstrations how to stabilize and act for a variety of tasks.
3

---

3 Stabilizing for Bimanual Tasks
Given a set of expert demonstrations, we aim produce a bimanual policy for executing a variety of manipulation tasks, such capping marker or zipping up a jacket. We formulate each bimanual task as a sequential decision making problem defined by components (O, A). Each observation or comprises an RGB image frame St IR [xx x and the proprioceptive state of each arm P: IR A is the action space for the two robot arms containing 14 degrees of freedom (DoF) joint actions We define a; = (af, af), where af of E R?7 are the stabilizing and acting arm actions respectively. We are a model-free setting, and make no assumptions on the unknown transition dynamics.
To perform these bimanual tasks, we use bimanual manipulator operating in a workspace that is reachable by both arms, along with standard (x, y, z) coordinate frame in this workspace. We use depth cameras with known intrinsics and extrinsics, which allows to obtain mapping (far fy) in pixel space to coordinate (x, y, z) in the workspace, which we later refer to keypoint.
To learn our bimanual policies, we first assume access to set of expert bimanual demonstra- tions D, and later relax this assumption to two sets of expert unimanual demonstrations D and D to avoid the challenges of collecting bimanual demonstrations. Each demonstration is a se- quence of observation, action pairs that constitute an expert trajectory. First, we consider bimanual demonstrations [[01, aj, of), (02, ad a.)....] E D discuss the challenges of learning Mono- lithic policy. Next, pivot to decoupling the bimanual policy with two unimanual datasets: [(01. af), (02. DDe and [(01, ag)...)... E D
3.1 Monolithic 14-DoF Policy
Let us first consider learning a monolithic 14-DoF policy mo(af; af (o)) parametrized by 0 via be- havioral cloning, which takes an observation or as input and outputs a bimanual action (a], az). We aim to find a policy that matches the expert demonstrations in D by minimizing this supervised loss:
(()) = - [o, ("o.a=,"")>> >D log (e(a a oo).).
(1)
While this is feasible in theory, in practice learning policies in this way is highly dependent on clean and consistent demonstration data for both arms acting in concert. However, as mentioned in Sec- tion 2, collecting such data is challenging and these difficulties are further exacerbated for precise and dexterous tasks. Motivated by stabilizing structures across many bimanual tasks, we sidestep these challenges by utilizing task-agnostic role-assignment while learning bimanual policies.
3.2 Stabilizing for Reducing Control Dimensionality
We observe that a wide variety of human bimanual tasks leverage a similar paradigm: one arm stabilizes objects in the scene to simplify the environment while the other arm acts accomplish the task. We translate this observation into generalizable robotics insight: assign either stabilizing or acting role to each arm to specify coordination framework. Thus, we decompose our bimanual policy (a7, a ~ T (-(o,) into two role-specific policies: stabilizing policy at ~ T&. (-lo,, af) and an acting policy of ~u w (-10;, of). These policies are co-dependent; we aim to disentangle them.
Given these roles, we make a crucial insight: for a given acting policy subtrajectory (af af there exists a single stabilizing action a" that works as a "fixture" for holding constant a task-specific part of the environment. For example, consider the role of a fork pinning a steak to a plate to facilitate cutting with the knife. These stabilizing fixtures act reduce the dimensionality of the control problem for the other arm, as the environment is less susceptible to drastic changes. We charac- terize this constant task-specific region with learned task-relevant representation ó O + R for some j, and we later instantiate a stabilizing fixture Æ" with a keypoint representation in Sec- tion 4.1 and execute non-compliant motions at this keypoint. Finally, we isolate our stabilizing policy zq. (aflo(or-)) (o:)) from the acting policy with loss that penalizes the expected change task-relevant region of the environment:
(((0)
Σ t=0
Bag aa (oo ())=-1)]|.
(2)

---

Given the stabilizing action az The (0)( we obtain an acting action of ~ Té (-) a.). This stabilizing action is valid for k timesteps, afterwards which the stabilizing fixture must be updated. To obtain this variable length k, we threshold the change in ((0;+n) from the initial observation c(o;) to indicate when a stabilizing fixture is no longer effective:
k inf {n
and [|@(o)+)) Φ(a,)) c)
In practice, we instantiate the task-relevant representation to be stabilized (b(ot) as a keypoint model learned from expert demonstrations (using learned mapping from an image to keypoint ft : i w*). We do not solve Eq. (2) but instead utilize a noncompliant controller to hold this point stationary over time (see Section 4. Given stabilizing fixture that is effective for acting actions [(s- we additionally learn a restabilizing classifier (o)) = {0. 1} that determines when & has been surpassed and new learned stabilizing action should be predicted. We describe this implementation
further in Section 4 and show in experiments in Section 5 that this approximation holds.
4 BUDS: BimanUal Dexterity from Stabilization
We describe Biman Dexterity from Stabilization (BUDS), which instantiates the stabilizing and acting role assignments in Section 3. As shown in Fig. 1, leam model for each role: f for stabilizing and z3 for acting, parameterized by weights 0 and é. We also learn restabilizing classifier J parameterized by weights v. All models are learned from human-annotated images or single-arm telcoperated robot demonstrations, avoiding the difficulties of collecting bimanual demonstrations. All labels and demonstrations are consistent across both arms for any given image.
4.1 Learning a Stabilizing Policy
Algorithm 1 Stabilizing with BUDS
From Section 3, aim find sta- bilizing policy #" (((o) = ag. Specifically, we aim to learn a task- specific representation s to be stabilized. We observe that when humans stabilize bimanual tasks, they hold point station- ary over time. Thus, D contains two ac- pero-actions that hold a point in place and transient actions that move between stabilizing positions. Additionally, this observation implies can be instantiated as mapping from an observation on to a stabilization position wy. We decompose the stabilizing role into two parts: (1) selecting a stabilization position w" to hold stationary and (2) sensing when to update the stabilization position (as in Eq. (3)).
1:
while Task Incomplete do
2:
ii = = (ot) while (or) do
3:
4:
of x² (ū ..) & faz of ~ wo
5:
0f -= ai))
6:
Execute at aZ. Observe ot+1, (oc+1). end while
7:
end while
tion types: stationary
We parameterize w" as keypoint on an overhead image of the workspace. We use a ResNet- R64004100 which takes as input an overhead image and outputs a Gaussian heatmap centered around the predieted stabilizing keypoint w This mapping is learned from the stationary actions in the demonstration data D=, indicating that the arm is at stabilizing position in this demonstration. In practice, we bypass the need for full trajectory demonstrations and provide supervision in the form of keypoint annotations. Given w and a depth value from the overhead camera, a non-compliant controller grasps this 3D point and holds it stationary. Thus, we approximate the stabilizing action at with the action that keeps the keypoint stationary, i.e., u R ú We then write x² =s((0)))) s(o:)) n (w w;), function of two consecutive keypoints learned from demonstrations: t f (o.). The learned keypoint mapping f is trained with a hand-labelled dataset of 30 image and keypoint pairs, where the keypoint is annotated as the stabilizing keypoint wi for the image. We fit a Gaussian heatmap centered at the annotation with a standard deviation of 8px. This dataset is augmented 10X with a series of label sabel-preserving image transformations [47] (see Appendix A). From this dataset, f learss to predict the keypoint w for the stabilizing policy to hold stationary.
34 [46] backbone learn mapping f/ RR610x18033
To determine when to update w², we close the feedback loop by learning restabilizing classifier f 1640x180x3 + {0, 1} that maps input workspace images to a binary output indicating whether
5

---

or not to update w This mapping is learned from the transient actions in the demonstration data DD D*-indicating that the stabilizing positions at these states need to be updated. In practice, we forgo using full trajectory demonstrations for supervision in the form of binary expert annotations. We instantiate I with a ResNet-34 [46] backbone and train this classifier with an expert-labelled dataset of 2000 images. For each rollout, an expert assigns when in the rollout a new stabilizing position w is needed; the preceding images are labelled while the following images are labelled
1. This dataset is augmented 2X with affine image transformations (See Appendix A for details). f learns to predict binary classification of when the stabilizing point is no longer effective and needs to be updated with fg. Together, f and fo define a stabilizing policy x² as outlined in Algorithm 1.
4.2 Learning an Acting Policy
Given a stabilization policy n² (#: an acting policy Th learns to accomplish the task in a simpler stationary environment. We instantiate T with a BC-RNN architecture that is trained on 20 single-arm demonstrations. A expert teleoperates the acting arm using SpaceMouse [48], a 3D joystick interface During data collection, the stabilizing arm is assumed to be in the expert-labelled stabilizing position w" and the environment is in simplified state. T optimizes the standard imitation learning loss as defined in Eq. (I), and we refer the reader to Appendix A for more details.
To further increase sample efficiency, we assume that our expert acting demonstrations start from pre-grasped initial position. To achieve this pre-grasped position, we train an optional grasping keypoint model f for the acting policy that maps input workspace images it E |640x480x3 to Gaussian heatmap centered around the grasp point. This grasping model is instantiated with the same ResNet-34 [46] and dataset parameters as used for the stabilizing keypoint model fø- The acting arm moves the keypoint position in fixed orientation, and grasps begin the task.
5 Experiments
We validate BUDS on four diverse bimanual tasks. We use two UR16e arms each with Robotig 2F-85 gripper, mounted at a 45 angle off a vertical mount, 0.3m apart. We use a RTDE-based impedance controller [49] and associated IK solver operating at 10Hz on an Intel NUC. End effectors move along linear trajectory between positions. All grasps use a grasping force of 225N and fixed orientation. We three Intel Realsense cameras: two 435 cameras mounted side view and on the robot wrist, and one 405 camera mounted overhead. For additional details, see Appendix B.
Bimanual Tasks. We consider four bimanual tasks, as shown in Fig. 3, and test the generalization of BUDS to unseen objects (Fig. 2). Each task requires both a high-precision acting policy and dynamic stabilizing policy that restabilizes multiple times during task execution. We emphasize the complexity of the coordination required of these dexterous tasks. Together, these four tasks represent a wide range of real- real-world bimanual manipulation tasks, which highlights the prevalence of the stabilizing and acting role assignments. For all tasks, we vary the initial position of all objects over each trial. For more details and videos, see Appendix B and our website.
Pepper Grinder. We grind pepper on three plates in order of color-yellow, pink, then blue as shown in Fig. 3. This task requires restabilizing the pepper grinder over each plate in succession. Jacket Zip. We zip a jacket by pinning down the jacket's bottom and pulling the zipper to the top. Due to the jacket's deformability, the robot must pin the jacket as close as possible to the zipper. We train all models with red jacket, and the keypoint models on two more jackets: dark grey and blue. We aim to generalize to light grey and black jackets with different material and zippers.
Marker Cap. We cap three markers in sequence from bottom to top of the workspace. This task requires restabilizing after each marker is capped. We train all policies on red, green, and blue Crayola markers and test generalization with Expo and Redimark markers.
Cut Vegetable. We cut a vegetable half (7-9em) into four 1 1-4cm pieces with three cuts. This task requires restabilizing the grasp on the vegetable each cut is made, as the stabilizing arm should hold the vegetable close possible to the cut prevent tearing and twisting. We train and celery sticks (8-10cm).
jalapeño and test generalization with zucchini halves

---

Pepper Grinder Stubilizing
Acting
Restabilizing
Acting
Restabilizing
Acling
!- soc
t-
20
t-
Jacket Zip
Stubilizing
Acting
Restabilzingg
Acting
Restabilizing
Acting
Restabiliziigg
Acting
2-
f-1
1=
==8
70
12
f- 15
F-
t=
28
Marker Cap
Stabilizing
Acting
Restubilizing
Acting
Restabilizing
Acting
- soc
Cut Vegetable
Stabiliziing
Acting
Restabiliziigg
Acting
Restabilizing
Acting
1- Bed
2-2
1=
f=
10
1:
[- 17
f=
2-26
Time
Figure 3: Experiment Rollouts: We visualize BUDS experiment rollouts. All tasks alternate be- tween updating stabilizing position w while the acting arm is paused and executing acting policy while the stabilizing arm holds steady. We visualize both w² and the acting actions.
Baselines. BC-Stabilizer illustrates the need for low-dimensional stabilizing representation by placing the stabilizing keypoint model f with policy learned from trajectory demonstrations. This policy is instantiated with the same BC-RNN ar- chitecture and training procedure as BUDS's acting policy. An oracle classifier determines when BC- Stabilizer has reached a valid stabilizing position, where noncompliant controller then holds the point stationary in BUDS while the pre-grasped acting policy from BUDS accomplishes the task. When the restabilizing classifier from BUDS f is triggered, the process repeats. No-Restable ablates BUDS's restabilizing classifier and only senses single sta- blizing point at the beginning each task. We evaluate No-Restable only on Jacket Zip and Cut Vegetable because other tasks require updated stabilizing position reach complete success. We do not compare Monolithic baseline (as in Section 3.1) as it achieves zero success for all tasks.
Jacket Zip
Marker Cap
B
Cut Vegetable
Jalapeño
Seen
Livaseen
Herd Easy
Zucchini
Figure Task Generalization: We present the Seen and Unseen objects in the Jacket Zip, Marker Cap, and Cut Vegetable tasks. We classify these OOD objects into two classes, Easy and Hard, based on their visual similarity to the objects seen during training.
BUDS Failures
Task
BC-Stabilizer
No-Restable
BUDS
T
fr.
Pepper Grinder
39.9 21
100
0
0
0
Jacket Zip (Clean)
28.2 =
58.8 39
72.1 + 18
0
3
Jacket Zip (Occluded)
21.6 = 17
51.1 + 37
55.7 + 37
1
2
1
2
Marker Cap
0.0 0
90.1 + 16
1
2
0
0
B
Cut Vegetable
15.0
46.6 28
66.8
24
2
0
3
We report average percent success and standard deviation across 10 trials of 4 bimanual tasks with randomly initialized object positions. For Jacket Zip. we classify initial configurations Clean Occluded, where none or up 30% of the zipper is occluded respectively. We report failure modes: i stabilizing keypoint, T. acting policy, f restabilizing, and (G) poor grasps. We compare to two baselines: BC-Stabilizer where a single-arm IL policy replaces the stabilizing keypoint model, and No-Restable, an ablation of BUDS that disregards restabilizing.
Table Physical Results:
We evaluate BUDS on four bimanual tasks that require dynamic restabilizing. Task success is mea- sured as the proportion of task completed over total amount be completed, for example zipped length over total zipper length. As shown in Table 1, BUDS achieves 76.9% success across four

---

tasks, visualized in Fig. 3. We report four failure modes: (1) an incorrect predicted stabilizing po- sition w², (2) an acting policy failure x², (3) restabilizing error S that does not detect when a stabilizing point needs updating, and (4) a failed grasp. The acting policy failure is the most com- mon, due to the low amount of data used to train the acting policy and the high precision required.
The stabilizing failures (w* and fo) are mostly due large visual differences from the training data, including occlusions, and cause the environment quickly move out of distribution from the stable, simplified states seen in the acting policy training data. Across all tasks, BUDS outperforms the unstructured BC-Stabilizer baseline due to the high precision required of a stabilizing role. Where BUDS and BC-Stabilizer both learn relevant point from a visual input, BC-Stabilizer must also learn the policy to reach this position. Thus, the BC-Stabilizer policy's 's primary failure mode is selecting a poor stabilizing position-it struggles to learn a stabilizing policy robust across many task configurations, as indicated by its 20.9% success rate. BUDS also outperforms No-Restable in Jacket Zip (Clean) and Cut Vegetable, highlighting the need for closed-loop restabilizing. BUDS and No- to-Restable achieve similar success on Jacket Zip (Occluded) because the biggest challenge in this task is the jacket's deformability and occlusions, which restabilizing alone cannot solve.
BUDS OOD
40-Demo Hard
BUDS Failures
Task
Easy
Hard
Ws
x²
G
Jacket Zip
62.3 40
28.8 + 27
23.1 25
10
0
2
Marker Cap
60.0 14
53.3 39
56.7 39
17
0
0
B Cut Vegetable
85.0 + 13
26.6 - 26
30.0
4
6
0
6
Table 2: Generalizability Results: We test BUDS's robustness to OOD objects of similar morphol-
ogy. The Easy and Hard OOD objects respectively more and less similar in visual appearance and dynamics to training objects (Fig. 2). We report average and standard deviation success over ten trials per object, along with failure modes over 20 trials. We compare to 40-Demo, whose acting policy is trained on 40 demonstrations, but do not observe performance difference on Hard objects. We test BUDS's generalizability to out-of-distribution (OOD) objects classified into two classes based on visual similarity to training objects (Fig. 2). We run 10 trials per object, and find BUDS achieves an average success rate of 52.7% (Table 2). In two of the three tasks, we observe slight performance drop compared to in distribution settings (Table 1), with worseming difference for Hard objects. With this expected performance drop, we observe more stabilizing failures (w² and f due to the stabilizing policy's high visual dependence, which struggles with novel object ap- pearances. For Jacket Zip, we attempt to improve performance by training the stabilizing keypoint model f on three jackets, but the policy still falls short on the vastly different Hard black jacket. 40-Demo aims to improve robustness by training the acting policy on double the data, but again does not significantly improve performance due to the Hard objects' large visual and dynamic differences compared to the training objects, which cannot be remedied with more in-distribution data. We note an exception: Easy zucchini in Cut Vegetable has higher success rate than that of the in-distribution jalapeño. The hollow jalapeño twists and tears, which is unforgiving of slight acting policy errors, while the solid zucchini can withstand shear forces from noisy policies, yielding more success.
Conclusion
We present BUDS, a system for dexterous bimanual manipulation that leverages novel role assign- ment paradigm: a stabilizing arm holds a point stationary for the acting arm act in a simplified environment. BUDS uses learned keypoint the stabilizing point and learns acting policy from unimanual trajectory demonstrations. BUDS also learns restabilization classifier to detect when a stabilizing point should be updated during rollouts. BUDS achieves 76.9% and 52.7% success on four bimanual tasks with objects seen and unseen from training respectively.
Limitations and Future Work. Because BUDS uses only visual inputs, it struggles with visu- ally different novel objects unseen during training BUDS can zip many jackets but struggles with dresses. Thus BUDS also falls short when tactile feedback is critical, such as plugging in a USB. BUDS assumes fixed roles in each task, which would not hold for tasks where the arms must alter- nate. In future work, we will explore policies for role assignment, which could be planned to avoid collisions or learned to enable more nuanced tradeoffs. We will incorporate tactile sensing for more sensitive stabilizing, towards tasks like buttoning a shirt.
8

---

Acknowledgments
This project was sponsored by NSF Awards 2006388, 2125511, and 2132847, the Office of Naval Research (ONR), Air Force Office of Scientific Research YIP award, and the Toyota Research Insti- tute. Jennifer Grannen is further grateful to be supported by NSF GRFP. Any opinions, findings, conclusions recommendations expressed in this material are those of the authors and do not neces- sarily reflect the views of the sponsors. We additionally thank our colleagues who provided helpful feedback and suggestions, especially Suneel Belkhale and Sidd Karamcheti.
References
[1] D. Losey, M. Li, Bohg, and D. Sadigh. Learning from My Partner's Actions: Roles in Decentralized Robot Teams. In Conf. on Robot Learming (CoRL), 2019.
[2] E. Ng, 7. Liu, and M. Kennedy III. It Takes Two: Learning to Plan for Human-Robot Cooper- ative Carrying, In Proc. IEEE Int. Conf. Robotics and Automation (ICRA), 2023.
[3] Grannen, Y. Wu, S. Belkhale, and D. Sadigh. Learning Bimanual Scooping Policies for Food Acquisition. In Conf. Robot Learning (CoRL), 2022.
[4] S. Höfer, K. Bekris, A. Handa, J. C. Gamboa, M. Mozifian, F. Golemo, C. Atkeson, D. Fox, K. Goldberg, Leonard, C. Karen Liu, Peters, S. Song, P. Welinder, and M. White. Sim2Real in Robotics and Automation: Applications and Challenges. IEEE Transactions on Automation Science and Engineering, 18(2):398-400, 2021.
[5] Z. Fu, X. Cheng, and D. Pathak. Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion. CoRL, 2022.
[6] Y. Chen, Yang, T. Wu, S. Wang, X. Feng. J. Jiang, S. M. McAleer, H. Dong, Z. Lu, and S.-C. Zhu. Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning, 2022.
[7] Kataoka, S. K. S. Ghasemipour, D. Freeman, and I. Mordatch. Bi-Manual Manipulation and Attachment via Sim-to-Real Reinforcement Learning, 2022.
[8] S. Stepputtis, M. Bandari, S. Schaal, and H. B. Amor. A system for imitation learning of contact-rich bimanual manipulation policies. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022.
[9] fñigo Elguea-Aguinaco, A. Serrano-Muñoz Chrysostomou, I. Inziarte-Hidalgo, S. Bøgh, and N. Arana-Arexolales A review reinforcement learning for contact-rich robotic ma- nipulation tasks. Robotics and Computer-Integraater Manufacturing, 81:102517, 2023. ISSN 0736-5845.
[10] O. Kroemer, S. Niekum, and G. Konidaris. A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms. Journal of Machine Learning Research, 22(30):
1- 82, January 2021.
[11] L. P. Ureche and A. Billard. Constraints extraction from asymmetrical bimanual tasks and their use in coordinated behavior. Robotics and Autonomous Systems, 103:222-233 235,, 2018. ISSN 0921-8890.
[12] C. Smith, Y. Karayiannidis, L.. Nalpantidis, X. Gratal, P. Qi, D. V. Dimarogonas, and D. Kragic. Dual arm manipulation- survey. Robotics and Autonomous Systems, 6001033134- 2012. ISSN 0921-8890. doithttps://ooi.or/// http::///oooo...100000o.oooo....
[13] R. Lioutikov, O. Kroemer, G. Maeda, and J. Peters. Learning manipulation by sequenc- ing motor primitives with two-armed robot. 302:1601-1611, 01 2016. doi:10.1007/ 978-3-319-0838--.115

---

[14] T. Z. Zhao, V. Kumar, S. Levine, and C. Finn. Learning fine the-ggained bimanual manipulation with low-cost hardware, 2023.
[15] Xie, A. Chowdhury, M. C. D. P. Kaluza, L. Zhao, L. L. S. Wong, and R. Yu. Deep imitation learning for bimanual robotic manipulation. 2020.
[16] N. Figueroa and A. Billard. Learning Complex Manipulation Tasks from Heterogeneous and Unstructured Demonstrations. In Proceedings of Workshop on Synergies between Learning and Interaction, JEEE/RSJ International Conference on Intelligent Robots and Systems, 2017.
[17] A. Batinica, B. Nemec, A. Ude, M. Raković, and A. Gams. Compliant movement primitives in bimanual setting. In 2017 IEEE-RAS 17th International Conference Humanoid Robotics (Humanoids), pages 365-375--377 -001,, 2017. doi:
[18] A. Colomé and C. Torras. Dimensionality Reduction for Dynamic Movement Primitives and Application to Bimanual Manipulation of Clothes. IEEE Transactions on Robotics, 34(3): 602-615, 2018.
109/TRO.2018.288093
[19] A. Colomé and C. Torras. Reinforcement Learning of Bimanual Robot Skills. Springer Cham, 2020.
[20] G. Franzese, L. de Souza Rosa, Verburg, Peternel, and J. Kober. Interactive imitation learning of bimanual movement primitives, 2022.
[21] R. Chitnis, S. Tulsiani, S. Gupta, and A. Gupta. Efficient Bimanual Manipulation Using Learned Task Schemas. In Proc. IEEE Int. Conf. Robotics and Automation (ICRA), 2020.
[22] W. Z. Wang, A. Shih, A. Xie, and D. Sadigh. Influencing towards stable multi-agent interac- tions. In Proceedings of the 5th Conference on Robot Learning (CoRL), 2021.
[23] Grannen, P. Sundaresan, B. Thananjeyan, J. Ichnowski, A. Balakrishna, M. Hwang,
V. Viswanath, M. Laskey, E. Gonzalez, and Goldberg. Untangling Dense Knots by Learn- ing Task-Relevan, Keypoints. In Conf. on Robot Learning (CoRL), 2020.
[24] Y. Avigal, L. Berscheid, T. Asfour, T. Kröger, and K. Goldberg. SpeedFolding: Learning Efficient Bimanual Folding of Garments. In Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), 2022.
[25] A. Ganapathi, P. Sundaresan, B. Thananjeyan, A. Balakrishna, D. Seita, J. Grannen, M. Hwang, R. Hoque, J. E. Gonzalez, N. Jamali, et al. Learning Dense Visual Correspondences in Simula- tion to Smooth and Fold Real Fabrics. Proc. IEEE Int. Conf. Robotics and Automation (ICRA), 2021.
[26] F. Amadio, A. Colomé, and C. Torras. Exploiting Symmetries in Reinforcement Learning of Bimanual Robotic Tasks. IEEE Robotics and Automation Letters, 4(2):1838-111 1845, 2019. 10. 2019/LR..209..28933...
[27] X. Yin and Q. Chen. Learning nonlinear dynamical system for movement primitives. In 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pages 3761-3766, 2014. doi:10.1109/SMC...... 1000.1009//MM22224666455666
[28) L. Fu, H. Huang, L.. Berscheid, H. Li, K. Goldberg, and S. Chitta. Safely Learning Visuo- Tactile Feedback Policies in Real For Industrial Insertion, 2022.
[29] R. Chitnis, S. Tulsiani, S. Gupta, and A. Gupta. Intrinsic Motivation for Encouraging Syner- gistic Behavior. In International Conference on Learning Representations, 2020.
[30] H. Ha and S. Song. Flingbot: The unreasonable effectiveness of dynamic manipulation for cloth unfolding. In Conf. on Robot Learning (CoRL), 2021.
10

---

[31] R. Zöllner, T. Asfour, and R. Dillmann. Programming by Demonstration: Dual-Arm Manipula- tion Tasks for Humanoid Robots. In IEFE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 479-484, 2004.
[32] P. Hsu. Coordinated control of multiple manipulator systems. JEEE Transactions on Robotics and Automation, 94):4000441, 1993. 1oi:10.1100 10.1109/70.24655.
[33] S. S. Mirrazavi Salchian, N. Figueroa, and A. Billard. Dynamical System-Based Motion Plan- ning for Multi-Arm Systems: Reaching for Moving Objects. In Proceedings of the Twenty- Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, pages 4914-4988, 2017. doi: 100.00/4996//icai22007/666
[34] P. Lertkultanon and Q.-C. Pham. A certified-comlltt bimanual manipulation planner. In IEEE Transactions on Automation Science and Engineering, pages 1355-1368, 2018. doi:
00.1199/AASE.2218.29999
[35] J. Gudiño Lau. Dynamic model and simulation of cooperative robots: A case study. Robotica, 23:615 - 624, 09 2005. doi: 10::10.100 0000000777002255770000021233
[36] N. Xi, T.-J. Tarn, and A. Bejezy. Intelligent planning and control for multirobot coordination: An event-based approach. IEEE Transactions on Robotics and Automation, 12(3))439-455)
1996. doi: 109/70.499825.
[37] C. Bersch, B. Pitzer, and S. Kammel. Bimanual robotic cloth manipulation for laundry folding. In 2011 IEEE/RS/ International Conference on Intelligent Robots and Systems, pages 1413- 1419, 2011. doi:
[38] S. H. Lee and M. R. Cutkosky. Fixture Planning With Friction. Journal of Engineering for Industry, 1 1333))32003327, 08 1991. ISSN 0022-0817. dol:10.1115/112899777 URL https: //doi. org 10. 1115/1. 2899703.
[39] H. Asada and A. By. Kinematic analysis of workpart fixturing for flexible assembly with automatically reconfigurable fixtures. IEEE Journal on Robotics and Automation, 1(2):86 94,
1985. doi: 10. 199//RA.198...1880000 1985000
[40] L. Shao, T. Migimatsu, and J. Bohg. Learning to Scaffold the Development of Robotic Manip- ulation Skills. In Proc. IEEE Int. Conf. Robotics and Automation (ICRA), 2020.
[41] R. Holladay, T. Lozano-Pérez, and A. Rodriguez. Robust planning for multi-stage forceful manipulation. In Int. Journal of Robotics Research (IJRR), 2022.
[42] L. Chen, L. F. C. Figueredo, and M. Dogar. Manipulation Planning under Changing External Forces. In Proc. JEEE/RSJ Int. Conf. Intelligent Robots and Systems (IROS), 2018.
[43] Y. Watanabe, K. Nagahama, K. Yamazaki, K. Okada, and M. Inaba. Cooking behavior with handling general cooking tools based on system integration for life-sized humanoid robot. Paladyn, Journal of Behavioral Robotics, 4(2):63-72, 2013. doi:doi: 1eflo:::0.2244 1::2411//////2220000 2013-0001.. 2013bbrr-000-01111
http::tt//////
//do:i////////////////// orgo 10.
[44] K. Zhang, M. Sharma, M. Veloso, and O. Kroemer. Leveraging Multimodal Haptic Sensory Data for Robust Cutting. In JEEE-RAS International Conference on Humanoid Robots, 2019.
[45] K. Yamazaki, Y. Watanabe, K. Nagahama, K. Okada, and M. Inaba. Recognition and ma- nipulation integration for a daily assistive robot working on kitchen environments. In 2010 IEEE International Conference Robotics and Biomimetics, pages 196-201, 2010. doi: 10.1109/ROBIO.CO 201..0000000000.72332.6666
[46] K. He, X. Zhang, S. Ren, and J. Sun. Identity Mappings in Deep Residual Networks. In B. Leibe, J. Matas, N. Sebe, and M. Welling, editors, European Conference Computer Vision, pages 645. Springer International Publishing, 2016.
11

---

[47] A. B. Jung, K. Wada, J. Crall, S. Tanaka, J. Graving. C. Reinders, S. Yadav, J. Banerjee, G. Vecsei, A. Kraft, Z. Rui, J. Borovec, C. Vallentin, S. Zhydenko, K. Pfeiffer, B. Cook, I. Fernández, F.-M. De Rainville, C.-H. Weng, A. Ayala-Acevedo, R. Meudec, M. Laporte, al. imgaug. httpp/////gtthbboooo..aaaaaaaggg 2020. Online: accessed 01-Feb-2020.
[48] Dec 2022. URL http://////////aaaaaa///////////
[49] 2023. URL httph: htt.lp/pp.//aaubbooo......a.....
[50] D. Kingma and J. Ba. Adam: A method for stochastic optimization. International Conference
on Learning Representations, 12 2014.
[51] A. Mandlekar, D. Xu, J. Wong, S. Nasiriany, C. Wang, R. Kulkarni, L. Fei-Fei, S. Savarese, Y. Zhu, and R. Martin-Martin. What matters in learning from offline human demonstrations for robot manipulation. In Conf. on Robot Learning (CoRL), 2021.
12

---

Stabilize to Act: Learning to Coordinate for Bimanual Manipulation Supplementary Material
A
Training Details
We provide details for training each of the mod- els for BUDS: fg and f for the stabilizing pol- icy and n and fo for the acting policy.
Augmentation
Parameters
LinearContrast
(0.95, 1.05)
Add
(-10, 10)
GammaContrast
(0.95, 1.05)
A.1 Stabilizing Policy Training
GaussianBlur
(0.0,
MultiplySaturation
(0.95, 1.05)
The keypoint models fe is trained with a hand- labelled dataset of pairs of images and human-annotated keypoints. We augment each image 10X with a series of label-preserving transformations from ImgAug library [47]. in- cluding rotation, blurring, hue and saturation changes, affine transformations, and adding Gaussian Noise. The detailed parameters for the transformations are listed in Table and we visualize the image augmentations in Fig. 5. The restabilizing classifier f is trained on dataset of images from 20 demonstration roll- outs with 100 images each. Each image is paired with binary expert annotation of whether or not restabilizing is needed and augmented by 2X with the same image transformations from above.
AaditieeGaussanNNisse
(0.3.1875)
Scale
(1.0,
Translate Percent
( (-0.08, 0.08)
Rotate
(-15%, 15%)
Shear
(-8 89)
Cval
(0,20)
Mode
['constant', 'edge']
Table 3: Image Data Augmentation Parame- ters: We report the parameters for the data aug- mentation techniques used to train the stabiliz- ing policy's 's stabilizing position and restabilizing classifier models in BUDS. All augmentations are used from the imgaug Python library [47].
Both the keypoint model and the restabilizing classifier are trained against a binary cross-entropy loss with an Adam [50] optimizer. The learning rate is 1.0e-4ande the weight decay is 1.0e- during the training process. We train these models for 25 epochs on a NVIDIA GeForce GTX 1070 GPU for hour.
A.2 Acting Policy Training
The acting policy starts from pre-grasped position, which achieve using optional grasping keypoint model. The training procedure of grasping keypoint model f is the same as that of stabilizing keypoint model fī. After the robotic gripper grasps the object, we collect 20 acting demonstration rollouts, each with between 50 and 200 steps. The variation of 20 demonstrations comes from the randomization of initial object position. differ- ences in object shape and dynamics, and variations in grasps. With these demonstrations,
set of hy- perparameters for all tasks to train BC-RNN model sim- ilar to prior work [5]]. We load batches of size 100 with a history length of 20. We learn policies from input images and use a ResNet-18 18 [46] architecture encoder which is trained end-to-end. These image encodings are of size and are then concatenated to the proprioceptive input Pt to be passed into the recurrent neural network which uses hidden size of 1000. We train against the standard imitation learning loss with learning rate of
Figure 4: Experimental Setup: We present our experimental setup, which uses three cameras due to heavy occlu- sion during manipulation. One camera is mounted overhead, one is on the wrist of the right arm, and one is facing the front of the workspace at an angle.
13

---

1e-4 and a weight decay of O. We train for 150k epochs on NVIDIA GeForce GTX 1070 GPU for 16 hrs.
Figure 5: Data Augmentation for Image Datasets: We visualize images from the augmentated dataset used to train the stabilizing position model and restabilizating classifier for the marker cap- ping task's stabilizing policy: f and fg. For fö. the dataset of expert-labelled image and keypoint annotations is augmented 10X construct final dataset of size 300. For f the dataset is aug- mented 2X for final size of 4000 image and binary classification pairs.
B Experiment Details
For all tasks, BUDS's acting policy uses a 3D action space. For the three tasks other than Pepper Grinder, this action space represents change in end effector position, (Az, Ay, A²). For the Pepper Grinder task, this action space instead represents the change in end effector roll, pitch, and yaw, due to safety concerns involving the closed chain constraint created by using both arms grasp the pepper grinder tool.
All tasks use the overhead camera for the sta- bilizing keypoint model and grasping model in- puts. Depending on the task and the types of occlusion present during manipulation, we use two of the three cameras for the acting policy and the restabilizing classifier outlined in Ta- ble 4.
Task
Cameras
Pepper Grinder
Overhead, Side
Jacket Zip
Overhead, Side
Marker Cap
Overhead, Wrist
We use the optional grasping model fa for all tasks except the Pepper Grinder task to ac- count for variations of the intial positions of the jacket, markers, and vegetables. Instead for the Pepper Grinder task, the acting arm instead moves to the point corresponding the end effector position of the stabilizing arm, and grasps fixed height above the stabilizing arm corresponding to the height of the pepper grinder. The pepper grinder begins pregrasped in the stabilizing robot hand, but the plate positions are randomly initialized.
Cut Vegetable
Wrist, Side
Table 4: Task-Specific Cameras: We report the cameras used for obtaining images input for the acting policy and restabilizing classifier by task.
In the BC-Stabilizer baseline, the stabilizing policy learned via imitation learning trained with the same procedure the acting policy for BUDS, with the exception of using an output of two Gaussian mixtures cover the 3D (Ax, Ay, Az) action space.
14


Reviews:
[{"review_id": "zpiGY1t5OqA", "reviewer": "Reviewer_13Nc", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "To make learning bimanual manipulation tractable, this paper proposes decomposing tasks such that one arm stabilizes an object while the other acts on it. Rather than having to learn a monolithic joint policy, and to collect complex data, this strategy enables the learning of a separate stabilizing policy and acting policy. Stabilizing is defined as statically holding an object in place, readjusting when needed. Thus the stabilizing policy is composed of a keypoint predictor, for where to hold, and a classifier on when to readjust. The acting policy can then be learned in the simpler environment where the object is assumed to be stabilized. Since acting is defined as grasping an object and then moving, the policy is composed of a grasping keypoint predictor and imitation learning. The paper considers 4 bimanual tasks and evaluates the results in many real world demos. Ablation studies show that the method struggles with objects visually different from the training data. \n", "summary_of_recommendation": "This paper focuses on the very complex and interesting task of bimanual manipulation and proposes separating the problem into stabilizing and acting. The paper demonstrates the method on four well-chosen tasks and conducts a large quantity of real world experiments. The paper would strongly benefit from clarifying how the stabilization method reasons over force (and if it doesn't, why this is sufficient) and how it generates solutions that are compatible with the acting policy. \n\n[My recommendation score has been updated as a result of the rebuttal. I have also added a comment below]", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "As further detailed above, there are a few questions about how the stabilizing policy operates in tandem with the acting policy: how is the orientation of the end effector chosen? How are the joint angles for the stabilizing keypoint selected such that it is collision-free with respect to the acting policy? How is the stabilizing policy informed about what it needs to stabilize with respect to? This is with respect to how the acting policy is interacting with the object, considering both geometry and force. ", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes role assignment to address the high-dimensionality of bimanual tasks, drawing from human inspiration. To help hammer in the difficulty of the problem, the paper explicitly discusses the challenges in trying to solve for the joint two-arm policy. By splitting up the problem into acting and stabilizing, the resulting learning problems are significantly easier. The stabilizing policy is further simplified by considering holding one keypoint \"still\" and discretely updating this keypoint. This framing is shown to be useful in an experimental comparison that considers learning stabilizing with imitation learning. The acting policy is further simplified by assuming the tasks involves grasping an object and then acting. \n\nThe paper proposes four really interesting bimanual tasks: grinding pepper on a series of plates, zipping up a jacket, capping a series of markers, cutting a vegetable in half. These tasks could serve as inspiration to future bimanual manipulation research. Additionally, the method is evaluated in entirely real world experiments which constitutes a significant amount of work (that this reviewer heavily commends!). The video provides a comprehensive summary of the paper and in general the paper is well-written.\n\nWithin the paper, the term \"stable\" is used colloquially as the method does not reason over the forces experienced during the task. This is even though we stabilize, or fixture, an object to hold it still in response to the forces that would perturb the object. For example, consider that for zucchini with a sufficiently large mass, fixturing would not be necessary. The paper would benefit by discussing this limitation and connecting the contribution to previous literature on fixturing (some references given below). \n\nThe idea of decomposing the task into acting and stabilizing is a powerful and useful idea that both allows the problem to be tractable and is general enough to work in a variety of settings. Since the stabilizing and acting policy must ultimately work together, it's slightly unclear how necessary information is shared. For example, both the stabilizing keypoint predictor and restabilizing classifier are trained on demonstration images. Given that how an object must be stabilized is conditioned on the details of the task (i.e. where the object is going to be cut), how is the task encoded in the image? The video describes a scenario where the stabilizing point would be updated as the acting arm cuts further up the zucchini but it is not clear how the stabilizing classifier would have access to this information. \n\nAdditionally, in choosing where to hold the object still, the stabilizing arm must not only choose a point that stabilizes the object but also a point (and an arm configuration) that enables the acting arm to complete its motion without collision. How is this handled? The keypoint predictor selects a position for the end effector, how is the end effector's orientation and joint configuration selected? In grasping to stabilize an object, how is the grasp force selected? Returning to the need to reason over force requirements, one can imagine situations where a stronger grasp is required to hold an object still. \n\nFor the four tasks, task success is measured as the proportion of the task completed. The definition of this is given for the zipper, is it fair to assume that for capping and grinding the success is measured as how many of the three items the robot successfully acts on? How is task success measured for cutting? It seems difficult to precisely measure, in the real world, what percentage of an object has been sliced through. \n\nWith respect to related work, [1] considered the idea of decomposing into an acting robot and fixturing robot, specifically in cutting. Both [2, 3] also considered the setting of one agent (a human and robot respectively) acting while the second agent (a robot) fixtures the object, while reasoning how the object needed to be fixtured to resist the exerted forces. \nWith respect to fixturing generally, both [4, 5] are foundational papers. Finally, [6] also demonstrates how stabilizing an object can make it easier to find an acting policy that operates on that object. \n\n[1] Watanabe, Yoshiaki, Kotaro Nagahama, Kimitoshi Yamazaki, Kei Okada, and Masayuki Inaba. \"Cooking behavior with handling general cooking tools based on a system integration for a life-sized humanoid robot.\" Paladyn, Journal of Behavioral Robotics 4, no. 2 (2013): 63-72.\n[2] Chen, Lipeng, Luis FC Figueredo, and Mehmet R. Dogar. \"Manipulation planning under changing external forces.\" Autonomous Robots 44 (2020): 1249-1269.\n[3] Holladay, Rachel, Tomás Lozano-Pérez, and Alberto Rodriguez. \"Robust Planning for Multi-stage Forceful Manipulation.\" arXiv preprint arXiv:2208.00319 (2022).\n[4] Asada, Haruhiko. \"Kinematics of workpart fixturing.\" In ICRA, vol. 2, pp. 337-345. IEEE, 1985.\n[5] Lee, Soo Hong, and M. R. Cutkosky. \"Fixture planning with friction.\" (1991): 320-327.\n[6] Shao, Lin, Toki Migimatsu, and Jeannette Bohg. \"Learning to scaffold the development of robotic manipulation skills.\" In ICRA, pp. 5671-5677. IEEE, 2020.", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "lXTmoPkjEg", "reviewer": "Reviewer_ku5C", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper introduces BimanUal Dexterity from Stabilization (BUDS), a strategy for bimanual robotic manipulation. The authors propose a division of labor between one stabilizing hand that keeps an object fixed in place and one active hand that interacts with the stabilized object in some way. The approach relies on several keypoint-based perception networks that determine where to grasp objects, what point on the object to hold stable, and when to switch to stabilizing a new object keypoint. The results show an improvement over behavior cloning with no explicit stabilization constraint.", "summary_of_recommendation": "Overall, the paper is well-presented and easy to understand with good robot demo videos. Additionally, it shows large improvement over a commonly-used behavior cloning baseline on the selected tasks. However, I think the paper could use some additional ablations and a few fairer baseline comparisons that make use of the extensive human annotations. The paper is also lacking explanation on some aspects of the system and a description of the failure modes of behavior cloning.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "How does the stabilization controller work? The authors only explain how the keypoint detection model is trained, but how is the robot actually controlled to stabilize a given keypoint? Does the hand move to reposition the keypoint if the object slips in the hand?\n\nWhat are the primary reasons for failure for BC-Stabilizer? A video was only shown for the cutting task and it looked like the gripper fingers bent and the pepper slipped. Could something like this be fixed with a stiffer controller? Are there other failure modes for the BC baseline?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Pros\nThe paper is well written and accompanied by a video that does a good job at explaining the general approach. The real-robot demo videos look cool and the diversity of tasks is a strong positive.\n\nCons\nThe proposed method is only compared to one baseline. Given the complexity of the system, I think the authors should compare to ablated versions of their system. Additionally, the one baseline that is evaluated is not exactly a fair comparison. BUDS is provided with a lot of additional task and problem specific human annotations (such as keypoints, pregrasp points) that the BC baseline isn't given.\n\nAlthough the method improves data efficiency for certain tasks, it also greatly increases the amount of expert annotations needed, resulting in fewer total datapoints per human expert hour.", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "dfD2-gdE1CO", "reviewer": "Reviewer_u9KV", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The authors present an approach for learning bimanual object interaction tasks from demonstrations. The main idea is to reduce the complexity of the problem by assigning roles to the two robot arms instead of learning a generic bi-manual policy. One arm is assigned the role of a \"stabilizer\" and has the task of keeping the object to be manipulated in a fixed position. The second arm is assigned the \"active\" role and has the task of actively manipulating the object. The advantage of the role assignment is that it simplifies the data collection process as well as the learning part due to the reduced overall complexity of the problem.", "summary_of_recommendation": "I chose my rating mainly because I did not find any major flaws in the paper. ", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "- If I understood correctly the roles of the arms are fixed. I'd appreciate if the authors could discuss what would be necessary in order to support tasks that require switching the stabilizer/actor roles during a task. I think it is a weakness of the approach that should be addressed in the limitations section.\n\n- Please clarify if the acting policy is stopped while the stabilizing policy is actively moving the robot arm to a new keypoint pose", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper in my opinion is well written and straightforward to understand. The line of argument behind the approach is reasonable and the design decisions made by the authors are well explained and reasonable as well. The approach is evaluated in various experiments and the results seem to be convincing. Citations seem to be adequate, however I am not an expert in this field and therefore can't really tell if important related work has not been cited.\n\nOne thing that was not explicitly discussed is if the two robot arms can move independently of each other. The stabilizing policy is not conditioned on the acting policy and at some point will decide to move to a new stabilizing point. From the videos, it looks like the acting policy is stopped during such a movement. The authors also mention the acting policy acts on a stationary environment (l. 224). I'd appreciate if the authors could explicitly clarify if the acting policy is stopped when the stabilizing policy moves to a new keypoint pose.", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning."}, "extra_text": {}}, {"review_id": "1biGziCm-zK", "reviewer": "Reviewer_bbdq", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper proposes simplifying the problem of bi-manual manipulation by leveraging the observation that for many tasks of interest, one of the arms is usually used to stabilize an object, while the other is used to act to accomplish the task. Under this premise, the method shows that it is possible to learn a sabilizing policy and then an acting policy sequentially, rather than simultaneously, which greatly simplifes the particular problem of demonstrations.\nThe method leverages both image-based learning with deep NNs and demonstrations via teleoperation and is evaluated in a series of tasks, using similar objects and out-of-distribution objects.\n", "summary_of_recommendation": "This paper provides some interesting observations on the roles of the arms during bi-manual manipulation and leverages this observation to provide a control solution. The idea is clear, and experiments have been well executed. I think the community will profit from the findings and propositions of the paper.\n", "technical_quality": "Excellent: Very convincing and thorough.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "\nI could have missed this in the paper, but what is the frequency of the control loop?\n\nWhat happens if the stabilizing arm cannot, in fact stabilize? For example, if the vegetable constantly slips from the stabilizing gripper as the acting gripper adds pressure on the object?\n\nThe cases of success rates less than 30% is quite disappointing. On the other hand, it seems the number of demonstrations was quite limited. I wonder why the authors decided to stop at only 20 demonstrations, and how the success rate would have increased if more demonstrations were provided.\n\nNot a question, but a comment. It looks like the stabilizing policy could have been profited from methods that explicitly aim at finding invariances among different morphologies such as [1].\n[1] P. Florence, L. Manuelli, and R. Tedrake. Self-supervised correspondence in visuomotor policy learning. IEEE RAL, 2019.\n\nNot a question, but another comment. The paper stresses the difficulty of bimanual demonstrations, but there are many teleoperation systems that allow for such tasks. Surely, using a 3D mouse as in the paper is hard, but there are many other examples of bi-manual teleoperation in which complex tasks have been achieved in different ways; from VR headsets/controllers to master-slave mechanisms. \n\n", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper is well written, easy to understand, and the motivation and narrative of the paper is quite well done and convincing. \nThe idea of separating the stabilizing policy from the action policy proves to be beneficial without feeling ad-hoc or limited in scope.\nVideo was very well done and provides a very nice entry point to the paper.\nPerhaps the main merit of the method is the ingenuity of framing a bi-manual task as supporting and acting tasks while not being restricitve on the various types of tasks that a bi-manual robot is expected to solve.\nThe validation was extensively done, although the task success rate in some cases is a bit disappointing.\n", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning."}, "extra_text": {}}]

Author rebuttal:

Meta-review: I agree with the reviewers highly positive remarks on this paper. The authors also did a great job in their rebuttal responding to the minor concerns brought up by the authors. I recommend the paper be accepted.
Decision: Accept (Oral)