InterACT: Inter-dependency Aware Action Chunking with Hierarchical Attention Transformers for Bimanual Manipulation
Authors: Andrew Choong-Won Lee
Ian Chuang
Ling-Yuan Chen
Iman Soltani
Venue: CoRL 2024 (2024)
Keywords: Robotics
Imitation Learning
Bimanual Manipulation
TL;DR: We presented InterACT, a framework for robust bimanual manipulation, which integrates hierarchical attention transformers to capture inter-dependencies between dual-arm joint states and visual inputs.

Abstract: We present InterACT: Inter-dependency aware Action Chunking with Hierarchical Attention Transformers, a novel imitation learning framework for bimanual manipulation that integrates hierarchical attention to capture inter-dependencies between dual-arm joint states and visual inputs. InterACT consists of a Hierarchical Attention Encoder and a Multi-arm Decoder, both designed to enhance information aggregation and coordination. The encoder processes multi-modal inputs through segment-wise and cross-segment attention mechanisms, while the decoder leverages synchronization blocks to refine individual action predictions, providing the counterpart's prediction as context. Our experiments on a variety of simulated and real-world bimanual manipulation tasks demonstrate that InterACT significantly outperforms existing methods. Detailed ablation studies validate the contributions of key components of our work, including the impact of CLS tokens, cross-segment encoders, and synchronization blocks.

Andrew Lee' Ian Chuang InterACT: Inter-dependency Aware Action Chunking with Hierarchical A Attention Transformers for Bimanual Manipulation
Ling- Ling-Yuan Yuan Chen' Iman Soltani'
University of California Davis
2 University of California Berkeley
Abstract: Bimanual manipulation presents unique challenges compared to uni- manual tasks due to the complexity of coordinating two robotic arms. this paper, we introduce InterACT: Inter-dependency aware Action Chunking with Hierarchical Attention Transformers, novel imitation learning framework de- signed specifically for bimanual manipulation. InterACT leverages hierarchical attention mechanisms to effectively capture inter-dependencies between dual-arm joint states and visual inputs. The framework comprises Hierarchical Attention Encoder, which processes multi-modal inputs through segment-wise and cross- segment attention mechanisms, and a Multi-arm Decoder that generates each arm's action predictions parallel, while sharing information between the arms through synchronization blocks by providing the other arm's intermediate out- put as context. Our experiments. conducted on various simulated and real-world bimanual manipulation tasks, demonstrate that InterACT outperforms existing methods. Detailed ablation studies further validate the significance of key compo- nents, including the impact of CLS tokens, sross-segment encoders. and synchro- nization blocks on task performance. We provide supplementary materials and videos on our project page'.
Keywords: Bimanual Manipulation, Imitation Learning
Introduction
Bimanual manipulation tasks, such as unscrewing a bottle cap or connecting two electrical cables, presents significant challenges due to the need for high precision and complex coordination between the two arms. Traditional approaches often rely on high-end robots and precise sensors, which be expensive and require meticulous calibration [1, 2, 3]. However, recent advances in learning- -based approaches offer the potential to perform such complex tasks using low-cost hardware. The ALOHA and the Action Chunking with Transformers (ACT) framework has shown that low-cost systems can tasks that were traditionally only possible with expensive setups. The ACT addresses compounding error problem in imitation learning by predicting sequences of actions rather than single steps, thereby reducing the task's effective horizon and mitigating emrors over time [4].
achieve
Despite recent advances, achieving the complex coordination between both arms required for sistent and successful execution of even relatively simple tasks remains challenge in bimanual robotics. In this work, we propose InterACT: a new policy designed for bimanual manipulation that emphasizes inter-dependencies between two arms by utilizing hierarchical attention mechanisms. In our designs, multimodal inputs are encoded through segment-wise and cross-segment encoders which handle the complex relationships between different segments in a manner similar to how long documents are processed NLP [5]. This combines the proprioceptive data of the robot arm joints and the visual features of the camera in coherent latent space that allows for coordinated detail-oriented and smooth action execution.
8th Conference Robot Learning (CoRL. 2024), Munich, Germany.

---

Our method advances beyond existing approaches by emphasizing the hierarchical structure of bi- manual tasks: while independent arm movements are managed at the lower level, coordination oc- curs at higher level, allowing each arm's 's actions to be informed by the other's movements as well as the environmental sensory feedback.. Leveraging the latest advances in attention mechanisms, this work presents novel solution for bimanual manipulation tasks.
2 Related Works
2.1 Bimanual Manipulation
Bimanual manipulation involves two robotic arms performing tasks that require dexterity and coor- dination. Inspired by the natural ability of humans to perform such tasks, researchers have been keen on modeling these skills in robots. Prevailing methodologies, including classical control methods, reinforcement learning, and imitation learning, have significantly advances the field.
Early research primarily relied on classical control methods, focusing on predefined trajectories and high-fidelity models to achieve coordinated movements [1, 2, 3]. However, these methods often required extensive calibration and were less adaptable to dynamic environments. The introduction of reinforcement learning (RL) made bimanual manipulation more adaptable and robust. RL-based approaches have proven effective in handling complex tasks, outperforming classical methods, and generalizing across different scenarios [6, 7, 8. 6, 9]. These methods leverage RL's ability to learn from interactions with the environment, improving performance in varied and unpredictable condi- tions [10, 11, 12]. Imitation learning has emerged as a prominent method for teaching robots tasks through human demonstrations. This approach enables robots to mimic complex human actions, facilitating the execution of intricate tasks [13, 14]. Research has demonstrated the effectiveness of imitation learning in training robots for coordinated dual-arm manipulation leveraging waypoints, hierarchical skill learning and force-based techniques [15, 16, 17, 18, 19, 20]. Frameworks like tasks using imitation learn-
ALOHA have shown that low-cost systems perform ing techniques traditionally reserved for expensive setups [4, 21]. Similarly, the Action Chunking with Transformers (ACT) algorithm addresses the compounding error problem in imitation learning by predicting sequences of actions, improving accuracy and efficiency [4].
Despite these advancements, bimanual manipulation remains challenging. Recent research has ex- plored stabilizing arm [22], simplifying actions [23], integrating additional multi-modal data, such as language sensory feedback [24, 25, 26], to enhance the robustness and efficiency of bi- manual manipulation systems. These efforts hold promise for developing more adaptable robotic systems capable of performing more complex manipulation tasks.
2.2 Hierarchical Attention Mechanisms
Hierarchical attention [27] mechanisms have gained prominence for their ability to process and inte- grate multi-modal inputs. These mechanisms aggregate information at multiple levels of granularity, making them well-suited for tasks requiring both local and global context understanding [28, 29].
Hierarchical attention has shown considerable success in other domains, such as natural language processing (NLP), where hierarchical attention transformers have been effectively used for long document classification [5, 30, 31]. These models focus on different parts of the input text at varying levels of abstraction, enhancing the ability to handle long and complex documents. Extensions of foundational attention mechanisms [32], such the Hierarchical Attention Network (HAN) [27] and hierarchical representations in BERT [33], further demonstrate their potential in managing multi- layered information.
In robotic manipulation, the concept of hierarchy has been explored to manage the complexity of long-horizon tasks by breaking down problems into manageable sub-tasks [34], but not at the pol- icy's 's attention layer level. As proven in long document classification, leveraging segment-wiss wise and cross-segment attention mechanisms, hierarchical attention models can capture dependencies within

---

and across sentences [5]. This makes hierarchical attention transformers particularly appealing for use in bimanual robotic manipulation tasks, where capturing dependencies between arms is crucial.
In this work, we tailor the transformer architecture featuring hierarchical attention the bimanual robotics tasks and explore its utility in extracting the complex inter-dependencies between the tions of the two arms. We hypothesize that this can lead to more coordinated actions and hence, more robust performance.
InterACT: Inter-dependency Aware Action Chunking with Hierarchical Attention Transformer
The ACT model [4] leverages transformer architecture to predict future steps in bimanual manipula- tion tasks, effectively handling sequences of actions by capturing temporal dependencies. However, does not explicitly model inter-dependencies between dual-arm joint states and visual inputs, which can limit its performance complex manipulation tasks.
InterACT builds upon the ACT model, enhancing it with hierarchical attention mechanisms to cap- ture inter-dependencies between dual-arm joint states and visual inputs. This section provides an overview of the InterACT model and its key components: the Hierarchical Attention Encoder, which processes multi-modal inputs to capture both intra- (corresponding to one arm or sensory in- put) and inter-segment (across arms or sensory inputs) dependencies, and the Multi-arm Decoder, which generates synchronized action predictions for both arms.
Anra Deceder
Armil Decoder Ieput
Coordintted
Frediceed sctic for
Preduad soton
Cross Sagment Encoder
..
Segment
Segmem-Wise
Segment-Wiie
crocs attendinn
24 cros sttensionn blods
bloca
Hiararchical Attention Encoder Blacks (u4)
Synchroniation Block
[UU Enbedded
[aS]] Brbedded
Amnt Decodar
Femil siquence
Fatten
Fostfon ombbedd
arm
Tonetll
BrbbediedCan1
Figure 1: Architecture of the InterACT. The Hierarchical Attention Encoder consists of multi- ple blocks of segment-wis encoders and cross- cross-segment encoder. The output is passed through the Multi-arm Decoder which consists of Arml and Arm2 specific decoders that process the input seg- ments independently. The synchronization block allows for information sharing between the two decoders.
3.1 Hierarchical Attention Encoder
Segments are defined as independent groups of data inputs that are processed individually before integration. In this context, segments include the joint states of each arm and visual features at a specific timestep. The Hierarchical Attention Encoder processes input segments and captures both intra-segment and inter-segment dependencies through hierarchical attention mechanism that consists of two primary components: the Segment-wise encoder and the Cross-segment encoder. A detailed pipeline for the Hierarchical Attention Encoder is illustrated in Figure 1 and Algorithm 1.

---

Algorithm 1: Hierarchical Attention Encoder
1: Given: Demo dataset D, segment- weie encoder Eug, cross-segment encoder Eemus CLS tokens CLS, number of layers L.
Let S; represent
input segment at index i, CLS; represent the CLS tokens for segment Svisul represent the visual features.
3: Initialize segment wise encoder Essg segment encoder Eeross
4: Initialize
for each segment Prepend CLS tokens: S 4- [CLSS, St]
D do
7: Add positional encoding to S
8:
for each layer I = 1, I, de
9: S < Bieg (S()
10: end for
11: end for
12: Extract CLS tokens: CLS + (CLS1, CLS2, CLSx)
13: Add positional encoding to CLSmkens
14: for each layer 1 = 1,
do CLSukees <- E (CLS
16: end for
17: Return final encoded states (CLSuml, Symn CLSsmuz, Syma, CLS Stisma
Each raw joint position is embedded into a single token through a linear layer. Visual features are extracted from RGB images using ResNet18 backbones, which convert and flatten the images along the spatial dimension form sequence of visual feature tokens. The joint sequences from both arms and the visual feature sequence are then concatenated form combined input sequence. Classification tokens (CLS tokens) are prepended each segment, allowing the model to capture and summarize segment information during attention [33, 35]. Positional embeddings are applied to the sequence to ensure the model can understand the order of the tokens within each segment sequence.
Segment- Wise encoder is responsible for capturing intra-segment dependencies by processing each segment individually. Each segment represents distinct modality, and the encoder's task is to aggregate the information within each segment. The self-attention mechanism allows each token within the segment to attend to every other token, including the CLS tokens, thereby allowing the CLS token to capture intra-segment dependencies. This aggregation of information enables the model to effectively leverage the relationships within different parts of the segment [5], such as the dependencies between joints or the spatial relationships within visual inputs.
Cross-segment encoder is responsible for capturing inter-segment dependencies by handling the CLS tokens generated from each segment by the segment-ve wise encoder. The CLS tokens serve as a condensed representation of each segment, summarizing the key information within the segment. By focusing on these CLS tokens, the cross-segment encoder reduces the computational overhead while effectively integrating information across different modalities.
Multiple segment-wisg and cross-segment encoders stacked, allowing the model progressively refine its understanding of both intra- and inter-segment relationships. We follow interleaved stacking approach [5], where segment-wise and cross rrggsetegment encoders alternate. This deep stack- ing enables the model to capture the complex dependencies across segments.
3.2 Multi-arm Decoder
Similar to multi-task learning in NLP [36, 37], the Multi-arm Decoder comprises two parallel paths of decoder blocks, each dedicated to processing the encoded states and target tokens to generate the predicted actions for one of the arms. This separation makes sense in bimanual manipulation tasks because, while the must coordinate, they often perform independent actions that require individual attention and control. By using separate decoders, each arm can process its specific actions while still sharing critical information through synchronization mechanisms.

---

This section details the components and workflow of the Multi-arm Decoder, highlighting how it employs enriched input sequence and CLS tokens from the encoder to coordinate actions between both arms effectively. A detailed pipeline for the Multi-arm Decoder illustrated Figure and Algorithm 2.
Algorithm 2: Multi-arm Decoder
1: Given; Target tokens with positional embeddings Tpo.. number of layers L. Let Duml, Dum2 represent decoders for Arml and Arm2 respectively. Dayne represent Synchronization block.
4: Initialize cross-attention blocks for Darml and Damm
5: Initialize synchronization block (multihead self-attention)
6: Arml Specific Decoder:
for each layer 2, .... do
8: Armliippt f- { (Saml, CLSum2, CLSvisual, Svisual)
9: Arm outpux f- Darmi (Arml input: Tpos)
10: end for
11: Arm2 Specific Decoder:
each layer 1 1, 2, do
13: Arm2Zing &- [CLS Same, CLSvisull Svisual)
14:
Arm2uurpus <- Dame 2iru, Too))
15: end for
Synchronization Block:
17: Concatenatedaupa f (Arml somllonn Loupurs Aouzuu
18: Sharedeotpul &- Dayne (Concaterattedauual
19: Split Shared Output:
20: Sharedsonuu.ami amm:: 2rtoruduupu.amm2 < Aplit(Sharedaupee
Specific Decoder
22: Arm cupput <- Dumi (Armrlagg Shar edleaga amm))
23: Arm2nua output <- Dumaz Armr2umun Share eddeagga
24: Return Arm ourpas, Arm output
Encoded states from the Hierarchical Attention Encoder are used as the context for decoding. Target tokens, initialized with positional embeddings, serve as the starting point for generating sequence of actions. The input for each decoder includes relevant segments and CLS tokens to facilitate cross-attention mechanisms.
Arm Specific Decoders responsible for generating intermediate decoder outputs for each arm. The Arml Decoder takes in Arml segments, Arm2 CLS tokens, and visual feature segments, while the Arm2 Decoder takes in Arm2 segments, Arm1 CLS tokens, and visual feature segments. Each layer processes these inputs through a cross-attention mechanism with the target tokens, incorporat- ing contextual information from both joint states and visual features. This design enables each arm use contextual information from the other arm ensuring synchronized actions.
Synchronization Block enhances coordination between both arms. Before generating the final out- from each decoder, the intermediate outputs from both decoders are concatenated and processed through a synchronization block, which uses self-attention to integrate shared information. This step ensures that both arms leverage the combined context from the other arm and visual inputs before passing through the reset of the decoder layers. The use of attention mechanisms for sharing in- formation across multiple decoders has also been explored in multi-task learning, demonstrating its effectiveness in improving performance and coherence within tasks [38].
3.3 Training and Evaluation
InterACT is trained using an end-to-end imitation learning framework adapted from the ACT al- gorithm. The training process involves collecting high-quality human demonstrations through a teleoperation system, capturing joint positions and RGB images at 50Hz. The collected data is pre- processed extract joint states and visual features using ResNet18 backbones, converting the RGB images into feature tokens. Each joint state and visual feature is tokenized, with multiple CLS tokens prepended to summarize each segment's information. Positional embeddings are added to retain
5

---

quence information. Action chunking is implemented to predict sequences of actions rather than single steps, reducing the task's s effective horizon and mitigating compounding errors. Additionally, a temporal ensemble method is employed to improve the temporal consistency and robustness of the action predictions by weighing predictions over multiple time steps [4].
Evaluation is conducted on both simulated and real-world tasks, measuring success rates assess the model's performance in generating accurate and coordinated actions.
4 Experiments and Results
For the real-robot setup, we modified the ALOHA [39] setup by adjusting the height of the top camera to improve the visibility of the tabletop environment. This adjustment ensures that the cam- era captures more comprehensive view of the workspace, which is essential for accurately tracking the bimanual manipulation tasks. Our robot setup is illustrated in Appendix B.
To evaluate our model, we conducted experiments on three simulation tasks and six Transfer Cube and Peg Insertion along with Slide Ziploc and Thread Velero are tasks adapted from ACT [4]. We also introduce five new tasks: one simulation task and four real-world tasks. The simulation task is Slot Insertion, where both arms need to grab each side of long peg together and place it in a slot on the table. The new real-world tasks include Insert Plug, Click Pen, Sweep, and Unserew Cap. We collected 50 demonstrations for each task. For the simulation tasks, the data used to train the model were all from human demonstrations and we did not evaluate performance of the models trained on scripted data. Detailed task definitions are provided in Appendix A, and the hyperparameters of both ACT and InterACT are provided in Appendix C.
tasks:
To Evaluate the performance of InterACT, we chose to compare our model against ACT [4] and the Diffusion policy [40]. However, with only 50 demonstrations, the Diffusion policy failed to execute any of the nine tasks. We did not perform any additional tuning on the Diffusion policy, as its poor performance suggested it was not well-suited for tasks with limited demonstration data. Therefore, we focus our comparisons exclusively on ACT, as ACT already significantly outperforms several other policies, including BC-ConvMLP [41, 14), BeT [42], and VINN [43], by large margin in bimanual manipulation tasks [4].
Transfer Cube (Sim)
Peg Insertion (Sim)
Slide Ziploc (Real)
Thread Velcro (Real)
Touch
Transfer
Grasp
Contact
Insert
Grasp
Pinch
Open
Lift
Grasp
Insert
ACT
82
60
50
76
66
20
96
92
88
88
42
16
InterACT
98
88
82
88
78
44
96
92
92
94
56
20
Slot insertion (Sim)
Insert Plug (Real)
Click Pen (Real)
Sweep (Real)
Unserew cap (Real)
Lift
Insert
Grasp
Insert
Grasp
Click
Grusp
Sweep
Touch
Unserew
ACT
96
88
92
88
84
InterACT)
100
100
94
Table 1: Success rate (%) for tasks adapted from ACT [4] (top) and our original tasks (bottom). For simulation tasks, we averaged the results across 3 random seeds over 50 episodes each. The real-world tasks were also evaluated over 50 episodes.
4.1 Results
The results of our experiments are summarized in Tables 1. Our InterACT model shows superior performance compared to ACT on all simulated and real-world tasks. In the simulated tasks, In- terACT outperformed ACT significantly, particularly in the Transfer Cube and Peg Insertion tasks where coordination and precision are crucial. The success rates for the "Transfer" stages in the Transfer Cube task, as well as the "Insert" stages in the Peg Insertion task, were notably higher with InterACT, demonstrating the effectiveness of our method in tasks that require coordination between the two arms. Moreover, in newly introduced tasks such Slot Insertion and Insert Plug, InterACT also demonstrated higher performance. The Slot Insertion task, which requires precise coordination

---

Transfer Cube
Peg Insertion
Slot Insertion
Touch
Lift
Transfer
Grasp
Contact
Insert
Lift
Insert
InterACT (no CLS Tokens)
98
84
84
70
68
22
100
86
InterACT CS Encoder)
72
72
80
InterACT (no Sync Block)
74
54
54
90
86
30
100
100
InterACT (all components)
98
88
84
88
78
44
100
100
Table 2: Success rate (%) for simulation tasks under different conditions, InterACT model with InterACT model without CLS tokens, cross-segment (CS) encoder, and synchronization block. Co- ordination subtasks are indicated in bold.
between arms carry the peg and adjust for alignment, showed 100% success rate with In- terACT, compared to 88% with ACT. Similarly, in the Insert Plug task, InterACT achieved better results in the coordination subtask "Insert". This highlights the robustness of our model in handling tasks that require precise coordination between the two arms.
Overall, the experimental results validate the effectiveness of our hierarchical attention framework. By improving coordination and precision in bimanual manipulation tasks, our InterACT model pro- vides more robust solution for complex bimanual manipulation challenges in both simulation and real-world scenarios.
4.2 Ablation Studies
In this section, we perform ablation studies to evaluate the contributions different components of InterACT framework. Specifically, we focus on the impact CLS tokens, the cross-segment encoder, and the synchronization block in the decoder.
Impact of CLS Tokens: To assess the impact of CLS tokens, we conducted experiments with and without CLS tokens as input to the decoder. The results, summarized in Table 2. showed no significant difference in the easier Transfer Cube task. However, there were notable improvements in the success rates of the more complex Peg Insertion task when CLS tokens were included. The aggregated information in the CLS tokens enhances the model's s ability to generate accurate and coordinated actions, particularly in tasks requiring higher precision and synchronization.
Attention Weights for CLS tokans Multi-arm Decoder
Attention Weights for CLS tokens in Multi-arm Decader
Ammt Decoder
Amm1 Decader
Aberten magettan
stteree
Timestep
Timestep
Figure 2: Attention weights for CLS tokens at the Multi-arm Decoder over time for Peg In- sertion (right) and Transfer Cube (right). The red highlighted sections correspond to specific timesteps in executing the task. Spikes in attention weights are observed during coordinated phase.
To gain deeper insights into the model's behavior, we studied how attention weights to CLS tokens at the decoder change over the timesteps. The results illustrated in Figure 2 showed that during the phase of interaction between the two arms, significant spikes in attention weights were observed. These spikes occurred at key moments where coordinated actions between the arms were neces- sary. This indicates that the model heavily relies on the CLS tokens process information when coordinating actions between the arms. This observation highlights the CLS tokens' importance in facilitating precise bimanual manipulation.
7

---

Impact of Cross-segment Encoder: The cross-segment encoder captures inter-segment dependen- cies, allowing the model to effectively integrate information from different joints across the two arms as well as the camera frames. The results indicate that removing the cross-segment encoder signifi- cantly decreases performance complex tasks. For example, in the Slot Insertion task, the success rate for the coordination subtask "Insert" dropped 24% from 44% without the cross-segment encoder. This highlights the importance of capturing inter-segment dependencies for generating accurate and coordinated actions.
The synchronization block enhances coordination between the two arms by sharing contextual information at the decoder. This is crucial for synchronized and efficient bimanual manipulation. The results show that the removal of the synchronization block leads significant drop in performance across all tasks, particularly in the Transfer Cube and Peg Insertion tasks. This demonstrates the necessity of the synchronization block for achieving coordinated and actions.
Impact of Synchronization Block:
Our ablation studies clearly illustrate that all components the proposed InterACT frame- York-LLS tokens, cross-segment encoder, and synchronization block- play- a critical role in achieving high success rates in coordination tasks. The best performance for the coordination sub- task is achieved when all components are utilized, underscoring the importance of the holistic inte- gration of these elements in the InterACT framework.
5 Conclusion and Future Work
In this work, we introduced InterACT, a framework for robust bimanual manipulation, which in- tegrates hierarchical attention transformers to capture inter-dependencies between dual-arm joint states and visual inputs. The key contributions our work include the development of Hierar- chical Attention Encoder and Multi-arm Decoder. The Hierarchical Attention Encoder aggregates intra-segment information using segment-wise encoder and integrates inter-segment dependen- cies through cross-segment encoder. The Multi-arm Decoder, while generating action sequence for each arm in parallel, ensures coordinated action sequence generation through synchronization blocks. Our experiment results on both simulated and real-world tasks demonstrate the superior performance of InterACT compared to the baseline ACT. The ablation studies further highlight the importance of these components in our framework. The use of CLS tokens, cross- cross-segment encoder and the synchronization block significantly enhances the model's 's' ability to generate accurate and coordinated actions, leading to higher success rates in bimanual manipulation tasks.
One of the limitations of InterACT is that the number of CLS tokens and the number of encoder blocks are influenced by the level of coordination required for a given task. While we set fixed number of CLS tokens across all tasks, finding the optimal hyperparameter requires heavy tuning, as there is no clear rule to guide this selection. Additionally, the level of coordination between the two arms is not straightforward to measure. Coordination can depend on task-specific factors such as temporal synchronization, dependency between arm movements, and precision, which are not easily quantifiable. As result, determining the optimal hyperparameters becomes more challenging, requiring manual adjustments and multiple experiments to ensure optimal performance. One poten- tial approach to mitigate this challenge could involve adding an auxiliary loss term that explicitly encourages coordinated actions between the two arms, which might help improve coordination with- out relying on extensive tuning. However, this may require quantitative definition of coordination and further experimentation. As it stands, the scalability of our approach to more complex tasks or scenarios with diverse coordination requirements remains a challenge.
While InterACT has shown promising directions in integrating robotic arm joints and visual inputs, it has not yet explored integration with other modalities. Future work will explore integrating addi- tional modalities, such as language or sensory feedback, to further improve the robustness of biman- ual manipulation tasks. Additionally, addressing the hierarchical nature tasks will be crucial for better task decomposition and execution. These enhancements could leverage the flexible attention mechanisms demonstrated this work to manage the added complexity and data integration.
8

---

Acknowledgments
We thank the members of the UC Davis Laboratory for AI, Roboties, and Automation (LARA) for their valuable feedback, discussions, and assistance with data collection that contributed to this work.
References
[1] S. S. Mirrazavi Salehian, N. Figueroa, and A. Billard. A unified framework for coordinated multi-arm motion planning. The International Journal of Robotics Research, 37(10); 1232, 2018.
[2] Y. Koga and J.-C. Latombe. Experiments in dual-arm manipulation planning. In Proceedings 1992 IEEE International Conference on Robotics and Automation, pages 2238-2239. IEEE Computer Society, 1992.
[3] C. Smith, Y. Karayiannidis, L. Nalpantidis, X. Gratal, P. Qi, D. V. Dimarogonas, and D. Kragic. Dual arm manipulation- survey. Robotics and Autonomous systems, 60(10):1340) (110):3440-15553 2012.
[4] T. Z. Zhao, V. Kumar, S. Levine, and C. Finn. Learning fine-grained bimanual manipulation with low-cost hardware. RSS, 2023.
[5] I. Chalkidis, X. Dai, M. Fergadiotis, P. Malakasiotis, and D. Elliott. An exploration of hi- erarchical attention transformers for efficient long document classification. arXiv preprint arXiv:2210.05529, 2022.
[6] R. Chitnis, S. Tulsiani, S. Gupta, and A. Gupta. Efficient bimanual manipulation using learned task schemas. In 2020 IEEE International Conference Robotics and Automation (ICRA), IEEE, 2020.
pages 1149-1112 1155.
[7] Y. Chen, T. Wu, Wang, X. Feng, Jiang, Z. Lu, S. McAleer, H. Dong, S.-C. Zhu, and Y. Yang. Towards human-level bimanual dexterous manipulation with reinforcement learning. Advances in Neural Information Processing Systems, 35:5150-5163, 2022.
[8] Kataoka, S. K. Ghasemipour, D. Freeman, and Mordatch. Bi-manual manipulation and attachment via sim-to-real reinforcement learning. arXiv preprint vwXiv:2203.08277 2022.
[9] K. S. Luck and H. B. Amor. Extracting bimanual synergies with reinforcement learning. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4805-4812. IEEE, 2017.
[10] S. Levine, C. Finn, Darrell, and P. Abbeel. End-to-end training of deep visuomotor policies. Journal of Machine Learning Research, 17(39):1-40, 2016.
[11] D. Quillen, E. Jang, O. Nachum, C. Finn, J. Ibarz, and S. Levine. Deep reinforcement learning for vision-based robotic grasping: A simulated comparative evaluation of off-policy methods. In 2018 IEEE international conference on robotics and automation (ICRA), pages 6284-6291. IEEE, 2018
[12] V. Mnih. Playing atari with deep reinforcement learning. arXiv preprint 1rXiv:1312.5602, 2013.
[13] C. Finn, T. Yu, Т. Zhang, P. Abbeel, and S. Levine. One-shot visual imitation learning via meta-learning. In Conference on robot learning, pages 357-368, PMLR, 2017.
[14] E. Jang, A. Irpan, M. Khansari, D. Kappler, F. Ebert, C. Lynch, S. Levine, and C. Finn. Be-z: Zero-shot task generalization with robotic imitation learning. In Conference Robot Learn- ing, pages 991-1002 PMLR, 2022.

---

[15] L. X. Shi, A. Sharma, T. Z. Zhao, and C. Finn. Waypoint- based imitation learning for robotic manipulation. arXiv preprint arXiv:2307. 14326, 2023.
[16] O. Kroemer, C. Daniel, G. Neumann, H. Van Hoof, and J. Peters. Towards learning hicrarchical skills for multi-phase manipulation tasks. In 20/5 IEEE international conference robotics and automation (ICRA), pages 1503-1510. IEEE, 2015.
[17] A. X. Lee, H. Lu, A. Gupta, S. Levine, and P. Abbeel. Learning force-based manipulation of deformable objects from multiple demonstrations, 2015 IEEE International Conference on Robotics and Automation (ICRA), pages 177-184. IEEE, 2015.
[18] T. Asfour, P. Azad, F. Gyarfas, and R. Dillmann. Imitation learning of dual-arm manipulation tasks in humanoid robots. International journal of humanoid robotics, 5(02): 000:1833200 202, 2008.
[19] J. Silvério, L. Rozo, S. Calinon, and D. G. Caldwell. Learning bimanual end-effector poses from demonstrations using task-parameterized dynamical systems. In 2015 IEEE/RSJ interna- lional conference on intelligent robots and systems (IROS), pages 464-470. IEEE, 2015.
[20] A.-L. Pais Ureche and A. Billard. Learning bimanual coordinated tasks from human demon- strations. In Proceedings of the Tenth Anmual ACM/IEEE International Conference on Human- Robot Interaction Extended Abstracts, pages 141-142, 2015.
[21] Z. Fu, T. Z. Zhao, and C. Finn. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation. arXiv, 2024.
[22] J. Grannen, Y. Wu, B. Vu, and D. Sadigh. Stabilize act: Learning to coordinate for bimanual manipulation. In Conference on Robot Learning, pages 563-576. PMLR, 2023.
[23] A. Bahery, P. Mandikal, B. Abbatematteo,, and Martin-Martin. Screwmimic: Bimanual imitation from human videos with screw space projection. arXiv preprint arXiv:2405.03666 2024.
[24] L. X. Shi, Z. Hu, T. Z. Zhao, A. Sharma, K. Pertsch, J. Luo, S. Levine, and C. Finn. Yell at your robot: Improving on-the-fly fly from language corrections. arXiv preprint 2rXiv:2403.19911 2024.
[25] N. Becker, E. Gattung, K. Hansel, T. Schneider, Y. Zhu, Y. Hasegawa, and J. Peters. Integrating visuo-tactile sensing with haptic feedback for teleoperated robot manipulation. arXiv preprint arXiv:2404. 19585, 2024.
[26] Varley, S. Singh, D. Jain, K. Choromanski, A. Zeng, S. B. R. Chowdhury, A. Dubey, and V. Sindhwani. Embodied ai with two arms: Zero-shot learning, safety and modularity. arXiv preprint arXiv:2404.03570 2024.
[27] Z. Yang, D. Yang, C. Dyer, X. He, A. Smola, and Hovy. Hierarchical attention networks for document classification. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies, pages 1480-1489, 2016.
[28] Y. Gu, K. Yang, S. Fu, S. Chen, X. Li, and Marsic. Multimodal affective analysis using hierarchical attention strategy with word-level alignment. In Proceedings of the conference. Association for Computational Linguistics. Meeting, volume 2018, page 2225. NIH Public Access, 2018.
[29] C. Wu, Y. Wei, X. Chu, S. Weichen, F. Su, and L. Wang. Hierarchical attention-based multi- modal fusion for video captioning. Neurocomputing. 315:362-370, 2018.
[30] I. Chalkidis, I. Androutsopoulos, and N. Aletras. Neural legal judgment prediction in english. arXiv preprint 1906v 2019.
10

---

[3]] C. Wu, F. Wu, T. Qi, and Y. Huang. Hi-transformer: Hierarchical interactive transformer for efficient and effective long document modeling. arXiv preprint arXiv:2/06.01040, 2021.
[32] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, Jones, A. N. Gomez, t.. Kaiser, and I. Polo- sukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[33] Devlin, M.-W. Chang, K. Lee, and Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprini
2018.
[34] J. Luo, C. Xu, X. Geng, G. Feng, K. Fang. L. Tan, S. Schaal, and S. Levine. Multi-stage cable routing through hierarchical imitation learning. IEEE Transactions on Robotics, 2024.
[35] A. Dosovitskiy. An image is worth 16x 16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11992 2020.
[36] H. Ivison and M. E. Peters. Hyperdecoders: Instance-specific decoders for multi-task nlp. arXiv preprint arXiv:2203.083004 2022.
[37] X. Xu, H. Zhao, V. Vineet, S.-N. Lim, and A. Torralba. Mtformer: Multi-task learning via transformer and cross-task reasoning. European Conference on Computer Vision, pages 304-321, Springer, 2022.
[38] I. Lopes, T.-H. Vu, and R. de Charette. Cross-task attention mechanism for dense multi-task learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2329-222222222 2338, 2023.
[39] J. Aldaco, T. Armstrong, R. Baruch, J. Bingham, S. Chan, K. Draper, D. Dwibedi, C. Finn, Florence, S. Goodrich, et al. Aloha 2: An enhanced low-cost hardware for bimanual teleop- eration.
arXiv preprint WxXi:2405.00292. 2024.
[40] C. Chi, Z. Xu, S. Feng, E. Cousineau, Y. Du, B. Burchfiel, R. Tedrake, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, page 02783649241223666, 2023.
[41] T. Zhang, 7. McCarthy, O. Jow, D. Lee, X. Chen, K. Goldberg, and P. Abbeel. Deep imita- tion learning for complex manipulation tasks from virtual reality teleoperation. In 2018 IEEE international conference robotics and automation (ICRA), pages
IEEE, 2018.
[42] N. M. Shafiullah, Cui, A. A. Altanzaya, and L. Pinto. Behavior transformers: Cloning modes with stone. Advances in neural information processing systems, 35:22955-22968, 2022.
[43] Pari, N. M. Shafiullah, S. Arunachalam, and Pinto. The surprising effectiveness of representation learning for visual imitation. arXiv preprint I 2021.

---

Appendix A: New task definitions
In this section, we define the new tasks we introduced in this work including one simulated task and four real weal-world tasks.
Slot Insertion (Sim): Slot insertion simulated task where both arms need to lift each side of long peg together (Lift) and place in slot on the table (Insert).
Insert Plug (Real): Insert Plug is real-world task where each arm grabs a male and female electrical plug respectively (Grasp) and connects the plugs above the table (Insert).
Click Pen (Real): Click Pen a real-world task where each one (left) arm grabs retractable pen in the middle (Grasp), and clicks the pen with the other (right) arm (Click).
Sweep (Real): Sweep is real-worke task where one arm grabs a brush and the other arm grabs dustpan (Grasp). The arms then move towards toy object lying on the table and sweep it into the dustpan (Sweep).
Unserew Cap (Real): Unscrew Cap is real-world task where one arm grabs a plastic water bot- lle while the other arm reaches and touches the bottle cap (Touch), then grabs the bottle cap and unscrews it (Unscrew).
12

---

Appendix B: Real-robot Setup
We utilize the ALOHA setup [39] for our real-world experiments. Rather than cropping the top camera frame, we lower the camera's height to focus on the table environment, thereby maintaining resolution and capturing the necessary details. Additionally, similar the ALOHA setup [4], tarp around the setup to block unnecessary background distractions. These modifications help enhance the quality of data collected by ensuring that the attention is solely on the manipulation tasks. A photo of our setup is shown in Figure 3.
Figure 3: Our Real-robot Setup. We have modified the ALOHA setup for our real-world experi-
ments. Modifications include adjusting the camera height and using tarp around the setup.
13

---

Appendix C: Hyperparameters
In this section, we summarize the hyperparameters InterACT and ACT models used for training and evaluation in this paper.
Hyperparameters
# Segment-Win Wise Encoder Layers
3
# Cross-Segment Encoder Layers
3
# Multi-arm Decoder Layers
4
Synchronization Block Layers
1
CL.S tokens for Arm Joints
# CLS tokens for Visual Features
5
Table 3: Unique hyperparameters of InterACT'
Hyperparameters
# Encoder Layers
4
# Decoder Layers
7
Table Unique hyperparameters of ACT
Hyperparameters
Learning Rate
le-5
Batch Size
8
Feedforward Dimension
3200
Hidden Dimension
512
# Heads
8
Chunk Size
50
Beta
10
Dropout
0.1
Table 5: Common hyperparameters of InterACT and ACT
14


Reviews:
[{"review_id": "tJMvwadnrk", "reviewer": "Reviewer_n8FP", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 4, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This work extends the ACT framework by introducing a hierarchical attention mechanism for bimanual manipulation tasks, named InterACT. InterACT includes a Hierarchical Attention Encoder and a Multi-arm Decoder. The multi-arm decoder consists of a synchronization block to facilitate the coordination of both arms.  The method is evaluated both in simulation as well in real-world. Ablation studies confirm the need for component, such as the synchronization block.", "summary_of_recommendation": "Good work, some parts require clarification, e.g. if it has been trained with scripted data or with human demonstrations", "technical_quality": 4, "clarity_of_presentation": 5, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "1.)\tACT reports different numbers if training with scripted data. It seems  that this work has copied the numbers when training with human data from ACT. Are the experiments for InterACT scripted or trained with human data?\n2.)  The work claims to “significantly outperforms existing methods” (L11). Which one besides ACT?\n3.) Can you elaborate on the design criteria of the new tasks? In what way do help to measure coordination?\n4.)\tHow often did you run the experiments?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1nqH1AdCt8", "reviewer": "Reviewer_KWL4", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 2, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper presents a new imitation learning method for bimanual manipulation. Built on top of the Action Chunking with Transformers (ACT) framework, the method introduces a hierarchical attention encoder and a multi-arm decoder. Specifically, the encoder consists of segment-wise encoders and a cross-segment encoder to better capture the interdependencies between dual-arm joint states and visual inputs. The decoder includes arm-specific decoders and a synchronization block. The method is demonstrated on both simulated and real-world bimanual manipulation tasks.", "summary_of_recommendation": "I recommend a weak reject based on the issues discussed above. My primary concern lies in the novelty of the proposed method, as the idea of a hierarchical attention mechanism has been introduced previously. Additionally, a more comprehensive method comparison and the details of the evaluation protocol need to be included.", "technical_quality": 2, "clarity_of_presentation": 3, "potential_impact": 2, "robotics_focus": 4, "quality_of_limitations": 1, "questions_for_rebuttal": "My main concerns are discussed in the weaknesses section. Additionally, several important details about the evaluation protocol are missing:\n* How many training seeds are used in the experiments?\n* How many demonstrations are used to train the models?\n* How many episodes are being evaluated? \n* What are the environment variations?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qRWQXMif3S", "reviewer": "Reviewer_Gcy9", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper \"InterACT\" introduces a novel framework for bimanual manipulation using Hierarchical Attention Transformers (HAT). It addresses the challenge of coordinating actions between both arms by integrating segment-wise and cross-segment attention mechanisms. The framework includes a Hierarchical Attention Encoder for multi-modal input processing and a Multi-arm Decoder for synchronized action prediction. Key contributions include significant performance improvements over existing methods in various tasks, validated through extensive experiments and ablation studies that highlight the effectiveness of its components like CLS tokens and synchronization blocks.", "summary_of_recommendation": "Overall, InterACT appears to be a significant contribution to the field of robotic manipulation, leveraging advanced attention mechanisms to improve coordination and performance in bimanual tasks. Further peer review and empirical validation will be crucial for confirming its robustness and practical utility across various applications.", "technical_quality": 3, "clarity_of_presentation": 4, "potential_impact": 4, "robotics_focus": 2, "quality_of_limitations": 1, "questions_for_rebuttal": "1. The Hierarchical Attention Transformers may introduce computational overhead, which could limit real-time application on real robot?  \n\n2. Could the authors provide more comparision with other baselines?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Strengths:
+ InterACT is a new approach to multi-arm coordination in bimanual robot-learning.
+ The approach is demonstrated on real-robot tasks.
+ The ablations confirm the contributions of the proposed changes to ACT.

Weaknesses:
- InterACT is only benchmarked against one baseline: ACT. Other visuomotor baselines like Diffusion Policy could be included. 
- Some details on the evaluation setting are missing. 
- The performance gains over ACT are marginal, given the added complexity of InterACT.

Post rebuttal:  
While there were some concerns regarding the novelty of the approach, InterACT's approach is non-trivial and an interesting contribution to bimanual manipulation.
Decision: Accept