InterACT: Inter-dependency Aware Action Chunking with Hierarchical Attention Transformers for Bimanual Manipulation

Authors: andrew  lee
ian  chuang
ling-yuan  chen
iman  soltani
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-06-07 09:50:23

Abstract: We present InterACT: Inter-dependency aware Action Chunking with Hierarchical Attention Transformers, a novel imitation learning framework for bimanual manipulation that integrates hierarchical attention to capture inter-dependencies between dual-arm joint states and visual inputs. InterACT consists of a Hierarchical Attention Encoder and a Multi-arm Decoder, both designed to enhance information aggregation and coordination. The encoder processes multi-modal inputs through segment-wise and cross-segment attention mechanisms, while the decoder leverages synchronization blocks to refine individual action predictions, providing the counterpart's prediction as context. Our experiments on a variety of simulated and real-world bimanual manipulation tasks demonstrate that InterACT significantly outperforms existing methods. Detailed ablation studies validate the contributions of key components of our work, including the impact of CLS tokens, cross-segment encoders, and synchronization blocks.

Main paper: 
 introduction: 
 Bimanual manipulation tasks, such as unscrewing a bottle cap or connecting two electrical cables, presents significant challenges due to the need for high precision and complex coordination between the two arms. Traditional approaches often rely on high-end robots and precise sensors, which can be expensive and require meticulous calibration [1, 2, 3] . However, recent advances in learning-based approaches offer the potential to perform such complex tasks using low-cost hardware. The ALOHA and the Action Chunking with Transformers (ACT) framework has shown that low-cost systems can achieve high-precision tasks that were traditionally only possible with expensive setups. The ACT addresses compounding error problem in imitation learning by predicting sequences of actions rather than single steps, thereby reducing the task's effective horizon and mitigating errors over time [4] .Despite recent advances, achieving the complex coordination between both arms required for consistent and successful execution of even relatively simple tasks remains a challenge in bimanual robotics. In this work, we propose InterACT: a new policy designed for bimanual manipulation that emphasizes inter-dependencies between two arms by utilizing hierarchical attention mechanisms. In our designs, multimodal inputs are encoded through segment-wise and cross-segment encoders which handle the complex relationships between different segments in a manner similar to how long documents are processed in NLP [5] . This combines the proprioceptive data of the robot arm joints and the visual features of the camera in a coherent latent space that allows for coordinated detail-oriented and smooth action execution.Our method advances beyond existing approaches by emphasizing the hierarchical structure of bimanual tasks: while independent arm movements are managed at the lower level, coordination occurs at a higher level, allowing each arm's actions to be informed by the other's movements as well as the environmental sensory feedback.. Leveraging the latest advances in attention mechanisms, this work presents a novel solution for bimanual manipulation tasks.2 Related Works 
bimanual manipulation: 
 Bimanual manipulation involves two robotic arms performing tasks that require dexterity and coordination. Inspired by the natural ability of humans to perform such tasks, researchers have been keen on modeling these skills in robots. Prevailing methodologies, including classical control methods, reinforcement learning, and imitation learning, have significantly advanced the field.Early research primarily relied on classical control methods, focusing on predefined trajectories and high-fidelity models to achieve coordinated movements [1, 2, 3] . However, these methods often required extensive calibration and were less adaptable to dynamic environments. The introduction of reinforcement learning (RL) made bimanual manipulation more adaptable and robust. RL-based approaches have proven effective in handling complex tasks, outperforming classical methods, and generalizing across different scenarios [6, 7, 8, 6, 9] . These methods leverage RL's ability to learn from interactions with the environment, improving performance in varied and unpredictable conditions [10, 11, 12] . Imitation learning has emerged as a prominent method for teaching robots tasks through human demonstrations. This approach enables robots to mimic complex human actions, facilitating the execution of intricate tasks [13, 14] . Research has demonstrated the effectiveness of imitation learning in training robots for coordinated dual-arm manipulation leveraging waypoints, hierarchical skill learning and force-based techniques [15, 16, 17, 18, 19, 20] . Frameworks like ALOHA have shown that low-cost systems can perform high-precision tasks using imitation learning techniques traditionally reserved for expensive setups [4, 21] . Similarly, the Action Chunking with Transformers (ACT) algorithm addresses the compounding error problem in imitation learning by predicting sequences of actions, improving accuracy and efficiency [4] . Despite these advancements, bimanual manipulation remains challenging. Recent research has explored stabilizing one arm [22] , simplifying actions [23] , integrating additional multi-modal data, such as language or sensory feedback [24, 25, 26] , to enhance the robustness and efficiency of bimanual manipulation systems. These efforts hold promise for developing more adaptable robotic systems capable of performing more complex manipulation tasks.Hierarchical attention [27] mechanisms have gained prominence for their ability to process and integrate multi-modal inputs. These mechanisms aggregate information at multiple levels of granularity, making them well-suited for tasks requiring both local and global context understanding [28, 29] .Hierarchical attention has shown considerable success in other domains, such as natural language processing (NLP), where hierarchical attention transformers have been effectively used for long document classification [5, 30, 31] . These models focus on different parts of the input text at varying levels of abstraction, enhancing the ability to handle long and complex documents. Extensions of foundational attention mechanisms [32] , such as the Hierarchical Attention Network (HAN) [27] and hierarchical representations in BERT [33] , further demonstrate their potential in managing multilayered information.In robotic manipulation, the concept of hierarchy has been explored to manage the complexity of long-horizon tasks by breaking down problems into manageable sub-tasks [34] , but not at the policy's attention layer level. As proven in long document classification, leveraging segment-wise and cross-segment attention mechanisms, hierarchical attention models can capture dependencies within and across sentences [5] . This makes hierarchical attention transformers particularly appealing for use in bimanual robotic manipulation tasks, where capturing dependencies between arms is crucial.In this work, we tailor the transformer architecture featuring hierarchical attention to the bimanual robotics tasks and explore its utility in extracting the complex inter-dependencies between the actions of the two arms. We hypothesize that this can lead to more coordinated actions and hence, more robust performance.3 InterACT: Inter-dependency Aware Action Chunking with Hierarchical Attention TransformerThe ACT model [4] leverages transformer architecture to predict future steps in bimanual manipulation tasks, effectively handling sequences of actions by capturing temporal dependencies. However, it does not explicitly model inter-dependencies between dual-arm joint states and visual inputs, which can limit its performance in complex manipulation tasks.InterACT builds upon the ACT model, enhancing it with hierarchical attention mechanisms to capture inter-dependencies between dual-arm joint states and visual inputs. This section provides an overview of the InterACT model and its key components: the Hierarchical Attention Encoder, which processes multi-modal inputs to capture both intra-(corresponding to one arm or sensory input) and inter-segment (across arms or sensory inputs) dependencies, and the Multi-arm Decoder, which generates synchronized action predictions for both arms.  
hierarchical attention encoder: 
 Segments are defined as independent groups of data inputs that are processed individually before integration. In this context, segments include the joint states of each arm and visual features at a specific timestep. The Hierarchical Attention Encoder processes input segments and captures both intra-segment and inter-segment dependencies through a hierarchical attention mechanism that consists of two primary components: the Segment-wise encoder and the Cross-segment encoder. A detailed pipeline for the Hierarchical Attention Encoder is illustrated in Figure 1 and Algorithm 1.Algorithm 1: Hierarchical Attention Encoder Each raw joint position is embedded into a single token through a linear layer. Visual features are extracted from RGB images using ResNet18 backbones, which convert and flatten the images along the spatial dimension to form a sequence of visual feature tokens. The joint sequences from both arms and the visual feature sequence are then concatenated to form a combined input sequence. Classification tokens (CLS tokens) are prepended to each segment, allowing the model to capture and summarize segment information during attention [33, 35] . Positional embeddings are applied to the sequence to ensure the model can understand the order of the tokens within each segment sequence.Segment-Wise encoder is responsible for capturing intra-segment dependencies by processing each segment individually. Each segment represents a distinct modality, and the encoder's task is to aggregate the information within each segment. The self-attention mechanism allows each token within the segment to attend to every other token, including the CLS tokens, thereby allowing the CLS token to capture intra-segment dependencies. This aggregation of information enables the model to effectively leverage the relationships within different parts of the segment [5] , such as the dependencies between joints or the spatial relationships within visual inputs.Cross-segment encoder is responsible for capturing inter-segment dependencies by handling the CLS tokens generated from each segment by the segment-wise encoder. The CLS tokens serve as a condensed representation of each segment, summarizing the key information within the segment. By focusing on these CLS tokens, the cross-segment encoder reduces the computational overhead while effectively integrating information across different modalities.Multiple segment-wise and cross-segment encoders are stacked, allowing the model to progressively refine its understanding of both intra-and inter-segment relationships. We follow an interleaved stacking approach [5] , where segment-wise and cross-segment encoders alternate. This deep stacking enables the model to capture the complex dependencies across segments.Similar to multi-task learning in NLP [36, 37] , the Multi-arm Decoder comprises two parallel paths of decoder blocks, each dedicated to processing the encoded states and target tokens to generate the predicted actions for one of the arms. This separation makes sense in bimanual manipulation tasks because, while the arms must coordinate, they often perform independent actions that require individual attention and control. By using separate decoders, each arm can process its specific actions while still sharing critical information through synchronization mechanisms.This section details the components and workflow of the Multi-arm Decoder, highlighting how it employs enriched input sequence and CLS tokens from the encoder to coordinate actions between both arms effectively. A detailed pipeline for the Multi-arm Decoder is illustrated in Figure 1 and Algorithm 2.Algorithm 2: Multi-arm Decoder Encoded states from the Hierarchical Attention Encoder are used as the context for decoding. Target tokens, initialized with positional embeddings, serve as the starting point for generating sequence of actions. The input for each decoder includes relevant segments and CLS tokens to facilitate cross-attention mechanisms.Arm Specific Decoders are responsible for generating intermediate decoder outputs for each arm. The Arm1 Decoder takes in Arm1 segments, Arm2 CLS tokens, and visual feature segments, while the Arm2 Decoder takes in Arm2 segments, Arm1 CLS tokens, and visual feature segments. Each layer processes these inputs through a cross-attention mechanism with the target tokens, incorporating contextual information from both joint states and visual features. This design enables each arm to use contextual information from the other arm ensuring synchronized actions.Synchronization Block enhances coordination between both arms. Before generating the final output from each decoder, the intermediate outputs from both decoders are concatenated and processed through a synchronization block, which uses self-attention to integrate shared information. This step ensures that both arms leverage the combined context from the other arm and visual inputs before passing through the reset of the decoder layers. The use of attention mechanisms for sharing information across multiple decoders has also been explored in multi-task learning, demonstrating its effectiveness in improving performance and coherence within tasks [38] .InterACT is trained using an end-to-end imitation learning framework adapted from the ACT algorithm. The training process involves collecting high-quality human demonstrations through a teleoperation system, capturing joint positions and RGB images at 50Hz. The collected data is preprocessed to extract joint states and visual features using ResNet18 backbones, converting the RGB images into feature tokens. Each joint state and visual feature is tokenized, with multiple CLS tokens prepended to summarize each segment's information. Positional embeddings are added to retain se-quence information. Action chunking is implemented to predict sequences of actions rather than single steps, reducing the task's effective horizon and mitigating compounding errors. Additionally, a temporal ensemble method is employed to improve the temporal consistency and robustness of the action predictions by weighing predictions over multiple time steps [4] .Evaluation is conducted on both simulated and real-world tasks, measuring success rates to assess the model's performance in generating accurate and coordinated actions. 
experiments and results: 
 For the real-robot setup, we modified the ALOHA 2 [39] setup by adjusting the height of the top camera to improve the visibility of the tabletop environment. This adjustment ensures that the camera captures a more comprehensive view of the workspace, which is essential for accurately tracking the bimanual manipulation tasks. Our robot setup is illustrated in Appendix B.To evaluate our model, we conducted experiments on three simulation tasks and six real-world tasks: Transfer Cube and Peg Insertion along with Slide Ziploc and Thread Velcro are tasks adapted from ACT [4] . We also introduce five new tasks: one simulation task and four real-world tasks. The simulation task is Slot Insertion, where both arms need to grab each side of a long peg together and place it in a slot on the table. The new real-world tasks include Insert Plug, Click Pen, Sweep, and Unscrew Cap. We collected 50 demonstrations for each task. For the simulation tasks, the data used to train the model were all from human demonstrations and we did not evaluate performance of the models trained on scripted data. Detailed task definitions are provided in Appendix A, and the hyperparameters of both ACT and InterACT are provided in Appendix C.To Evaluate the performance of InterACT, we chose to compare our model against ACT [4] and the Diffusion policy [40] . However, with only 50 demonstrations, the Diffusion policy failed to execute any of the nine tasks. We did not perform any additional tuning on the Diffusion policy, as its poor performance suggested it was not well-suited for tasks with limited demonstration data. Therefore, we focus our comparisons exclusively on ACT, as ACT already significantly outperforms several other policies, including BC-ConvMLP [41, 14] , BeT [42] , and VINN [43] , by a large margin in bimanual manipulation tasks [4] . Table 1 : Success rate (%) for tasks adapted from ACT [4] (top) and our original tasks (bottom).For simulation tasks, we averaged the results across 3 random seeds over 50 episodes each. The real-world tasks were also evaluated over 50 episodes.The results of our experiments are summarized in Tables 1. Our InterACT model shows superior performance compared to ACT on all simulated and real-world tasks. In the simulated tasks, In-terACT outperformed ACT significantly, particularly in the Transfer Cube and Peg Insertion tasks where coordination and precision are crucial. The success rates for the "Transfer" stages in the Transfer Cube task, as well as the "Insert" stages in the Peg Insertion task, were notably higher with InterACT, demonstrating the effectiveness of our method in tasks that require coordination between the two arms. Moreover, in newly introduced tasks such as Slot Insertion and Insert Plug, InterACT also demonstrated higher performance. The Slot Insertion task, which requires precise coordination between two arms to carry the peg and adjust for alignment, showed a 100% success rate with In-terACT, compared to 88% with ACT. Similarly, in the Insert Plug task, InterACT achieved better results in the coordination subtask "Insert". This highlights the robustness of our model in handling tasks that require precise coordination between the two arms.Overall, the experimental results validate the effectiveness of our hierarchical attention framework. By improving coordination and precision in bimanual manipulation tasks, our InterACT model provides a more robust solution for complex bimanual manipulation challenges in both simulation and real-world scenarios.In this section, we perform ablation studies to evaluate the contributions of different components of the InterACT framework. Specifically, we focus on the impact of CLS tokens, the cross-segment encoder, and the synchronization block in the decoder.Impact of CLS Tokens: To assess the impact of CLS tokens, we conducted experiments with and without CLS tokens as input to the decoder. The results, summarized in Table 2 , showed no significant difference in the easier Transfer Cube task. However, there were notable improvements in the success rates of the more complex Peg Insertion task when CLS tokens were included. The aggregated information in the CLS tokens enhances the model's ability to generate accurate and coordinated actions, particularly in tasks requiring higher precision and synchronization. To gain deeper insights into the model's behavior, we studied how attention weights to CLS tokens at the decoder change over the timesteps. The results illustrated in Figure 2 showed that during the phase of interaction between the two arms, significant spikes in attention weights were observed. These spikes occurred at key moments where coordinated actions between the arms were necessary. This indicates that the model heavily relies on the CLS tokens to process information when coordinating actions between the arms. This observation highlights the CLS tokens' importance in facilitating precise bimanual manipulation.Impact of Cross-segment Encoder: The cross-segment encoder captures inter-segment dependencies, allowing the model to effectively integrate information from different joints across the two arms as well as the camera frames. The results indicate that removing the cross-segment encoder significantly decreases performance in complex tasks. For example, in the Slot Insertion task, the success rate for the coordination subtask "Insert" dropped to 24% from 44% without the cross-segment encoder. This highlights the importance of capturing inter-segment dependencies for generating accurate and coordinated actions.Impact of Synchronization Block: The synchronization block enhances coordination between the two arms by sharing contextual information at the decoder.. This is crucial for synchronized and efficient bimanual manipulation. The results show that the removal of the synchronization block leads to a significant drop in performance across all tasks, particularly in the Transfer Cube and Peg Insertion tasks. This demonstrates the necessity of the synchronization block for achieving coordinated and actions.Our ablation studies clearly illustrate that all components of the proposed InterACT framework-CLS tokens, cross-segment encoder, and synchronization block-play a critical role in achieving high success rates in coordination tasks. The best performance for the coordination subtask is achieved when all components are utilized, underscoring the importance of the holistic integration of these elements in the InterACT framework. 
conclusion and future work: 
 In this work, we introduced InterACT, a framework for robust bimanual manipulation, which integrates hierarchical attention transformers to capture inter-dependencies between dual-arm joint states and visual inputs. The key contributions of our work include the development of a Hierarchical Attention Encoder and a Multi-arm Decoder. The Hierarchical Attention Encoder aggregates intra-segment information using a segment-wise encoder and integrates inter-segment dependencies through a cross-segment encoder. The Multi-arm Decoder, while generating action sequence for each arm in parallel, ensures coordinated action sequence generation through synchronization blocks. Our experiment results on both simulated and real-world tasks demonstrate the superior performance of InterACT compared to the baseline ACT. The ablation studies further highlight the importance of these components in our framework. The use of CLS tokens, cross-segment encoder and the synchronization block significantly enhances the model's ability to generate accurate and coordinated actions, leading to higher success rates in bimanual manipulation tasks.One of the limitations of InterACT is that the number of CLS tokens and the number of encoder blocks are influenced by the level of coordination required for a given task. While we set a fixed number of CLS tokens across all tasks, finding the optimal hyperparameter requires heavy tuning, as there is no clear rule to guide this selection. Additionally, the level of coordination between the two arms is not straightforward to measure. Coordination can depend on task-specific factors such as temporal synchronization, dependency between arm movements, and precision, which are not easily quantifiable. As a result, determining the optimal hyperparameters becomes even more challenging, requiring manual adjustments and multiple experiments to ensure optimal performance. One potential approach to mitigate this challenge could involve adding an auxiliary loss term that explicitly encourages coordinated actions between the two arms, which might help improve coordination without relying on extensive tuning. However, this may require quantitative definition of coordination and further experimentation. As it stands, the scalability of our approach to more complex tasks or scenarios with diverse coordination requirements remains a challenge.While InterACT has shown promising directions in integrating robotic arm joints and visual inputs, it has not yet explored integration with other modalities. Future work will explore integrating additional modalities, such as language or sensory feedback, to further improve the robustness of bimanual manipulation tasks. Additionally, addressing the hierarchical nature of tasks will be crucial for better task decomposition and execution. These enhancements could leverage the flexible attention mechanisms demonstrated in this work to manage the added complexity and data integration. 
 
 Appendix: https://soltanilara.github.io/interact/ 8th Conference on Robot Learning (CoRL2024), Munich, Germany. 


Decision: Strengths:
+ InterACT is a new approach to multi-arm coordination in bimanual robot-learning.
+ The approach is demonstrated on real-robot tasks.
+ The ablations confirm the contributions of the proposed changes to ACT.

Weaknesses:
- InterACT is only benchmarked against one baseline: ACT. Other visuomotor baselines like Diffusion Policy could be included. 
- Some details on the evaluation setting are missing. 
- The performance gains over ACT are marginal, given the added complexity of InterACT.

Post rebuttal:  
While there were some concerns regarding the novelty of the approach, InterACT's approach is non-trivial and an interesting contribution to bimanual manipulation.

Reviews: review: title: title: The research is well-executed but needs clarification on marginal improvement over ACT
paper_summary: paper_summary: This work extends the ACT framework by introducing a hierarchical attention mechanism for bimanual manipulation tasks, named InterACT. InterACT includes a Hierarchical Attention Encoder and a Multi-arm Decoder. The multi-arm decoder consists of a synchronization block to facilitate the coordination of both arms.  The method is evaluated both in simulation as well in real-world. Ablation studies confirm the need for component, such as the synchronization block.
main_review: main_review: The research is well-executed, my major concern is the marginal improvement compared to ACT if trained with scripted data (See question 1). It would be good if the authors can clarify this point if the numbers have been copied from the original ACT work and how the new method was trained.

Other than that, writing can be improved by adding numbers and supporting some claims made in this work.
While the related works section is comprehensive, it could be further improved by including a discussion of other bimanual manipulation methods beyond ACT. Many existing methods, although often tailored to specific tasks, could provide a broader context for the contributions of this work.

The figures in the paper could be revised for clarity, and additional figures could be added to provide more visual insight into the results and methodology.


### Minor:

- Line 36, 210: Seems to be generated by GPT. Some additional information Is missing. E.g. citations or numbers, e.g. the relative improvement compare to ACT? 
- Line 252: synchronizatin
questions: questions: 1.)	ACT reports different numbers if training with scripted data. It seems  that this work has copied the numbers when training with human data from ACT. Are the experiments for InterACT scripted or trained with human data?
2.)  The work claims to “significantly outperforms existing methods” (L11). Which one besides ACT?
3.) Can you elaborate on the design criteria of the new tasks? In what way do help to measure coordination?
4.)	How often did you run the experiments?
limitations: limitations: 2
review_summary: review_summary: Good work, some parts require clarification, e.g. if it has been trained with scripted data or with human demonstrations
title: title: The proposed method has shown noticeable performance improvements over the baseline; however, the novelty of this method may be limited.
paper_summary: paper_summary: This paper presents a new imitation learning method for bimanual manipulation. Built on top of the Action Chunking with Transformers (ACT) framework, the method introduces a hierarchical attention encoder and a multi-arm decoder. Specifically, the encoder consists of segment-wise encoders and a cross-segment encoder to better capture the interdependencies between dual-arm joint states and visual inputs. The decoder includes arm-specific decoders and a synchronization block. The method is demonstrated on both simulated and real-world bimanual manipulation tasks.
main_review: main_review: Strengths:
* The method is demonstrated on real-world, fine-grained bimanual manipulation tasks.
* The figures and algorithms are clear and helpful in understanding the method and results.


Weaknesses:
* Although the proposed method has shown noticeable performance improvements over the baseline, the novelty of this method is somewhat lacking since the idea of a hierarchical attention mechanism is not new.
* The proposed method is only being compared to one baseline (ACT), which may be limited. A baseline that could be included in the comparison is a bimanual Diffusion Policy. 
* Many important details about the evaluation protocol are missing. Please refer to the “Questions for Rebuttal” section.


Minor weaknesses:
* The related works section is missing some recent papers on bimanual manipulation: [1, 2, 3, 4].
* Typo: “decoder” on line 128.


[1] J. Grannen, Y. Wu, B. Vu, and D. Sadigh. Stabilize to act: Learning to coordinate for bimanual manipulation. In Conference on Robot Learning (CoRL), 2023.

[2] L. X. Shi, A. Sharma, T. Z. Zhao, and C. Finn. Waypoint-based imitation learning for robotic manipulation. In Conference on Robot Learning (CoRL), 2023. 

[3] A. Bahety, P. Mandikal, B. Abbatematteo, and R. Martín-Martín. ScrewMimic: Bimanual Imitation from Human Videos with Screw Space Projection. In Robotics: Science and Systems (RSS), 2024. 

[4] J. Varley, S. Singh, D. Jain, K. Choromanski, A. Zeng, S. B. R. Chowdhury, A. Dubey, and V. Sindhwani. Embodied AI with Two Arms: Zero-shot Learning, Safety and Modularity. arXiv preprint arXiv:2404.03570, 2024.
questions: questions: My main concerns are discussed in the weaknesses section. Additionally, several important details about the evaluation protocol are missing:
* How many training seeds are used in the experiments?
* How many demonstrations are used to train the models?
* How many episodes are being evaluated? 
* What are the environment variations?
limitations: limitations: 1
review_summary: review_summary: I recommend a weak reject based on the issues discussed above. My primary concern lies in the novelty of the proposed method, as the idea of a hierarchical attention mechanism has been introduced previously. Additionally, a more comprehensive method comparison and the details of the evaluation protocol need to be included.
title: title: good paper
paper_summary: paper_summary: The paper "InterACT" introduces a novel framework for bimanual manipulation using Hierarchical Attention Transformers (HAT). It addresses the challenge of coordinating actions between both arms by integrating segment-wise and cross-segment attention mechanisms. The framework includes a Hierarchical Attention Encoder for multi-modal input processing and a Multi-arm Decoder for synchronized action prediction. Key contributions include significant performance improvements over existing methods in various tasks, validated through extensive experiments and ablation studies that highlight the effectiveness of its components like CLS tokens and synchronization blocks.
main_review: main_review: #### Strengths
- **Hierarchical Attention Mechanisms**: Effective in capturing inter-dependencies.
- **Multi-arm Coordination**: Addressing challenges specific to bimanual tasks.
- **Experimental Validation**: Robust testing across simulated and real-world scenarios.
- **Ablation Studies**: Detailed analysis of framework components.
- **Potential Impact**: Application in practical robotics domains.

#### Weaknesses
- **Complexity**: The effectiveness of hierarchical attention and synchronization blocks may introduce computational overhead, which could limit real-time application on less powerful hardware.
- **Limited Comparison**: The authors only compare with ACT. Although the author mentioned "We focus our comparisons exclusively on ACT as ACT already outperforms BC-ConvMLP [30, 31], BeT [32], and VINN [33] by a large margin in bimanual manipulation tasks [4]." It will be better to compare more potential baselines. For example, the reviewer found that "Hierarchical Attention Transformers" is also studied for general RL [1,2,3] and used in other fileds [4,5], which could be possible baselines.

[1] Transformer in Transformer as Backbone for Deep Reinforcement Learning. Arxiv 2022.  
[2] PDiT: Interleaving Perception and Decision-making Transformers for Deep Rinforcement Learning. AAMAS 2024.   
[3] Sequential Asynchronous Action Coordination in Multi-Agent Systems: A Stackelberg Decision Transformer Approach. ICML 2024.   
[4] CoSLight: Co-optimizing Collaborator Selection and Decision-making to Enhance Traffic Signal Control. KDD 2024.   
[5] X-Light: Cross-City Traffic Signal Control Using Transformer on Transformer as Meta Multi-Agent Reinforcement Learner. IJCAI 2024.
questions: questions: 1. The Hierarchical Attention Transformers may introduce computational overhead, which could limit real-time application on real robot?  

2. Could the authors provide more comparision with other baselines?
limitations: limitations: 1
review_summary: review_summary: Overall, InterACT appears to be a significant contribution to the field of robotic manipulation, leveraging advanced attention mechanisms to improve coordination and performance in bimanual tasks. Further peer review and empirical validation will be crucial for confirming its robustness and practical utility across various applications.