Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes to use a Transformer to perform multi-task manipulation via language-conditioned behavior-cloning. The key challenge is Transformer is known to be data inefficient. By casting behavior-cloning as an action detection task, PREACT achieves great efficiency on a range of simulated and real-world tasks.
strengths_and_weaknesses: Strength:
* The paper is well-organized and easy to follow.
* The proposed method is succinct and easy to understand.
* The tasks are challenging, and the proposed method achieves good performance while maintaining good data efficiency.
* The real robot experiments make the paper promising.

Suggestions and Improvements：
* As the author mentioned, both Transformers and 3D Convs are data inefficient, but I did not see why detecting voxel actions is extremely data efficient (i.e. achieving ~$60\%$ trained with only 53 demos across 7 tasks.) Could you provide more analysis?
* It is hard to evaluate the generalizability of PREACT. As mentioned in L213, all objects are seen during training. Could you please provide more detailed differences between training and testing? (e.g. the size differences as shown on the website; training: stack yellow block on blue block, testing: stack red block on yellow block; training: turn left tap, testing: turn right tap, etc)
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: I'd recommend "weak accept". The paper studies difficult problems and achieves good performance. It would be great if the author can provide more analysis about why action detection (classification) makes such a huge difference from regression (BC). 
issues: Strength:
* L45, please consider defining action-centric representation when it first shows up.
* L125, please consider providing an intuitive understanding of keyframes, e.g. "the pose of the end-effector at the point where object interaction should begin"
* L174, one task may require several keyframes to finish, are the language goals remain the same?
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The paper introduces a language-conditioned behavior-cloning agent for multi-task 6-DoF visual manipulation. In Leyman terms, the paper is a variation of the Coarse-to-fine ARM paper, where the key differences are:  

- (1) The addition of the Perceiver Transformer [1], rather an a 3D convolution.
- (2) Using behavior cloning rather than reinforcement learning. 
- (3) Language conditioning to enable multi task/variation learning.  The paper evaluated in both simulation (on 18 RLBench task) and in 7 real world tasks.
strengths_and_weaknesses: **Strengths**

- An efficient BC algorithm that performs well on 53 demonstrations.  
- A large number of simulation experiments for reproducibility, and also real world experiments to show that the method is suitable for real-word training.   
- The authors did well to state clearly any modifications to the tasks in RLBench , ensuring reproducibility.
- A detailed description of the limitations (in appendix).  
- The addition of the perceiver seems to clearly help learning these complex tasks.   
- Fig 2 gives a good overview of the system.   
- Other than the introduction, which could be a little clearer, the rest of the paper is well written.   

**Weaknesses**

- The introduction could be improved to better frame the proposed work. There seems to be quite a lot of similarities with the C2F-ARM work through the paper (next-best pose action, 3D voxelization, motion planning, demo aug, keyframe discovery) but it would be good to perhaps front-load a summary of these similarities and differences, rather than having the reader have to go through the entire paper. Try to make it instantly clear what separates this work.  

- There isn’t anything in the paper that is BC specific that has been introduced in this paper, and so it would have been nice to include results of the effect of adding the transformer for the RL agent, rather than just the BC one. 

- Ablation on patch size (other than the 5^3 used) would be very useful. 


**Other minor comments**

- ln 69: “… has a limited receptive field that cannot look at the entire scene at the finest level”. This is not exactly true; worth noting that this can be eleviated with [b].  

- ln 71:  “PERACT does BC instead of RL, which enables us to easily train a multi-task agent for several tasks”, but [a] has trained a multi-task policy with C2F-ARM. I’d suggest amending this comment and elaborate on how these two works differ. For one, the PERACT can do both multi task and multi-variation because it conditions on language, while [a] only does multi task because it does not use task conditioning.

- [c] is relevant work on using a next-best action space for multi-task BC on RLBench tasks. 

- Fig 2: Could benefit from having the RGB views to make it clear what is in the voxel grid scene. 

- ln 136: [71] should be cited when discussing collide prediction.  

- I wonder if PERACT training/memory overhead could be reduced by applying a coarse-to-fine structure as in C2F-ARM? Fig 4 evaluated multiple hierarchies for C2F-ARM, and it would be nice to see something similar also for PERACT. If we can get similar performance from having this hierarchy, but reduce the memory required, then that would be great.

[a] Mandi et al.  "On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learning."

[b] James at al. "Coarse-to-fine Q-attention with Tree Expansion."

[c] Liu et al. "Auto-Lambda: Disentangling Dynamic Task Relationships."
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I believe the work would be valuable to the community. I would consider increasing my recommendation if the majority of the issues are addressed in the rebuttal phase. 
issues: See weaknesses
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes a language-guided robot manipulation framework built on top of Perceiver IO. The proposed framework – Perceiver-Actor or PerAct in short, performs multi-stage manipulation tasks in a multi-task manner, showing superior performance in both simulated environments (RLBench) and in real-world settings trained with few demonstrations.   
strengths_and_weaknesses: Strength: This paper shows a promising direction to scale up vision-based manipulation tasks with transformers. It combines modern large-scale pre-training frameworks, using CLIP to encode language instructions, and using Perceiver IO for learning discretized actions. Using Perceiver IO is significantly helpful to reduce memory required for encoding high-resolution colored voxels inputs, (attended with a much smaller latent embedding) compared to previous methods like C2FARM alternatively require to multi-stage (coarse-to-fine) to encode fine-level voxels with limited memory. 

Weakness: Though the proposed framework is cleaner in design, it poses some other limitations.

-- The training setting for keyframe discovery and learning Q-attention values is pretty much following ARM and C2F ARM. The authors did mention in keyframe discovery but missed the related references in action parametrization (L162 - 170).

-- The training time requirement is significantly larger than the related works. In L185-186, the authors stated that the training needs 8 V100 for 16 days (ImageNet experiments with the same amount of compute takes only 2 days, but with 1M training data, here the number of training data would be definitely less than 100k), whilst C2F ARM only requires less than 1 hour for each task in a single GPU. Considering each task is only augmented with 10 demonstrations, I hope the authors could slightly explain why training takes such long time? Is it simply because transformers in general take longer time to train?

-- Design details in baseline methods. Are two mentioned baseline methods: Image-BC / C2FARM-BC single-task or multi-task learning methods? L223-224 indicates all methods are multi-task agents, how the sense of “task” then is fed to these baselines? Or the authors just treat everything is an image-to-action type of learning tasks without introducing the idea of task in training? This is slightly unfair, since the sense of task is encoded in language in the proposed framework.

-- Compared to other multi-task methods. Multi-task robot learning has also been heavily investigated in the multi-task learning community. For example, notable works like: PCGrad [1], Auto-Lambda [2] also evaluated multi-task robot learning which might be more suitable baselines compared to Image-BC / C2F ARM-BC which were designed as single-task learning methods. In particular, Auto-Lambda also performs multi-task learning on RLBench (though not language guided), but with the performance of same evaluated tasks like: slide block: 77 (72), money in safe: 64 (44), place wine: 19 (12), push buttons: 95 (48), trained with the same number of 100 demonstrations, achieved better performance than the proposed method. I would suggest to do a thorough literature review on multi-task learning on manipulation and should compare with at least 1 or 2 related baselines designed specifically for multi-task manipulation.

[1] Yu et al.  Gradient Surgery for Multi-Task Learning. NeurIPS 2020

[2] Liu et al. Auto-Lambda: Disentangling Dynamic Task Relationships.  TMLR 2022.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are not well addressed
summary_of_recommendation: In general, I quite like the idea as it shows a promising direction to bring transformers to the manipulation community. I would further raise the score if the authors could resolve my concerns stated in the weaknesses section.
issues: See weaknesses.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper presents a transformer-based language-conditioned multi-task imitation learning method for 6-DoF manipulation. Unlike prior language-conditioned multi-task BC methods operated on 2D images, this paper encodes both the language instructions and the RGB-D voxel observations. The authors show that such a method improves over prior approaches in both simulated and real-world robotic manipulation tasks.
strengths_and_weaknesses: **Strengths:**

1. This paper presents a new language-conditioned multi-task BC method operated on RGB-D voxels. The authors utilize the Perceiver to handle the high-dimensional observation space, which seems novel.

2. The empirical results appear rather convincing. The method is able to outperform prior methods such as Image-BC (similar to BC-Z) and C2FARM-BC in the multi-task simulated robotic manipulation tasks with limited demonstrations. The authors also show that the method can work well on the real robot.

3. The paper is well-written and easy to understand.

**Weakness:**

1. The paper extends prior works such as SayCan and BC-Z to the setting with the extra RGB-D voxel in the observations but the main technique is not too different from prior works, which makes the contribution a bit less novel. 

2. The paper does not compare to SayCan in the experiments, which seems to be necessary. Moreover, there are no baseline results in the real-world setting.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: Based on the summary in the Strengths And Weaknesses section, I think while the paper lacks some additional comparisons and is a bit incremental, the good empirical results and novel usage of the Perceiver architecture make senses and I would give the paper a weak accept.
issues: 1. Compare to SayCan.

2. Add baseline results in the real-world experiments.
review_title: null

Meta-review: metareview: Strengths:
- The contributions are interesting and relevant to the CoRL community
- The experiments are generally convincing.
- The paper includes experiments on a real robot
- The paper is well-written
- There is a detailed discussion of the limitations in the appendix

Weaknesses:
- There are a lot of similarities with the ARM and C2F-ARM works, which is okay -- much of research is incremental on top of prior work. But, the paper could have done a better job at framing the contributions in the context of these prior works.
- There are some additions to the experiments that would make them more thorough and informative, e.g. including comparisons on the real robot, including RL experiments, including ablations on patch size, and adding more comparisons in simulation such as to multi-task methods or SayCan.

The author response and discussion has helped address the concerns. The paper makes a valuable contribution to the CoRL conference.
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)