Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposed a model architecture for behavior cloning. The key element of the architecture is to feed region proposals from an RPN into a transformer. Results in simulation and real suggest this model works well.
strengths_and_weaknesses: # Strengths

- Cool real-world videos.
- Nice looking architecture figure.
- Results in sim and real suggest the model works pretty well.
- Figure 5 attention rankings are interesting.

# Weaknesses

- An inexperienced researcher would read this introduction and think that this paper has come up with the idea of using object centric representations in visuomotor policy learning. But of course this is not the case. See for example Transporter Networks, CoRL 2020, which isn’t an object centric method but reviewed different prior works in this area. 
- This also goes for the title, “VIOLA: Object-Centric Imitation Learning for Vision-Based Robot Manipulation” which seems to suggest the authors have invented this idea. A more appropriate title might be “Region Proposal Transformers for Behavior Cloning” or something like that.
- The paper refers to its method as an algorithm but I think it should probably be referred to as a model architecture. The algorithm it’s evaluated with is behavior cloning.
- Why does section 3.1 talk about MDPs? There is no concept of rewards, or transition functions, or discount factors that is used in this paper. This paper just studies behavior cloning. The MDP paragraph copy paste is irrelevant to this paper. “Our goal is to maximize the expected return…” is not true, instead the method’s goal is to fit human demos.
- The details of the architecture are probably not sufficiently described in order to implement it from reading the paper.
- Why is [22] cited for doing behavior cloning with an RNN? This has been done well before [22], for example:
  - R. Rahmatizadeh, P. Abolghasemi, L. Bo ̈lo ̈ni, and S. Levine, “Vision-based multi-task manipulation for inexpensive robots using end-to-end learning from demonstration,” ICRA 2018
  - P. Florence, L. Manuelli, R. Tedrake, “Self-supervised Correspondence in Visuomotor Policy Learning,” RA-L 2019
- IL methods suffering from covariate shift has been studied at least since DAgger, Ross et al, 2011. Not a thing only from 2019 onwards, as it is cited here.
- Even specifically RNN for BC with a GMM (also called MDN) was done in Rahmatizadeh et al noted above.
- Overall I believe the academic question of the text should be more specifically “do region proposals help do more effective behavior cloning?” With this more precise framing of the text, I’m not sure the experimentation is very thorough for this question. The only comparison that gives a hint here is the Patch rather than RPN question. 
- Although the real world demos are in some ways impressive looking, they don’t seem to require much closed loop feedback.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I am conflicted on this paper. On the one hand the robot demos are pretty cool, and it seems to work pretty well. Meanwhile, I listed several key weaknesses with the presentation and novelty of the paper in the Weaknesses section. I’m pretty borderline on this paper, I would currently lean reject but could consider moving up to accept if the authors address the weaknesses I listed.
issues: See Weaknesses discussion.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper presents VIOLA, an object-centric imitation learning method for manipulation. The idea is that object- and task-specific knowledge should be attended to for a manipulation policy. To achieve this, VIOLA uses a pre-trained object detector to find regions of interest, which are fed into a transformer-based policy. The paper introduces three simulated tasks (sorting, stacking, kitchen) and three real-world tasks (plate+fork setting, bowl retrieval+setting, and making coffee). VIOLA is evaluated on these tasks along with 3 baseline imitation learning methods (BC, OREO, BC-RNN).
strengths_and_weaknesses: Strengths
* Object-centric information seems useful and a transformer gives a path for how to use such information.
* The evaluation strategy is thorough. I appreciate the out-of-distribution tests (placement, distractor, camera jitter) and the A/B testing paradigm to mitigate human bias (L283).
* The make coffee task seems challenging and linked to a realistic application.
* VIOLA-Patch (a variant of VIOLA without object-specific patches) is a great baseline to differentiate performance due to transformers.

Weaknesses
* The method is somewhat complex, which makes it challenging to determine the key aspect of VIOLA's success. Data augmentations are not all ablated or used consistently across baselines (unclear). I'm glad the random erasing was included in the ablation, but I'm not sure how much the other augmentations (color, pixel shifting, etc.) contribute to VIOLA's success. It would be nice if all baselines used the same augmentations, or if not, include motivation for why the augmentations are specific to the method. The eye-in-hand image should also be used consistently.
* Does K (the number of object proposals) need tuning for each environment? It would be more compelling to use the same K in simulation and the real world (just stick with either 15 or 20). L205-206 says "we set K=15 which are found to attain a high recall rate on real images" - but I'm not sure how this was selected. Can K be easily inferred from the demonstrations or was any online tuning done here?
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: The motivation for this work is strong, and I like the experimental setup. I have some concerns regarding the complexity of the method (see weaknesses above), but if these are addressed then I believe the work can be a strong contribution to CoRL.
issues: I'm surprised at how small the object proposal patches are (visualized in figure 5). For example, I would consider the coffee maker as a whole to be a single object, but one patch cannot capture this. Would the method struggle to be scale-invariant (e.g. if the size of objects or distractors becomes much larger or smaller in the image)?

It appears as though Dining-PlateFork and Dining-Bowl were initially going to be a single sequential task. Was this the case? Was there a failure here due to its long-horizon nature?

L117-118: "Our goal is to learn the policy... that maximizes the expected return." This phrasing makes it sound like reinforcement learning methods will be evaluated, but to my knowledge, the reward is not being maximized anywhere (and there is no reward function defined for any task). Furthermore, I can't find a description of how successes are determined (in computing the success rate).

Some of the descriptions could be clearer (no need to respond to these):
* I don't understand what the global context feature is doing (L161).
* Figure 2 is overly complex, perhaps partially due to the complexity of the method and partially due to too many details being included in the figure.
* Is "sinusoidal position encoding of the temporal positions" (L174) common? I don't entirely follow the purpose of this here. Do we expect a sinusoidal nature in these manipulation tasks?
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper presents an object-centric imitation learning approach to learning closed-loop visuomotor policies. The policy relies on object region proposal and features and a transformer based policy, as network structure, to improve generalization across view points, initializations and distractors. The policy outperform naive baselines and several ablated methods in both simulation and the real world. 
strengths_and_weaknesses: Strength: 

The paper introduces an interesting object-centric approach for closed-loop visuomotor policy learning, and demonstrates robustness across viewpoints, initializations, and distractors under systematic simulation and real world evaluations. 

Compared to the naive end-to-end approaches, VIOLA is more structured and makes use of transformers to attend to task relevant parts in the spatial-temporal domains.

The implementation details for making the transformer policy to work with temporal-spatial data, if released, can benefit the community in general.

Weakness:

One of the main contributions seem to be using a pretrained FPN for proposing object bounding boxes, which is claimed to be more generalizable. Are there comparisons of this intermediate representation to other vision pipelines such as mask predictions or object detections? I can imagine the recall rates for these methods for finding interesting objects would be lower.

The applicability of pretrained object detectors can be limited in some cases such as deformable or granular object manipulations. The region proposals seem to make the network focus less on the irrelevant parts, but the generality and robustness of the approach still need more experiments to verify. A depth camera might also be able to help filter out backgrounds in some cases. 

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This paper presents a well-thought and clearly written approach to imitation learning through object-centric representation and transformer backbones. The method outperforms baseline methods in simulation and in the real world. There are still several weaknesses regarding the general applicability and performance of the methods. 
issues: The architecture design seems to contribute a lot to the success rates in the network ablation study. I wonder if the performance gap to BC-RNN is still big if we vary the number of demonstrations or increase the BC network size. 

Similarly, I wonder if the context information from spatial feature map, eye-in-hand image, and proprioception also provided to the simple BC baselines. I.e. how much the region proposals help with the transformer policy, v.s. How much the transformer policy outperforms vanilla networks. 

The fact that random erasing helps a lot in table 2 is unclear to me. Is it because that the policy is overfitting to the K patches and some random masking would help with generalization?
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The paper presents a transformer-based object-centric Visual Imitation Learning method, topping relevant state-of-the-art methods. The method is shown in several simulated tasks, introduced in the paper, including a multi-stage environment, and in the real world. Ablations show that the method is able to effectively deal with perturbations, jitter and object placement variations.
strengths_and_weaknesses: Strengths:
- This work is very good and shows a significant improvement over previous works. 
- The design of the method is simple and elegant, with ablations justifying most design choices
- Clear and well presented

Weaknesses:
- The works lacks experimentation with more practical concerns such as generalization across similar objects, severe occlusion, and even more significant distractors than the ones shown.
- A concern is that the distractors shown in the supplementary files are minimal.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I have recommended strong accept, because I am willing to argue for this paper’s acceptance, and believe it is a positive contribution dealing with an important problem. 

---

Update: Most of the concerns raised were answered by the authors. My evaluation remains unchanged as a Strong Acceptance.
issues: Technical issues and questions:
- Hand-eye camera images are used as input to deal with local occlusion, but this choice is not discussed in detail. Would it be self-sufficient to have only the eye-in-hand camera? How much does the agent benefit from it being there? 
- Random erasing, which seems to be key to using regional features, could use more explanation within the background/method of the paper. 
- As per L151-L154, the model would be relying on a trainable RPN instead of a frozen one. Is the ResNet also not pre-trained? Wouldn’t this reduce the ability of the agent to generalize across similar objects (e.g., trained on one screwdriver, and then manipulating another). Moreover, it would be interesting to compare the frozen networks versus the trainable one used in the paper’s ablations (rather than in supplementary materials).
- Choice of ResNet18 over other backbones itself is unclear – modern transformer models might provide better features, it would have been interesting to see. 

Minor issues:
- L20: Typo: “its success of it”
- Related Works, the cited Sieb et al. [24] has relevance in more than just the bounding box representation it uses since its very closely related to the proposed method, in that it is also vision-based object-centric imitation learning and the given summary of it doesn’t do it justice. 
- L187: MHSA is redefined
review_title: null

Meta-review: metareview: Below is a summary of the strengths and weaknesses of the paper, according to the reviewers.

Strengths:
- The use of transformers is interesting for attending to task-relevant parts of the image.
- Results demonstrate robustness across viewpoints, initialisations, and distractors (although there is concern that the distractors are not that significant).
- The ablation studies are very useful.
- The real-world demos are interesting and impressive.

Weaknesses:
- The paper appears to makes overly grand claims about introducing object-centric representations for imitation learning.
- The baselines do not use the same data augmentations as the proposed method, making it an unfair comparison.
- The use of pre-trained object detectors could make the method impractical for granular or deformable objects.

In the rebuttal, please address the above weaknesses, as well as the other concerns and questions raised by the reviewers.

----------

Update after rebuttal:

Following the rebuttal, opinions were still divided on whether the paper should be accepted. There is a consensus that the overall idea is sensible and performs well, with strong real-world demos. But there is still concern about what we can really learn from this paper, other than the strength of transformers and region proposals. However, some of the questionable claims made in the original paper have now been rephrased in an updated paper. After a private discussion, the opinion amongst the reviewers is generally more positive than negative, and I believe that the simplicity of the method, together with the compelling real-world demo, would be appealing to the community.
confidence: 3: The area chair is somewhat confident
Decision: Accept (Poster)