Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a method for exploration of 3D articulated environments that alternates between collecting interaction data with RL while maximizing a combination of extrinsic and intrinsic rewards, and training visually conditioned action maps, image conditioned manipulation trajectory priors, and success predictors, that further guide the intrinsic reward prediction during data collection.
main_review: I have two main concerns regarding the paper: 

1)The link between the visual perception and the RL policy appears weak as the only feedback is through exploration rewards for the RL policy to try out interactions on places where the visual perception models assigns low success probability. But is this a good exploration bonus? What if indeed these are simply not good places to act and succeed? Shouldn’t the certainty of the visual model be taken into account as opposed to low probability?

2)The naïve RL baseline seems to be training one RL policy across multiple tasks (if I understood correctly) that operates on a point cloud input. This baseline is designed to fail. It can be much improved by training a separate RL policy in each environment separately, and simply disabling the visual perception representations? Then, at the end, we can train actors that operate directly from images, similar to an ``asymmetric actor critic" setup.




summary_of_the_review: Could the authors show results of the baseline suggested above?
Could the authors explain the rational of the exploration reward?

It is possible that I am missing something from my understanding of the paper. I will be careful during discussion period to clarify any misunderstandings.



Post rebuttal : the authors have put together the requested baseline and they show significant performance margins over it. Thank you very much for this, I raise my score accordingly.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper is solving the problem of pushing and pulling objects (mostly things like cabinets) by learning visual action trajectories proposals via a curiosity-driven RL / Perception joint training.  The system input point clouds the object and outputs the actionable score and the per-trajectory success likelihood score on the most likely approach to interacting with the object.  This approach is validated both in simulations but also in simulation and with real results.  
main_review: Overall Comments: 
The main novelty of this paper is a slight twist on the Where2Act paper that, instead of generating grip orientations, this paper is generating trajectories.  This difference necessitates the difference in the models.  However, they do show a comparison against the Where2Act approach that shows that their modifications to the network (including the curiosity exploration and how they find the trajectories) are superior, for this task.  

This paper is easy to understand and well written.  The ideas are easy to follow and build well.  

The appendix was very useful for clarifying some parts of the paper in more detail which was very helpful such as for the curiosity-driven explorations and where the heuristic-based method failed.  

Smaller comments: 

The term step seems to refer to waypoint step but also can be confused for timestep.  Can you clarify when you use it? 

Cite prior cVAE work this is based on in the Trajectory Proposal Module.  
summary_of_the_review: This paper has some interesting elements to it and their approach is validated by both real and simulated results.  While the task isn’t very novel, they at least validated their approach to show that it does better than previous approaches thus contributing to the field.  They also do a good job of explaining each of the steps in the appendix to make it easy to understand what exactly they are doing.  For these reasons, it merits inclusion in the conference.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper extends work on static term action generation (Where2Act, ICCV21) for 3D articulated objects to 1) long-term action trajectory generation by learning from data generated data via RL exploration,  2) action trajectory conditioned with task-awareness.
main_review: Pros：
The paper proposes the problem of long term action trajectory generation for 3D articulated objects which is not well studied. 
The paper is well written.

Cons:
The method itself is not very novel, more about extending the existing Where2Act and a combination of Where2Act and curiosity guidance for RL Policy for Interactive Trajectory Exploration.
Baseline for the Curiosity Guidance for RL Policy for Interactive Trajectory Exploration, which is one of the main components of the method, is not compared in the trajectory generation.

summary_of_the_review: The paper proposes to solve a new problem of long term action trajectory generation for 3D articulated objects, but the method to solve the problem is more about an extension and combination of existing work. The overall quality of the paper, writing and experiment, is good.
review_title: null

Meta-review: The paper claims to present actionable visual representations for manipulating 3D articulated objects. Specifically, the approach learns to estimate the spatial affordance map as well as the trajectories and their scores. After checking the rebuttal from the authors, all reviewers agree that the paper adds value to the research area. In the end, it got three borderline accept ratings. The initial criticism included lacking (experimental) comparison to baselines, and the authors successfully corresponded to the request from the reviewer. One reviewer commented that the proposed approach is a combination of Where2Act and curiosity guidance for RL Policy for Interactive Trajectory Exploration, which we believe is a valid point. Still, the paper extends the previous Where2Act and successfully demonstrates its success on difficult tasks.

We recommend accepting the paper.
Decision: Accept (Poster)