Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper describes, on the one hand, a scenario for robot learning, planning, and acting inspired in a toy for cognitive development for children, and, on the other hand, a learning and planning framework tailored to this specific scenario to let a robot learn to execute a given task. The authors claim that one of the main contributions resides in the characteristics of the environment that allows a robot to learn cause-effect relations through interaction and to use these relations to select a sequence of action to bring the scenario from an initial configuration to a target (goal) one. They also propose a set of algorithms for learning and representing these cause-effects in a robotic application.


strengths_and_weaknesses: The tasks used for validation of learning and planning approaches are too simple and can be solved by traditional approaches. The integration of sense-plan-act mechanisms with learning through interaction is not novel.  The perception mechanisms using vision and deep learning may entail some novelty but this is not developed in the paper. A clear presentation of the theoretical background and (theoretical) improvements achieved with respect to the state of the art is still missing. 

There are no interdependencies between consecutive actions in a plan. The search problem is simple and straightforward. 

One potentially interesting contribution of this work might be to let the robot learn the primitives to manipulate objects that requires small displacement. Unfortunately, these methods are not properly described and contrasted with respect to the state of the art. The experimental evaluation does not allow to fully assess the validity of the method. The comparisons are carried out only with ablated versions of their system, but not against the state of the art.

The contributions are difficult to distil from the paper. The authors condense a large number of algorithms without providing enough details and justifications. The paper in its current state gives the impression that the proposed methods are rather ad-hoc, with arguable transferability to other problems and scenarios.

The supplementary video is well-edited and clear.


robotics_focus: Relevant but unlikely to deploy to hardware in near future
quality_of_the_limitations_section: Limitations are not well addressed
summary_of_recommendation: I suggest the authors to avoid presenting the BusyBoard scenario as a contribution and to elaborate on methods that potentially entail scientific contributions with respect to the state of the art.

The description of Sec. 4.1 is incomplete and confusing. It is not clear for me how the action policy is defined/learned. I encourage the authors to better describe this method: What actions are considered in the action space? What is the state/object space? How the interaction policy is initialized? The authors mention that the robot-object interaction points are learned from depth images. How this takes place? Is there any previous knowledge used to guide the learning? How the per-pixel affordance is learned/defined? I think these could entail appealing contributions but unfortunately it is not possible to assess their validity with the current state of the paper. 

The word "prediction" already entails future events. I suggest to avoid using the phrase "prediction of the future". I encourage the authors to explore the large literature in task and motion planning and learning of prediction models and use the same terminology.

The description of the limitation is insufficient. I recommend the authors to better discuss scalability, transferability, and robustness.
issues: Reduce the description of the scenario (e.g. Sec. 3) and use the gained space to elaborate on the scientific contributions.

The paper is missing a section where the terminology, problem definition, and basic tools are properly introduced. It will be helpful if the authors provide concrete examples of complete interact-learn-plan processes.

The reward is defined from differences in images. This will generate problems in real scenarios with changing light conditions and non-causal events taking place from external contingencies. How the authors are planning to address this problem? Would the proposed algorithms fail in these cases?

The explanation of the exploration strategy is unclear and insufficient.

Line 200. It is not clear is BusyBot is just the planning approach or the framework comprising all the learning algorithms (interaction policy, cause-effect relations, etc.).

Justify how the values of the parameters in the experimental set-up are defined (23 images, training test 32/9, etc. Why these arbitrary values?)

Lines 243 and 245. Typos.

Some references are incomplete (volume, issues, pages,…).
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: Self-contained busyboard environment with articulated objects and functional relations.
Self-supervised strategy: 1- learn policy to interact with environment, 2- collect data with policy and learn inter-object relations with causal discovery model 3- combine policy and causal model to learn goal-conditioned manipulation model.
motivation: busyboards amplify sensory feedback and have causal relations and so are ideal self-learning env for toddlers and robots.
8 sub-models: position network, direction network, 3 GNNs, 1 GRN, 2 agents, trained with thresholded image difference reward, MSE, and L2. Trained on 10k boards, eval’d on novel boards (2k) and objects (2k).
Results: interaction module: on par with “oracle”, benefits from responders vision and exploration. Relation reasoning: high accuracy even for novel configs and objects. goal-conditioned manipulation: good for 1-1, less for 1-many. real: high scores for 3 triggers and 3 lights.

strengths_and_weaknesses: strengths:
- A self-contained diverse environment for learning interactions, reasoning and planning is good for robotics research.
- publicly available code and simulation is nice (have not seen it available yet).
- The proposed solution seems to yield good results.

weaknesses:
- thresholded image difference as reward function: how would this be deployed in the real world?
- The learning method seems overly engineered and tuned for this specific problem, it seems unlikely that it would be useful for broad learning in the real world as is, a more unified model (as opposed to the 8 modules used here) or more end-to-end design. I understand that some engineering is ok for the sake of solving this problem, but it doesn’t seem that the proposed method is advancing research beyond this narrow setup.
- The real setup is pretty narrow, I wonder how much tuning went into the real setup given its small size and how easy it is to obtain good results with tuning.
- There is no mention or precise articulation / plan of how the techniques used here are transferable to the real world besides the analogy to how human infants learn from busyboards.
- Although busyboard is a more relevant env to the real world than Atari games, it is not necessarily that much more convincing in terms of relevance to general real world robotics. Maybe authors can improve that front.
- Future work discussion could benefit from a vision on how to expand the learnings of this work beyond the narrow setup of the busy board.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: The science seems sound. The motivation of self-supervision with simplified toy-like setups with multiple abilities being tested (interaction, reasoning, planning) is good. While a new public benchmark is good as well, the issue I see with the paper is the proposed learning solution not being general enough to be of value beyond this narrow setup.
issues: - minor: Lacking references for GNN papers?
- more articulate vision on how the proposed approach is applicable beyond this setup.
- code and sim available?
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: There are 2 main contributions of the paper.
* The first is the BusyBoard environment which allows generations of many tasks. The environment places articulated objects and creates inter-object functional relationships between the objects and responder objets.
* The second is a self-supervised learning framework called BusyBot that proposes to solve goal conditioned tasks in the above environment. The framework proposes a interaction policy using supervised learning, a reason module which collects interactions with the environment to create a scene graph using GNNs and a future dynamics model, and a planning agent which uses the above modules to perform goal conditioned tasks in the environment.

strengths_and_weaknesses: * The environment is interesting because it allows exercising integrated skills for embodied agents. Interaction, reasoning, and planning fused into one environment.
* The simplicity of the observations and action spaces (topdown depth and SE(3) pose) could be seen as a plus, to focus on the reasoning and planning parts of the problem.
* The ability of the environment to generate new task configurations to test novel placements and novel objects.
* The proposed BusyBot framework has a high success rate in sim and real and beats a few baselines proposed by the authors.
* The solution for each reasoning/planning/interaction components seems well thought out.
* Real world experiments.
* The paper overall is well written and clear.


Weaknesses
* The environment does not contain many-to-one relationships, which would be a very interesting setup to test for embodied agents, requiring agents with memory and potentially longer sequences and interactions.
* The observation is a top-down depth image. This modality may not be available in all cases for real world robotic systems. There are no ablations on this modality. It would be nice to see how the system compares with rgb observations for example, or a perspective view of the board.
* No vanilla RL or BC baselines. It would be interesting to see the performance of even a simple BC sequence model, trained on a sim oracle or human dataset. It could help answer the question: how necessary are the separation of interaction/reasoning/planning models?
* BusyBot is not evaluated on environments other than BusyBoard, so it’s unclear how generally useful this system could be. Is it “overfit” to this particular setup? A few details in particular that could be a concern for it’s usefulness:
* As mentioned above, many-to-one interactions are not included and could be common in real world tasks. Unclear if the method would work for this case.
* A few parts of the system, such as the k-means clustering hyperparam, could require tuning in different environment setups.
* Baselines are not tested on the real world.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: I think the paper is borderline and I would consider changing my recommendation with a few clarifications and/or results. The environment and proposed method are solid and the experimental results support the authors conclusions overall. The environment is a good contribution, as a single environment that evaluates several skills in embodied agents.
The BusyBot results could be made more convincing. I think testing the proposed method on more complex many-to-one tasks or with more realistic modalities, or against a more diverse set of baselines would be more convincing.
I recommend addressing some of the weaknesses above to improve the paper.

issues: * Clarify the issue of many-to-one tasks - were these tasks attempted or could the environment be modified to generate these sorts of tasks? Would the proposed method address these tasks?
* Could the authors clarify the choice of observation space. Why was only topdown depth chosen?
* It's hypothesized that the relation agent might have trouble handling 1-to-m relations (L196). Why is the 1-to-m result higher for novel objects for the relation agent than the predictive agent?
* Could the authors clarify how BusyBot could be useful outside of the 1-to-1 or 1-to-m scenarios presented in the env?
* Addressing or explaining limitations mentioned above in limitations section.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This work proposes a framework for generating simple learning environments both in simulation and in the real world, dubbed BusyBoard. BusyBoard consists of sets of switches and objects, some of which are articulated, that can be interacted with and uses lights to amplify the effects of many of the small actions (e.g. pushing switches) making them more observable. BusyBoard instances are generated procedurally from URDF files making the environments amenable to many variations on the BusyBoard theme.

The authors also propose a baseline for the BusyBoard environment dubbed BusyBot. BusyBot learns action affordances from depth, learns inter-object interactions, and then creates and executes plans in the BusyBoard enviornment.
strengths_and_weaknesses: Strengths
- The BusyBoard framework complements typical toy domains, such as gridworlds and block stacking, by providing a relatively rich set of inter-object interactions in a small and feasibly explorable space with relatively dense reward.
- BusyBoard's procedural nature lends itself nicely to explorations of transfer learning and distribution shift.
- BusyBot appears to be a strong baseline for this domain (with one of the biggest assumptions being the sufficiency of single-step actions to realize meaningful change in the world).
- The paper contains an interesting hardware implementation of the both the BusyBoard domain and BusyBot.

Weaknesses
- By design the BusyBoard domain does not attempt to expose challenges such as partial-observability and sparse long-horizon rewards. While the authors are explicit about this design choice, these are two extremally fundamental difficulties in robotics.
- The BusyBot baseline is not well explored in the evaluation. While the design appears reasonable, virtually no comparison was made with existing learning methods that may be applicable to this task. As a result, this paper's primary contribution centers on BusyBoard; it's hard for a reader to judge the effectiveness of BusyBot.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I advocate for acceptance of this work. I think the BusyBoard domain, while simple, has some value to the community as a platform for exploring learning in constrained settings. That being said, BusyBoard, by design, hides much of the complexity present in less structured environments. As a result, progress in the BusyBoard domain may not be a particularly good proxy for progress in real-world tasks.
issues: - Consider adding comparisons to existing agent-learning approaches for BusyBot. As is, it's difficult to evaluate the value of BusyBot and most of the paper's contribution centers around the BusyBoard domain.
review_title: null

Meta-review: metareview: The paper proposes a method to learn cause-effect relationships on Busyboards in a self-supervised way. 

The reviewers see strength in the paper of creating the Busyboard simulation environment as a benchmark and evaluating their approach on a real hardware setup.
However, the reviewers also raised many concerns about the generality and novelty as well as comparison to other methods.
I invite the authors to respond to the reviewers.  

Post-rebuttal:
The authors have cleared many concerns by providing baseline evaluations and experiments on additional environments (2 envs from AI2THOR). I have read the revised version in detail and came to the following conclusion: 
Although the reviews are still mixed, I believe that the paper adds value to the community, both by providing the Busyboard environments, as well as by creating a relatively simple algorithm for such interaction tasks, that shows good performance.

Comments: please rearrange the paper to fit the page limit and make sure the important aspects mentioned in the reviews make it to the main paper, whereas details are moved to the supplementary. 
(Typo: L386: broads)

confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)