Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper introduces a new task, mobile construction, to evaluate the abilities of an agent to localise within a dynamic environment. The goal is to navigate an environment to build a structure, either in 1, 2 or 3D specified as a goal. The observation available to the agent is the state of the environment in its current neighbourhood. The fact that agents modify themselves the appearance of the environment lead to challenges in terms of localization and mapping. 

Different baselines from the literature are considered, and compared against humans and handcrafted baselines. The main takeaway is that the task is challenging, and thus an interesting direction to pursue. Authors also provide a study of the current limitations of agents, by varying the available given inputs.
strength_and_weaknesses: Strengths:
1. The introduced task targets an important problem, i.e. localization, and challenges current SLAM and sfM methods due to the dynamic status of the environment caused by construction.
2. Authors evaluate a variety of baselines from the literature, and experiments are thorough. The ablation study on the impact of modifying the given input to the agent is interesting, and allows to better understand the source of current agent limitations.
3. The paper is generally well written.

Weaknesses:
1. Compared with other tasks, e.g. in Embodied AI, the environment the agent evolves in is not photorealistic. However, the simple nature of the task can be a good thing to isolate core problems to be solved before to move to environments that are closer to what humans experience in the real world. Moreover, as shown in the paper, the task can not be considered as trivial as most policies do not get strong performance. I would thus tend not to consider lack of realism as a weakness here.
2. Handcrafted methods beat all other baselines in most cases. However, authors mention they have access to more prior knowledge. I would like authors to elaborate on this, as it is an important concern in this work.
clarity,_quality,_novelty_and_reproducibility: The paper is well written, and conducted experiments are through. The introduced task is novel, and targets an important problem.
summary_of_the_review: This work introduces an interesting task, along with an evaluation of diverse baselines from the literature. Compared with other robotics, Embodied AI tasks that also necessitate localisation and mapping, the observations to the agent are much further from real-world observations. However, this should not be considered as a weakness as it allows to properly isolate the studied capacity, i.e. localization in a dynamic environment. Moreover, the ability of the agent to modify its environment makes it an interesting and challenging task for current SLAM methods.

I am however concerned with the much stronger performance of handcrafted baselines, compared with all other agents, and in particular the agent equipped with the L-Net module introduced in this work. Authors should elaborate on what additional prior knowledge these handcrafted baselines have access to, as it looks to me like the design of L-Net is also guided by some prior knowledge about the task at hand.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper presents the mobile construction POMDP to formulate the task of a mobile agent placing blocks according to some design, while using observations of its (dynamic) environment to navigate. The paper introduces various instantiations of, and complications to, this task, such as operating in 1/2/3D settings and with/without obstacles. Finally, the paper demonstrates that a deep recurrent Q-network (DRQN) in concert with a localization network (L-net) outperforms baseline approaches and achieves good results on some test cases but leaves room for future work to succeed in others.
strength_and_weaknesses: Strengths
------------
Novelty
- The mobile construction POMDP is interesting and novel
Significance
- RL for mobile construction is highly relevant to a growing subfield in robotics
- The proposed DRQN+L-Net approach significantly outperforms the baselines in the most challenging test cases
- The problem specification is quite general and extensible for future works to build upon and compare against
- The ablation study is very well done and points towards promising future directions
Clarity
- The paper is mostly clear and concise
- Figures are detailed and effective at visualizing the task and results
- Tables 1, 2, and 4 are effectively summarize a great deal of information clearly
Soundness
- The POMDP formulation, choice of evaluation metric (IoU), and other design decisions seem sound

Weaknesses
-----------------
- Marginal novelty algorithmic contribution
- Proposed POMDP is similar enough to existing grid-world type problems that there are not many totally new/unexpected insights
- Feels like a small step towards a more realistic model/simulation of mobile construction
- Font sizes in tables and figures are way too small (unreadable when printed, hard to read on screen without extreme zoom)
- Some grammatical and spelling issues to address
- "So we removed the penalty for better performance" -- better performance on what algorithm(s)? If this is to improve the performance only of the proposed approach, it seems inappropriate to make it part of the experimental setup.
clarity,_quality,_novelty_and_reproducibility: The formulation of mobile construction as a POMDP appears to be a novel and significant contribution. It is an interesting new class of relevant problems for future works to continue to explore. However, the algorithmic contribution is only marginally novel; it is based on recent work suggesting that the separation of representation learning and reinforcement learning can be an effective practice, and makes use of two existing architectures to achieve these respective tasks. There is not a particularly compelling argument made for why this should be considered a novel contribution that is interesting in its own right, and not just an engineered solution. Similarly, the proposed POMDP formulation of the problem is quite simplistic and quite closely resembles many other POMDPs / grid-world environments from RL, as noted by the paper, so the formulation's novelty is not particularly exciting or insightful. As such, it feels like a small step towards a more complete and realistic formulation of mobile construction (especially the 1D and 2D cases -- the 3D test scenario is a bit better but still seems like a minor step beyond the 2d scenario).

The quality of the experimental results and analysis is a good strength of the paper. Several strong baselines are compared against and the results (and appendix) demonstrate a wide range of different classes of baselines on the new task. The inclusion of a human baseline is also impressive and highly informative, as is the inclusion of handcrafted task algorithms. The ablation study is also detailed and insightful. The entire results section does an excellent job of providing qualitative and quantitative analysis on the performance of different algorithms and on the challenge of the novel task. In terms of further improvement, more discussion of how the problem might change as it takes into account more real-world complexities would be interesting (the analysis of and comparison to baselines, as well as the ablation study, seem largely specific to the test environments explored).

Overall, the clarity of the paper is good. The figures and tables are all excellent at efficiently and effectively conveying various ideas and results. Unfortunately, the legends and text in most of the figures and tables is unreadable unless zoomed in significantly on a screen. Most figures, and table 2, are completely illegible when printed. The font size needs to be increased significantly in all cases, which likely requires enlarging figures. Work also needs to be put into editing the paper; section 4.2 has some grammatical mistakes (e.g. ''train LSTM", "the similar process") as does the Table 2 caption ("2D matrice"), and there other more minor mistakes elsewhere (e.g. "planning" is often misspelled as "planing").

There is no reproducibility statement included in the main text. I think this would be far more useful to the community if the test cases and baselines were implemented in a way to be easily accessed and reproduced.
summary_of_the_review: This paper serves as a solid foundation for future works to explore more sophisticated approaches to the mobile construction task, as well as more sophisticated formulations of the task itself that capture more real world complexities. The paper does an excellent job of exploring the performance and limitations of existing approaches, and presents a new and somewhat novel approach that outperforms the baselines. The clarity could use some work, and there is some concern about the reproducibility of the experiments. If these issues were addressed, I would feel quite comfortable upgrading my rating.

Update: Based on author responses and revised manuscript, I am upgrading my rating to an 8. Please see comments for details.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new challenge for AI -- Mobile Construction. In Mobile Construction an agent is initialized in an environment and tasked with building a given structure. The challenge of the task is that it is long horizon and the agent must operate from local observations.

There are 3 versions of the task that are proposed, 1 in 1D, 1 in 2D, and 1 in 3D. This gives an increasing amount of difficulty.

Towards solving this task, the authors propose to explicitly learn localization and incorporate it into the agent by cropping and transforming the target design to be relative to the agents current location.

The proposed method outperforms all the various baselines in the most challenge 3D case.
strength_and_weaknesses: ## Strengths

- The proposed task seems challenging for RL but conceptually quite simple. A nice mix.
- The comparisons to existing methods are well done and thorough.
- The proposed method is effective and the way of incorporating positional information is interesting.
- Ablations are well done.

## Weaknesses

No simulation speed numbers are given for the environment itself. Given that this seems to be a visually simplistic environment, I imagine it can be quite fast, but specific numbers would be very useful.

On that note, it also seems like the environment doesn't support batched simulation. I encourage the authors to think about supporting this as such an environment seems idea for this and a faster environment almost always gains more traction.
clarity,_quality,_novelty_and_reproducibility: The work is original, although similar ideas for explicit learning localization have been used before, i.e. "Integrating Egocentric Localization for More Realistic Point-Goal Navigation Agents, CoRL 2020".

The work is clear and reproducible, assuming the authors release code for the environment, as they say they will.
summary_of_the_review: Overall, my main concern with this paper is lack of evaluation of the environment's simulation performance. This seems like a key part of characterize given that the environment is one that, at least in my understand, should be able to be simulated very quickly.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors introduce a new problem of mobile construction, where an autonomous agent must construct a grid world according to a design (target grid world state) that is fed as input to the agent. To tackle this problem, the authors de-couple the approach into (1) learning to localize the agent via an L-Net, and (2) zoom in on the design based on the localization info and feed that, along with the observation, as input to a DNN to predict the actions.
strength_and_weaknesses: ## Strengths
* The authors introduce an interesting problem of mobile construction, and develop a grid world setup for experimenting on this task.
* The authors propose a novel approach of localization via an L-Net and action prediction via Deep Recurrent Q Network.
* The authors include extensive experiments: comparisons against human performance, ablations, and comparisons against standard RL baselines (e.g. PPO, DQN) and hand-crafted policies under different setups (e.g. obstacles, no location uncertainty, etc.)

## Weaknesses
* It's not clear to me that the findings in the grid world experiments will transfer easily to more complicated setups, such as mobile construction and mobile manipulation in the real world, in a 3D physics-based simulator, or in minecraft.
* Moreover, most of today's learning-based approaches for learning-based navigation/mobile manipulation (e.g. Savinov et al. semiparametric topological mapping; learning to explore via active neural SLAM) already utilize localization information (and same with non-learning based approaches ie. SLAM), so the finding that decoupling localization from action is helpful for mobile construction is also not very surprising.

## Notes
* Regarding the reward and adding penalties: "we found this led to more frequent actions of moving instead of dropping bricks". It may be useful to add a "time" penalty for each action that doesn't result in a positive reward so that the agent can be more efficient in construction.
clarity,_quality,_novelty_and_reproducibility: The writing and clarity of the paper is sufficient. The work is original.
summary_of_the_review: My main concern is how the findings and contributions of this paper can be applied more generally. For example, the authors propose the L-Net as a useful decoupling, but this approach is specific to the grid world setup. In more realistic setups, such as iGibson or Habitat, many navigation approaches already predict localization. Also, the experimental findings which show that explicit localization prediction is helpful for the mobile construction task, and that adding obstacles in the grid worlds reduces performance drastically, are somewhat obvious.

Update: Based on the other reviewers' comments and the authors' responses, I've updated my review to be 6.  I agree with the authors' responses that decoupled representations are helpful for this task, and that the findings are not as obvious as I initially thought.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper introduces a new task, mobile construction in a dynamic environment and tests various baselines against it. The paper has been generally appreciated and a consensus was reached quickly. The reviewers also appreciated well executed evaluations and ablations including qualitative and quantitative analysis on the performance of different algorithms.

On the downside, reviewers noted that the task is not photo-realistic as other navigation tasks are, that generalization to non-grid world environments is not straightforward and that no methodological contribution has been provided.

The AC considered these issues outweighted by the interesting benchmark and the well executed evaluation and recommends acceptance.
justification_for_why_not_higher_score: The paper introduces a new benchmark but no new method.
justification_for_why_not_lower_score: -
Decision: Accept: poster