Paper: Mobile Object Rearrangement with Learned Localization Uncertainty (ICLR-2024)

Reviewer comment (Experiments):
Missing baselines or explanations about related works. Can the authors explain why Neural SLAM [1] is not included as a baseline? In NeuralSLAM, the NeuralSLAM module estimates the 2D map and relative pose as what the L-Net does in this paper, and a global policy is trained to explore the map but can be adapted to achieve arrangement tasks as what the P-Net does in this paper. Besides, [2] showcases that using explicitly estimated poses can be helpful for learning mobile manipulation.

Author response:
There are two primary reasons why neural SLAM may not be suitable for our task. Firstly, the dynamic nature of our environment, where object positions frequently change due to rearrangement actions, contradicts the static environment assumption inherent in neural SLAM. While in room environments like Habitat or AI2Thor, a slight change in the pose of a small object may still allow SLAM-based methods to be effective, our MOR environment simulates scenarios such as a warehouse where the substantial position change of large objects dramatically influences the scene, rendering the environment highly dynamic. Therefore, the application of neural SLAM in the MOR context becomes challenging.

Secondly, the additional pose sensor used in neural SLAM is not involved in the MOR task. While it's possible to include this pose sensor in our simulation, doing so would make the results not directly comparable to ours.

We appreciate the reviewer for recommending this paper, as it aligns with our approach of explicitly estimating poses for rearrangement tasks. However, our unique contribution lies in the exploration of additional pose uncertainties, demonstrating that they provide the agent with more information than poses alone, facilitating the learning of an effective rearrangement policy. This assertion is substantiated by our ablation study on uncertainty, as detailed in the experiment section. Our work extends beyond the mere estimation of poses by emphasizing the importance of uncertainty in enhancing the agent's performance in rearrangement tasks.

Reviewer comment (Writing):
Does the "global coordinate" in Sec 4.1 mean "episodic coordinate"? For example, the initial agent position is considered (0, 0).

Author response:
The global coordinate is corresponding to the world coordinate where the agent is always reset at the center of a 12x12 size environment with position (6,6).

Reviewer comment (Experiments):
How do the authors acquire "relative pose" for the baseline "ResNet + $\Delta$Pose"? And why are results for these baselines missing for "Pick"?

Author response:
Cause we know the GT agent position and objects’ positions from the simulation environment. So we can easily compute the relative position between agent and objects. The reason why we provide the relative position to the Drop task is that the baselines fail to learn any policy by only RGB input. Therefore, we give the baselines more information to learn a reasonable policy. However, in the pick task, the baselines can already learn a reasonable policy. Therefore the relative pose is not involved in the pick task.

Reviewer comment (Experiments):
Is the recurrent network used for baselines like ResNet + Img and ResNet + Landmark?

Author response:
No, we do not use recurrent network for the baselines. As we stated in the experiment section, we find that the baselines have the same level of performance with or without recurrent networks. Therefore, for training efficiency, we don’t use recurrent networks for baselines.

Reviewer comment (Evaluation):
Can the authors qualitatively and quantitatively show how well the estimated poses are?

Author response:
The demo videos in the supplementary showcases that when the objects are not in the view of the agent, the uncertainties of these objects are large. As the agent perceives the objects, the uncertainties drop to near zero. This qualitatively demonstrates the L-Net could effectively represent the uncertainty. For the quantitative results, we could not think about a way to quantify how well the L-Net could estimate the uncertainty. Following some related works like [1] which uses downstream task performance to quantitatively evaluate the uncertainties, we could use the performance of downstream rearrangement task to verify the effectiveness of L-Net for the uncertainty estimation quantitatively. 

[1] Su, Sanbao, et al. "Uncertainty quantification of collaborative detection for self-driving." 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023.

Reviewer comment (Experiments):
The environment used is not visually realistic and does not support their claim that "MOR adopts visual inputs from 3D simulations that are easier to transfer to real-world robotics systems." The environment itself is a simple convex shape with no obstacles or visual features besides a black-and-white checkerboard pattern on the walls. The reviewer is concerned that these two extreme simplifications in the environment mean that any conclusions drawn won't transfer to more realistic settings. This concern is reinforced by the result that CLIP features were not appropriate for this environment.

Author response:
“The reviewer is concerned that these two extreme simplifications in the environment mean that any conclusions drawn won't transfer to more realistic settings.”

We agree that our simulation environment is a simplified toy-like environment. The intuition of conducting all experiments in this simplified, yet challenging environment is rooted in the belief that the environmental setups can be readily transferred to real-world scenarios, as demonstrated in the supplementary material. The showcased real-world demonstration illustrates the zero-shot transferability of the trained policy net from simulation to reality. While we acknowledge the importance of eventually working in a more photorealistic environment, such as Habitat, for greater relevance to the broader robotic community, we recognize this as a facet of our future work. The current emphasis on a simplified environment serves as a foundational step, demonstrating the feasibility of transferring policies from simulation to real-world scenarios.

We acknowledge the noted failure of the CLIP-agent in our tasks, primarily attributed to the domain gap between our simplified environment and a more realistic setting. It is important to underscore that the primary contribution of this paper does not lie in the feature extraction domain. Rather, the core innovation resides in our proposed solution, which can be readily adapted to a more photorealistic environment by simply substituting the visual feature extraction module, such as CLIP, with a more suitable alternative. This flexibility in adapting the proposed solution underscores its robustness and applicability across varying environments.

Reviewer comment (Experiments):
Additional baselines are needed for L-NET. The embodied AI literature has a large number of tracking methods that were developed for PointGoal navigation, see Partsey et al 2022 for one example.

Author response:
“Additional baselines are needed for L-NET.”

We express our gratitude to the reviewers for suggesting related papers. However, it is important to note that our paper specifically focuses on the rearrangement task, as opposed to navigation, with L-Net serving as just one component of our proposed method. Given this focus, we believe it may not be necessary to compare our work directly with navigation baselines.

Furthermore, we would like to highlight another contribution of our paper, the explicit estimation of uncertainty and its beneficial impact on rearrangement performance. This aspect sets our work apart, as many navigation papers typically do not address uncertainty explicitly.

Reviewer comment (Writing):
The name "Mobile Object Rearrangement (MOR)" is confusing as the word mobile could also be used to describe the agent being mobile and differentiate the task from table-top rearrangement.

Author response:
“The name "Mobile Object Rearrangement (MOR)" is confusing.”	

We thank the reviewer for pointing this out. The word “mobile” refers to the agent being mobile. As we stated in the abstract, Mobile object rearrangement (MOR) stands for the embodied AI task for a mobile agent to move objects to their target locations.

Reviewer comment (Presentation):
The figure 3 caption is very long and has key method details. Method details should not be in figure captions as the reader does not know when they are supposed to read these details.

Author response:
“The figure 3 caption is very long and has key method details.”

Thank you for your comments. We have shorten the caption of Figure 3 and moved the details to the supplementary.

Reviewer comment (Experiments):
(Motivation) Why couldn't the agent utilize depth information and conduct simultaneous localization and mapping (SLAM)? It doesn't seem necessary to completely forgo the use of internal GPS and depth sensors.

Author response:
We acknowledge the potential involvement of other sensors, such as internal GPS and depth cameras, in facilitating simultaneous localization and mapping (SLAM)-type techniques within our MOR task. However, it should be noted that possessing depth information or employing SLAM does not equate to achieving perfect localization. This paper concentrates on the challenge of developing an effective rearrangement policy without assuming perfect localization knowledge.

As illustrated in the experimental section, our proposed L-Net demonstrates the ability to effectively estimate the poses of both the agent and objects, along with their associated uncertainties. Notably, these uncertainties play a pivotal role in enhancing the performance of MOR. In contrast, SLAM methods lack the capability to provide such uncertainty information for informing the rearrangement policy. Furthermore, the static environment assumption becomes invalid due to the dynamic nature of rearrangement actions, particularly the positional changes of large objects, significantly altering the scene in MOR scenarios. Consequently, deploying SLAM-based methods in our tasks proves challenging due to these dynamic and evolving environmental conditions.

Reviewer comment (Experiments):
(Method) The proposed method heavily relies on the localization network (L-Net). However, training the L-Net necessitates extensive pre-training (e.g., 10,000 episodes in the proposed simplified environment), raising questions about its suitability for deployment in more complex environments like Habitat, AI2Thor, or the real world.

Author response:
Regarding the applicability of L-Net in more photorealistic environments, such as Habitats or AI2Thor, we contend that its deployment in these simulators should not pose significant challenges. Notably, simulators like Habitat[1] are efficient, achieving over 1,000 steps per second (SPS), enabling the completion of an entire episode in approximately one second. Consequently, the collection of substantial datasets within the Habitat simulator can be accomplished in a matter of hours.

Comparatively, for training L-Net, we rely on a real-world dataset comprising only 20,000 steps. Acquiring this amount of data would typically take less than a day. This efficiency underscores the suitability of L-Net for real-world setups.

[1] Szot, Andrew, et al. "Habitat 2.0: Training home assistants to rearrange their habitat." NIPS 2021

Reviewer comment (Experiments):
(Evaluation) The evaluation of the proposed method takes place in toy-like environments, and real-world experiments assume ground-truth poses of agents and objects.

Author response:
We agree that the evaluation of baselines and our method is in the simplified environment, but dynamic involved in this simulation environment such as wall texture, the types and colors of each object changed every episode, and the introduced motion uncertainty makes MOR still non-trivial for baselines and even challenging for proposed method according to the experiment results. 

Presently, our focus extends to real-world experiments, where the agent relies solely on RGB inputs for rearrangement tasks. We anticipate releasing these real-world experiment findings on our project webpage shortly. The current version serves as a demonstration, employing ground truth (GT) information to showcase the setup of our real-world platform. This choice is made to exemplify the potential for transferring the trained policy seamlessly from simulation to the real-world setting without the need for any fine-tuning.

Reviewer comment (Writing):
(Writing) I recommend that the authors enhance the clarity of the writing in Sections 3 and 4. These sections currently contain an overwhelming amount of details and lack structured and logically coherent expressions.

Author response:
We thank the reviewer's valuable suggestion. The fundamental logic underlying Section 3 is to start with the definition of the MOR, treating it as a Partially Observable Markov Decision Process (POMDP). Subsequently, we decompose MOR into a series of subtasks. The section concludes by introducing evaluation metrics utilized in this paper to assess performance.

In Section 4, the logical progression commences with the introduction of L-Net, followed by a detailed exploration of the policy net. The inclusion of these specifics aims to provide a comprehensive definition of the MOR task and elucidate the proposed method thoroughly.

Reviewer comment (Experiments):
How does the agent perceive the target poses of the objects? Does it estimate these poses from another unshuffled environment?

Author response:
The target pose of each object is directly given to the agent (in the global coordinate, similar as the task definition in Habitat[1]). So there is No need to estimate the target poses. 

[1] Szot, Andrew, et al. "Habitat 2.0: Training home assistants to rearrange their habitat." NIPS 2021