Learning to Act with Affordance-Aware Multimodal Neural SLAM
Authors: Zhiwei Jia
Kaixiang Lin
Yizhou Zhao
Qiaozi Gao
Govind Thattai
Gaurav S. Sukhatme
Venue: ICLR 2022 Submitted (2022)
Keywords: Language-Guided Task Completion
Multimodal learning
Neural SLAM.

Abstract: Recent years have witnessed an emerging paradigm shift toward embodied artificial intelligence, in which an agent must learn to solve challenging tasks by interacting with its environment. There are several challenges in solving embodied multimodal tasks, including long-horizon planning, vision-and-language grounding, and efficient exploration. We focus on a critical bottleneck, namely the performance of planning and navigation. To tackle this challenge, we propose a Neural SLAM approach that, for the first time, utilizes several modalities for exploration, predicts an affordance-aware semantic map, and plans over it at the same time. This significantly improves exploration efficiency, leads to robust long-horizon planning, and enables effective vision-and-language grounding. With the proposed Affordance-aware Multimodal Neural SLAM (AMSLAM) approach, we obtain more than 40% improvement over prior published work on the ALFRED benchmark and set a new state-of-the-art generalization performance at a success rate of 23.48% on the test unseen scenes.

Under review as a conference paper at ICLR 2022
LEARNING TO ACT WITH AFFORDANCE-AWARE MUL- TIMODAL NEURAL SLAM
Anonymous authors Paper under double-blind review
ABSTRACT
Recent years have witnessed an emerging paradigm shift toward embodied ar- tificial intelligence, in which an agent must learn solve challenging tasks by interacting with its environment. There are several challenges in solving em- bodied multimodal tasks, including long-horizon planning, vision-and-languuge grounding, and efficient exploration. We focus critical bottleneck, namely the performance of planning and navigation. To tackle this challenge, we pro- pose Neural SLAM approach that, for the first time, utilizes several modalities for exploration, predicts an affordance-aware semantic map, and plans over at the same time. This significantly improves exploration efficiency, leads bust long-horizon planning, and enables effective aadion-and-lnguage grounding. With the proposed Affordance-aware Multimodal Neural SLAM (AMSLAM) ap- proach, we obtain more than 40% improvement over prior published work on the ALFRED benchmark and set a new state-of-the thefthhe-rtt generalization performance a success rate of 23.48% on the test unseen scenes.
INTRODUCTION
There is significant recent progress in learning simulated embodied agents Pashevich et al. (2021); Zhang & Chai (2021); Blukis al. (2021); Nagarajan & Grauman (2020); Singh et al. (2020); Suglia et al. (2021) that follow human language instructions, process multi-sensory inputs and act to complete complex tasks Anderson al. (2018): Das al. (2018); Chen al. (2019); Shridhar al. (2020). Despite this, challenges remain before agent performance approaches satisfactory levels, including long-horizon planning and reasoning Blukis et al. (2021), effective language grounding in visually rich environments, efficient exploration Chen et al. (2018) 8), and importantly, generalization to unseen environments. Most prior work Singh et al. (2020); Pashevich et al. (2021); Nguyen et al. (2021); Suglia al. (2021) adopted end-to-end deep learning models that map visual and language inputs into action sequences. Besides being difficult to interpret, these models show limited generalization, suffering from significant performance drop when tested new tasks and scenes.
In contrast, hierarchical approach Zhang & Chai (2021): Blukis et al. (2021) achieve better general- ization performance and interpretabilityy Although hierarchical structure is helpful for long-horizon planning, its key impact is an expressive semantic representation of the environment acquired via Neural SLAM-based approaches Chaplot et al. (2020a:c): Blukis al. (2021). However, missing component in these methods is fine-grained affordance Kim & Sukhatme (2015); Qi et al. (2019). To build a robotic assistant that can follow human instructions to complete task (e.g., Open the fridge and grab me soda), it is essential that the agent can perform affordance- navigation: it must navigate reasonable position and pose near the fridge that enables follow-on actions open and pick-up. Operationally, the agent has to move to location where the fridge is within reach yet without arresting the fridge door from being opened. Ideally, it should also position itself so that the soda is in its first person viewing field to allow the follow-on pick-up action. This is challenging compared to pure navigation (where navigating to any location close to the fridge is acceptable). To achieve this, propose a sophisticated affordance-aware semantic representation that leads to accurate planning for navigation setting up subsequent object interactions for success.
Efficient exploration of the environment Ramakrishnan et al. (2021); Chen al. (2018) needs to be addressed to establish this semantic representation it is unacceptable for a robot to wander around for extended period of time to complete single task in real-world setting. To resolve

---

Under review as a conference paper at ICLR 2022
this issue, we propose the first multimodal exploration module that takes language instruction as guidance and keeps track of visited regions to explore the area of interest effectively and efficientll. This lays foundation for map construction, which is critical to long-horizon planning.
Here, we introduce Affordance-awart Multimodal Neural SLAM (AMSLAM), which implements two key insights address the challenges of robust long-horizel planning, namely, efficient ex- ploration and generalization: 1. Affordance-aware semantic representation that estimates object information in terms of where the agent can interact with them support sophisticated affordance- aware navigation, and 2. Task-driven multimodal exploration that takes guidance from language instruction, visual input, and previously explored regions to improve the effectiveness and efficiency of exploration. AMSLAM is the first Neural SLAM-basee approach for Embodied AI tasks to uti- lize several modalities for effective exploration and an affordance-awann semantic representation for robust long-horizon planning. We conduct comprehensive empirical studies on the ALFRED benchmark Shridhar et al. (2020) to demonstrate the key components of AMSLAM, setting new state-of-the-ar generalization performance at 23.48%. ~40% improvement over prior published state-of-the-ar approaches.
2 RELATED WORK
Recent progress in Embodied Artificial Intelligence, spans both simulation environments Kolve al. (2017); Li et al. (2021); Savva et al. (2019); Gan et al. (2020): Puig et al. (2018) and sophisticated tasks Das et al. (2018); Anderson et al. (2018); Shridhar et al. (2020). Our work is most closely related to research in sanguage-guided task completion, Neural SLAM, and exploration.
Guided Task Completion. ALFRED Shridhar et al. (2020) is benchmark that enables a learning agent to follow natural language descriptions to complete complex household tasks. The agent's goal is to learn a mapping from natural language instructions to a sequence of actions for task completion in a simulated 3D environment. Various modeling approaches have been proposed falling into roughly two families of methods. The first focuses on lcarning large end-to-end mod- els that directly translate instructions to low-level agent actions Singh et al. (2020); Suglia et al. (2021); Pashevich et al. (2021). However, these agents typically suffer from poor generalization performance, and are difficult to interpret. Recently, hierarchical approaches Zhang & Chai (2021); Blukis et al. (2021) have attracted attention due to their better generalization and interpretability. We also adopt hierarchical structure, focusing on affordance-aware navigation thereby achieving significantly better generalization than all existing approaches.
Neural SLAM and Affordance-aware Semantic Representation. Neural SLAM Chaplot et al. (2020a;b:c), constructs an environment semantic representation enabling map-based long- long-horizon planning Chaplot al. (2021). However, these are tested in pure navigation tasks instead of complex household tasks, and does not consider affordance Qi et al. (2019); Nagarajan & Grauman (2020); Xu al. (2020), which is required for tasks involving both navigation and manipulations. In Blukis et al. (2021), the authors utilize SLAM for 3D environment reconstruction in language-guided task completion. Their approach relies heavily on accurate depth prediction (less robust in unseen en- vironments). Instead, we propose waypoint-oriented representation which associates each object with the locations on the floor from where the agent can interact with the object. Furthermore, dif- ferent from the 2D affordance map in Blukis et al. (2021) that directly predicts affordance type, our semantic representation supports more fine-grained control of the robot's position and pose, which facilitates significantly better generalization. The approach in Qi al. (2019) assumes direct access to the ground truth depth information (not available in our setup) and the method in Nagarajan & Grauman (2020) only focuses on pure navigation problems.
Learning to Explore for Navigation. An essential step in Neural SLAM-based approaches is learn- ing to explore the environment for map building Ramakrishnan et al. (2021); Chen al. (2018); Jayaraman & Grauman (2018); Chaplot et al. (2020a). Multiple approaches have been proposed to tackle aspects of exploration in the reinforcement learning Schmidhuber (1991); Pathak et al. (2017); Burda et al. (2018) 8); Chen et al. (2018); Jayaraman & Grauman (2018), computer vision Ra- makrishnan et al. (202 I); Nagarajan & Grauman (2020), and robotics Blukis et al. (2021); Harrison et al. (2018) communities. The central principle of prior methods is learning to reduce environment uncertainty: different definitions of uncertainty lead to the following types of methods Ramakrish- nan et al. (2021). Curiosity- -diven Schmidhuber (1991): Pathak al. (2017); Burda al. (2018) approaches learn forward dynamics and reward visiting areas that are poorly predicted by the model. Count-based exploration Tang et al. (2017); Bellemare et al. (2016); Ostrovski et al. (2017); Rashid
2

---

Under review as a conference paper at ICLR 2022
et al. (2020) encourages visiting states that are less frequently visited. Coverage-basee Chen et al. (2018); Jayaaama & Grauman (2018) approaches reward visiting all navigable areas by searching in task-agnostic manner. In contrast, we propose multimodal exploration approach utilizing ego- centric visual input, language instructions, and memory of explored areas to reduce task-specific uncertainty of points of interest (areas important complete the task). We show this to be more efficient, leading to more effective map prediction and robust planning.
3 PROBLEM FORMULATION
We focus on the ALFRED challenge Shridhar et al. (2020), where an agent is asked follow hu- man instructions to complete long-horizon household tasks in indoor scenes (simulated in A12Thor Kolve et al. (2017)). Each task in ALFRED consists of several subgoals for either navigation (mov- ing in the environment) object interactions (interacting with at least one object). Language inputs contain a high-level task description and a sequence of low-level step-by-step instructions (cach cor- responding to subgoal). The agent is simulated robot with access to the states of the environment only through a front-view RGB camera with a relatively small field of view. The agent's 's own state is a 5-tuple (a,, y. r. h o). where x. y are its 2D position, the horizontal rotation angle, h the vertical camera angles (also called "horizon") ") and o the type of object held in its hand. The state space of the agent is discrete, with navigation actions: MoveAhead (moving forward by 0.25mm) RotateLeft & eetateRight (rotating in the horizontal plane by 909) and LookUp & LookDown (adjusting the horizon by 15%). Formally, r E {0p, 909 180% 2709) and E {600, 45. - 155 -30", where positive h indicates facing downward. With these discrete actions, the agent has full knowl- edge of the relative changes Ar. Ay, Ar and Ah. Each of the object interaction actions (P i ickup, Open, Slice, etc.) is parametrized by binary mask for the target object, which is usually pre- dicted with a pre-trained instance segmentation module. Featuring long-horizon tasks with a range of interactions, the ALFRED challenge evaluates agent's ss ability to perform tasks over unseen test scenes, while only allowing <1000 steps and <10 action failures for each task at inference time.
4 AFFORDANCE-AWARE MULTIMODAL NEURAL SLAM
Affordance-awarr navigation is a major challenge in solving complex and long-horizon indoor tasks such as ALFRED with both navigation and object interactions. Specifically, given each object of interest in the scene, the agent required to not only find and approach it but also end up at a pose (I, y. r, h). that is feasible for subsequent interactions with the object. For instance, to open fridge, the robot should approach the fridge closely enough (so the door is within reach), look at it (so that the fridge is in the field of view), and leave enough room to open the door. To solve a long-horizon task involving multiple navigation and object interaction subgoals, it is natural to use an explicit semantic map, either 2D 3D, of the environment (similar to Neural Active SLAM Chaplot et al. (2020a)), together with moodll-based planning (c.g. as in HLSM Blukis et al. (2021)). This line of work tends to generalize better than models that directly learn mappings from human instructions to navigation & interaction actions (e.g., E.T. Pashevich et al. (2021)). With perfect knowledge of possible achieve (nearly) perfect performance. In practice, however, the semantic map acquired at inference time is usually far from ideal, primarily due to Incompleteness (missing information due to insufficient exploration of the scene) and Inaccuracy (erroneous object location prediction on the map, especially for small objects).
the environment,
To improve exploration performance, we propose multimodal module that, at each step, predicts an exploration action a E (MoveAhead, Rotat eLeff, RotateRi } by taking visual obser- vations& & actions in the past, step-by-step language instructions, and the explored area map which indicates where the agent has visited. We show that, compared to existing model-based approaches on ALFRED (e.g., HLSM Blukis et al. (2021) which applies random exploration), our use of low- level language instructions leads to more efficient exploration. The proposed exploration module operates at the subgoal level and only predicts exploration actions (in contrast to E.T. which directly predicts actions for the entire task). The extra modality (the explored area) facilitates exploration by providing the agent with explicit spatial information. We illustrate the exploration module in Figure 3, elaborate its details in Section 4.3, and empirically demonstrate its advantages in Section 5.
3

---

Under review as a conference paper at ICLR 2022
Multimodal Exploration
Iragecc
long &
Multimodal Exploration Module
Move Ahead
Entromment
No
Ves
Exploration
Explored area
Instruction
Head forward other sids room.
Navigable area
Environment
Form left and walk to the alove.
Subgoal Parser
Language instructions
Novigation
Interaction
Tablc
Tarpet
Object Interaction
Object Parser
Semantic Map
Transformer
up
Aggregation
Egocentric Observations
Move
Execution
Exploration Flow
Execution Flow
The agent aims to explore the environment given the guidance from low-level language instructions, previous exploration actions, and the explored area where the agent has visited and made observations. Execution Flow: Given the language instructions and the affordance-awar semantic representation (i.e., semantic map) acquired during exploration, the agent executes the subgoals sequentially. It uses planning module (which consumes the semantic map) for navigation subgoals and an object interaction transformer for other subgoals.
Figure 1:
Exploration Flow:
To deal with the inaccuracy in map prediction, carefully design affordance-aware semantic representation for the environments. On one hand, knowing the precise spatial coordinates of ob- jects requires precise depth information, which is difficult acquire due 3D sensor noise and/or inaccuracy in predicting depth from 2D images. On the other hand, affordance-awan essentially asks for poses (z. y, r, h)) of the agent suitable for interactions with the target objects, thus requiring only coarse-grained spatial information. Given object type o, we define such cor- responding poses as waypoints W,, and then treat navigation as a path planning problem among different waypoints. To generate such waypoints, and small objects (apples, mug, etc.) differently. The waypoints for large objects are computed using 2D grid maps predicted and aggregated from front- view camera images by CNN-based network; for small objects, we directly search over all observations acquired during the exploration phase with the help of pre-trained Mask RCNN He al. (2017) (detailed below in Section 4.2).
navigation
handle large objects (fridges, cabinets, etc.)
4.1 OVERALL PIPELINE
We illustrate the overall inference pipeline of our proposed framework in Figure Given a task T specified by high-level goal description and low-level human instructions, method proceeds two phases: exploration and execution. During exploration, the agent navigates across the room (guided by the language instructions) for sufficient exploration of the indoor scene, where mul- timodal transformer predicts the exploration actions sequentially. In the meantime, an affordance- aware semantic representation is acquired given the egocentric observations (images) at each step by a carefully designed neural SLAM system. This representation can be used to derive waypoints for all target objects (from which the agent can interact with the objects of interest). When the explo- ration phase ends (by the agent predicting Stop), it moves to the execution phase, where the agent carries out actions predicted for each subgoal of the task T in sequential manner. Specifically, since each step-by-step language instruction corresponds to subgoal, we use Transformer based Vaswani et al. (2017) subgoal parser to process the text and predict the subgoal g E T for naviga- tion not. For navigation subgoal, we further use another Transformer-basee target object parser to process the same text and predict the categories of the target objects, which are consumed together with the affordance-aware semantic representation by Dijkstra- bijsstr--based planner to generate naviga- tion actions. Otherwise. an object interaction transformer takes charge the action predictions given the visual and language inputs of the object interaction subgoal.

---

Under review as a conference paper at ICLR 2022
Floor
Table
CounterTop
Fridge
Unexplored area
Figure 2: Illustration of the affordance-aware semantic representation used in our framework. (left) A top-down view of an indoor scene used in ALFRED (this view is not available to the agent at test time). (middle) A visualization of the corresponding semantic map; only shown are the side table (in green), counter top (in blue) and the navigable area (in red). Two waypoints (drawn in white stars and arrows) are displayed for side table (E large object) and lettuce (E small object), respectively. (right) While the waypoint for side table is computed from the predicted map. the waypoint for lettuce is obtained by searching among all exploration steps. The visual observation and its mask prediction on the waypoint for lettuce are shown. For reference, the high-level goal description of the task used in this example is "pick up the lettuce and place it on a table".
4.2 AFTORDANCE-AWARE SEMANTIC REPRESENTATION
We empirically show in Section 5.1 that a major bottleneck for solving long-horizon navigation & interaction tasks the affordance-aware navigation. To so, the agent needs position and pose (defined waypoints previously in this section) from which the potential follow-on actions for the target object are feasible, rather than the exact location of the target object. Accordingly, our goal is to develop map representation that supports waypoint generation solved reasonably well by path planning. ALFRED supports more than 100 types of objects (one way in which it mimicks real-world complexity), and we propose to handle small and large objects differently, We detail our design below and give an example in Figure 2. Further implementation details are available in the Appendix.
that navigation can be
For a class Clarge out of Niarge large object types, we compute its waypoint """,""" " r* ht)) in 3 steps. First, we find all positions {(x, y)} that might contain an instance of class Garge and all navigable locations for the robot, both represented on 2D grid map of dimension G G x (Niorge + 37 and unit length 0.25m. Each grid point in the map has binary multi-
1), with grid size hot vector to represent whether each object class appears there and whether the point is navigable. Specifically, each (a, y. visited in the exploration phase, we use a pre-trained CNN (whose inputs are images at 3 different horizons observed at (at, r)) to predict small partial map of the 2D map. We then aggregate across exploration steps by max-pooling over these partial maps, each translated and rotated via Spatial Transformer Jaderberg et al. (2015). Similar to Neural Active SLAM Chaplot et al. (2020a), as we know the changes (Az, Ay, Ar) of the agent after each action,
directly compute the parameters of the Spatial Transformer. After applying some post-processing, we obtain the final 2D map estimation. Details about the CNN architecture, its pre-training process and the post-processing steps are in the Appendix. Second, we find (Pas Py) on this 2D map as the most confident position predicted for class e and then find the navigable position (x', y') closest (p2: Third, we choose the rotation r" to be the one, suppose the agent stands at (a'. vV), that an object at (Pa, appears closest to the center of the agent's 's field of view. To leave room for object interactions (by the agent's backing up few steps), we compute xx x' 5y (c) and y" y' 82(c, r*) where 5, are rule-based functions to (see Appendix A.2). We defer the estimation the horizon h² (the camera angle relative to the horizontal) to the exccution phase described in Section 4.4.
For a small object type Csmall, predicting its 3D coordinates precisely is rather challenging (even 2D object detection for small objects is hard Kisantal et al. (2019)). In ALFRED, this is especially true since only RGB images are given at test time and many types of small objects occur rarely. To deal with this challenge, by searching through all observations (RGB images) at each step during exploration. Specifically, compare all instance masks for Csmall predicted by pre-trained Mask RCNN He al. (2017)
propose to directly find the waypoint (x*. """""""",, h²) for Comall
5

---

Under review as a conference paper at ICLR 2022
that are of confidence 2 Te- Then (2",y","" "",r".h) .h) mask prediction has the largest area. Similar to the waypoint generation for large objects. we do not estimate the horizon h* until the execution phase. Directly finding the waypoint of Camall (without estimating its location) relies heavily on how well the exploration is carried out (discussed later in Section 4.3). In case the agent finds such waypoint (no valid observation of Csmall during exploration), we instead use the waypoint for the container (normally of large object type such as fridge, side table) of the small object, which is much easier find. Most small objects appear on (or are contained in) containers, whose object types are predicted by the target object parser (see Section 4.4). While the proposed affordance-awarc semantic representation is critical for robust long-horizon planning, we also add brief discussion on its limitations in the Appendix.
is computed as the where the aforementioned
4.3 TASK-DRIVEN MULTIMODAL EXPLORATION
The multimodal exploration module consists of several sub-modules, either learned or pre- trained/fixed. At a high level, given a task, the exploration go through its navigation subgoals one by one, with the subgoal parser (see Section 4.4) predicting whether a subgoal is for navigation. Specifi- cally, the module predicts exploration actions a E (MoveAhead, or Stop auto-regressively for each navigation subgoal. The agent switches to the next navigation subgoal whenever it predicts Sp until the last one to end the exploration phase. Instead of ran- dom exploration (as in HLSM) or exploration for maximum coverage (as in Active Neural SLAM), our module utilizes low-level language instructions to achieve task-driven exploration for better ef- ficiency. ALFRED the exploration done individually for each task and the steps count towards the total steps (which has limit of 1000).
potateLefte RotateRittggio.... ght} }
There are 4 modalities (and 4 corresponding branches). At each time step, the first branch takes two low-level human instructions, one for the current navigation subgoal, and one for the subsequent object interaction subgoal. The second and third branches consume the egocentric observations (im- ages) and the previous exploration actions, respectively, Inspired by E.T. Pashevich et al. (2021), use a cross-modal transformer (also see Lu et al. (2019); Zhang & Chai (2021)), which aggregates inputs of these 3 modalities across all previous exploration steps to output 21 The fourth modal- ity, the explored area, is G x G 2D grid map marked for the regions observed by the agent in the past and elsewhere except that the center of the map (which always indicates the agent's position) is marked 2. This map is constructed in 2 steps. First, at each previous exploration step, since the agent can only observe small area in front of it, we define single-step explored region as binary map where there is a 5 x 3 rectangle grid of I's representing where the agent has observed (see details in Appendix). Second, we aggregate the maps across all previous steps. Each single-step map is translated and rotated by Spatial Transformer and then merged by max max-pooling (similar to the semantic map described in Section 4.2). Given this explored area map, we use CNN to extract fa E 32 Together with f1,2,3, we finally use MLP to predict the next exploration action. Illus- trated in Figure 4.3 and evaluated in Section 5.3, our design to represent the action history explicitly and geometrically improves both the effectiveness and efficiency of the exploration.
We train the multimodal exploration module supervisedly. A common practice (e.g. in HLSM Blukis et al. (2021), EmBERT Suglia et al. (2021), LWIT Nguyen et al. (2021)) during inference is to augment the exploration by injecting actions periodically. We manually inject RotateRis ght after every 2 MoveAhead predicted by module to acquire 360 views of the scenes. Moreover, the semantic representation produced by the neural SLAM (in Section 4.2) requires input images of 3 different horizons. So we further inject two LookUp or two LookDown actions alternately after every exploration action. This zigzagging scheme is an efficient way to acquire images of multiple horizons, which only triples the total number of steps in the exploration phase. Since each exploration step counts towards the total steps for a task, there is a trade-off between exploration (a better view of the environment) and exploitation (there is an upper limit of 1000 for total steps). Full implementation details and examples are provided in the Appendix.
4.4 OTHER MODULES
Subgoal Parser & Target Object Parser Both the subgoal parser and the target object parser take the language instruction as the only input. Both models use the same Transformer based architecture (not sharing weights, though) and are trained supervisedly. The subgoal parser performs binary

---

Under review as a conference paper at ICLR 2022
Next Navigation Action: Rotate left, Rotate right, Move ahead or Stop
Fully Connected Layers
Multimodal Transformer Encoder
CNN FC Layers
Transformer Encoder
FC Layers
Word Embedding
Word Embedding
ResNet-50
Move ahead, Turn left,
Spatial Transformer
Head forward to the sofa front of you.
Previous navigation actions
Language modality
Image modality
Explored area
The exploration module. Consisting several transformer-basee networks, the ex- ploration module operates subgoal level. Given navigation subgoal during inference, it takes multiple modalities as input including language instructions for the current and subsequent sub- goals, egocentric observations from the agent, and exploration actions in the past as well as their corresponding explored area. It predicts the next exploration action carried out in the environment auto-regressivell Illustrated is an example where given previous actions the model tries predict the 5-th. Note that for brevity the positional (and temporal) encoding layers right before the trans- former encoders are omitted in the figure.
Figure 3:
classification, predicting whether each subgoal for navigation (based on human instruction). The target object parser predicts both the target object and its container (if it has onc) by taking the language instructions for the navigation subgoal and for the next subgoal.
Online Planner for Affordance-aware Navigation Given the affordance-avee aware semantic repre-
sentation that supports waypoints generation, we deal with a navigation subgoal in steps: (1) Obtain waypoint (Ir, V,, *) for the object type predicted by the target object parser. (2) Derive action sequence from the path connecting the current location and pose (x', y', r h²) t to (I, h²) using Dijkstra's algorithm. (3) Decide the horizon by online exploration by first navigating to (I, y. (.rr,)), N'). The agent then goes through 6 horizons {60", 750% .... 0º - 15%) (essentially search over most horizons allowed the agent) and obtains the mask prediction with confidence 0.8 (selected from {0.5, 0.6, 0.7, 0.8, 0.9} using the valid unseen data) of the target object type by pre-trained Mask RCNN. Finally, we select with the largest mask area. See ablation studies of this scheme in Section 5.3. We also include a two-step backtracking mechanism for our planner (with more details in the Appendix).
Object Interaction Transformer We adopt the cross-modal transformer again the subgoal level for object interaction subgoals, which maps language instructions and visual observations (only for the current object interaction subgoal) actions. The module trained by imitation learning on the ALFRED training data at the subgoal level.
EXPERIMENTS
We evaluate our method for language-guider task completion on the ALFRED challenge Shridhar al. (2020), which supports task evaluation in unseen environments (i.e., room layouts), the main focus of our method. ALFRED provides both validation (with ground truth provided) and test set (evaluation occurs the server) for tasks sampled in indoor scenes unseen during training. The training split contains 21,023 tasks sampled from 108 scenes (i.e., rooms), the valid unseen split and the test unseen split contain 821 tasks sampled from 4 scenes and 1,529 tasks sampled from scenes, respectively. For all experiments, report the commonly used evaluation metrics: the task

---

Under review as a conference paper at ICLR 2022
level Success Rate and the subgoal level Goal Condition. Notice that each task in ALFRED allows up to 1000 total steps (including both exploration and execution phases in our method).
We organize the presentation of the numerical results into 3 parts. Firstly, demonstrate that the affordance-awar navigation is the major bottleneck for language-guided task completion. Sec- ondly, we present main numerical results showing that with better affordance-aware navigation, our method significantly outperforms previous state-of-the-ar methods on ALFRED. Finally, we perform series of ablation studies justify our design."
5.1 AFFORDANCE-AWARR NAVIGATION
We analyze the generalization performance of our approach by using ground truth actions for igation subgoals and actions generated by the object interaction transformer otherwise. Similar to solving indoor tasks in real-world settings, the major bottleneck in tackling ALFRED is affordance- aware navigation. A navigation subgoal succeeds only if the agent stays close enough to the target object (so that it is within reach), sets camera angle that the object is the field of view, and leaves room for object articulation (c.g., open door of a fridge). We perform several experiments on the valid unseen data of ALFRED with numerical results reported in Table Specifically, when the ground truth navigation actions (denoted GT navi.) are used during inference, our framework achieves an extremely high success rate (64.6). However, the performance drops significantly per- turbations (random displacement is adding 1 to the coordinates) are added to the target (z, v, h) of each navigation subgoal, verifying our claim about the need for affordance-awarc navigation.
Valid Unseen (%)
Success Rate
Goal Cond.
HLSM Blukis al. (2021)
11.8
24.7
GT navi. Obj. Int. Transformer
64.6
74.2
GT navi. Obj. Int. Transformer + rand. horizon
21.7
35.9
GT navi. Obj. Int. Transformer + rand. displacement
47.3
65.4
Table Generalization performance our framework with ground truth navigation (denoted GT navi.) inserted together with different types of pose perturbations. For reference, we also include results from HLSM Blukis et al. (2021), the previous state-of-theaat method on ALFRED.
5.2 MAIN RESULTS
We present the main results our proposed method, evaluated on the test unseen and valid unseen data of ALFRED. Our framework uses ground truth or metadata about the environment during inference. As introduced Section 4.1, our framework performs exploration to acquire knowledge about the indoor scenes and then predict actions in hierarchical approach with both rule-based planner and few learning-base based modules. We set new state-of-the-ar performance with a sub- stantial improvement (>40%) over previously published methods (see Table 2). See additional qualitative results in Appendix.
Test Unseen (%)
Valid Unseen (%)
Success rate
Goal Cond.
Success rate
Goal Cond.
EmBERT Suglia al. (2021)
7.52
16.33
5.73
15.91
E.T.+ Pashevich et al. (2021)
8.57
18.56
7.32
20.87
LWIT Nguyen et al. (2021)
9.42
20.91
9.70
23.10
HiTUT Zhang & Chai (2021)
13.87
20.31
12.44
23.71
ABP Kim et al. (2021)
15.43
24.76
HLSM Blukis et al. (2021)
16.29
27.24
11.80
24.70
AMSLAM (ours)
23.48
34.64
17.68
33.96
Table 2: Performance valid & test unseen data from ALFRED. Our method achieves new -fate-of-the-an results with a substantial improvement over previously published methods.
TTe code reproduce all results in this work will be made publicly availahle.
8

---

Under review as a conference paper at ICLR 2022
While AI2THOR adopts a discrete action/state space for the agent, believe the idea behind our multimodal exploration design and waypoint-base based semantic representation is applicable to generic embodied tasks. In the presence of motion noise and pose sensor noise, Active Neural SLAM shows that such inaccuracy can be solved to an acceptable level by simply learning a pose estimator.
5.3 ABLATION STUDIES
We perform ablation studies to justify our framework design. In Ablation Studies we examine the components in our multimodal exploration module. We choose four variants. In variant AMSLAM rand. exploration, replace the multimodal exploration by random exploration strategy similar to HISMM In variant AMSLAM - lang., we do not use the language instructions at all to guide the exploration process In variant AMSLAM lang. (partial), we only use language instructions associated with the navigation subgoal (i.e.. without the next object interaction subgoal) to guide the exploration. In variant AMSLAM explored area, we do not use the extra modality in the exploration module. For all variants, the exploration phase ends when pre-defined upper limit of action fails or exploration steps reached. In Table 3. we compare the valid unseen performance on ALFRED, the coverage, defined as the number of distinct (ar, y) pairs visited during the exploration phase (average per task), and coverage efficiency (Cov. Eff.), defined as the coverage divided by total number of (un-augmented) exploration steps. More details are available in the Appendix.
In Ablation Studies II, we examine the components in the affordance-awart map representation and in the online planner. We justify our approach of handling small (denoted instance mask waypoints) and large objects (denoted as object map waypoints) in different ways by showing that the generalization performance degrades drastically when adopting only strategy for waypoint generation. We show that the backing up rule in the planner (to leave room for articulated objects) important. Moreover, perturbed for all subgoals to justify our online exploration (and backtracking) strategy for finding the best h. See numerical results in Table 4 and implementation details in Appendix.
evaluate variant (AMSLAM rand. horizon) where horizon h is
Valid Unseen (%)
Coverage Analysis
Success Rate
Goal Cond.
Coverage
Cov. Eff. (%)
AMSLAM + rand. exploration
9.73
22.10
20.40
58.39
AMSLAM lang.
4.30
15.90
9.10
43.50
AMSLAM lang. (partial)
13.66
28.93
24.37
66.05
AMSLAM explored area
15.17
30.90
27.13
65.48
AMSLAM (ours)
17.68
33.96
28.70
67.09
Table 3: Ablation Studies the multimodal exploration module in AMSLAM.
Valid Unseen (%)
Success Rate
Goal Cond.
AMSLAM object map waypoints
12.10
26.73
AMSLAM instance mask waypoints
8.90
18.70
AMSLAM back up steps
12.50
28.85
AMSLAM + rand. horizon
9.81
20.60
AMSLAM (ours)
17.68
33.96
Table 4: Ablation Studies II: affordance- semantic representation & planner in AMSLAM.
6 CONCLUSION
This work presents comprehensive empirical results that substantiate the importance of affordance- aware navigation for language-guided task completion. We propose Affordance-aware Multimodal Neural SLAM (AMSLAM) that constructs an accurate affordance-awaae semantic representation and collects data efficiently through novel multimodal exploration module. We conduct thorough ablation studies to demonstrate that the various aspects of our design choices is essential to the per- formance. Last but not least, AMSLAM achieves more than 40% improvement over prior published work on the ALFRED benchmark and sets a new state-of-the-as generalization performance at a success rate of 23.48% test unseen scenes.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton Van Den Hengel. Vision-and-langee ann-lane navigation: Interpreting visually-groune navigation instructions in real environments. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, pp. 3674-3683, 2018
Mare Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton, and Remi Munos. Unifying count-based exploration and intrinsic motivation. Advances in neural information pro- cessing systems, 29:1471-1479, 2016.
Valts Blukis, Chris Paxton, Dieter Fox, Animesh Garg, and Yoav Artzi. A persistent spatial semantic representation for high-level natural language instruction execution. arXiv preprint arXiv:2107.05612, 2021.
Yuri Burda, Harri Edwards, Deepak Pathak, Amos Storkey, Trevor Darrell, and Alexei A Efros. Large-scale study of curiosity-driven learning. arXiv preprint arXiv: 04335, 2018.
Devendra Singh Chaplot, Dhiraj Gandhi, Saurabh Gupta, Abhinav Gupta, and Ruslan Salakhutdinov. Learning explore using active neural slam. arXiv preprint 0rXiv:2004.05755, 2020a.
Devendra Singh Chaplot, Dhiraj Prakashchand Gandhi, Abhinav Gupta, and Russ R Salakhutdinov. Object goal navigation using goal-oriented semantic exploration. Advances in Neural Information Processing Systems, 33, 2020b.
Devendra Singh Chaplot, Ruslan Salakhutdinov, Abhinav Gupta, and Saurabh Gupta. Neural topo- logical slam for visual navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 12875-12884, 2020c.
Devendra Singh Chaplot, Deepak Pathak, and Jitendra Malik. Differentiable spatial planning using transformers. In International Conference Machine Learning, pp. 1495. PMLR, 2021.
Howard Chen, Alane Suhr, Dipendra Misra, Noah Snavely, and Yoav Artzi. Touchdown: Natural language navigation and spatial reasoning in visual street environments. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12538-12547, 2019.
Tao Chen, Saurabh Gupta, and Abhinav Gupta. Leanning exploration policies for navigation. In International Conference Learning Representations, 2018.
Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra. Embod- ied question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-10, 2018.
Chuang Gan, Jeremy Schwartz, Seth Alter, Martin Schrimpf, James Traer, Julian De Freitas, Jonas Kubilius, Abhishek Bhandwaldar, Nick Haber, Megumi Sano, et al. Threedworld: A platform for interactive
physical simulation. arXiv preprint arXiv:2007. 04994. 2020.
Brent Harrison, Upol Ehsan, and Mark O Riedl. Guiding reinforcement learning exploration using natural language. In Proceedings of the 17th International Conference Autonomous Agents and MultiAgeni Systems, pp. 1956-1958, 2018
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference computer vision, pp. 2961-2969, 2017.
Max Jaderberg, Karen Simonyan, Andrew Zisserman, Spatial transformer networks. Advances in neural information processing systems, 28:2017-2025, 2015.
Dinesh Jayaraman and Kristen Grauman. Learning look around: Intelligently exploring unseen environments for unknown tasks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1238-1247, 2018.
Byeonghwi Kim, Suvaansh Bhambri, Kunal Pratap Singh, Roozbeh Mottaghi, and Jonghyun Choi. Agent with the big picture: Perceiving surroundings for interactive instruction following. In Embodied Al Workshop CVPR, 2021.
10

---

Under review as a conference paper at ICLR 2022
David Inkyu Kim and Gaurav S Sukhatme. Interactive affordance map building for robotic task. In 2015 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4581- 4586. IEEE, 2015.
Mate Kisantal, Zbigniew Wojna, Jakub Murawski, Jacek Naruniec, and Kyunghyun Cho. Augmen- tation for small object detection. arXiv preprint urXiv:1902.07290 2019.
Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Herrasti, Daniel Gordon, Yuke Zhu, Abhinav Gupta, and Ali Farhadi. Ai2-thor: An interactive 3d environment for visual ai. arXiv preprint arXiv:1712.05477, 2017.
Chengshu Li, Fei Xia, Roberto Martin-Martin, Michael Lingelbach, Sanjana Srivastava, Bokui Shen, Kent Vainio, Cem Gokmen, Gokul Dharan, Tanish Jain, et al. Igibson 2.0: Object-centric simula- tion for robot learning of everyday household tasks. arXiv preprint arXiv:2 03220 2021.
Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. Vilbert: Pretraining aank-sgi visiolin- guistic representations for vision- vision-and-land and-language tasks. arXiv preprint 1908 2019.
Tushar Nagarajan and Kristen Grauman. Learning affordance landscapes for interaction exploration in 3d environments. Advances in Neural Information Processing Systems, 33:2005-2015, 2020.
Van-Quang Nguyen, Masanori
Suganuma, and Takayuki Okatani. Look wide and interpret
twice: Improving performance on interactive instruction-following tasks. arXiv preprint 005569, 2021.
Georg Ostrovski, Marc G Bellemare, Aäron Oord, and Rémi Munos. Count-based exploration with neural density models. In International conference on machine learning, pp. 2721-2730. PMLR, 2017.
Alexander Pashevich, Cordelia Schmid, and Chen Sun. Episodic transformer for vision-and- language navigation. arXiv preprint 2Xiv:2105.06453. 2021.
Deepak Pathak, Pulkit Agrawal, Alexci A Efros, and Trevor Darrell. Curiosity-driven exploration by sulf-supervised prediction. In International conference machine learning, pp. 2778-2787 PMLR, 2017.
Xavier Puig, Kevin Ra, Marko Boben, Jiaman Li, Tingwu Wang, Sanja Fidler, and Antonio Tor- ralba. Virtualhome: Simulating household activities via programs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8494-8503 8502, 2018.
William Qi, Ravi Teja Mullapudi, Saurabh Gupta, and Deva Ramanan. Learning to move with affordance maps. In International Conference on Learning Representations, 2019.
Santhosh K Ramakrishnan, Dinesh Jayaraman, and Kristen Grauman. An exploration of embodied visual exploration. International Journal of Computer Vision, 12965) 2021.
Tabish Rashid, Bei Peng, Wendelin Boehmer, and Shimon Whiteson. Optimistic exploration even with a pessimistic initialisation. arXiv preprint arXiv:2002.12174, 2020.
Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, et al. Habitat: A platform for embodied ai research. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9339 9347, 2019.
Jürgen Schmidhuber. Curious model-building control systems. In Proc. international joint confer- ence on neural networks, pp. 1458-1463, 1991.
Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10740-10749, 2020.
Kunal Pratap Singh, Suvaansh Bhambri, Byeonghwi Kim, Roozbeh Mottaghi, and Jonghyun Choi. Moca: A modular object-centric approach for interactive instruction following. arXiv preprint arXiv:2012 032008. 2020.
11

---

Under review as a conference paper at ICLR 2022
Alessandro Suglia, Qiaozi Gao, Jesse Thomason, Govind Thattai, and Gaurav Sukhatme. Embodied bert: A transformer model for embodied, language-gunn guided visual task completion. arXiv preprint 00ff::2200.049927,, 2021.
Haoran Tang, Rein Houthooft, Davis Foote, Adam Stooke, Xi Chen, Yan Duan, John Schulman, Filip De Turek, and Pieter Abbeel. # exploration: A study of count-based exploration for deep reinforcement learning. In 31st Conference on Neural Information Processing Systems (NIPS), volume 30, pp. 1-18, 2017.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017.
Danfei Xu, Ajay Mandlekar, Roberto Martín- Yuke Zhu, Silvio Savarese, and Li Fei-Fei. Deep affordance foresight: Planning through what can be done in the future. arXiv preprint 1rXiv:2011.08424, 2020.
Yichi Zhang and Joyce Chai. Hierarchical task learning from language instructions with unified transformers and self ceff-monitoring. arXiv preprint Stti::2000.0342277 2021.
A VALIDATION OF OBJECT INTERACTION TRANSFORMER
To begin with, we first validate the hierarchical approach (i.e., subgoal level task execution) adopted by our method. We takes hierarchical approach where each subgoal is executed sequentially by either Dijkstra-based planner (for navigation subgoals) or the object interaction transformer (oth- erwise). In this section, we justify the use of the object interaction transformer, which is a cross- modal transformer trained at the subgoal level. In specific, we pre-process the training data from the original training fold of ALFRED such that each trajectory contains inputs (low-level language instructions and visual observations) and ground truth actions for single object interaction subgoal. We evaluate our proposed module on the valid unseen split of ALFRED. Compared to E.T.+, which adopts cross modal transformer trained in full task level and with extra synthesized data, the object interaction transformer generalize better on nearly all subgoals, as reported in Table 5.
Valid Unseen (%)
Toggle
Pickup
Cool
Put
Heat
Clean
Slice
E.T.+ Pashevich et al. (2021)
83.2
69.0
99.1
69.6
99.3
91.2
65.8
Obj. Int. Transformer (ours)
86.1
72.0
100.0
75.3
98.5
91.7
72.1
Table 5: Success rates (%) of all 7 object interaction subgoals evaluated valid unseen data in ALFRED. Overall, our method (object interaction transformer) generalizes better than E.T.+.
B AFFORDANCE-WARE SEMANTIC REPRESENTATION
B. NEURAL SLAM MODULE
CNN Architecture The CNN used for predicting the 2D grid map of size G x G (Ntarg (with grid size G = 37 and unit length 0. 0.25m) has the following architecture. The inputs are 3 images, which are first processed by the ResNet-50 Faster RCNN feature extractor (we use the checkpoint provided by E.T.) Then the three 512-d features are each processed by the same 4 layer CNN (filter size 3, stride size 2, number of features as 256, 256 256, 256). The flattened and con- catenated features from the previous step are then fed into a 4-layer FC network (number of features are 512, 512, 512, 128 7- 10). Next the output features are reshaped into a 128 x 7 X 10 tensor. Finally another 3-layer CNN (filter size 3, stride size 1, number of features 256. 256, Niarge + 1) takes in the tensor and output the 7 x 10 x (Narge + 1) map, which represents the prediction of small region in front of the agent (in its egocentric view).
12

---

Under review as a conference paper at ICLR 2022
Aggregation The 2D grid map predictions at each exploration step are aggregated via the use of a spatial transformer. The parameters to the spatial transformer are computed in the same way as the Neural Action SLAM (since know exactly what each exploration action is and thus where the agent was headed). Each predicted 21D grip map is first rotated and translated by the spatial transformer (with the initial location considered the center of the 21) map and the initial rotation angle considered as the direction facing upward) and then max-pooled to form the complete semantic map of the environment. During this process, since each region at single step has a limited field of view, we mask the prediction at each step by hard-coded 7 x 10 binary map (starting from the row the agent is standing at towards another 9 rows facing forward). We have tried multiple combinations for the shape of the binary map, with height E {5, 6, 7] and width E {9, 10, 11j. We choose 7 x 10 using valid unseen data in ALFRED by the average prediction accuracy.
Model Training We train the model by minimizing the cross-entropy distance between the ground truth semantic map and the predicted one. The training is performed for each single step map prediction. The ground truth for the navigable area is generated by using the API from A12Thor. The ground truth for the object map is not available for scenes in ALFRED, which uses a version of A12Thor that does not support bounding box information for general objects in the scenes. However, later A12Thor versions support such functionality. There are some scene layout mismatches between later versions of AI2Thor and the version used by ALFRED, though. We solve this by mamually inspecting all the 108 training scenes and fixing bugs by hand. We will release the code as well as the processed training data for our Neural SLAM module. We use the Adam optimizer to train the CNN model with an initial learning rate of 0.005 and a linear decaying schedule (starting from the second half of the training) for a total of 10 epochs. We find the best checkpoint using the prediction accuracy evaluated on the valid seen and valid unseen data of ALFRED.
Post-processing To have more robust navigable area for long-horizon planning, we further apply some post-processing steps to the aggregated navigable map by the Neural SLAM module (i.e., the last dimension of the 2D grid map). Specifically, we consider map A as the binary navigable area map where only the predicted confidence greater than 0.95 will be considered as a valid prediction for a navigable point (i.e.. a value of 1). We also consider map B as the binary map where location with greater or equal to 3 nearby points (i.e., the one whose LI distance to it is 1) being navigable (here we use confidence threshold 0.5) is marked as a navigable point. We then perform an element- wise product of the two maps to obtain the final navigable area.
B.2 WAYPOINT GENERATION
Small and Large Objects As mentioned in the paper, we handle small and large objects differ- ently when designing our affordance-awart semantic representation. In specific, we consider large object types in the following list and otherwise small object types:
armchair, chair, cart, sofa, shelf, drawer, cabinet, countertop, sink, stove burner fridge, bed, dresser, toilet, bathtub, ottoman, diningtable, sidetable, coffeetable, desk
The Back up Steps The benefit of handling the large objects in a 2 step process is that we can compute their waypoints in a more flexible manner. In our framework, we find backing up a few steps to be a very effective strategy, which leaves some margin between the target position of the agent and the target object the agent needs interact with This strategy is particularly critical for articulated objects. We use simple heuristics, denoted ô and 89 as introduced in the paper. Specifi- cally, the agent will back up steps if the target object fridge. 2 steps if is a safe, a cabinet or a drawer, and step for everything else. The implementation of the 8_ (., is simply an integer 1, or for either x or y coordinate given the different rotation angles E (09, 90" 180 270").
B.3 LIMITATIONS
The existing framework regarding the affordance-aware semantic representation can be further im- proved by:
Introducing an instance- representation such that multiple instances of the same object type can be handled better.
13

---

Under review as a conference paper at ICLR 2022
Figure 4: An illustration of the single-step explored area for four exploration steps (left) and the aggregated one (right). The actual size of a single-step explored area is 5 x 3 instead of 3 x 2 shown here.
Training with more room layouts to prevent overfitting as currently there are only 108 scenes in the training data of ALFRED.
C MULTIMODAL EXPLORATION
C.I MODEL TRAINING
Explored Area The extra modality introduced in our multimodal explo- ration module essentially tracks the action history explicitly and geometrically. Specifically, during each step in the exploration phase, we hard-code a 3 binary mask to indicate where the agent has observed in the current egocentric view. We find the exact shape of this region does not matter much (we have tried 3x2, 4x2, 5x4) as long as helps to track where the agent has visited. As the agent always stands at the center of the explored region map, we set the binary mask as starting from the center row and extending facing forward towards another 4 rows. Then we merge these single-step explored area by using the spatial transformer and max-pooling, the same way when we aggregate the 2D object map in our semantic representation. We finally mark the center of the aggregated explored area as 2, indicating where the agent is standing An illustration is shown in Figure (for simplicity, we only draw a x 2 binary mask), where four single-step explored area maps are aggregated into one. The CNN used to process the explored area is of 4 layers (filter size 3, stride size 2, number of features as 128, 128, 64, 32). The output of the CNN is flattened and fed into 2-layer FC network (number of features are 128, 32). Then we concatenate its output with the output from the multimodal transformer (which extract features for the other 3 modalities) and feed it into the final 3-layer FC network to predict the next exploration action (number of features are 256, 128, Neap + 1 where Neap 3 and the extra 1 is for the Stop action).
The Extra Modality:
Training Data We regenerate the trajectories from the training data of ALFRED by ignoring all object interaction actions and LookUp/ Down. We train the exploration module by imitation learn- ing on this new training set. Each sample in the new training set corresponds to one navigation subgoal in a trajectory of the original training set. As each trajectory the training data AL- FRED starts with initial horizon of 30,, all visual observations used for training of such horizon (i.e., vertical camera angle).
Hyper- pareer-prrametes We train the exploration module by Adam optimizer with an initial learning rate of 0.001 and linear decaying schedule (kicking in only for the second half of the training) to 0 for a total of 20 epochs. We find the hest checkpoint using the coverage and coverage efficiency computed the valid seen and valid unseen data of ALFRED.
C.2 ACTION AUGMENTATION
Zigzagging As introduced in the paper, we inject two LockUp or two LookDown actions al- ternately after every exploration action so that acquire images of 3 different horizons at each (x, y. r) observed during exploration. We choose this zigzagging scheme as it is the most efficient way to perform exploration in the vertical camera angle space. An example is listed below with the action sequence (after the periodic injection of Rozated RozateRight) shown in the first line and the zigzagged sequence shown in the second line.
14

---

Under review as a conference paper at ICLR 2022
Move, Right, Move, Right, Right, Right, Right, Move, Left
Down, Up. Up. Move, Down, Down, Right, Up, Up. Move, Down, Down, Right, Up, Up, Right, Down, Down, Right, Up, Up. Right, Down, Down, Move, Up, Up. Left, Down, Down
The injected actions are bolded, with the first LookDown inserted to handle the beginning of the exploration.
Other Details Since each augmented trajectory (i.e., the sequence injected with RotateRi tiittiiight and LookÜp/ (rokkp/Doww) is strict superset of the original unaugmented ones predicted directly via the multimodal exploration module, these injection does not interfere with the normal inference pipeline of the exploration module. Specifically, we mask the imputs (observations, actions history, and the explored area) corresponding to the injected actions in all of the 4 branches.
D OTHER MODULES OR MODELS
D.1 SUBGOAL PARSER AND TARGET OBJECT PARSER
Network Architecture We use a transformer-buc based architecture similar to the multimodal trans- former used for exploration. Since the input to both the subgoal parser and the target object parser the language instruction, we mask out the inputs to the other branches (i.e., we use uni-modal transformers). The two models share the same architecture while not sharing the weights. The sub- goal parser essentially binary classifier. The target object parser, on the other hand, predicts two pieces of information, namely the target object for the navigation subgoal and its container (if it has one). This prediction involves two steps; we, therefore, model it in an auto uuto-regressive manner.
Model Training We train the subgoal parser and the target object parser by minimizing the cross- entropy loss using the ground truth object information. Specifically, we use APIs provided by A12Thor acquire spatial relationships and use
decide the container object type for each instance of the small objects (which the target object parser is required to predict). We train the 2 models by Adam optimizer with an initial learning rate of 0.001 and linear decaying schedule (kicking in in t=the second half of the training) to O for total of 10 epochs. We find the best checkpoint using the prediction accuracy (both for the binary prediction problem and the 2-step classification problem) evaluated on the valid seen and valid unseen data of ALFRED.
Pre-trained Mask RCNN We use a pre-trained Mask RCNN in multiple occurrences our framework. We directly the checkpoint provided by E.T., which is trained for predicting the instance and segmentation masks of objects in ALFRED.
D.2 ONLINE PLANNER
During online planning, we always keep track of the agent's current position, pose, and so on (to be more precise, the (x, r. h) tuple). Then at each navigation subgoal, we can perform path planning using Dijkstra's 's algorithm given the acquired waypoints our affordance-aware semantic representation. To decide the target horizon of the agent, we perform online exploration to cover 6 different vertical camera angles and select the one with the largest mask area for the target object type (predicted by pre-trained model, with confidence 0.8). We also add a backtracking mechanism in case that the estimated best horizon ht does not work for the subsequent object interaction subgoals. In specific, if the subsequent object interaction subgoal fails (i.e., the agent encounters action failure), we find another horizon and perform inference for the object interaction subgoal again. In total, we try times for 1t", hh 15 h² 15".
E ABLATION STUDIES
Details for Ablation Studies I For all the four variants, stop the exploration process if (1) the module predicts a Stop action, or (2) the number of action failures reaches 4, or (3) the total number of exploration steps (including the injected actions) reaches 500. In the variant AMSLAM rand.
15

---

Under review as a conference paper at ICLR 2022
Figure 5: For the task "Move watch the inside of small safe", the first row and second row corresponds to trajectories predicted by ours and E.T. Each column corresponds to a different time step with 0, 0, 12, 18, 24, 32, 35, 36 from left to right.
Figure For the task "Place two spray bottles on toilet tank.", the first row and second row corresponds to trajectories predicted by ours and E.T. Each column corresponds to a different time step with 0, 5, 10, 15, 20, 30, 35, 41 from left to right.
exploration, we adopt the following random exploration strategy. First, we make Rot Botigtiight and adjust the horizon of the agent acquire 360 and horizon view of the environment. Next, we obtain the navigable area estimated by our Neural SLAM system. Then, we randomly sample point on the boundary of navigable area and navigate that point (during which process we inject the botateRis ght and LockUp Down similar to our multimodal exploration strategy). We repeat the previous step until the aforementioned stop condition satisfied.
Details for Ablation Studies IT For the variant AMSLAM + rand. horizon, first disable the online exploration and backtracking strategy of the planner for finding the best horizon h. We then randomly choose a final horizon for each navigation subgoal one of {60", 45" -15 -30%
Remark: We will release the code as well as relevant model files soon.
F QUALITATIVE EVALUATIONS
We also illustrate qualitative results from our proposed method compared to the E.T. baseline. We display pair of trajectories predicted from both models (for ours, only show results from the execution phase) for data the valid unseen and test unseen split, respectively. By utilizing task- driven exploration to acquire the affordance- object interaction modules, our model is capable of completing long-horizon cnstruction-folloviii tasks. In the first task where it asks the agent to grab a watch and then find and store it inside a safe, E. fails to navigate to position such that it can successfully open the safe, whereas ours succeeds. the second task for moving two spray bottles from a single shelf to the same toilet tank, E.T. fails to find the second sprayer while ours can move both sprayers to the right place.
map of the scene, together with the planning and
16


Reviews:
[{"review_id": "usPN-RzIGB5", "reviewer": "Reviewer_pomV", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "strengths:\n- results are much better than state-of-the-art and the improved margins are significant.\n- the proposed technical module of learning an affordance-aware semantic representation is valid, reasonable, and somewhat novel.\n- the system designs are really solid in mixing up many well-designed heuristics and learned modules into a big system that produces very good results.\n\nweaknesses:\n- my biggest concern is that the paper presentation is not selling the framework well as an easily-readable research paper, for example, a) there are no qualitative figures showing the performance of the work, only tables with numbers are presented. It is quite important to show qualitative results and your qualitative comparisons to baselines, additionally with some analysis figures clearly proving the effectiveness of each proposed module, for readers to be better convinced where your superior performance comes from. b) the writing for Sec. 4 can be improved. The current writing is quite messy mixing up the two contributions together. Also, many important details (e.g. the architecture to learn the affordance maps and the aggregation steps) are put in supplementary. c) for the experiments, the authors spent many more words on Sec 5.1 and 5.2 than Sec 5.3 (the main results). The main results subsection is very short and does not contain analysis over the tables or show qualitative figures to analyze the results.\n- another of my major concerns is whether or not using the language instructions during the exploration stage is a reasonable and realistic setting. My opinion is that we should not assume we know the task (goal) and the instructions for subgoals beforehand. and during the exploration. Otherwise, we need different exploration steps given different tasks. Or, at least, you need to report your exploration steps as part of the task execution steps, since you need different exploration steps given any new test task. Please clarify if I'm wrong with this. But, the confusion regarding this point prevents me from recognizing the claimed technical contribution on leveraging multimodal inputs during exploration.\n- though there are some novel designs for the affordance learning part, there are previous works (Qi et al, 2019, and Nagarajan & Grauman, 2020) that have proposed the essential ideas, and I don't see too much difference other than adding some heuristics-derived waypoint definitions. Please clarify if I missed or misunderstood the main differences from previous works.\n- though I recognize that the presented system achieves amazing results and significantly refreshes the state-of-the-art, it's unclear how much performance gains come from combining many state-of-the-art architectures, say transformers or bert, and how much comes from the claimed two novel technical design points. Can you compare to baseline methods with the same architectures or report the parameter amounts for different models, to give us a sense of this?\n- one minor issue is to use \\citep instead of \\cite in latex for a more valid citation format of papers", "summary_of_the_paper": "This paper proposes a new framework that improves the state-of-the-art performance of the ALFRED benchmark (accomplishing navigation and interaction tasks given language instructions in AI2THOR environments) by 40% relatively. Two claimed main technical contributions are: 1) leveraging multimodal inputs, specifically the newly incorporated language instructions and visited area maps, during exploration, 2) proposing an affordance-aware semantic representation that marks the object locations, heights, and possible agent interaction spots. Results that are 40% relatively better than the state-of-the-art are observed and several ablation studies over key network modules are also presented.", "summary_of_the_review": "Overall, I have a mixed feeling. On one hand, the performance is really good and significant. On the other hand, the two claimed technical contributions are not stated clearly about their significance of differences than the previous methods or the validity of the design. Also, this paper is more like a system report other than a well-written research paper with clearly presented and analyzed novel techniques. Adding qualitative figures and analysis will definitely help on this front.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Tq6L2NPDwhu", "reviewer": "Reviewer_kMKr", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths**\n\n- The method achieves the state-of-the-art performance on the ALFRED benchmark, which is quite challenging.\n\n- The set of ablation experiments are informative.\n\n\n**Weaknesses**\n\n- The paper is heavily engineered for the ALFRED benchmark. I am not sure if any of these modules can be used for generic embodied tasks.\n\n- The paper makes a big deal about the \"affordance-aware\" representation in the introduction and other places in the text, but it is actually a simple heuristic that the agent backs up a few steps when it is next to an object. According to the text \"the agent will back up 3 steps if the target object is a fridge, 2 steps if it is a safe, a cabinet or a drawer, and 1 step for everything else\". These types of heuristics are not generalizable or scalable. The affordance-aware representation should be learned instead.\n\n- There is a strong assumption that \"we know the changes (∆x, ∆y, ∆r) of the agent after each action\". These types of assumptions are valid only in simulation. Several previous works have made similar assumptions. However, those approaches are suitable only for simulation. In reality, the estimates for agent pose are quite noisy. I would like to see how robust the proposed approach is against noise in the agent pose.\n\n- It is not clear if the exploration steps and the steps used for capturing panoramic views are included in the evaluations.", "summary_of_the_paper": "The paper proposes a method for solving the ALFRED task (following language instructions to perform a set of household tasks). The model has two main components: (1) Affordance-aware Semantic Representation (2) Multi-modal exploration. The former component estimates the location of the objects in the scene. The latter provides an exploration strategy by using instructions, images, previous actions and previously explored areas as input. ", "summary_of_the_review": "The paper proposes an approach which is heavily engineered for the ALFRED benchmark and I do not see a major novelty that generalizes to other embodied tasks. The method is mainly a mixture of heuristics and a set of pre-trained components that are glued together for the ALFRED tasks. However, it achieves good performance on the ALFRED benchmark, hence, my borderline rating. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "099LIraOPpL", "reviewer": "Reviewer_2ghY", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- This submission is well motivated. Training robotic assistants to follow human instructions to complete an interactive task is an important and challenging problem. \n\n- The proposed method achieves good performance, improving 40% over prior published approaches to achieve 23.48% success rate. \n\nWeaknesses:\n- The writing clarity and readability can be improved significantly. The method seems extremely complicated and is very difficult to understand. There are way too many forward and backward references in Section 4. Even after reading over the methods section twice and going over the details in the Appendix, I am unable to understand many details in the method. For example:\n    - In sec 4, what is meaning of module operating at the \"subgoal level\"? In contrast to what in the E.T. paper? Why is the explored area considered an extra modality?\n    - In sec 4.1, how does the agent predict the task described in the language instructions? How does the agent know the subgoals in each task? How does the agent switch between exploration and execution phases? What is the difference between planned and prediction actions?  \n    - In Figure 1, it is unclear where and how the semantic map is used. It is unclear where the explored area is coming from (my understanding is that the environment does not provide the explored area). The caption refers to \"affordance-aware semantic representation\" which is not shown in the figure. What is the meaning of the \"actual\" task? \n    - In Sec 4.2, what are the delta rule-based functions?\n    - In Sec 4.3, how does the agent know the which low-level instruction corresponds to the current navigation and subsequent object interaction subgoals?\n\n- The method seems to use of lot of arbitrary choices, rules and hacks. For example, \"we define a single-step explored region as a binary map where a 5×3 rectangle grid (as a hard-coded field of view)\", \"We manually inject 4 RotateRight after every 2 MoveAhead predicted by our module to acquire 360◦ views of the scenes\", \"we further inject two LookUp or two LookDown actions alternately after every exploration action\", \"The agent then goes through 6 horizons {60◦, 45◦, ..., 0◦, −15◦} and obtains the mask prediction (with confidence > 0.8)\". All these choices seem to be specific to the Alfred benchmark. I can not imagine a robot operating in the real-world in this manner. My worry is that the authors have exploited unrealistic approximations in simulation environments (such as discrete action and state space, no noise in motion and pose sensors and use of high-level interaction actions), and over-optimized the design choices specific to this benchmark, which are likely to not result in better performance in more realistic settings.\n\n- The technical contributions of this paper are unclear:\n   - The authors claim in the introduction \"we propose the first multimodal exploration module that takes language instruction as guidance and keeps track of visited regions.\" I believe the Active Neural SLAM model, which AMSLAM is based on, also keeps track of visited regions to learn exploration. Based on authors' definition, even Active Neural SLAM is multimodal as it used pose, explored area and visual observations as input. Is the addition of language instructions as input the only change to the model?\n    - The authors claim they introduce \"Affordance-aware semantic representation that estimates object positions, heights, and relative spatial relationships\". I believe the semantic representations in Chaplot et al. 2020b and Blukis et al. 2021 also estimate object positions, heights and relative spatial relationships. It is unclear what makes the proposed representation \"affordance-aware\". ", "summary_of_the_paper": "This paper presents a Neural SLAM-based approach for tackling embodied multimodal tasks in ALFRED benchmark. The approach, called Affordance-aware Multimodal Neural SLAM (AMSLAM), utilizes several modalities for exploration, predicts an affordance-aware semantic map, and plans over it at the same time. The approach achieves 40% improvement over prior published work.", "summary_of_the_review": "Overall, the motivation is good and the method achieves a good performance on a public ALFRED benchmark. However, the authors seem to have made many design choices specific to this benchmark which are likely to not result in better results on realistic tasks. The technical contributions of this paper are unclear and the writing clarity needs to be improved significantly.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a SLAM based approach for the ALFRED benchmark. The presented method, Affordance aware Multimodal Neural SLAM has two key advantages over past works: It uses a multimodal exploration strategy and it predicts an affordance aware semantic map. It also obtains a very large performance improvement over the ALFRED benchmark. The reviewers for this paper were quite impressed by the large improvements obtained by this technique. However, there were two major concerns across the reviews: (1) Are the design choices made in this paper heavily engineered towards ALFRED ? (2) Does the work make too many assumptions about the setting (unrealistic assumptions that may not really hold in more realistic environments or the real world) ? The authors have provided a detailed response and answered many questions posed to them, but the reviewers continue to have concerns about the generalizability of the proposed method. Another point of concern pointed out by a reviewer is whether it is reasonable in a realistic setting to perform exploration with a knowledge of the downstream task. This point has not really been answered satisfactorily by the authors. My takeaway is that the method presented by the authors clearly works on ALFRED. But it contains several design choices that are largely ALFRED specific and in some cases unrealistic. This provides fewer benefits to readers looking for more general insights that can be valuable across a suite of tasks. As a result of this, and in spite of the large gains, I recommend rejecting this paper.
Decision: Reject