DANLI : Deliberative Agent for Following Natural Language Instructions (2022)

Abstract: Recent years have seen an increasing amount of work on embodied AI agents that can perform tasks by following human language instructions. However, most of these agents are reactive, meaning that they simply learn and imitate behaviors encountered in the training data. These reactive agents are insufficient for long-horizon complex tasks. To address this limitation, we propose a neuro-symbolic deliberative agent that, while following language instructions, proactively applies reasoning and planning based on its neural and symbolic representations acquired from past experience (e.g., natural language and egocentric vision). We show that our deliberative agent achieves greater than 70% improvement over reactive baselines on the challenging TEACh benchmark. Moreover, the underlying reasoning and planning processes, together with our modular framework, offer impressive transparency and explainability to the behaviors of the agent. This enables an in-depth understanding of the agent’s capabilities, which shed light on challenges and opportunities for future embodied agents for instruction following. The code is available at https://github.com/sled-group/DANLI.

## introduction natural language instruction following with embodied ai agents @xcite @xcite @xcite ) is a notoriously difficult problem, where an agent must interpret human language commands to perform actions in the physical world and achieve a goal. especially challenging is the hierarchical nature of everyday tasks, foot_0 which often require reasoning about subgoals and reconciling them with the world state and overall goal. however, despite recent progress, past approaches are typically reactive @xcite in their execution of actions: conditioned on the rich, multimodal inputs from the environment, they perform actions directly without using an explicit representation of the world to facilitate grounded reasoning and planning @xcite @xcite . such an approach is inefficient, as natural language instructions often omit trivial steps that a human may be assumed to already know @xcite . besides, the lack of any explicit symbolic component makes such approaches hard to interpret, especially when the agent makes errors. inspired by previous work toward deliberative agents in robotic task planning, which apply longterm action planning over known world and goal states @xcite @xcite , we introduce danli, a neuro-symbolic deliberative agent for following natural language instructions. danli combines learned symbolic representations of task subgoals and the surrounding environment with a robust symbolic planning algorithm to execute tasks. first, we build a uniquely rich semantic spatial representation (section 3.1), acquired online from the surrounding environment and language descriptions to capture symbolic information about object instances and their physical states. to capture the highest level of hierarchy in tasks, we propose a neural task monitor (section 3.2) that learns to extract symbolic information about task progress and upcoming subgoals from the dialog and action history. using these elements as a planning domain, we lastly apply an online planning algorithm (section 3.3) to plan low-level actions for subgoals in the environment, taking advantage of danli's transparent reasoning and planning pipeline to detect and recover from errors. our empirical results demonstrate that our deliberative danli agent outperforms reactive approaches with better success rates and overwhelmingly more efficient policies on the challenging task-driven embodied agents that chat (teach) benchmark @xcite . importantly, due to its interpretable symbolic representation and explicit reasoning mechanisms, our approach offers detailed insights into the agent's planning, manipulation, and navigation capabilities. this gives the agent a unique self awareness about the kind of exceptions that have occurred, and therefore makes it possible to adapt strategies to cope with exceptions and continually strengthen the system. ## problem definition the challenge of hierarchical tasks is prominent in the recent task-driven embodied agents that chat (teach) benchmark for this problem @xcite . here, language instructions are instantiated as a task-oriented dialog between the agent and a commander (who has comprehensive knowledge about the task and environment, but cannot perform any actions), with varying granularity and completeness of guidance given. we focus on the execution from dialog history (edh) setting in teach, where the agent is given a dialog history as input, and is expected to execute a sequence of actions and achieve the goal set out by the commander. this setting allows us to abstract away the problem of dialog generation, and focus on the already difficult problem of instruction following from task-oriented dialog. as shown in figure 1 , a task, e.g., make a sand-wich, may have several subtasks that the agent must achieve in order to satisfy the overall task goal. the success of a task/subtask is achieved by meeting a set of goal conditions, such as slicing a bread and toasting two pieces. at each timestep, the agent receives an egocentric visual observation of the world and the full dialog history up to that time, and may execute a single low-level action. actions can either involve navigation, e.g., to step forward, or manipulation, e.g., to pick up an object. manipulation actions additionally require the agent to identify the action's target object by specifying a pixel in its field of view to highlight the object. the execution continues until the agent predicts a stop action, otherwise the session will terminate after 1000 timesteps or 30 failed actions. at this time, we can evaluate the agent's completion of the task. it is worth noting that while we focus on teach, our approach is largely transferable between benchmark datasets and simulation environments, albeit requiring retraining of some components. ## a neuro-symbolic deliberative agent an overview of our neuro-symbolic deliberative agent is shown in figure 2 . we first introduce the symbolic notions used in our system. we use the object-oriented representation @xcite to represent the symbolic world state. each object instance is assigned an instance id consisting of its canonicalized class name and ordinal. we define a state in the form of predicate(arguments) as an object's physical state or a relation to another object. we define subgoals as particular states that the agent should achieve while completing the task, represented by a symbolic form (patient, predicate, destination) foot_1 , where the patient and destination are object classes, and the predicate is a state that can be applied to the patient. we define an action in the agent's plan as actiontype(arguments) where each argument is an object instance. to complete tasks, our agent reasons over a learned spatial-symbolic map representation (section 3.1) to generate a hierarchical plan. at the high level, it applies a neural language model to the dialog and action history to predict the complete sequence of completed and future subgoals in symbolic form (section 3.2). for each predicted subgoal, it then plans a sequence of low-level actions using both the symbolic subgoal and world representations online, with robustness to various types of planning and execution failures (section 3.3). next, we describe how each component works and highlight our key innovations. ## related work the topics most relevant to this work include: semantic scene representation, hierarchical policies, and symbolic planning. ## conclusion and discussion this paper introduced danli, a deliberative instruction following agent which outperformed reactive agents in terms of both success rate and efficiency on a challenging benchmark teach. the use of explicit symbolic representations also supports a better understanding of the problem space as well as the capabilities of the agent. despite the strong results, we are still far away from building embodied agents capable of following language instructions, and we identify three major bottlenecks that will motivate our future work. first, the capability to handle uncertainty in perception and grounding of language is essential when constructing symbolic representations for planning and reasoning. we hope to explore tighter neurosymbolic integration in order to allow better learning of symbolic representations, while still keeping the advantage of providing transparent reasoning and decision-making in agent behaviors. second, the capability to robustly handle exceptions encountered during execution is vital. humans proactively seek help if they get stuck in completing a task, while current ai agents lack this capability. integrating dialog-powered exception handling policies is our next step to extend danli. third, future work needs to improve the scalability of symbolic representations for use in novel situations. in this work, the symbol set for predicates and subgoals is predefined, which requires engineering effort and may not be directly applicable to other domains. as recent years have seen an exciting application of large language models for acquiring action plans (ahn et al., 2022), we will build upon these approaches for more flexible learning of action planning in an open-world setting in the future.