Abstract: In this work, we analyze the performance of general deep reinforcement learning algorithms for a task-oriented language grounding problem, where language input contains multiple sub-goals and their order of execution is non-linear.  ::: We generate a simple instructional language for the GridWorld environment, that is built around three language elements (order connectors) defining the order of execution: one linear - "comma" and two non-linear - "but first", "but before". We apply one of the deep reinforcement learning baselines - Double DQN with frame stacking and ablate several extensions such as Prioritized Experience Replay and Gated-Attention architecture.  ::: Our results show that the introduction of non-linear order connectors improves the success rate on instructions with a higher number of sub-goals in 2-3 times, but it still does not exceed 20%. Also, we observe that the usage of Gated-Attention provides no competitive advantage against concatenation in this setting. Source code and experiments' results are available at this https URL

Introduction: Task-oriented language grounding refers to the process of extracting semantically meaningful representations of language by mapping it to visual elements and actions in the environment in order to perform the task specified by the instruction BIBREF0. Recent works in this paradigm focus on wide spectrum of natural language instructions' semantics: different characteristics of referents (colors, relative positions, relative sizes), multiple tasks, and multiple sub-goals BIBREF0, BIBREF1, BIBREF2, BIBREF3, BIBREF4, BIBREF5, BIBREF6. In this work, we are interested in the language input with the semantics of multiple sub-goals, focusing on the order of execution, as the natural language contains elements that may lead to the non-linear order of execution (e.g. “Take the garbage out, but first wash the dishes“). We refer to this kind of elements as non-linear order connectors in the setting of task-oriented language grounding. In particular, we want to answer: what is the performance of general deep reinforcement learning algorithms with this kind of language input? Can it successfully learn all of the training instructions? Can it generalize to an unseen number of sub-goals? To answer these questions, we generate an instruction's language for a modified GridWorld environment, where the agent needs to visit items specified in a given instruction. The language is built around three order connectors: one linear – “comma“, and two non-linear – “but first“, and “but before“, producing instructions like “Go to the red, go to the blue, but first go to the green“. In contrast to BIBREF6, where the sub-goals are separated in advance and the order is known, in this work, we specifically aim to study whether the agent can learn to determine the order of execution based on the connectors present in the language input. We apply one of the offline deep reinforcement learning baselines – Dueling DQN and examine the impact of several extensions such as Gated-Attention architecture BIBREF0 and Prioritized Experience Replay BIBREF7. First, we discover that training for both non-linear and linear order connectors at the same time improves the performance on the latter when compared to training using only linear ones. Second, we observe that the generalization to an unseen number of sub-goals using general methods of deep reinforcement learning is possible but still very limited for both linear and non-linear connectors. Third, we find that there is no advantage of using gated-attention against simple concatenation in this setting. And fourth, we observe that the usage of prioritized experience replay in this setup may be enough to achieve the training performance near to perfect.

Question: What are the baselines?
Answer: CNN, LSTM, BERT