Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learning (2020)

Abstract: Interactive Fiction (IF) games with real human-written natural language texts provide a new natural evaluation for language understanding techniques. In contrast to previous text games with mostly synthetic texts, IF games pose language understanding challenges on the human-written textual descriptions of diverse and sophisticated game worlds and language generation challenges on the action command generation from less restricted combinatorial space. We take a novel perspective of IF game solving and re-formulate it as Multi-Passage Reading Comprehension (MPRC) tasks. Our approaches utilize the context-query attention mechanisms and the structured prediction in MPRC to efficiently generate and evaluate action outputs and apply an object-centric historical observation retrieval strategy to mitigate the partial observability of the textual observations. Extensive experiments on the recent IF benchmark (Jericho) demonstrate clear advantages of our approaches achieving high winning rates and low data requirements compared to all previous approaches.

## introduction interactive systems capable of understanding natural language and responding in the form of natural language text have high potentials in various applications. in pursuit of building and evaluating such systems, we study learning agents for interactive fiction (if) games. if games are world-simulating software in which players use text commands to control the protagonist and influence the world, as illustrated in figure 1 . if gameplay agents need to simultaneously understand the game's information from a text display (observation) and generate ⇤ primary authors. 1 source code is available at: https://github.com/ xiaoxiaoguo/rcdqn . ## experiments we evaluate our proposed methods on the suite of jericho supported games. we compared to all previous baselines that include recent methods addressing the huge action space and partial observability challenges. ## conclusion we formulate the general if game playing as mprc tasks, enabling an mprc-style solution to efficiently address the key if game challenges on the huge combinatorial action space and the partial observability in a unified framework. our approaches achieved significant improvement over the previous state-of-the-art on both game scores and training data efficiency. our formulation also bridges broader nlu/rc techniques to address other critical challenges in if games for future work, e.g., common-sense reasoning, noveltydriven exploration, and multi-hop inference.