Original: Modeling Worlds in Text
Revised: Modeling Worlds in Text

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: Afﬁliation Address email
Revised: Afﬁliation Address email

Original: Abstract
Revised: Abstract

Original: We provide a dataset that enables the creation of learning agents that can build knowledge graph-based world models of interactive narratives.
Revised: We provide a dataset that enables the creation of learning agents that can build knowledge graph-based world models of interactive narratives.

Original: 1 Interactive narratives—or text-adventure games—are partially observable environments structured as long puzzles or quests in which an agent perceives and interacts with the world purely through textual natural language.
Revised: 1 Interactive narratives—or text-adventure games—are partially observable environments structured as long puzzles or quests in which an agent perceives and interacts with the world purely through textual natural language.

Original: Each individual game typically contains hundreds of locations, characters, and objects—each with their own unique descriptions—providing an opportunity to study the problem of giving language-based agents the structured memory necessary to operate in such worlds.
Revised: Each individual game typically contains hundreds of locations, characters, and objects—each with their own unique descriptions—providing an opportunity to study the problem of giving language-based agents the structured memory necessary to operate in such worlds.

Original: Our dataset provides 24198 mappings between rich natural language observations and: (1) knowledge graphs that reﬂect the world state in the form of a map; (2) natural language actions that are guaranteed to cause a change in that particular world state.
Revised: Our dataset provides 24198 mappings between rich natural language observations and: (1) knowledge graphs that reﬂect the world state in the form of a map; (2) natural language actions that are guaranteed to cause a change in that particular world state.

Original: The training data is collected across 27 games in multiple genres and contains a further 7836 heldout instances over 9 additional games in the test set.
Revised: The training data is collected across 27 games in multiple genres and contains a further 7836 heldout instances over 9 additional games in the test set.

Original: We further provide baseline models using rules-based, question-answering, and sequence learning approaches in addition to an analysis of the data and corresponding learning tasks.
Revised: We further provide baseline models using rules-based, question-answering, and sequence learning approaches in addition to an analysis of the data and corresponding learning tasks.

Original:  Introduction
Revised:  Introduction

Original: Observation: West of House You are standing in an open ﬁeld west of a white house, with a boarded front door.
Revised: Observation: West of House You are standing in an open ﬁeld west of a white house, with a boarded front door.

Original: There is a small mailbox here.
Revised: There is a small mailbox here.

Original: Action: Open mailbox
Revised: Action: Open mailbox

Original: Observation: Opening the small mailbox reveals a leaﬂet.
Revised: Observation: Opening the small mailbox reveals a leaﬂet.

Original: Action: Read leaﬂet
Revised: Action: Read leaﬂet

Original: Observation: (Taken) "WELCOME TO ZORK!
Revised: Observation: (Taken) "WELCOME TO ZORK!

Original: ZORK is a game of adventure, danger, and low cunning.
Revised: ZORK is a game of adventure, danger, and low cunning.

Original: In it you will explore some of the most amazing territory ever seen by mor- tals.
Revised: In it you will explore some of the most amazing territory ever seen by mor- tals.

Original: No computer should be without one!"
Revised: No computer should be without one!"

Original: Action: Go north
Revised: Action: Go north

Original: Observation: North of House You are facing the north side of a white house.
Revised: Observation: North of House You are facing the north side of a white house.

Original: There is no door here, and all the windows are boarded up.
Revised: There is no door here, and all the windows are boarded up.

Original: To the north a narrow path winds through the trees.
Revised: To the north a narrow path winds through the trees.

Original: We seek to create agents that exhibit human-like capabilities such as commonsense reasoning and natural language understanding in interactive and situated settings.
Revised: We seek to create agents that exhibit human-like capabilities such as commonsense reasoning and natural language understanding in interactive and situated settings.

Original: Interactive narrative environments provide a critical stepping stone in this pursuit towards creating learning agents that can produce contextually relevant and goal-driven natural language [Côté et al., 2018, Urbanek et al., 2019, Hausknecht et al., 2020].
Revised: Interactive narrative environments provide a critical stepping stone in this pursuit towards creating learning agents that can produce contextually relevant and goal-driven natural language [Côté et al., 2018, Urbanek et al., 2019, Hausknecht et al., 2020].

Original: They require agents to observe textual descriptions and then act upon the world using natural language with the aim of completing a long term goal or quest as seen in Figures 1, 2.
Revised: They require agents to observe textual descriptions and then act upon the world using natural language with the aim of completing a long term goal or quest as seen in Figures 1, 2.

Original: We focus on two of the core challenges faced by learning agents in these environments—as identiﬁed in prior work—knowledge representation and a combinatorially sized state-action space.
Revised: We focus on two of the core challenges faced by learning agents in these environments—as identiﬁed in prior work—knowledge representation and a combinatorially sized state-action space.

Original: The knowledge representation challenge rises from the fact that interactive narratives span many distinct locations, each with unique descriptions, objects, and characters as seen can be seen in Figure 2.
Revised: The knowledge representation challenge rises from the fact that interactive narratives span many distinct locations, each with unique descriptions, objects, and characters as seen can be seen in Figure 2.

Original: Players move by issuing navigational commands, which can convey Euclidean space like go West or non-Euclidean span like step into portal , warping the agent to an entirely new section of the world.
Revised: Players move by issuing navigational commands, which can convey Euclidean space like go West or non-Euclidean span like step into portal , warping the agent to an entirely new section of the world.

Original: To cope with such challenges, humans often create structured memory aids such as hand drawn maps when attempting to play these games.
Revised: To cope with such challenges, humans often create structured memory aids such as hand drawn maps when attempting to play these games.

Original: A good knowledge representation can assist with long-term action dependencies that often arise in game quests (as well as real world environments).
Revised: A good knowledge representation can assist with long-term action dependencies that often arise in game quests (as well as real world environments).

Original: An example of a long-term dependency is a key being found in one location that opens a lock on a chest in an entirely different section of the map.
Revised: An example of a long-term dependency is a key being found in one location that opens a lock on a chest in an entirely different section of the map.

Original: For an agent to learn this relationship, it must be able to replicate the sequence of picking up the key and unlocking the chest while not being distracted by interstitial actions and states.
Revised: For an agent to learn this relationship, it must be able to replicate the sequence of picking up the key and unlocking the chest while not being distracted by interstitial actions and states.

Original: Long-term action dependencies are made challenging by two aspects of interactive narrative environments, which are also present in real-world environments.
Revised: Long-term action dependencies are made challenging by two aspects of interactive narrative environments, which are also present in real-world environments.

Original: First, these environments are partially observable in the sense that an agent only has local observability.
Revised: First, these environments are partially observable in the sense that an agent only has local observability.

Original: Second, interactive narrative environments have a combinatorially-sized natural language state-action space .
Revised: Second, interactive narrative environments have a combinatorially-sized natural language state-action space .

Original: For example, in the cannonical game Zork1 an action can consist of up to ﬁve-words from a relatively modest vocabulary of 697 words, resulting in O (697 5 )
Revised: For example, in the cannonical game Zork1 an action can consist of up to ﬁve-words from a relatively modest vocabulary of 697 words, resulting in O (697 5 )

Original: = 1 .
Revised: = 1 .

Original: 64 × 10 14 possible actions at every step—though the number of valid actions that are gramatically coherent and contextually relevant is signiﬁcantly smaller.
Revised: 64 × 10 14 possible actions at every step—though the number of valid actions that are gramatically coherent and contextually relevant is signiﬁcantly smaller.

Original: This makes exploration sample-inefﬁcient, making it harder to learn the relationship between actions that are temporally distant from each other.
Revised: This makes exploration sample-inefﬁcient, making it harder to learn the relationship between actions that are temporally distant from each other.

Original: The knowledge representation challenges inherent to interactive narrative games give rise to the Textual-SLAM problem, a textual variant of Simultaneous Localization And Mapping (SLAM)
Revised: The knowledge representation challenges inherent to interactive narrative games give rise to the Textual-SLAM problem, a textual variant of Simultaneous Localization And Mapping (SLAM)

Original: [Thrun et al., 2005] problem of constructing a map by inferring information from one’s surroundings while navigating a novel environment.
Revised: [Thrun et al., 2005] problem of constructing a map by inferring information from one’s surroundings while navigating a novel environment.

Original: As in humans, the creation of such world models or memory aids in agents—in the form of knowledge graphs—has been shown to be critical in helping automated learning agents operate in these textual worlds
Revised: As in humans, the creation of such world models or memory aids in agents—in the form of knowledge graphs—has been shown to be critical in helping automated learning agents operate in these textual worlds

Original: [Ammanabrolu and Riedl, 2019,
Revised: [Ammanabrolu and Riedl, 2019,

Original: Murugesan et al., 2020, Adhikari et al., 2020, Ammanabrolu and Hausknecht, 2020].
Revised: Murugesan et al., 2020, Adhikari et al., 2020, Ammanabrolu and Hausknecht, 2020].

Original: Despite the success of knowledge graphs in addressing these problems, a broad dataset across a diverse set of games mapping text game observations to knowledge graphs does not exist—hindering progress in building of world modeling agents with structured memory.
Revised: Despite the success of knowledge graphs in addressing these problems, a broad dataset across a diverse set of games mapping text game observations to knowledge graphs does not exist—hindering progress in building of world modeling agents with structured memory.

Original: Building off the popular text game simulator Jericho [Hausknecht et al., 2020], we have constructed a dataset dubbed JerichoWorld that maps text game state observations to both the underlying ground truth knowledge graph representations of the game and the set of contextually relevant actions that can be performed in that state.
Revised: Building off the popular text game simulator Jericho [Hausknecht et al., 2020], we have constructed a dataset dubbed JerichoWorld that maps text game state observations to both the underlying ground truth knowledge graph representations of the game and the set of contextually relevant actions that can be performed in that state.

Original: Using this data, we seek to enable development of agents that focus on answering the questions of “What actions make sense for me to perform right now?”
Revised: Using this data, we seek to enable development of agents that focus on answering the questions of “What actions make sense for me to perform right now?”

Original: and “What have I already done and how will the world change now if I perform a particular action?”—questions relating to the problems natural language understanding, commonsense reasoning, and structured memory.
Revised: and “What have I already done and how will the world change now if I perform a particular action?”—questions relating to the problems natural language understanding, commonsense reasoning, and structured memory.

Original: The training set contains 24198 instances across 27 games and the heldout test set contains 7836 instances from 9 games.
Revised: The training set contains 24198 instances across 27 games and the heldout test set contains 7836 instances from 9 games.

Original: We further formally deﬁne two initial tasks for this dataset focusing on the questions mentioned: (1) Given a textual observation, predict the underlying knowledge graph of the world.
Revised: We further formally deﬁne two initial tasks for this dataset focusing on the questions mentioned: (1) Given a textual observation, predict the underlying knowledge graph of the world.

Original: (2) Given a textual observation, predict the set of actions that are contextually relevant.
Revised: (2) Given a textual observation, predict the set of actions that are contextually relevant.

Original: Results for three baselines—using rules-based, question-answering, and sequence-learning approaches—are provided in addition to an analysis of the dataset and results themselves.
Revised: Results for three baselines—using rules-based, question-answering, and sequence-learning approaches—are provided in addition to an analysis of the dataset and results themselves.

Original:  Related Work
Revised:  Related Work

Original: We constrain our related work section to three primary areas: current interactive narrative benchmarks, world modeling and model-based reinforcement learning, and the use of knowledge graphs in text games.
Revised: We constrain our related work section to three primary areas: current interactive narrative benchmarks, world modeling and model-based reinforcement learning, and the use of knowledge graphs in text games.

Original: Currently, three primary open-source platforms and baseline benchmarks have been developed so far to help measure progress in this ﬁeld:
Revised: Currently, three primary open-source platforms and baseline benchmarks have been developed so far to help measure progress in this ﬁeld:

Original: Jericho [Hausknecht et al., 2020] 2 a learning environment for human-made interactive narrative games; TextWorld [Côté et al., 2018] 3 a framework for procedural generation in text-games; and LIGHT [Urbanek et al., 2019] 4 a large-scale crowdsourced multi-user text-game for studying situated dialogue.
Revised: Jericho [Hausknecht et al., 2020] 2 a learning environment for human-made interactive narrative games; TextWorld [Côté et al., 2018] 3 a framework for procedural generation in text-games; and LIGHT [Urbanek et al., 2019] 4 a large-scale crowdsourced multi-user text-game for studying situated dialogue.

Original: Further extensions and adaptation to some of these benchmarks have been proposed for use in neighboring domains such as vision-and-language navigation [Shridhar et al., 2021], commonsense reasoning [Murugesan et al., 2021], and procedural text understanding [Tamari et al., 2021].
Revised: Further extensions and adaptation to some of these benchmarks have been proposed for use in neighboring domains such as vision-and-language navigation [Shridhar et al., 2021], commonsense reasoning [Murugesan et al., 2021], and procedural text understanding [Tamari et al., 2021].

Original: Our work builds on the Jericho environment.
Revised: Our work builds on the Jericho environment.

Original: Work on world models in learning agents have recently been inspired by theories of how humans form mental models of the world
Revised: Work on world models in learning agents have recently been inspired by theories of how humans form mental models of the world

Original: [Jancke, 2000, Ha and Schmidhuber, 2018].
Revised: [Jancke, 2000, Ha and Schmidhuber, 2018].

Original: When in the form of predictive probabilistic generative models of the world, they can be used in model-based reinforcement learning tasks [Sutton and Barto, 1998, Arulkumaran et al., 2017, Schrittwieser et al., 2019].
Revised: When in the form of predictive probabilistic generative models of the world, they can be used in model-based reinforcement learning tasks [Sutton and Barto, 1998, Arulkumaran et al., 2017, Schrittwieser et al., 2019].

Original: In such cases, a learning agent attempts to learn the underlying environment dynamics at the same time as a policy, often using information about one to inform the other.
Revised: In such cases, a learning agent attempts to learn the underlying environment dynamics at the same time as a policy, often using information about one to inform the other.

Original: Ha and Schmidhuber [2018] take this one step further by replacing a environment entirely with the agent’s own learned world model and training a control policy there.
Revised: Ha and Schmidhuber [2018] take this one step further by replacing a environment entirely with the agent’s own learned world model and training a control policy there.

Original: All of these methods have been shown to have the added beneﬁts of signiﬁcantly improving sample efﬁciency as the agent is now able to (at least partially) simulate the environment via the world model.
Revised: All of these methods have been shown to have the added beneﬁts of signiﬁcantly improving sample efﬁciency as the agent is now able to (at least partially) simulate the environment via the world model.

Original: In all of the world modeling cases mentioned, the state representations that the models are conditioned on are drawn directly from the existing base environments, e.g. raw pixel game screens in the case of the Arcade Learning Environment [Bellemare et al., 2013] or other visual games such as
Revised: In all of the world modeling cases mentioned, the state representations that the models are conditioned on are drawn directly from the existing base environments, e.g. raw pixel game screens in the case of the Arcade Learning Environment [Bellemare et al., 2013] or other visual games such as

Original: Sokoban [Bamford and Lucas, 2020].
Revised: Sokoban [Bamford and Lucas, 2020].

Original: In the case of human-made text games, however, knowledge graphs—not directly provided by existing text game learning frameworks—have been shown to be superior state representations when compared to just the textual observations by themselves.
Revised: In the case of human-made text games, however, knowledge graphs—not directly provided by existing text game learning frameworks—have been shown to be superior state representations when compared to just the textual observations by themselves.

Original: They aid in the challenges of partial observability/knowledge representation [Ammanabrolu and Riedl, 2019, Adhikari et al., 2020, Sautier et al., 2020], combinatorial state-action spaces [Ammanabrolu and Hausknecht, 2020, Ammanabrolu et al., 2020b], and commonsense reasoning [Ammanabrolu and Riedl, 2019, Murugesan et al., 2020, 2021, Dambekodi et al., 2020].
Revised: They aid in the challenges of partial observability/knowledge representation [Ammanabrolu and Riedl, 2019, Adhikari et al., 2020, Sautier et al., 2020], combinatorial state-action spaces [Ammanabrolu and Hausknecht, 2020, Ammanabrolu et al., 2020b], and commonsense reasoning [Ammanabrolu and Riedl, 2019, Murugesan et al., 2020, 2021, Dambekodi et al., 2020].

Original: Closest in spirit to this work is the Jericho-QA dataset [Ammanabrolu et al., 2020b], a questionanswering dataset tuned to text games that enables agents to identify common objects in the world and their attributes.
Revised: Closest in spirit to this work is the Jericho-QA dataset [Ammanabrolu et al., 2020b], a questionanswering dataset tuned to text games that enables agents to identify common objects in the world and their attributes.

Original: It does not have information regarding the full underlying knowledge graph state or valid actions, however.
Revised: It does not have information regarding the full underlying knowledge graph state or valid actions, however.

Original: As far as we know, ground truth knowledge graph state representation dataset across a diverse set of human-made text games is not currently available in any of the primary text game benchmarks mentioned previously, hindering the ability to create agents with structured memory in the form of graph-based world models.
Revised: As far as we know, ground truth knowledge graph state representation dataset across a diverse set of human-made text games is not currently available in any of the primary text game benchmarks mentioned previously, hindering the ability to create agents with structured memory in the form of graph-based world models.

Original:  JerichoWorld
Revised:  JerichoWorld

Original: Côté et al.
Revised: Côté et al.

Original: [2018] and Hausknecht et al.
Revised: [2018] and Hausknecht et al.

Original: [2020] deﬁne text games as Partially-Observable Markov
Revised: [2020] deﬁne text games as Partially-Observable Markov

Original: Decision Processes.
Revised: Decision Processes.

Original: A game can be represented as a 7-tuple of (cid:104) S, T, A, Ω , O, R, γ (cid:105) representing the set of environment states, mostly deterministic conditional transition probabilities between states , the vocabulary or words used to compose text commands, observations returned by the game, observation conditional probabilities, reward function, and the discount factor respectively.
Revised: A game can be represented as a 7-tuple of (cid:104) S, T, A, Ω , O, R, γ (cid:105) representing the set of environment states, mostly deterministic conditional transition probabilities between states , the vocabulary or words used to compose text commands, observations returned by the game, observation conditional probabilities, reward function, and the discount factor respectively.

Original: Drawing from this deﬁnition, each instance of our dataset takes the tuples of (cid:104) s t , a t , s t +1 , r t +1 (cid:105) where s t and s t +1 are two subsequent states with a t being the action used to transition states and r t
Revised: Drawing from this deﬁnition, each instance of our dataset takes the tuples of (cid:104) s t , a t , s t +1 , r t +1 (cid:105) where s t and s t +1 are two subsequent states with a t being the action used to transition states and r t

Original: +1 is the observed reward for some step t .
Revised: +1 is the observed reward for some step t .

Original: To collect the (cid:104) s t , a t , s t +1 , r t +1 (cid:105) tuples
Revised: To collect the (cid:104) s t , a t , s t +1 , r t +1 (cid:105) tuples

Original: we implement a basic agent that explores the game along a trajectory corresponding to a game walkthrough .
Revised: we implement a basic agent that explores the game along a trajectory corresponding to a game walkthrough .

Original: Game walkthroughs are texts describing the solutions to games, generally retrieved from the internet, but already part of the Jericho framework.
Revised: Game walkthroughs are texts describing the solutions to games, generally retrieved from the internet, but already part of the Jericho framework.

Original: Walkthroughs, however, only present one possible solution to a game and solve all the core puzzles required to complete a game with the maximum possible score.
Revised: Walkthroughs, however, only present one possible solution to a game and solve all the core puzzles required to complete a game with the maximum possible score.

Original: To achieve greater coverage of the game’s state space, our data collection agent stops off to explore by executing random valid actions for n steps before resetting to the walkthrough.
Revised: To achieve greater coverage of the game’s state space, our data collection agent stops off to explore by executing random valid actions for n steps before resetting to the walkthrough.

Original: One such collected state—a part of the full tuple mentioned—is detailed below
Revised: One such collected state—a part of the full tuple mentioned—is detailed below

Original: The textual observations consist of descriptions of the location and inventory as well as the game engine response to the previous action performed.
Revised: The textual observations consist of descriptions of the location and inventory as well as the game engine response to the previous action performed.

Original: For example:
Revised: For example:

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Inventory:
Revised: Inventory:

Original: [Table]
Revised: [Table]

Original: carrying: a brass lantern (providing light) a pair of glasses four candy bars: a ZM$ a Multi-Implementeers a Forever Gores a Baby Rune a cheaply-made sword Prev Act: put on gloves
Revised: carrying: a brass lantern (providing light) a pair of glasses four candy bars: a ZM$ a Multi-Implementeers a Forever Gores a Baby Rune a cheaply-made sword Prev Act: put on gloves

Original: We further provide the set of objects that are found in both the agent’s inventory and surroundings, including textual descriptions for each of the objects.
Revised: We further provide the set of objects that are found in both the agent’s inventory and surroundings, including textual descriptions for each of the objects.

Original: Attributes for each of these objects are also included are acquired by decompiling the games, following [Ammanabrolu et al., 2020b].
Revised: Attributes for each of these objects are also included are acquired by decompiling the games, following [Ammanabrolu et al., 2020b].

Original: For example:
Revised: For example:

Original: Inventory Objects: candy: Which do you mean, the ZM$100000, the Multi Implementeers, the Forever Gores or the Baby Rune?
Revised: Inventory Objects: candy: Which do you mean, the ZM$100000, the Multi Implementeers, the Forever Gores or the Baby Rune?

Original: Implementeers: The profiles on the wrapper of this delicacy look more like Moe, Larry, and Curly than those of your favorite Implementeers (presumably, Marc, Mike, and David.)
Revised: Implementeers: The profiles on the wrapper of this delicacy look more like Moe, Larry, and Curly than those of your favorite Implementeers (presumably, Marc, Mike, and David.)

Original: Forever/Gores: The wrapper of this bar pictures the Milky Way, but the stars are all blood red.
Revised: Forever/Gores: The wrapper of this bar pictures the Milky Way, but the stars are all blood red.

Original: Kids love them.
Revised: Kids love them.

Original: sword: This is a cheaply made sword of no antiquity whatsoever.
Revised: sword: This is a cheaply made sword of no antiquity whatsoever.

Original: With regard to grues or other underworldly denizens, your weapon is as likely to engender laughter as fear.
Revised: With regard to grues or other underworldly denizens, your weapon is as likely to engender laughter as fear.

Original: rune: The label is covered with mystical runes, the meanings of which elude you.
Revised: rune: The label is covered with mystical runes, the meanings of which elude you.

Original: glasses: The owner of these glasses had an indeterminate vision problem, because the lenses have both been crushed underfoot.
Revised: glasses: The owner of these glasses had an indeterminate vision problem, because the lenses have both been crushed underfoot.

Original: The vision problem, of course, has been solved.
Revised: The vision problem, of course, has been solved.

Original: lantern:
Revised: lantern:

Original: The lantern, while of the cheapest construction, appears functional enough for the moment.
Revised: The lantern, while of the cheapest construction, appears functional enough for the moment.

Original: Your best hope is that it stays that way.
Revised: Your best hope is that it stays that way.

Original: It looks like the lamp has gone through a few cycles of impact revitalization.
Revised: It looks like the lamp has gone through a few cycles of impact revitalization.

Original: Inventory Attributes: glasses: clothing gloves: clothing sword: animate, equip lantern: animate, equip Surrounding Objects: gargoyles: Unless you are inordinately masochistic, the less time spent examining the artwork, the better.
Revised: Inventory Attributes: glasses: clothing gloves: clothing sword: animate, equip lantern: animate, equip Surrounding Objects: gargoyles: Unless you are inordinately masochistic, the less time spent examining the artwork, the better.

Original: east: You see nothing special about the east wall.
Revised: east: You see nothing special about the east wall.

Original: tunnel: The tunnel leads west.
Revised: tunnel: The tunnel leads west.

Original: gloves: The razor like gloves would be very attractive for an axe murderer.
Revised: gloves: The razor like gloves would be very attractive for an axe murderer.

Original: And they’re just your size.
Revised: And they’re just your size.

Original: south: You see nothing special about the south wall.
Revised: south: You see nothing special about the south wall.

Original: sign: The sign indicates today’s performance, which (in honor of the festivities in the Convention Center) is "A Massacre on 34th Street."
Revised: sign: The sign indicates today’s performance, which (in honor of the festivities in the Convention Center) is "A Massacre on 34th Street."

Original: Surrounding Attributes: gloves: clothing tunnel: animate sign: animate
Revised: Surrounding Attributes: gloves: clothing tunnel: animate sign: animate

Original: We further provide the ground truth knowledge graph representing the world state corresponding to these textual observations.
Revised: We further provide the ground truth knowledge graph representing the world state corresponding to these textual observations.

Original: The ground truth knowledge graph is a set of tuples (cid:104) s, r, o (cid:105) such that s is a subject, r is a relation, and o is an object.
Revised: The ground truth knowledge graph is a set of tuples (cid:104) s, r, o (cid:105) such that s is a subject, r is a relation, and o is an object.

Original: It reﬂects information on the current state such as objects and attributes and is extracted from the game engine by traversing the engine’s internal representation and converting it to human readable form.
Revised: It reﬂects information on the current state such as objects and attributes and is extracted from the game engine by traversing the engine’s internal representation and converting it to human readable form.

Original: Relations are deﬁned on the basis of traversal operations in the game engine’s internal representation, e
Revised: Relations are deﬁned on the basis of traversal operations in the game engine’s internal representation, e

Original: [Table]
Revised: [Table]

Original: g. “in” and “have” signify parent-child ownership for locations and inventory respectively.
Revised: g. “in” and “have” signify parent-child ownership for locations and inventory respectively.

Original: For example:
Revised: For example:

Original: [Table]
Revised: [Table]

Original: Valid actions are deﬁned by Hausknecht et al.
Revised: Valid actions are deﬁned by Hausknecht et al.

Original: [2020] as the set of actions guaranteed to cause a change in the current world state and are identiﬁed by the Jericho framework.
Revised: [2020] as the set of actions guaranteed to cause a change in the current world state and are identiﬁed by the Jericho framework.

Original: For example in one particular state me might have the following valid actions:
Revised: For example in one particular state me might have the following valid actions:

Original: [Table]
Revised: [Table]

Original: 3.1 Dataset Analysis
Revised: 3.1 Dataset Analysis

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 52 tryst
Revised: 52 tryst

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 38 enter
Revised: 38 enter

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: omniquest
Revised: omniquest

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 30 zork
Revised: 30 zork

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 26 zork
Revised: 26 zork

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 73 inhumane
Revised: 73 inhumane

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 24 loose
Revised: 24 loose

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 00 murdac
Revised: 00 murdac

Original: [Table]
Revised: [Table]

Original: moonlit
Revised: moonlit

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: 61 dragon
Revised: 61 dragon

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: jewel
Revised: jewel

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: weapon
Revised: weapon

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 68 karn
Revised: 68 karn

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 44 zenon
Revised: 44 zenon

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 95 acorncourt
Revised: 95 acorncourt

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 08 ballyhoo
Revised: 08 ballyhoo

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 11 yomomma
Revised: 11 yomomma

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 52 enchanter
Revised: 52 enchanter

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 40 gold
Revised: 40 gold

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 92 huntdark  162.
Revised: 92 huntdark  162.

Original: [Table]
Revised: [Table]

Original: Table 1 presents statistics for our data in the form of showing vocab sizes, and average lengths of different data ﬁelds.
Revised: Table 1 presents statistics for our data in the form of showing vocab sizes, and average lengths of different data ﬁelds.

Original: The dataset as a whole has an input vocabulary size of 11056 —this is the superset of the vocabulary that can be used to act in any of these games.
Revised: The dataset as a whole has an input vocabulary size of 11056 —this is the superset of the vocabulary that can be used to act in any of these games.

Original: It is worth noting, that the output vocabulary size—determined by the observations—is not restricted.
Revised: It is worth noting, that the output vocabulary size—determined by the observations—is not restricted.

Original: As seen later when we introduce models for these tasks, this means
Revised: As seen later when we introduce models for these tasks, this means

Original: that subword based tokenization
Revised: that subword based tokenization

Original: [Kudo, 2018] for processing inputs is the most effective way of avoiding unknown tokens.
Revised: [Kudo, 2018] for processing inputs is the most effective way of avoiding unknown tokens.

Original: The training and testing games both cover a wide range of genres as noted by Hausknecht et al.
Revised: The training and testing games both cover a wide range of genres as noted by Hausknecht et al.

Original: Ammanabrolu et al.
Revised: Ammanabrolu et al.

Original: [2020b]—e.g. 905 is a everyday slice-of-life simulator in which a character walks around a house preparing for work, afﬂicted is a monster horror game, ballyhoo, detective are murder mysteries, and karn, zork1 are traditional fantasies.
Revised: [2020b]—e.g. 905 is a everyday slice-of-life simulator in which a character walks around a house preparing for work, afﬂicted is a monster horror game, ballyhoo, detective are murder mysteries, and karn, zork1 are traditional fantasies.

Original: On average, the observation token, graph, and valid action lengths are comparable across both the training and testing games.
Revised: On average, the observation token, graph, and valid action lengths are comparable across both the training and testing games.

Original: Outliers in these metrics usually represent game-speciﬁc challenges.
Revised: Outliers in these metrics usually represent game-speciﬁc challenges.

Original: For example, acorncourt has the highest observation token and graph length counts by far.
Revised: For example, acorncourt has the highest observation token and graph length counts by far.

Original: This is because the game is focused heavily on object collection and so contains more entities on average than others.
Revised: This is because the game is focused heavily on object collection and so contains more entities on average than others.

Original: In a similar vein, enchanter has signiﬁcantly more valid actions than other games.
Revised: In a similar vein, enchanter has signiﬁcantly more valid actions than other games.

Original: This is due to the game being focused on constantly discovering valid actions in the form of spells and their effects by casting them—everything from healing yourself to causing an object to give off light.
Revised: This is due to the game being focused on constantly discovering valid actions in the form of spells and their effects by casting them—everything from healing yourself to causing an object to give off light.

Original: It is worth noting that many of these spells appear and have similar effects in other fantasy text games.
Revised: It is worth noting that many of these spells appear and have similar effects in other fantasy text games.

Original: These are some examples of challenges that players must overcome to be successful in these worlds.
Revised: These are some examples of challenges that players must overcome to be successful in these worlds.

Original:  Benchmarks
Revised:  Benchmarks

Original: This section introduces the two primary tasks using JerichoWorld required for world modeling in learning agents: knowledge graph prediction and valid action prediction .
Revised: This section introduces the two primary tasks using JerichoWorld required for world modeling in learning agents: knowledge graph prediction and valid action prediction .

Original: We then introduce baseline models for each of the tasks, report zero-shot results on the testing games, and analyze performance.
Revised: We then introduce baseline models for each of the tasks, report zero-shot results on the testing games, and analyze performance.

Original: 4.1 Knowledge Graph Prediction
Revised: 4.1 Knowledge Graph Prediction

Original: The ﬁrst world modeling task involves predicting a knowledge graph from the current set of textual observations.
Revised: The ﬁrst world modeling task involves predicting a knowledge graph from the current set of textual observations.

Original: Recall that our dataset takes the form of tuples of (cid:104) s t , a t , s t +1 , r t +1 (cid:105) where s t and s t +1 are two subsequent states with a t being the action used to transition states and r t
Revised: Recall that our dataset takes the form of tuples of (cid:104) s t , a t , s t +1 , r t +1 (cid:105) where s t and s t +1 are two subsequent states with a t being the action used to transition states and r t

Original: +1 is the observed reward.
Revised: +1 is the observed reward.

Original: This task is to predict s graph t +1 , a set of knowledge graph relations, given the textual observations s obs t , the previous state’s graph s graph t , and action a t for all samples in the dataset.
Revised: This task is to predict s graph t +1 , a set of knowledge graph relations, given the textual observations s obs t , the previous state’s graph s graph t , and action a t for all samples in the dataset.

Original: We present three baseline models for this task.
Revised: We present three baseline models for this task.

Original: Rules.
Revised: Rules.

Original: Following Ammanabrolu and Hausknecht [2020], we extract graph information from the observation using information extraction tools such as OpenIE [Angeli et al., 2015] in addition to some hand-authored rules to account for the irregularities of text games.
Revised: Following Ammanabrolu and Hausknecht [2020], we extract graph information from the observation using information extraction tools such as OpenIE [Angeli et al., 2015] in addition to some hand-authored rules to account for the irregularities of text games.

Original: Question-Answering.
Revised: Question-Answering.

Original: This baseline comes from the Q*BERT agent described in Ammanabrolu et al.
Revised: This baseline comes from the Q*BERT agent described in Ammanabrolu et al.

Original: [2020b].
Revised: [2020b].

Original: It is trained on the SQuAD 2.0 [Rajpurkar et al., 2018], the Jericho-QA text game question answering dataset  on the same set of training games as found in JerichoWorld and then on JerichoWorld itself by formatting our dataset in the style of questions and answers when possible.
Revised: It is trained on the SQuAD 2.0 [Rajpurkar et al., 2018], the Jericho-QA text game question answering dataset [Ammanabrolu et al., 2020b] on the same set of training games as found in JerichoWorld, and then on JerichoWorld itself by formatting our dataset in the style of questions and answers when possible.

Original: It uses the ALBERT [Lan et al., 2020] variant of the BERT [Devlin et al., 2018] natural language transformer to answer questions and populate the knowledge graph via a few hand-authored rules from the answers.
Revised: It uses the ALBERT [Lan et al., 2020] variant of the BERT [Devlin et al., 2018] natural language transformer to answer questions and populate the knowledge graph via a few hand-authored rules from the answers.

Original: Examples of questions asked include: “What is my current location?”, “What objects are around me?”.
Revised: Examples of questions asked include: “What is my current location?”, “What objects are around me?”.

Original: Seq2Seq.
Revised: Seq2Seq.

Original: We further introduce a encoder-decoder based sequence-to-sequence learning approach [Sutskever et al., 2014] inspired by the transformer model BART [Lewis et al., 2020].
Revised: We further introduce a encoder-decoder based sequence-to-sequence learning approach [Sutskever et al., 2014] inspired by the transformer model BART [Lewis et al., 2020].

Original: The model architecture is shown in Figure 3 and consists of a bidirectional encoder such as BERT [Devlin et al., 2018] that takes the full set of textual observations—including location and inventory descriptions—as input and an autoregressive decoder such as GPT-2 [Radford et al., 2019] which takes in the current graph and learns to predict the graph sequence shifted by a token.
Revised: The model architecture is shown in Figure 3 and consists of a bidirectional encoder such as BERT [Devlin et al., 2018] that takes the full set of textual observations—including location and inventory descriptions—as input and an autoregressive decoder such as GPT-2 [Radford et al., 2019] which takes in the current graph and learns to predict the graph sequence shifted by a token.

Original: The weights of the encoder are ﬁne-tuned from BERT’s original weights on both the graphs, in triple form, and the textual observations taken from the training games using a masked language modeling loss.
Revised: The weights of the encoder are ﬁne-tuned from BERT’s original weights on both the graphs, in triple form, and the textual observations taken from the training games using a masked language modeling loss.

Original: The decoder is not pre-trained.
Revised: The decoder is not pre-trained.

Original: During test time, only the starting token is given to the decoder and it decodes the graph token by token via bean search until an end-of-sequence token is reached.
Revised: During test time, only the starting token is given to the decoder and it decodes the graph token by token via bean search until an end-of-sequence token is reached.

Original: Metrics.
Revised: Metrics.

Original: For this task, we report two types of metrics (Exact Match or EM and F1) operating on two different levels—at a graph tuple level and another at a token level.
Revised: For this task, we report two types of metrics (Exact Match or EM and F1) operating on two different levels—at a graph tuple level and another at a token level.

Original: EM checks for accuracy or direct overlap between the predictions and ground truth, while F1 is a harmonic mean of predicted precision
Revised: EM checks for accuracy or direct overlap between the predictions and ground truth, while F1 is a harmonic mean of predicted precision

Original: [Figure]
Revised: [Figure]

Original: and recall.
Revised: and recall.

Original: The graph level metrics are based on matching the set of (cid:104) subject, relation, object (cid:105) triples within the graph, all three tokens in a particular triple must match a triple within the ground truth graph to count as a true positive.
Revised: The graph level metrics are based on matching the set of (cid:104) subject, relation, object (cid:105) triples within the graph, all three tokens in a particular triple must match a triple within the ground truth graph to count as a true positive.

Original: The token level metrics operate on measuring unigram overlap in the graphs, any relations or entities in the predicted tokens that match the ground truth count towards a true positive.
Revised: The token level metrics operate on measuring unigram overlap in the graphs, any relations or entities in the predicted tokens that match the ground truth count towards a true positive.

Original: Analysis.
Revised: Analysis.

Original: Table 2 presents a breakdown of the results for this task across the testing games on all the baseline models presented.
Revised: Table 2 presents a breakdown of the results for this task across the testing games on all the baseline models presented.

Original: There are a few main trends to note in these results.
Revised: There are a few main trends to note in these results.

Original: The ﬁrst is that the question-answering (QA) approach signiﬁcantly outperforms both the Rules and
Revised: The ﬁrst is that the question-answering (QA) approach signiﬁcantly outperforms both the Rules and

Original: Seq2Seq approaches on average across all the testing games.
Revised: Seq2Seq approaches on average across all the testing games.

Original: The QA method used is extractive.
Revised: The QA method used is extractive.

Original: This means that the system is trained to pick out answers by highlighting spans in the input context that best answers a question.
Revised: This means that the system is trained to pick out answers by highlighting spans in the input context that best answers a question.

Original: The Rules approach also functions similarly but is not trained in any way on our data.
Revised: The Rules approach also functions similarly but is not trained in any way on our data.

Original: This is inherently a simpler problem formulation than the Seq2Seq approach—which seeks to generate the graph by decoding token by token—but has its limitations.
Revised: This is inherently a simpler problem formulation than the Seq2Seq approach—which seeks to generate the graph by decoding token by token—but has its limitations.

Original: These limitations are seen in the relative differences between the magnitudes of the graph and token metrics for these approaches.
Revised: These limitations are seen in the relative differences between the magnitudes of the graph and token metrics for these approaches.

Original: Both QA and Rules have signiﬁcantly lower graph metrics than token metrics, a phenomenon not observed in the Seq2Seq model.
Revised: Both QA and Rules have signiﬁcantly lower graph metrics than token metrics, a phenomenon not observed in the Seq2Seq model.

Original: In other words, the right information is extracted but is potentially not well shaped into knowledge graph form.
Revised: In other words, the right information is extracted but is potentially not well shaped into knowledge graph form.

Original: We hypothesize that this implies two things.
Revised: We hypothesize that this implies two things.

Original: (1) That these systems likely over-extract by extracting more information than is strictly necessary.
Revised: (1) That these systems likely over-extract by extracting more information than is strictly necessary.

Original: Take for example the sample observation seen in Figure 3: “You are standing in an open ﬁeld west of a white house, with a boarded front door.” .
Revised: Take for example the sample observation seen in Figure 3: “You are standing in an open ﬁeld west of a white house, with a boarded front door.” .

Original: QA when asked the question “What is my location?” answers: “open ﬁeld west of a white house, with a boarded front door”.
Revised: QA when asked the question “What is my location?” answers: “open ﬁeld west of a white house, with a boarded front door”.

Original: Seq2Seq, in contrast, is trained to map this sentence more tersely to: (cid:104) you, in, West of House (cid:105) .
Revised: Seq2Seq, in contrast, is trained to map this sentence more tersely to: (cid:104) you, in, West of House (cid:105) .

Original: (2) Both QA and
Revised: (2) Both QA and

Original: Rules use hand-crafted rules to put the graphs together once information has been extracted either through the core QA model or OpenIE.
Revised: Rules use hand-crafted rules to put the graphs together once information has been extracted either through the core QA model or OpenIE.

Original: We see here that while over-extraction can be beneﬁcial for the token metrics—it makes it difﬁcult to create a set of graph construction rules that generalize well across games with different structures, resulting in relatively lower graph metrics.
Revised: We see here that while over-extraction can be beneﬁcial for the token metrics—it makes it difﬁcult to create a set of graph construction rules that generalize well across games with different structures, resulting in relatively lower graph metrics.

Original: On the other hand, the main advantage of the Seq2Seq approach is that it is not extractive and trained directly on the graphs found in the dataset.
Revised: On the other hand, the main advantage of the Seq2Seq approach is that it is not extractive and trained directly on the graphs found in the dataset.

Original: This means that it is potentially able to infer facts that are not directly present in the input context.
Revised: This means that it is potentially able to infer facts that are not directly present in the input context.

Original: Recall that text games are partially observable and so the textual observations themselves may potentially be incomplete.
Revised: Recall that text games are partially observable and so the textual observations themselves may potentially be incomplete.

Original: An example of such an observation is: “You see a locked chest in front of you in the cellar.” .
Revised: An example of such an observation is: “You see a locked chest in front of you in the cellar.” .

Original: The ground truth graph for this would be: (cid:104) you, in, Cellar (cid:105) , (cid:104) chest, in, cellar (cid:105) , (cid:104) chest, is, lockable (cid:105) , (cid:104) sword, in, chest (cid:105) .
Revised: The ground truth graph for this would be: (cid:104) you, in, Cellar (cid:105) , (cid:104) chest, in, cellar (cid:105) , (cid:104) chest, is, lockable (cid:105) , (cid:104) sword, in, chest (cid:105) .

Original: The last fact in the graph, the sword being in the chest, is not revealed to you via the observation until you open the chest and thus cannot be predicted by extractive approaches like Rules and QA.
Revised: The last fact in the graph, the sword being in the chest, is not revealed to you via the observation until you open the chest and thus cannot be predicted by extractive approaches like Rules and QA.

Original: This gives models like
Revised: This gives models like

Original: Seq2Seq—that are trained directly on the graph—the ability to perform commonsense inference by potentially ﬁlling in information missing from the partially observable text inputs.
Revised: Seq2Seq—that are trained directly on the graph—the ability to perform commonsense inference by potentially ﬁlling in information missing from the partially observable text inputs.

Original: It further implies that extractive models, in their current form, will not be able to achieve perfect performance.
Revised: It further implies that extractive models, in their current form, will not be able to achieve perfect performance.

Original: The main limitation of the Seq2Seq model, however, is that this non-extractive framing—given that every token is decoded autoregressively, requiring a prediction at every step over the entire combined
Revised: The main limitation of the Seq2Seq model, however, is that this non-extractive framing—given that every token is decoded autoregressively, requiring a prediction at every step over the entire combined

Original: Expt.
Revised: Expt.

Original: [Table]
Revised: [Table]

Original: All experiments are evaluated over three random seeds with standard deviations not exceeding ±
Revised: All experiments are evaluated over three random seeds with standard deviations not exceeding ±

Original: in any overall category.
Revised: in any overall category.

Original: entity and relation vocabulary length of 7002 —is a signiﬁcantly more difﬁcult problem than the other approaches.
Revised: entity and relation vocabulary length of 7002 —is a signiﬁcantly more difﬁcult problem than the other approaches.

Original: It is likely that that such non-extractive approaches will have to simplify the problem by adding constraints that account for properties of knowledge graphs (e.g. graph are sets of tuples and the same tuple in a set cannot be decoded twice).
Revised: It is likely that that such non-extractive approaches will have to simplify the problem by adding constraints that account for properties of knowledge graphs (e.g. graph are sets of tuples and the same tuple in a set cannot be decoded twice).

Original: 4.2 Valid Action Prediction
Revised: 4.2 Valid Action Prediction

Original: The second world modeling task involves predicting the set of valid actions from the current set of textual observations.
Revised: The second world modeling task involves predicting the set of valid actions from the current set of textual observations.

Original: Given the data (cid:104) s t − 1 , a t − 1 , s t , r t (cid:105) (note the change in indexing), this task is formally deﬁned as: predict the set of valid actions for the subsequent state s valid t given the current state text observation s obs t , current knowledge graph s graph t , previous valid actions s valid t − 1 , and action a t − 1 that caused the state change for all individual samples across the dataset.
Revised: Given the data (cid:104) s t − 1 , a t − 1 , s t , r t (cid:105) (note the change in indexing), this task is formally deﬁned as: predict the set of valid actions for the subsequent state s valid t given the current state text observation s obs t , current knowledge graph s graph t , previous valid actions s valid t − 1 , and action a t − 1 that caused the state change for all individual samples across the dataset.

Original: This task requires linguistic priors in the form of commonsense reasoning and a knowledge of affordances—e.g. open mailbox is a more reasonable action to take in most situations than eat mailbox .
Revised: This task requires linguistic priors in the form of commonsense reasoning and a knowledge of affordances—e.g. open mailbox is a more reasonable action to take in most situations than eat mailbox .

Original: We present a single baseline for this task.
Revised: We present a single baseline for this task.

Original: We developed a Seq2Seq model that is identical to that presented for the Knowledge Graph Prediction task, except adapted to Valid Action Prediction.
Revised: We developed a Seq2Seq model that is identical to that presented for the Knowledge Graph Prediction task, except adapted to Valid Action Prediction.

Original: That is, it performs sequence learning on the valid actions token by token.
Revised: That is, it performs sequence learning on the valid actions token by token.

Original: Extractive approaches like
Revised: Extractive approaches like

Original: QA are not possible for valid action prediction given that the verbs in the action—e.g. take, get, put, swing, go —are not often found anywhere within the observation.
Revised: QA are not possible for valid action prediction given that the verbs in the action—e.g. take, get, put, swing, go —are not often found anywhere within the observation.

Original: The Seq2Seq approach thus decodes actions token by token from the entire combined output vocab of 11056 (see Table 1) at every step until a special end-of-sequence tag is reached.
Revised: The Seq2Seq approach thus decodes actions token by token from the entire combined output vocab of 11056 (see Table 1) at every step until a special end-of-sequence tag is reached.

Original: Metrics.
Revised: Metrics.

Original: For this task, we adapt the graph level Exact Match (EM) and F1 metrics as described in the previous task to actions.
Revised: For this task, we adapt the graph level Exact Match (EM) and F1 metrics as described in the previous task to actions.

Original: In other words, positive EM or F1 happens only when all tokens in a predicted valid action match one in the gold standard set.
Revised: In other words, positive EM or F1 happens only when all tokens in a predicted valid action match one in the gold standard set.

Original: Given that most valid actions have less than four tokens, we do not use standard Seq2Seq metrics—such as
Revised: Given that most valid actions have less than four tokens, we do not use standard Seq2Seq metrics—such as

Original: BLEU [Papineni et al., 2002]—intended for measuring n -gram overlap in longer sequences.
Revised: BLEU [Papineni et al., 2002]—intended for measuring n -gram overlap in longer sequences.

Original: [Table]
Revised: [Table]

Original: report
Revised: report

Original: Analysis.
Revised: Analysis.

Original: Table 3 shows the results for the valid action prediction task on all the testing games for the Seq2Seq baseline.
Revised: Table 3 shows the results for the valid action prediction task on all the testing games for the Seq2Seq baseline.

Original: Recall that an EM of 20 means that if there were 100 gold standard valid actions in an instance, the model predicted 20 of them exactly.
Revised: Recall that an EM of 20 means that if there were 100 gold standard valid actions in an instance, the model predicted 20 of them exactly.

Original: Based on this, we further note a trend in Table 3 that negative correlation between the as seen in Table 1.
Revised: Based on this, we further note a trend in Table 3 that negative correlation between the as seen in Table 1.

Original: That is, the more the average number of gold standard valid actions per instance in a game, the more predicted actions match.
Revised: That is, the more the average number of gold standard valid actions per instance in a game, the more predicted actions match.

Original: Games like ztuu, deephome, balances have a high number of gold standard average valid actions and lower performance than games like pentari, ludicorp, detective, temple which have a low number of average valid actions.
Revised: Games like ztuu, deephome, balances have a high number of gold standard average valid actions and lower performance than games like pentari, ludicorp, detective, temple which have a low number of average valid actions.

Original: This is counter intuitive as the expected result would be that a model is able to learn a smaller sequence more effectively than a larger one—implying that a smaller number of gold standard valid actions per instance would lead to more matches.
Revised: This is counter intuitive as the expected result would be that a model is able to learn a smaller sequence more effectively than a larger one—implying that a smaller number of gold standard valid actions per instance would lead to more matches.

Original: We hypothesize that this is likely due to the fact that the model best learns common actions found across all games ﬁrst before learning potentially more ﬁne grained actions—effectively a label imbalance issue across the valid actions in the dataset.
Revised: We hypothesize that this is likely due to the fact that the model best learns common actions found across all games ﬁrst before learning potentially more ﬁne grained actions—effectively a label imbalance issue across the valid actions in the dataset.

Original: E.g. navigation actions like go north are found much more often than actions like hit monster with sword —which are usually found in only a handful of fantasy games.
Revised: E.g. navigation actions like go north are found much more often than actions like hit monster with sword —which are usually found in only a handful of fantasy games.

Original: When performing zero-shot prediction on testing games, the model thus predicts these common actions with higher conﬁdence than the more ﬁne grained ones.
Revised: When performing zero-shot prediction on testing games, the model thus predicts these common actions with higher conﬁdence than the more ﬁne grained ones.

Original: Testing games with a smaller number of average gold standard valid actions also tend to have a larger proportion of uncommon actions—thus posing more of a challenge for the Seq2Seq model.
Revised: Testing games with a smaller number of average gold standard valid actions also tend to have a larger proportion of uncommon actions—thus posing more of a challenge for the Seq2Seq model.

Original:  Conclusions and Future Work
Revised:  Conclusions and Future Work

Original: This paper presents the JerichoWorld dataset and corresponding benchmarks that seek to drive progress in textual world modeling.
Revised: This paper presents the JerichoWorld dataset and corresponding benchmarks that seek to drive progress in textual world modeling.

Original: This primarily involves two key challenges behind the creation of agents that can understand and generate natural language in a diverse set of interactive and situated settings such as text games.
Revised: This primarily involves two key challenges behind the creation of agents that can understand and generate natural language in a diverse set of interactive and situated settings such as text games.

Original: Our dataset provides mappings from textual observations to ground truth knowledge graph states to enable agents to learn to infer the state of the world—alleviating the knowledge representation or Textual-SLAM challenge.
Revised: Our dataset provides mappings from textual observations to ground truth knowledge graph states to enable agents to learn to infer the state of the world—alleviating the knowledge representation or Textual-SLAM challenge.

Original: A key insight from an comparison of baseline models shows that a promising future direction lies in inferring the knowledge graph world state through commonsense reasoning rather than extracting this information due to the partial observability of text games.
Revised: A key insight from an comparison of baseline models shows that a promising future direction lies in inferring the knowledge graph world state through commonsense reasoning rather than extracting this information due to the partial observability of text games.

Original: A second world modeling task revolves around tacking the combinatorially-sized action space of text games.
Revised: A second world modeling task revolves around tacking the combinatorially-sized action space of text games.

Original: The dataset also provides mappings from textual observations to valid actions—i.e. the set of contextually relevant actions guaranteed to change the world in any state.
Revised: The dataset also provides mappings from textual observations to valid actions—i.e. the set of contextually relevant actions guaranteed to change the world in any state.

Original: A qualitative analysis of a state-of-the-art Seq2Seq model adapted to the domain and trained for this task suggests that while learning to conditionally generate commonly occurring actions across a large set of games might be relatively easy, learning to generate speciﬁc and contextually relevant actions provides a signiﬁcantly more difﬁcult challenge.
Revised: A qualitative analysis of a state-of-the-art Seq2Seq model adapted to the domain and trained for this task suggests that while learning to conditionally generate commonly occurring actions across a large set of games might be relatively easy, learning to generate speciﬁc and contextually relevant actions provides a signiﬁcantly more difﬁcult challenge.

Original: Current performance by state-of-the-art models across both these tasks suggests that there is much space for improvement.
Revised: Current performance by state-of-the-art models across both these tasks suggests that there is much space for improvement.

Original: There are many more tasks that can be framed for other challenges related to world modeling from this dataset.
Revised: There are many more tasks that can be framed for other challenges related to world modeling from this dataset.

Original: Some immediate examples: (1) ofﬂine reinforcement learning for game agents through imitation learning—predicting the sequence of actions that ﬁnish the game based on walkthroughs and reward information; (2) knowledge graph verbalization, a form of the standard data-to-text natural language processing task [Wiseman et al., 2017], in which we learn to generate text that is conditioned on a knowledge graph; and (3) description generation conditioned on the names of various objects, locations, and characters—with applications in long-form text generation domains such as automated storytelling [Martin et al., 2018, Fan et al., 2019] and procedural generation of interactive narratives [Ammanabrolu et al., 2020a, Walton et al., 2020].
Revised: Some immediate examples: (1) ofﬂine reinforcement learning for game agents through imitation learning—predicting the sequence of actions that ﬁnish the game based on walkthroughs and reward information; (2) knowledge graph verbalization, a form of the standard data-to-text natural language processing task [Wiseman et al., 2017], in which we learn to generate text that is conditioned on a knowledge graph; and (3) description generation conditioned on the names of various objects, locations, and characters—with applications in long-form text generation domains such as automated storytelling [Martin et al., 2018, Fan et al., 2019] and procedural generation of interactive narratives [Ammanabrolu et al., 2020a, Walton et al., 2020].

Original:  Broader Impacts
Revised:  Broader Impacts

Original: Text games are simpliﬁed analogues for systems capable of long-term dialogue with humans, such as in assistance with planning complex tasks, and also discrete planning domains such as logistics.
Revised: Text games are simpliﬁed analogues for systems capable of long-term dialogue with humans, such as in assistance with planning complex tasks, and also discrete planning domains such as logistics.

Original: Our focus is on helping agents to better model such worlds, enabling greater efﬁciency for agents training to produce such contextually relevant language.
Revised: Our focus is on helping agents to better model such worlds, enabling greater efﬁciency for agents training to produce such contextually relevant language.

Original: The data is collected from games containing situations of non-normative language usage—describing situations that ﬁctional characters may engage in that are potentially inappropriate, and on occasion impossible, for the real world such as running a troll through with a sword.
Revised: The data is collected from games containing situations of non-normative language usage—describing situations that ﬁctional characters may engage in that are potentially inappropriate, and on occasion impossible, for the real world such as running a troll through with a sword.

Original: Instances of such scenarios are mitigated by careful curation of the games that the data is collected from.
Revised: Instances of such scenarios are mitigated by careful curation of the games that the data is collected from.

Original: The original Jericho framework [Hausknecht et al., 2020]—further veriﬁed by us in this work—uses a curated set of games found not to contain extreme examples of non-normative language usage.
Revised: The original Jericho framework [Hausknecht et al., 2020]—further veriﬁed by us in this work—uses a curated set of games found not to contain extreme examples of non-normative language usage.

Original: This is based on manual vetting and (existing) crowd-sourced reviews on the popular interactive narrative forum IFDB.
Revised: This is based on manual vetting and (existing) crowd-sourced reviews on the popular interactive narrative forum IFDB.

Revised: A Appendix

Revised: We would ﬁrst like to note the presence of a complementary dataset of observation-action pairs created by humans on the ClubFloyd online Interactive Narrative forum.

Revised: This dataset appears in both Ammanabrolu and Hausknecht [2020] and Yao et al.

Revised: [2020] with the latter using it to tune a

Revised: GPT-2 model for valid action prediction.

Revised: To prevent data leakage from human transcripts to our test games, we do not use this dataset to pre-train or tune our models.

Revised: The rest of this Appendix ﬁrst provides additional samples for the dataset for qualitative purposes and then provides training details for the baseline models.

Revised: A.1 Dataset

Revised: The games used in the Jericho suite and here are all open sourced freeware.

Revised: The walkthroughs required to create the oracle agents for the collection of data for the games were drawn from various sources on the internet and errors were corrected manually.

Revised: We provide our data at https: //github.com/JerichoWorld/JerichoWorld under an MIT license.

Revised: We provide 3 samples drawn from different games in the full dataset to help the readers better understand the diversity of text there.

Revised: [Table]

Revised: Secret

Revised: [Table]

Revised: This is a rather dark and small room, having only two exits, back north the way you came, from the ancestral homes of Tana, or through the heavily barred wooden door before you that leads southwest and inward to the abandoned Deephome, abode of the Dwarves in Telleen.

Revised: It has been three hundred years since your people lived here.

Revised: The heavy door stands open, admitting you into Deephome.

Revised: Observation: As you touch the finely etched symbol, you hear a click and a whir.

Revised: Then the door swings open before you, opening into the abandoned city of Deephome.

Revised: Your score has just gone up by five points.

Revised: Inventory

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: A.2 Baselines

Revised: The baseline models that are adapted from other works, i

Revised: [Table]

Revised: e. the Rules and QA systems, are trained using hyperparameters and methodologies described in their respective works.

Revised: A.2.1 Rules

Revised: Following Ammanabrolu and Hausknecht [2020]

Revised: [Table]

Revised: the exact details regarding knowledge graph updates are found as follows.

Revised: At every step, given the current state and possible attributes as context.

Revised: • Linking the current room type (e.g. “Kitchen”, “Cellar”) to the items found in the room with the relation “has”, e.g. (cid:104) kitchen, has, lamp (cid:105) • All attribute information for each object is linked to the object with the relation “is”.

Revised: e.g. (cid:104) egg, is, treasure (cid:105) •

Revised: Linking all inventory objects with relation “have” to the “you” node, e.g. (cid:104) you, have, sword (cid:105) •

Revised: Linking rooms with directions based on the action taken to move between the rooms, e.g. (cid:104) Behind House, east of, Forest (cid:105) after the action “go east” is taken to go from behind the house to the forest

Revised: A.2.2 Question-Answering

Revised: The QA models are trained on the SQuAD 2.0 [Rajpurkar et al., 2018], the Jericho-QA text game question answering dataset on the same set of training games as found in JerichoWorld

Revised: [Table]

Revised: and then on

Revised: JerichoWorld itself by formatting our dataset in the style of questions and answers when possible.

Revised: Our dataset is formatted in the style of Jericho-QA by templating questions that ask about location, objects (including characters)

Revised: [Table]

Revised: and attributes.

Revised: An example of a JerichoWorld dataset example converted to Jericho-QA format is seen below—though we would like to note that this removes much of the information present naturally within our dataset.

Revised: All other model architecture and hyperparameter details are as seen in Ammanabrolu et al.

Revised: [2020b]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: A.2.3 Seq2Seq

Revised: For both tasks, models were trained until validation accuracy (picked to be a random 10% subset of the training data) did not improve for 5 epochs or 72 wall clock hours on a machine with 4 Nvidia

Revised: GeForce RTX 2080 GPUs, three times with three random seeds.

Revised: All models decode using beam search with a beam width of 15 at test time until the end-of-sequence tag is reached.

Revised: The size of the decoding vocabulary for the action prediction task is 11056 and for the graph prediction task is 6985 .

Revised: Hyperparameters were not tuned and were taken from BART [Lewis et al., 2020].

Revised: [Figure]