S cience W orld: Is your Agent Smarter than a 5th Grader? (2022)

Abstract: We present ScienceWorld, a benchmark to test agents’ scientific reasoning abilities in a new interactive text environment at the level of a standard elementary school science curriculum. Despite the transformer-based progress seen in question-answering and scientific text processing, we find that current models cannot reason about or explain learned science concepts in novel contexts. For instance, models can easily answer what the conductivity of a known material is but struggle when asked how they would conduct an experiment in a grounded environment to find the conductivity of an unknown material. This begs the question of whether current models are simply retrieving answers by way of seeing a large number of similar examples or if they have learned to reason about concepts in a reusable manner. We hypothesize that agents need to be grounded in interactive environments to achieve such reasoning capabilities. Our experiments provide empirical evidence supporting this hypothesis – showing that a 1.5 million parameter agent trained interactively for 100k steps outperforms a 11 billion parameter model statically trained for scientific question-answering and reasoning from millions of expert demonstrations.

## introduction question answering (qa) has seen rapid progress recently. standardized elementary and middle school science exams have served as a challenge task for qa @xcite , as these questions require combining science-domain knowledge with world knowledge in complex reasoning procedures to solve. as large language models have toppled these benchmarks @xcite @xcite , the focus has shifted away from simply answering questions toward producing human-readable explanations for a model's 1 website: https://sciworld.apps.allenai.org 2 code: https://github.com/allenai/scienceworld scienceworld task 3 (test electrical conductivity subtask) ## related work science-domain inference: standardized science exams are a challenging task for question answering due to their diverse knowledge and inference requirements @xcite @xcite . top performing models can answer more than 90% of multiple-choice questions correctly @xcite , typically through the use of large language models @xcite . a number of corpora of structured and semistructured science exam explanations exist for training the explanation-generation task @xcite @xcite . evaluating explanations is challenging, and typically done by comparing generated explanations to a single gold explanation. this has been shown to substantially underestimate explanation generation performance by up to 40% @xcite . while others have worked to mitigate this by generating multiple alternate explanations for each question, this is expensive, and has only been demonstrated for adding one or two additional explanations per question @xcite . here, we propose a partial solution to this evaluation problem by treating the action sequences of agents as structured manner explanations for how to solve a task. these action sequences can be directly run in the scienceworld simulator to automatically determine their correctness (i.e., whether they accomplish the task), independent of any variations in their solution methods. for example, whether an agent melts ice by using a stove, building a campfire, or leaving the ice out on a kitchen counter, the end result is the same and directly measurable through the formal semantics of the simulator. environments: interactive text environments are becoming a vehicle for research in natural language processing (see @xcite for a review), primarily because of their reduced development costs relative to 3d environments, combined with their ability to easily implement high-level tasks with large action spaces. in spite of these benefits, implementation costs can still be substantial for large complex environments, and text agents are frequently evaluated on a small set of exist-ing interactive fiction games such as zork @xcite using an unified interface like jericho @xcite . a few purpose-built environments provide simple tasks for studying text agents, typically on procedurally generated pick-and-place or object-combination tasks (e.g., cooking, @xcite . kitchen cleanup @xcite and textworld common sense @xcite require agents to tidy up one or more rooms in a house environment by putting objects in their typical locations (e.g., a hat should be placed on the hat rack), testing an agent's declarative knowledge of common object locations with the procedural knowledge required for this pick-and-place task. the closest existing interactive text environment to scienceworld is textlabs @xcite which simulates chemistry wet-lab protocols with actions such as pipetting and centrifuging. compared to these existing environments, scienceworld is generally a larger and more dynamic environment, populated with more complex objects with greater depth of physical simulation. this simulation depth enables more complex tasks associated with elementary science (e.g., thermodynamics, electrical circuits, etc.) to be tested, and a greater variety of solutions. simulators: nearly all text-based world simulations are currently implemented as z-machine games @xcite , frequently through higher-level application-specific languages (such as inform7, @xcite ) that compile to z-machine code. textworld @xcite creates environments using linear-logic statements that specify action preconditions and postconditions @xcite and generate inform7 code. existing tooling is designed for simpler simulations than scienceworld, with primarily agent-centered state changes that make modeling autonomous physical processes (e.g., thermodynamics) difficult. as such, in this work we build a novel simulator to model physical processes in text environments. agents: a variety of agent models have been proposed for reasoning in interactive text environments. most approaches frame reasoning as a partially-observable markov decision process (pomdp), and model inference using reinforcement learning (rl). this includes rl-based models that learn a policies to pick relevant actions from lists of candidate actions @xcite , or models that mix rl with knowledge graphs @xcite or language models @xcite to aid in next-action selection. action selection has also been modelled using case-based reasoning @xcite , or directly as a sequence-prediction imitation learning task, using language models trained on gold action sequences to predict the next action given the current state @xcite @xcite . in general, agent performance on solving interactive fictions is still modest, with only easier games close to completion @xcite . in this work, we benchmark state-of-the-art agents on scienceworld as well as introduce novel agents. we empirically show that these elementary science tasks are difficult for current agents, and also that smaller and simpler agents can outperform billion-scale parameter language models trained on gold sequences, highlighting the difficulty of this task for transformer-based models. ## scienceworld scienceworld is a simulation of the world abstracted through a complex interactive text environment in english with many objects, actions, and simulation engines. the framework consists of 40k lines of scala (speed) with a python interface. the scienceworld environment contains 10 interconnected locations (see figure 1 ), populated with up to 200 types of objects, including devices, instruments, plants/animals, electrical components, substances, containers, and common environment objects such as furniture, books, and paintings. the scienceworld action space contains 25 high-level actions, including both science-domain actions (e.g., using thermometer) and common actions (e.g., moving, opening containers, picking up items), with approximately 200k possible action-object combinations per step (though only a limited subset of these will be meaningful). see appendix a for details about scienceworld, including the object model, actions, and input parser. ## experiments to understand how contemporary approaches to text agents perform at scienceworld tasks, we benchmark a selection of recent architectures. tasks. to support our goal of generating a diverse set of tasks, we identified a candidate set of 10 broad science exam topics from the list of 400 fine-grained science curriculum topics of @xcite . topics were chosen that would be amenable to text-based simulation, and that did not have critical fine-grained spatial reasoning requirements, and include: changes of state, temperature measurement, electrical circuits, friction, object classification, chemical mixtures, plants and pollinators, life spans, life stages, and mendelian genetics. each topic was further divided into between 2 and 4 specific tasks for agents to perform, producing a total of 30 science-domain tasks. these topics and tasks are described in appendix b.2. to prevent overfitting and encourage generalization, each subtask contains between 10 and 1400 parametric variations (with 7200 total variations across all 30 subtasks). variations change critical task objects (e.g., the specific substance to be melted), the agent's starting location in the environment, as well as randomly vary the contents of the environment itself (e.g., whether the living room contains a bookshelf, or a painting, or both). train, development, test sets: for a given subtask, variations are split into 50% training, 25% development, and 25% test sets. variations are sorted such that critical unseen variations (e.g., substances, animals, or plants unseen during training) are found in development and test sets. ## discussion elementary science tasks are challenging for text agents. with top-performing agents reaching normalized average scores of 0.17 across tasks, performance on scienceworld is comparable to the current best-performing agents on mediumdifficulty interactive fiction games such as zork @xcite . much as in interactive fiction games, examining agent trajectories reveals that while agents appear to struggle with science-domain inference procedures such as how to heat a substance or how to grow a seed, they also currently lack a fluency with commonsense skills such as navigating the environment or storing liquids in containers. this underscores the need for models that can incorporate commonsense and science-domain knowledge, and integrate that declarative knowledge into actionable procedures to progress towards goals in the environment. larger models are not necessarily better. while larger models generally perform better in question answering tasks (e.g., @xcite , here we observe that larger models do not always increase performance. our best-performing model, the drrn, has only 1.5 million parameters -four orders of magnitude less than the t5 models. both models also receive the same number of gradient updates (10 6 ) with respect to scienceworld training tasks-though the t5 models have the added benefit of pre-training both from science exam qa and a large number of expert demonstrations. 4 this underscores that how a model approaches modeling state spaces and action sequences may be more important than the scope of its pre-training. online, interactive training enables models such as the drrn and kg-a2c to perform tasks requiring long action sequences more efficiently in terms of both samples and parameters. limitations of agents and environments. while agents still find text environments challenging, it is important to recognize that even this modest performance is achieved through a number of simplifying properties. for example, because agents frequently generate plausible but invalid actions, all but two agents benchmarked here depend on sci-enceworld's valid action detection aid at test time, substantially simplifying their search problem in the action space. similarly, while science-world achieves a high environment fidelity for a text simulation, this is still tempered by pragmatic concerns, such as generating comparatively short descriptions of environments that can fit into the sequence lengths of most transformer models. as such, even environments with complex physical, chemical, and biological processes underlying their simulations (such as scienceworld) still ultimately must limit the vividness of their descriptions, until these technical limitations in modelling can be surpassed. hybrid environments (e.g., shridhar et al., 2020) that concurrently model the same environment as both a high-fidelity 3d world and comparatively low-fidelity text-based simulation have shown that text environments can be used to provide useful task pre-training that can transfer back to the 3d environment with relatively low simulation compute cost. explanations as action sequences. explanations take on a variety of roles @xcite , from detailed human-readable descriptions of classification processes that describe how a decision was made (e.g., @xcite , to higher-level appeals to scientific processes to explain why an answer is correct (e.g., @xcite . here, the action procedures generated by agents act as manner explanations for how to solve a particular task -but while they describe how to accomplish something, they don't explain at a high-level why those actions accomplish the task. for example, action sequences lack high-level goal information such as "melting a substance requires heating it, so first the agent needs to heat the substance with a heating device, like a stove.". similar to how cuing agents to answer contextual questions can help improve their task performance @xcite , cuing agents to generate these explanatory scaffolds may help future agents increase task performance, while structuring their action sequence explanations for better human interpretability. ## conclusion despite recent progress in both interactive text agents and scientific text processing via transformers, current models are unable to reason about fundamental science concepts in a grounded and reusable manner-calling into question how much they are actually understanding the tasks at hand. to better measure such scientific reasoning abilities, we introduce scienceworld, an interactive text environment derived from an elementary school science curriculum-with tasks ranging from electrical conductivity to mendelian genetics. we evaluate three state-of-the-art reinforcement learning text game agents: drrn, kg-a2c, and calm; and further introduce two large-scale transformer-based agents inspired by recent ad-vances such as behavior cloning and the decision transformer and trained for scientific reasoning in scienceworld. while we find that overall performance on unseen tasks that require using science-domain knowledge is low across all agents, our results also suggest that agents that learn interactively in a grounded environment are more sample and parameter efficient than large language models that learn offline by reading text from static sources. the best agent performance is still modest -and on-par with medium-difficulty interactive fiction environments such as zork -highlighting the need for agents that can integrate declarative scientific and world knowledge with procedural action sequences in virtual environments.