Exploring the Capacity of Pretrained Language Models for Reasoning about Actions and Change
Year: 2023

Abstract:
Reasoning about actions and change (RAC) is essential to understand and interact with the ever-changing environment. Previous AI research has shown the importance of fundamental and indispensable knowledge of actions, i.e., preconditions and effects. However, traditional methods rely on logical formalization which hinders practical applications. With recent transformer-based language models (LMs), reasoning over text is desirable and seemingly feasible, leading to the question of whether LMs can effectively and efficiently learn to solve RAC problems. We propose four essential RAC tasks as a comprehensive textual benchmark and generate problems in a way that minimizes the influence of other linguistic requirements (e.g., grounding) to focus on RAC. The resulting benchmark, TRAC, encompassing problems of various complexities, facilitates a more granular evaluation of LMs, precisely targeting the structural generalization ability much needed for RAC. Experiments with three high-performing transformers indicate that additional efforts are needed to tackle challenges raised by TRAC.

heading: 1 Introduction
text: Reasoning about actions and change (RAC) has been a central issue since the early days in AI (McCarthy, 1963) and received much attention from the NLP community (Li et al., 2021; Zellers et al., 2021; Dalvi et al., 2018). Classical AI has long recognized the importance of actions and characterized the fundamental knowledge about actions: preconditions and effects (Reiter, 2001). Preconditions are the conditions that must be satisfied when actions are executed, while effects define the result. Consider the example where an agent is moving containers in a dock: Suppose a red container is on top of a green one, and the agent is given an instruction to “move the green container to another
∗Corresponding author
platform”. To achieve such a goal, it needs to know the preconditions of moving a container (there is not any other container on top of it) and the effect of such actions.
While traditional approaches provide sound and effective reasoning by capturing preconditions and effects, they rely on expensive and difficult formalization. Consequently, inducing knowledge from text and reasoning directly is becoming more preferable. As transformers have shown their promising potential in many linguistic tasks (Liu et al., 2019; Raffel et al., 2020), we set out to probe whether pretrained language models can master the ability of reasoning about actions and changes.
Previous approaches involving actions in NLP are more application-centric and usually contain specific tasks (e.g. instruction following, prediction) (Li et al., 2021; Zellers et al., 2021) without considering both preconditions and effects. In general these tasks are not verified logically. On the other hand, there exist some template-based datasets, such as the one in (Clark et al., 2020), which are logically sound and generated from several simple rules. Unfortunately, they pay more attention to general deduction in a static perspective, while RAC problems require repeated changes.
We propose Textual Reasoning about Actions and Change (TRAC), a comprehensive suite of four fundamental and granular RAC reasoning tasks, inspired by traditional reasoning problems (Brachman and Levesque, 2004). Recognizing the utmost importance of preconditions and effects, we include the two fundamental tasks, Projection and Executability, which directly target the essential knowledge of RAC. We also provide two composite tasks, Plan-Verification and Goal-Recognition for more comprehensive problem settings. Together, the four aspects of TRAC enable a more granular evaluation. We position TRAC as a diagnostic benchmark that is a more direct and precise embodiment of RAC abilities:
4629
• Projection: anticipate the effects of actions; • Executability: decide if actions are applicable; • Plan-Verification: decide if actions form a valid
plan; • Goal-Recognition: recognize the goal from ob-
servations of actions.
We employ two principles in dataset construction. Firstly, we minimize the impact of other important abilities (e.g. grounding and language variance) for a “clean-room” evaluation that solely focuses on RAC. Secondly, we desire reasoning problems with controlled complexities for testing structural generalization. We design a framework that takes the action domain knowledge and the textual template as input, generates symbolic problems, and synthesizes the textual problems. In this paper, we select the blocks world (Cook and Liu, 2002), a typical action domain for a proof-of-concept. However, the framework is domain-agnostic and is thus extensible to other action domains.
We design generalization tests based on the observation that once the classical AI systems are endowed with knowledge of actions, they can effectively solve structurally more complex problems. Therefore, we lay out four aspects of generalization experiments: 1) with more objects; 2) with longer action sequences; 3) with unseen names of objects; 4) with unseen conjunctive conditions. We train neural language models (LMs) of three highperforming transformer architectures, testing their efficiency and effectiveness on the TRAC datasets. The result shows that while transformers are able to induce and utilize knowledge from a large number of training examples, it remains a great challenge to efficiently generalize to structurally more complex problems on TRAC.

heading: 2 Related Work
text: Recent NLP tasks are embracing reasoning. While machine reading tasks such as SQuAD (Rajpurkar et al., 2018) require inference to some extent, our work is the first one as we know to systematically study Textual RAC. Reasoning about dynamic worlds is also needed in commonsense tasks such as the Winograd Schema Challenge (Levesque et al., 2012; Wang et al., 2019). But environments in such tasks did not model well-defined actions and change. Compared to these tasks, our concern is formal reasoning that supports sound conclusions.
Our work is distinct from previous attempts of formal reasoning in natural language. Natural Logic studies valid inference over natural language (Lakoff, 1970), but its recent developments focus on reasoning about semantic relations between lexical terms (Angeli et al., 2016). Meanwhile, transformers have achieved promising results on textual deduction tasks (Clark et al., 2020; Saha et al., 2020). While they target general deduction such as “round people are nice; Bob is round; Thus Bob is nice”, we dedicate our work to RAC and requires systems to learn the knowledge from examples.
Previous work has been exploring NLP tasks involving actions, especially with instruction following, action outcome prediction and procedural text generation tasks. TRAC is unique in its focus on fundamentals, broad coverage of reasoning tasks, and the granularity. Linguistic requirements (e.g. the ability of grounding and handling languge variance), though important, are foreign to RAC and are thus avoided. Compared to application-centric tasks, our work provides a more abstract view of tasks that embody the fundamental requirements of RAC. Moreover, this paper proposes a suite of generalization tests that target structural complexities in RAC, which enables more fine-grained tests. Instruction Following (IF): Agents are given textual directives to execute actions in an environment. In (Zhou et al., 2021; Dan et al., 2021), agents receive visual features or spatial coordinates as input. More importantly, IF does not target directly preconditions and effects as we do. Prediction: Projection and prediction both concern the effect of actions. In (Zellers et al., 2021), symbolic attributes represent the world instead of language. Questions in bAbI (Weston et al., 2016) and ProPara (Dalvi et al., 2018) ask about change of objects. However, they neglect the equally important preconditions. Procedure Generation: Predicting the next instruction (Li et al., 2021) or next step (Bosselut et al., 2018) might relate to the executability of actions, but they only cover partial RAC illustrated in this paper. While understanding the preconditions is needed, it could also implicate other extraneous factors such as the preferences of the instructor. Generalization has always been a desirable trait. In (Dan et al., 2021), the authors attacked IF systems with adversarial examples (e.g. with slight perturbations). Long-horizon problems are shown to be non-trivial for neural planners (Zhou et al.,
2021; Xu et al., 2019). Our evaluation with transformers further corroborate the observation. Additionally, TRAC imposes more structural generalization requirements in a more granular way.

heading: 3 TRAC
text: In this section, we first introduce the theoretical background before we lay out the four tasks and discuss dataset generation. The dataset is available at https://github.com/sysulic/trac.

heading: 3.1 Theoretical Preparation
text: In RAC, the essential issues include understanding 1) the state of the environment, 2) which actions are applicable, and 3) how actions affect the environment. For precise definitions of states and actions, we use the semantics of STRIPS (Fikes and Nilsson, 1971), a typical formal language that enables sound reasoning. In STRIPS, an action domain specifies the types of objects, predicates for describing the states, and how actions change the environment. In our exploration, we limit the action domains to be deterministic and noise-free. We also assume the unique name axioms, where different objects have different names. TRAC is built upon the following concepts:
• A state is a set of atomic expressions (e.g., {clear(A), on(A,B), . . .}) that represents a snapshot of the environment at a specific time point. Atoms not in the state are considered false. • An action consists of four parts: name, precondition, add list and delete list, the last three of which are sets of atoms. An action is applicable to a state iff the atoms of the precondition are all contained in the state. When it is applied to a state s, expressions in the delete list will be removed from s and those in the add list will be added into s, resulting in a new state s′. • An action sequence is an ordered sequence of actions. It is applicable to a state if and only if every action can be applied consecutively. • A goal is a ground formula that describes the objective state. We limit goals to be either a literal (an atom or its negation) or a conjunction of two. We only consider achievable goals. • A plan (wrt a goal and a state) is an applicable action sequence. It is goal-achieving if the execution of the plan achieves the goal. A goalachieving plan is optimal if there is no shorter plan.
We use a variant of the blocks world (BW) as an example where 1) a table has infinite space to hold blocks, 2) blocks have identical sizes and can be stacked as towers (a block is either on another one or immediately on the table), and 3) a block can be moved only if certain conditions are met. Example 1 and 2 show the predicates and actions in BW. Given an action domain and an initial state, the action and expression space can be determined, based on which the TRAC problems are generated.
Example 1. Predicates in BW: 1. on(x, y) states that x is on y; 2. onTable(x) says x is on the table; 3. clear(x) declares that there is no block on x.
Actions in BW. We provide the definitions of the actions.
Example 2. Actions in BW:
move(x, y, z): Move block x that is on block y onto block z. • Precondition: clear(x), clear(z), on(x, y). • Add list: clear(y), on(x, z). • Delete list: clear(z), on(x, y).
moveToTable(x, y): Move block x that is on block y onto the table. • Precondition: clear(x), on(x, y). • Add list: onTable(x), clear(y). • Delete list: on(x, y).
moveFromTable(x, y): Move block x that is on the table onto block y. • Precondition: onTable(x), clear(x), clear(y). • Add list: on(x, y). • Delete list: onTable(x), clear(y).

heading: 3.2 Reasoning Tasks in TRAC
text: For a comprehensive and granular evaluation, we propose four different reasoning tasks, each focusing on an aspect of RAC. All four tasks in TRAC are formulated as text classification problems, similar to the deduction task (Clark et al., 2020) and Natural Language Inference task. Given the input of two texts, a context and a query, the system is asked to classify if the query is true accordingly. Concrete examples can be seen in Table 1.
Projection. The projection task directly asks about the effects of actions: Given an initial state s and an applicable sequence a⃗ of N actions, decide whether the projection query q, a proposition, would hold after the execution of a⃗. The context is s and a⃗, and the query is q.
Executability. This task directly targets the preconditions of actions: Given an initial state s and a sequence a⃗ of N actions, decide whether a⃗ can be executed consecutively in s. The context is s and the query is a⃗.
Plan-Verification (PV). Planning is the task of formulating actions to fulfill a certain goal. In TRAC, we use the verification version that asks systems to recognize if the provided actions can achieve the goal: Given an initial state s and a goal g, a proposition, and a sequence a⃗ of N actions, decide if a can achieve g. The context is s and g, and the query is a⃗.
Goal-Recognition (GR). GR is the task to recognize the goal from the partial observation of actions. We use a simplified version, where systems observe a partial action sequence and need to figure out if the given goal is the true objective: Given an initial state s, a potential goal g, and a sequence a⃗ of N actions as the observation, decide if g is the true objective. That is, decide if a⃗ is a prefix of any optimal plans to achieve g. The context is s and a⃗, and the query is g.

heading: 3.3 Dataset Generation
text: We provide a framework to generate TRAC problems. It takes both the action domain and the
language template as input to construct symbolic forms and translation, respectively. For each task in TRAC, we generate three basic datasets, each having action sequences of different lengths (denoted as L1, L2, and L3 for lengths 1, 2, and 3 respectively). All examples in the datasets have M = 5 objects. For the generalization tests discussed in detail in the next section, we also construct additional datasets with different parameters. Each problem is first generated in the symbolic form before being translated into textual form in English.
Symbolic Instance Generation. Commonly existing in all examples are the initial state and the action sequence, which serve as the foundation for all four tasks. While the initial state is always part of the context, the action sequence is either part of the context or is the query, depending on the task. Firstly we generate the state space with M objects according to the action domain. In the blocks world where blocks have different colors, their names (e.g. “the red block”) are randomly chosen from a pre-specified range. Secondly, the action space is computed, which includes all grounded possible actions with respect to each possible state. With these spaces, we construct the context and query for each TRAC problem.
Projection. The context includes an initial state and an action sequence. From the action space, we randomly sample N actions to form a sequence that is executable in the initial state. For the query, we randomly generate a formula of the following form: l1 or l1 ∧ l2, (1) where l1 and l2 are literals (atoms or their negations), e.g., onTable(Blue), ¬on(Green,Blue). The query is true if and only if it holds after executing the action sequence.
Executability. The context is an initial state. For the query, we sample N actions from the action space as a sequence. The query is true if and only if it is executable consecutively in the initial state.
Plan-Verification. The context consists of an initial state and a goal that is achievable with N actions or less. The query is an action sequence of length N . Both the goal and the action sequence are generated at random. Goals share the same form as projection queries, shown in Formula 1. We only include achievable goals in this task. The query is true if and only if it is a valid goal-achieving plan.
Goal-Recognition. The context comprises an initial state and a sequence of N actions as the partial observation. The GR query is a plausible goal of the same form as in the plan-verification task. Both the action sequence and the goal are randomly generated. The query is true if and only if the observation is a prefix of any optimal plans that achieve the query.
Textual Form Synthesis. We use templates to generate the actual problems, following the guideline of “clean-room” evaluation, which strives to focus on RAC instead of other linguistic requirements such as grounding and language variance. To maintain readability, our framework utilizes handcrafted templates for the specific action domain. These templates specify the translation of each predicate, action, goal, and projection query. A symbolic state will be converted into several sentences, each of which is a direct translation of the atomic expression. Similarly, an action sequence is translated into a concatenation of the action sentences. Projection queries are synthesized in the same fashion, but goals are processed differently to accommodate conjunctions with “and” if necessary. For example, the textual form of the goal onTable(Blue)∧¬on(Green,Blue) is “the blue
block is on the table and the green block is not on the blue block”.
As a result, we generate four datasets (four tasks, each containing various lengths of action sequences). Each dataset contains 15,000 labelbalanced examples. We split the 15k examples into 12k training examples (where 2k are used as a dev set) and 3k testing examples.

heading: 4 Experiments
text: We conduct experiments to address the following questions:
1. Can transformers induce knowledge to effectively solve TRAC problems? 2. Can they generalize to problems that are structurally more complex? 3. How data-efficient are they?
The datasets, code, and hyper-parameters are available in the supplementary materials.

heading: 4.1 Baseline Models
text: We use three different LMs as our baseline models, each with different architectures: RoBERTa (Liu et al., 2019), GPT-2 (Radford et al., 2019), and T5 (Raffel et al., 2020). These architectures compose transformer layers in various typical fashions:
1. RoBERTa features transformer-encoder layers; 2. GPT-2 contains transformer-decoder layers; 3. T5 combines both encoder and decoder layers.
Driving by the classification nature of TRAC, the first two baseline models are built by adding a linear layer upon the transformer layers for both
RoBERTa and GPT-2. As for T5, we use its textto-text pre-training objective to generate the labels. The input for RoBERTa is organized as 
 context 
 query 
 where 
 and 
 are separator symbols, following previous work (Clark et al., 2020). We tarin these LMs to predict the truth of the query using the cross-entropy loss function. Accuracy is used as the metric for evaluation, and we report the mean values and standard deviations of the repeated experiments. Appendix A.4 gives details of the LMs.

heading: 4.2 Effectiveness of Transformers
text: We first evaluate the effectiveness of the baseline models on TRAC problems of the same structural complexity. Both the training set and the test set contain problems with M = 5 objects and N actions (N ∈ {1, 2, 3}, where the ratio of L1:L2:L3 is 1:1:1). For each architecture, we train a baseline model separately for each reasoning task, resulting in twelve such models, shown in Table 2. In this setting, transformers are given enough training data and are required to induce knowledge about actions and change from examples.
In Table 2, transformers have shown capable performances, with all accuracies above 80%. While they excel at Executability and Goal-Recognition, there is considerable room for improvement on Projection and Plan-Verification. Although these different transformer architectures have their own wins on different tasks, they do have rather similar performances, demonstrating that the challenge of TRAC is universal for transformers.

heading: 4.3 Structural Generalization Experiments
text: Targeting structural generalization ability, we design four out-of-distribution tests. Novel test examples that are more structurally complex are generated to this end. For the first three generalization experiments (GEs), we re-use models trained on previous datasets (with complexities M = 5, N ∈ {1, 2, 3}). Since the last GE requires additional training, both training and test sets are necessary. In total, we created twenty additional datasets beyond the normal ones. Table 3 shows the results for the RoBERTa-base models. The other two tranformer architecture are also evaluated.
GE1: More Objects. In this test, we examine whether the baselines can handle TRAC problems containing more objects (blocks in the BW domain). We generate a new dataset for each reasoning task, all of which involve ten objects. We evaluate the baselines trained on the standard datasets. The results show that baselines have significantly worse performances on GE1 datasets for all tasks, most notably in Projection. This is expected, as the GE1 problems have longer state descriptions and more complicated states. Compared to the training examples, the state descriptions in GE1 datasets contain 6.1 more sentences with 52.1 more words on average. Nonetheless, the results show that the baselines do not generalize well to more objects.
GE2: More Actions. Naturally, we are interested in the generalization to longer action sequences, as the length often plays a vital role in both formal reasoning tasks and natural language tasks (Zhou et al., 2021; Xu et al., 2019). For Projection, Executability, and Plan-Verification tasks, we generated the L4 and the L5 datasets that have ac-
tion sequences of length four and five, respectively. Goal-Recognition is left out as there are not enough test examples in this setting. The results from Table 3 confirm that the length of the action sequence is a vital factor, as the accuracies degrade for longer sequences. It is more apparent for Plan-Verification problems, with an appalling 40% accuracy loss on L5. This observation further verifies the universality of the long-horizon problem (Xu et al., 2019), even for transformer-based LMs.
GE3: Unseen Names. Changing the names of the objects is supposed to have negligible impact. The ability of generalization to unseen symbols is desirable in reasoning over both formal and natural languages. Seeing this, we substitute the names of objects for previously unseen ones, resulting in four more datasets. They differ from the standard ones only in the object names. The results from Table 3 show minor differences between the performances as expected, demonstrating the capability of the LMs to generalize beyond unseen names.
GE4: Compositionality in Goals and Projection Queries. Reasoning requires the capability of compositionality: to understand or manipulate higher-level structures composed of known components. One such ability is to combine two conditions as a conjunction. If systems understand conditions A and B separately, they are expected to realize that the conjunction A∧B is true iff both are true. In TRAC tasks, the ideal testbeds are Projection, Plan-Verification, and Goal-recognition, where the projection queries and goals could be partitioned into literals and conjunctions.
In Projection, conjunctions take the form of two separate sentences. For example, a projection query could be “The red block is on top of the green block. The green block is on the table.” Whereas in PlanVerification and Goal-Recognition, conjunctions have the specific “and” surface form. For example, a goal could be “the red block is clear and the red block is on top of the green block”.
In this experiment, we train baselines using examples with only literals in conditions and see if they can handle the examples with conjunctions. Therefore, new training datasets are needed. For each of the three tasks, we generate GE4-literals, a dataset of 15k instances with only literals in target conditions (10k for training, 2k as dev set, and the other 3k for testing), along with GE4-conjunctions, a dataset of 3k problems with only conjunctions.
After training the baselines on GE4-literals, we compare their performances on test sets of literals and conjunctions.
The results from Table 3 show that the baselines do not generalize well to conjunctions, losing more than 20% accuracies on all tasks. Such a phenomenon suggests that compositionality is not trivial in TRAC for the transformers. It is also noteworthy that conjunctive conditions have various forms: While conjunctions are represented by two sentences in Projection, they are of the form “condition1 and condition2” in Plan-Verification and Goal-Recognition. This leads us to believe that the performance loss is not about the introduction of the surface form “and”, but about the conceptual understanding of conjunctive compositionality, without which the models cannot generalize well.

heading: 4.4 Data Efficiency
text: In reality, humans need only a few examples to adapt to novel environments. To explore how many training samples are needed for the transformers, we plot the accuracies of the RoBERTa baselines with increasing numbers of training samples. In Figure 1, we notice that the baselines require at least 3000 samples to have acceptable accuracies (above 80%) on standard datasets. The inefficiency is even more obvious when it comes to GE1 and GE2 examples in Figure 2 and Figure 3, respectively. Moreover, the Plan-Verification task seems to be the most challenging one when training data is limited.
Other Transformers. We also evaluate GPT-2, T5, and a larger RoBERTa model on the GEs and the full results are provided in Appendix A.1.
• GPT-2 and T5 have rather similar performances on GEs, suggesting that structural generalization is a universal challenge for transformers; • Although the larger RoBERTa model outperforms smaller models, it also suffers when facing structurally more complex problems.

heading: 5 Discussion
text: Although we are optimistic about the future of transformers, their performances in the generalization tests indicate that transformers alone are not enough for RAC: Firstly, the reasoning problems in this proof-of-concept evaluation involve few actions objects that would be quite effortless for humans. Secondly, the scale of generalization
is rather minor (from three actions to four or five actions; from five objects to ten objects). Yet we could observe the struggle of transformers with such minute structurally complex problems. As illustrated in (Li et al., 2021), transformers capture meaning in texts to some extent, which indicates that they have potential in modeling actions and change. Such potentials can also be seen in our experiments when transformers are given more than abundant training examples. Meanwhile, (Zellers et al., 2021) showed that dedicated neural components other than transformers could help model state changes and predictions. These lead us to the conjecture that additional mechanisms that model the preconditions and effects could be the next objectives towards solving TRAC problems.

heading: 6 Limitations
text: More Complex Domains. We choose the BW for its intuitive and simplistic nature (with one kind of object, three types of actions, and three predicates). Although the generalization experiments suffice currently to challenge transformers, realworld situations are more complicated. With the improvement of the algorithms, the need for a better arrangement of actions domains is emerging. In time, it could be beneficial to include several domains with various levels of complexities.
Balance Between Rigor and Natural. For now, the synthesized English sentences are generated using a fixed template. Whilst being accurate without ambiguity, the resulting text is still quite formal. It would be valuable to add variety in the expressions without losing precision.
Better Solvers. As our demonstrations suggest, current LMs still fall short on the generalization tests. We hope that our work will pique interests in the community towards reasoning about actions and change, and challenge approaches to undertake the fundamental reasoning tasks.

heading: 7 Conclusion
text: In a time where language models excel at many natural language tasks, including deductive ones, we revisit the key reasoning abilities for dynamic worlds with actions and change. While preserving the essence of traditional formal reasoning, we set out to investigate how well transformers can reason rigorously over textual input, which avoids
the need for a complete formalization of each specific problem. Using the semantics of STRIPS, we characterize four essential reasoning tasks about actions and change to form the TRAC benchmark. We devise a framework to generate symbolic problems and transform them into text, resulting in a suite of datasets of various complexities. We also design four further experiments that target different aspects of structural generalization. Built upon the high-performing transformers, the baselines are put to the test under different settings. Although they show promising results on in-distribution problems provided with more-than-abundant training examples, it is the out-of-distribution generalization tests that cause troubles. We argue that TRAC tasks could be used to 1) expand our understanding of the limitations of transformers and, 2) serve as a challenge for generalization in RAC over text. In the future, we expect to see more interesting work based on TRAC, such as better solvers with mechanisms to learn both preconditions and effects, and novel generalization tests that call for more specific reasoning abilities.

heading: Acknowledgement
text: We acknowledge support from the Natural Science Foundation of China under Grant No. 62076261 and 61906216.

heading: A Experiment Details
text: A.1 GEs for RoBERTa-large, GPT-2, and T5
The results for GEs for RoBERTa-large, GPT-2, and T5 can be seen from Table 4, Table 5, and Table 6 respectively, which suggests:
• Number of parameters matters. Compare the results of RoBERTa-base and RoBERTa-large, we can clearly see that the larger models outperform the smaller ones consistently. However, training larger LMs is notoriously more time-consuming and expensive. Additionally, larger models hinder practical applications in the real world where inference time is typically short.
• The overall generalization performances are similar for different transformer architectures. As the parameters of models (RoBERTa-base, GPT2 and T5) are at the same level, their struggle facing the generalization tests are alike. The data efficiency test for GE2 shows that transformers plateau once given more-than-abundant training examples.
A.2 Computing Infrastructure
We use a workstation with Intel i9-10980XE CPU, 128GiB of RAM, and RTX 3090 GPU. The experiments took about 100 hours.
A.3 License of TRAC
The datasets and the code are released under the CRAPL license (the Community Research and Academic Programming License). The full text of the license is included in the supplmentary material.
A.4 Transformers
We use the HuggingFace transformers (Wolf et al., 2020) implementation. The following shows the LMs with the number of parameters:
• RoBERTa-base: 125M; • GPT-2-small: 117M; • T5-base: 220M; • RoBERTa-large: 770M.
The following hyper-parameters are used:
• The learning rate is 1e-5 for RoBERTa-base and 1e-4 for the others;
• The maximum sequence length is 256; • The batch size is 16; • The weight decay is 0.01; • The warmup ratio is 0.06.
We repeated every experiment five times to calculate the mean values and standard deviations using five different seeds. We list the input-output examples in Listing 3.

heading: B The Blocks World
text: B.1 Symbolic Forms We list the symbolic forms of both the BW domain and the TRAC examples in the paper. • Symbolic forms of the problems in Table 1 are
shown in Listing 1. • The action domain BW is defined in a PDDL file.
Its content is shown in Listing 2.
Listing 1 TRAC Symbolic Examples 1 Projection Example 2 Initial state = { 3 onTable(Green), 4 clear(Red), 5 clear(Blue), 6 clear(Green), 7 onTable(Red), 8 onTable(Blue) 9 } 10 Action sequence = [ 11 moveFromTable(Green , Red) 12 ] 13 Query = on(Blue , Red) 14 15 Executability Example 16 Initial state = { 17 onTable(Olive), 18 on(Yellow , Olive), 19 clear(Indigo), 20 on(Indigo , Yellow) 21 } 22 Query = [ 23 moveToTable(Indigo , Yellow) 24 ] 25 26 Plan -Verification Example 27 Initial state = { 28 clear(Blue), 29 on(Blue , Magenta), 30 on(Magenta , White), 31 onTable(White) 32 } 33 Goal = !on(Blue , Magenta) 34 Query = [ 35 moveToTable(Blue , Magenta) 36 ] 37 38 Goal -Recognition Example 39 Initial state = { 40 clear(Blue), 41 on(Blue , Magenta), 42 on(Magenta , White), 43 onTable(White) 44 } 45 Action Sequence = [ 46 moveToTable(Blue , Magenta) 47 ] 48 Query = on(Blue , Magenta)
Listing 2 The action domain BW in PDDL 1 (define 2 (domain blocksworld) 3 (: requirements :strips :typing) 4 (: types block - object) 5 (: predicates (clear ?x - block) 6 (on ?x - block ?y - block) 7 (ontable ?x - block)) 8 (: action move 9 :parameters (?x - block ?y - block ?z - block) 10 :precondition (and (clear ?x) 11 (clear ?z) 12 (on ?x ?y)) 13 :effect (and (not (clear ?z)) 14 (not (on ?x ?y)) 15 (on ?x ?z) 16 (clear ?y))) 17 18 (: action movetotable 19 :parameters (?x - block ?y - block) 20 :precondition (and (clear ?x) 21 (on ?x ?y)) 22 :effect (and (not (on ?x ?y)) 23 (clear ?y) 24 (ontable ?x))) 25 26 (: action movefromtable 27 :parameters (?x - block ?y - block) 28 :precondition (and (ontable ?x) 29 (clear ?x) 30 (clear ?y)) 31 :effect (and (not (ontable ?x)) 32 (not (clear ?y)) 33 (on ?x ?y))) 34 )
Listing 3 Input And Output for Transformers 1 RoBERTa 2 Input: "
 The yellow block is on
the table. The magenta block is on top of the pink block. The gray block is clear. The gray block is on the table. The magenta block is clear. The pink block is on top of the green block. The green block is on the table. The yellow block is clear. Jane moves the yellow block from the table to the gray block. </s > The green block is clear. The gray block is not on top of the yellow block. 
"
3 Output: 0 4 5 GPT -2 6 Input: "The yellow block is on the
table. The magenta block is on top of the pink block. The gray block is clear. The gray block is on the table. The magenta block is clear. The pink block is on top of the green block. The green block is on the table. The yellow block is clear. Jane moves the yellow block from the table to the gray block. The green block is clear. The gray block is not on top of the yellow block."
7 Output: 0 8 9 T5
10 Input: (Same as GPT -2) 11 Output: "No"
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
6
A2. Did you discuss any potential risks of your work? Not applicable. Left blank.
3 A3. Do the abstract and introduction summarize the paper’s main claims? 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Left blank.
B1. Did you cite the creators of artifacts you used? Not applicable. Left blank.
3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Left blank.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Not applicable. Left blank.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. 3
C 3 Did you run computational experiments? 4
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? A
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? A
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 4
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Not applicable. Left blank.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: In the future, we expect to see more interesting work based on TRAC, such as better solvers with mechanisms to learn both preconditions and effects, and novel generalization tests that call for more specific reasoning abilities