C a T -Bench: Benchmarking Language Model Understanding of Causal and Temporal Dependencies in Plans (2024)

Abstract: Understanding the abilities of LLMs to reason about natural language plans, such as instructional text and recipes, is critical to reliably using them in decision-making systems. A fundamental aspect of plans is the temporal order in which their steps need to be executed, which reflects the underlying causal dependencies between them. We introduce CaT-Bench, a benchmark of Step Order Prediction questions, which test whether a step must necessarily occur before or after another in cooking recipe plans. We use this to evaluate how well frontier LLMs understand causal and temporal dependencies. We find that SOTA LLMs are underwhelming (best zero-shot is only 0.59 in F1), and are biased towards predicting dependence more often, perhaps relying on temporal order of steps as a heuristic. While prompting for explanations and using few-shot examples improve performance, the best F1 result is only 0.73. Further, human evaluation of explanations along with answer correctness show that, on average, humans do not agree with model reasoning. Surprisingly, we also find that explaining after answering leads to better performance than normal chain-of-thought prompting, and LLM answers are not consistent across questions about the same step pairs. Overall, results show that LLMs’ ability to detect dependence between steps has significant room for improvement.

## introduction planning is central to decision making and has been studied in various domains such as robotics and embodied environments @xcite . to follow, revise, or customize a plan, one must be able to reason about the steps involved as well as their causes and effects @xcite . recent work on evaluating reasoning in plans focuses on classical problems ## related work early work in text understanding argued for the importance of understanding plans and goals @xcite . generating plans (aouladomar and saint-dizier, 2005) involves different types of understanding such as temporal reasoning and entity state tracking. naturalplan (zheng et al., 2024) present real-world tasks with natural language interaction, but are only limited to three tasks. planbench @xcite showed that llms were unable to generate executable and effective plans, but focused on simulated worlds with restrictive pddl syntax. @xcite proposed the goal-oriented script construction task, where a model produces a sequence of steps (or a plan) to accomplish a given goal. chattychef @xcite uses the conversational setting to generate cooking instructions and iteratively refine its step ordering. coplan @xcite collects conditions associated with a revised list of steps for the task of plan revision to satisfy constraints. lal et al. ( 2024 ) study the use of llms for plan customization according to user requirements. llms have been shown to generate plans well but it is unclear how well they truly understand all aspects of these plans. plan understanding tasks involve multiple aspects such as tracking entity states @xcite , linking actions @xcite @xcite , next event prediction @xcite @xcite and more. openpi @xcite enables entity tracking in how-to procedures. propara @xcite focuses on describing and understanding scientific processes. xpad @xcite extend propara by adding the new task of explaining actions by predicting their dependencies. @xcite formalize several multiplechoice tasks related to step-and goal-relations in procedures. @xcite explore predicting dependencies in cooking recipes and related tasks. similar work has been done on identifying dependencies in multimodal instructions with im-ages and text @xcite . pizzacommonsense @xcite is a dataset for learning commonsense about intermediate and implicit steps for cooking recipes, and contains explicit input/output pairs of each action with finegrained annotations. choice-75 @xcite aims to study decision branching in plans by generating user scenarios and choices for various steps within the plan. crepe @xcite measures how well llms understand the comparative likelihood of two events occurring in a procedure. there are a variety of datasets evaluating different aspects of plans, but there is a lack of one that clearly studies the prediction and explanation of temporal ordering constraints on the steps of an instructional plan. ## cat-bench understanding plans requires reasoning about how different steps in a plan relate to each other. in this work, we focus on the ability to recognize temporal dependencies between steps i.e., deciding if a one step must happen before another. typically, step i must happen before a step j if the effects (outcomes) of step i satisfy one or more preconditions necessary for the proper execution of step j, or if the effects of step j aggregate or modify the effects of step i in service of accomplishing a (sub-)goal. for example, in the plan for baking shortcakes shown in figure 2 , step 10 which involves moving the (implicitly mentioned) baked cake to the wire rack for cooling, requires that the cake be baked first, which in turn requires the dough to be placed in the baking tray. thus, recognizing such dependencies requires the ability to infer many important logical connections such as preconditions, causes, sub-goals, and effects of the steps. this suggests that a simple test of whether a step must happen before another step (or after) can be an effective test of reasoning about the various logical dependencies between the steps in a plan. we build on this idea to create cat-bench, a new dataset of causal dependency questions defined over cooking recipes. specifically, we make use of the recipe flow graph corpus @xcite containing 300 english language cooking recipes annotated with substep procedure dependencies. for each recipe, this dataset provides a directed acyclic graph (dag), in which the nodes are steps and directed edges indicate the temporal edge between those steps. if the nodes correspond-goal: lemon zested strawberry shortcakes steps: ... 6. divide dough in half. 7. add sugar; beat until stiff peaks form. 8. place 5cm apart on an ungreased baking tray. 9. bake at 200 c / gas 6 for 8-10 minutes. 10. remove to a wire rack; cool for 15 minutes. 11. in bowl, combine butter and lemon zest; set aside. 12. in mixing bowl, beat cream until it begins to thicken. ing to two steps are not connected by a directed path, then they can be performed in any order (with respect to themselves) without changing the recipe result. in other words, two steps are temporally dependent if and only if there is a directed path from one to the other, and independent otherwise. for all ordered pairs of steps (i, j) in a plan, we create two binary (yes/no) questions: (i) must step i happen before step j ? (ii) must step j happen after step i ?. these questions primarily test for for precondition relations (e.g. first question in figure 2 ), and the ability to understand effects of steps and how they relate to sub-goals or overall goals of the plan (e.g second question in figure 2 ). we pool all such questions from dependent pairs of steps (i.e, the steps where there is a directed path from one step's node to the other in the recipe dag) into dep, and the rest into nondep. 2 in total, cat-bench contains 2,840 questions about causal dependencies of steps for 57 unique plans. we undersample the non-dependent questions to ensure that nondep and dep are of the same size (i.e., 1,420 questions each). half of cat-bench tests the "before" temporal relation and the other half tests the "after" relation. on the distance between the pairs of steps. two steps are deemed close if they are within 3 steps of each other, (ji) ≤ 3, there are 1,256 questions about close steps and 1,584 about distant steps. cat-bench enables two tasks. step order prediction elicits binary judgments about dependencies between pairs of steps in a plan, and performance on this task can be evaluated automatically. step order explanation requires models to provide explanations for their judgments about step dependencies. this involves understanding causal relationships and expressing relevant knowledge about actions in the steps being asked about. since this is a free-form generation task, these explanations require human evaluation. note that cat-bench does not contain gold, human-written explanations, and we advocate for reference-free human evaluation since there can be multiple valid explanations. ## benchmarking models on cat-bench we benchmark the performance of a variety of models on cat-bench. ## analysis to better understand the strengths and weaknesses of these models, we analyze their performance on cat-bench organized by different characteristics of the questions and model prompts. ## conclusion understanding plans requires reasoning about its different aspects such as preconditions and effects. this paper introduces cat-bench, a new benchmark to evaluate the causal and temporal reasoning abilities about plans. despite the remarkable strength of current sota llms, we find that none of them are very good at understanding whether one step in a plan must precede (or succeed) another. particularly, they are much worse at knowing when there is not a dependency between steps. we also find that llm predictions are not robust as measured by two metrics of consistency. prompting llms to provide an answer and then to explain it improves performance significantly, and is even better than chain-of-thought (reasoning followed by answering). human evaluation of these explanations shows that models have a long way to go at understanding dependencies. our results highlight the current deficiencies of llms in plan-based reasoning, and there is need for improvement before they can be reliably used in the many applications requiring such reasoning.