Distilling Script Knowledge from Large Language Models for Constrained Language Planning (2023)

Abstract: In everyday life, humans often plan their actions by following step-by-step instructions in the form of goal-oriented scripts. Previous work has exploited language models (LMs) to plan for abstract goals of stereotypical activities (e.g., “make a cake”), but leaves more specific goals with multi-facet constraints understudied (e.g., “make a cake for diabetics”). In this paper, we define the task of constrained language planning for the first time. We propose an over-generate-then-filter approach to improve large language models (LLMs) on this task, and use it to distill a novel constrained language planning dataset, Coscript, which consists of 55,000 scripts. Empirical results demonstrate that our method significantly improves the constrained language planning ability of LLMs, especially on constraint faithfulness. Furthermore, Coscript is demonstrated to be quite effective in endowing smaller LMs with constrained language planning ability.

## introduction to accomplish everyday goals, humans usually plan their actions in accordance with step-by-step instructions. such instructions are discovered as goal-oriented scripts @xcite , involving a set of prototypical event sequences to achieve goals. for the example in figure 1 , to achieve the goal (make a cake), one usually has to follow certain steps of instructions, e.g., gather ingredients, preheat the oven, etc. the planning for such step-by-step scripts chains up reasoning toward complex goals @xcite . therefore, the automation of planning envisions more intelligent and reasonable ai 1) gather your ingredients. script for the goal: make a cake for diabetics sugar for diabetics?! instructgpt (text-davinci-002) ## related work language planning language planning aims to decompose a goal into sequences of steps @xcite , which is widely used in robotics @xcite @xcite and procedural text generation @xcite . early studies approach language planning with syntactic parsing for the context @xcite . recently, researchers have investigated the planning capability of language models in various domains @xcite . however, they mainly focus on generating scripts for stereotypical activities toward abstract goals. for example, @xcite proposes to plan for the general-typed tasks for embodied agents, while @xcite edits actions for abstract goals to video retrieval. in contrast, we explore planning for specific goals (e.g., "make a cake for diabetics"). @xcite has benchmarked llms for planning with included/excluded objects, but they merely study this problem in a limited scope (only dozens of cases) without further in-depth analysis. scripts a structure describing a sequence of events in a particular scenario is script @xcite , consisting of two types: 1) narrative script: a narrative chain of events describing a particular scenario derived from narrative texts such as recipes @xcite or stories (tandon et al., 2020); 2) goal-oriented script @xcite : an appropriate sequence of steps as instructions to achieve a goal. in this work, the steps for achieving a given goal in language planning can be categorized into the second class. many datasets for goal-oriented scripts have been proposed to improve the language planning ability of lms @xcite . however, they mainly consist of abstract goals with prototypical instructions and thus are not built to train lms for planning with more specific goals. constraint type 1: modifier definition: a word, an adjective or a phrase that modifies or constrains an abstract goal. ex.1: make a chocolate cake. ex.2: make a pink cake. constraint type 2: method definition: a tool or specified mode that controls the process for achieving the goal. ex.1: make a cake with an oven . ex.2: make a cake by using cake mix . ## definitions before diving into technical details, we first clarify some important terms used in the paper. scripts a goal-oriented script is a list of steps (@xmath0, s 2 , • • • , s |s| }) that fulfill a certain goal (g) (e.g., "make a cake") @xcite . the language planning task is defined as m : @xmath1, where m is the planning model. goals different from previous studies that focus mostly on abstract goals with prototypical scripts, abstract goal: make a cake filtered scripts ## constrained language planning with llms in this section, we evaluate and enhance the constraint language planning ability of llms. the overall workflow is illustrated in figure 2 . we first extend the specific goals g c from the abstract ones g a using a human-in-the-loop acquisition approach with llms ( § 4. ## script distillation from llms since llms are costly to deploy, it is essential to enable language planning ability for smaller, specialized models. creating datasets is an inevitable step to this end. however, previous datasets do not enable planning for specific goals @xcite , and manual dataset annotation is expensive and highly demanding. thus, we follow the idea of symbolic knowledge distillation @xcite to distill constrained language planning datasets from llms. ## constrained language planning with specialized models with coscript, we can train smaller but specialized models for constrained language planning. ## conclusion in this paper, we define planning toward specific goals with constraints. we propose a better prompting method for llms, and distill a novel dataset from llms (coscript) to improve the constrained language planning ability of specialized models. experiments show that our method improves the planning quality of llms for specific goals, and smaller models trained on coscript even outperform llms. we hope the coscript dataset will be a valuable resource to advance the research on language planning with more complex and diverse goals and constraints.