LLM s as Planning Formalizers: A Survey for Leveraging Large Language Models to Construct Automated Planning Models (2025)

Abstract: Large Language Models (LLMs) excel in various natural language tasks but often struggle with long-horizon planning problems requiring structured reasoning. This limitation has drawn interest in integrating neuro-symbolic approaches within the Automated Planning (AP) and Natural Language Processing (NLP) communities. However, identifying optimal AP deployment frameworks can be daunting and introduces new challenges. This paper aims to provide a timely survey of the current research with an in-depth analysis, positioning LLMs as tools for formalizing and refining planning specifications to support reliable off-the-shelf AP planners. By systematically reviewing the current state of research, we highlight methodologies, and identify critical challenges and future directions, hoping to contribute to the joint research on NLP and Automated Planning.

## introduction the advent of large language models (llms) has marked a significant paradigm shift in ai, sparking claims regarding emergent reasoning capabilities within llms @xcite and their potential integration into automated planning for agents @xcite . while llms, due to the prowess of distributed representation and learning, excel at system i tasks, planning-an essential aspect of system ii cognition @xcite remains a significant bottleneck @xcite . furthermore, llms face challenges with long-term planning and reasoning, often producing unreliable plans @xcite @xcite , frequently failing to account for the effects and requirements of actions as they scale @xcite , with performance degrading under self-iterative llm feedback @xcite @xcite . state-of-the-art llms have shown limited planning capability by directly generating action sequences-as the correctness, optimality, and reliability of their outputs are not guaranteed. classical automated planning (ap) synthesizes plans through structured representation, logic, and search methods which are not subject to the weaknesses mentioned. meanwhile, llms possess promising capabilities at extracting, interpreting, and refining planning model specifications from natural language (nl), acting as complementary components that can enable classical planners to generate robust solutions. bringing the advantages of ap and llms together-llms for constructing planning specifications, ap systems for execution-defines the focus of this survey, a paradigm we call llmsas-formalizers for constructing ap models. this paper is driven by the fragmented landscape in the current literature, where many surveys lack a cohesive overview of llm integration within this field. our focus stems from the need to address these gaps and provide a clear framework, highlighting the importance of aligning llm capabilities with areas where they offer tangible benefits. the motivation for this approach is threefold: (i) planning accuracy: llms can help ensure that all relevant factors are considered, reducing the risk of overlooked constraints @xcite . (ii) adaptability: llms can aid systems in adapting to dynamic environments to capture real-world nuances better, reducing the need for manual edits @xcite . (iii) agnostic modeling: llms trained on large corpora of diverse data can generalize across various domains without needing task-specific tuning, which reduces the reliance on specialized expertise, a major bottleneck in realworld ap integration @xcite . our survey's taxonomy is divided into three key areas: model generation, the process of extracting natural language input-originating from users or the environment-into structured planning model formalizations, further consisting of (i) task generation (sec. 3.1.1), which translates initial and goal states along with object assignments; (ii) domain generation (sec. 3.1.2), constructing predicates and action schemas; and (iii) hybrid generation (sec. 3.1.3), encapsulating both task instance and domain generation. model editing (sec. 3.2) systematically addresses code refinement and the repair of errors and inconsistencies in ill-defined planning formalizations. finally, model benchmarks (sec. 3.3) encompass assessments of both the performance of llms in planning tasks and the quality of llm-generated planning formalizations. to our knowledge, this is the first comprehensive survey of llm-driven ap-model specification. our contributions can be summarized as follows: ## llms for constructing automated planning models we consider the research on leveraging powerful llms to assist in constructing planning models to be of critical importance. verifiable planning modules remain the backbone of planning, ensuring reliability, robustness, and explainability. note that deploying llms themselves in an end-to-end manner to perform planning still falls short of providing soundness guarantees @xcite and may have principled weaknesses. we organize the existing works into a taxonomy comprising three key areas: model generation, model editing, and model benchmarks-where the term model, in this context, refers to ap specifications such as pddl, as illustrated in figure 2 . the tasks require joint efforts from the nlp and automated planning community. ## language-to-plan (l2p) with the proliferation of related techniques to convert nl to pddl, we are seeing an ever-increasing set of related methods. to bring them together under a single computational umbrella, and beyond just relating the work together conceptually as we have done thus far in this survey, we created a unified platform: l2p foot_0 , which re-implements landmark papers covered in this survey. this python library is open source and has the capability of encapsulating the generalized version of the proposed "llm-modulo" framework @xcite , which ensures soundness via iterative plan refinement with external verifiers, shifting focus from direct planning to pddl generation with integrated verifiers and user-guided refinement through complete, planner-executable specifications. l2p offers three major benefits: figure 3 : a shortened l2p reconstruction of the 'actionby-action algorithm' @xcite , which iteratively generates pddl actions while updating a dynamic predicate list. output found in figure 8 . appendix c demonstrates examples of l2p usage. to demonstrate the flexibility of the framework, l2p re-implements some key papers covered in this survey (refer to appendix a). we hope to maintain the l2p framework as a repository of existing advancements in llm model acquisition and relevant papers, ensuring that users have access to the most current research and tools. ## discussion revisiting rq1: while frameworks can generate parsable and solvable pddl files, it remains uncertain if these specifications align with human goals. simple domains like blocksworld are easier to verify. still, scaling complex domains requires users to understand how llms generate these specifications, emphasizing the need for explainable planning to yield robust, transparent, and correctable outputs @xcite . corrective feedback loops notably improve failure handling, such as resolving action precondition errors @xcite , or re-planning in case of unexpected failures during plan execution @xcite . ensuring alignment with user goals involves breaking down pddl model construction into pre-processing steps with human-inthe-loop feedback @xcite . very reminiscent of the "critics" process in the llm-modulo framework @xcite , setting up a sort of external verifier checklist and using llms to provide feedback is demonstrated by @xcite . a potential idea is analyzing the semantic correctness of plans generated and using that as feedback to refine the llm-generated pddl specifications @xcite . additionally, intermediate representation (i.e. asp, python, json) that are easier for llms to process before converting to pddl @xcite can also enhance accuracy. revisiting rq2: llms have demonstrated that they are significantly sensitive to promptingraising questions about whether they are better off functioning as translators or generators. @xcite demonstrate that highly explicit descriptions improve translation accuracy, while @xcite leverage minimal descriptions, relying on llms' internal world knowledge to enrich outputs; however, this excess freedom often leads to inconsistent or inexecutable domain models. @xcite ; @xcite ; @xcite recognize that specifying a precise predicate set in nl is crucial and addresses the common problem of evaluating across different methods. the challenge of operating with minimal to no textual guidance beyond the initial task prompt underscores the importance of standardizing prompt granularity for initial generation and iterative feedback @xcite . @xcite provide a promising, organized, and standardized paradigm for automatically generating pddl descriptions that can be applied to llms. ## conclusion extracting planning models has long been recognized as a major barrier to the widespread adoption of planning technologies @xcite . even with the emergence of llms, this remains a persistent challenge, introducing a new suite of obstacles. in this survey, we examine nearly 80 scholarly articles that propose their frameworks and some other subsidiary works delegating model acquisition tasks to llms. by identifying the research distribution and gaps within these categories, we aim to provide a higher generalization from each framework's methodologies into broader aspects for future architectures. additionally, we hope researchers can apply these methodologies to more advanced planning languages with the support of our l2p library.