Planning Anything with Rigor: General-Purpose Zero-Shot Planning with LLM-based Formalized Programming
Authors: Yilun Hao
Yang Zhang
Chuchu Fan
Venue: ICLR 2025 Poster (2025)
Keywords: LLM Planning
Code generation
LLM Tool-Use
Primary area: foundation or frontier models, including LLMs
TL;DR: We propose a general-purpose planning framework with zero-shot generalization capability, which enable LLMs to build and solve diverse types of planning problems as optimization problems with no task-specific examples or external critics.

Abstract: While large language models (LLMs) have recently demonstrated strong potential in solving planning problems, there is a trade-off between flexibility and complexity. LLMs, as zero-shot planners themselves, are still not capable of directly generating valid plans for complex planning problems such as multi-constraint or long-horizon tasks. On the other hand, many frameworks aiming to solve complex planning problems often rely on task-specific preparatory efforts, such as task-specific in-context examples and pre-defined critics/verifiers, which limits their cross-task generalization capability. In this paper, we tackle these challenges by observing that the core of many planning problems lies in optimization problems: searching for the optimal solution (best plan) with goals subject to constraints (preconditions and effects of decisions). With LLMs' commonsense, reasoning, and programming capabilities, this opens up the possibilities of a universal LLM-based approach to planning problems. Inspired by this observation, we propose LLMFP, a general-purpose framework that leverages LLMs to capture key information from planning problems and formally formulate and solve them as optimization problems from scratch, with no task-specific examples needed. We apply LLMFP to 9 planning problems, ranging from multi-constraint decision making to multi-step planning problems, and demonstrate that LLMFP achieves on average 83.7\% and 86.8\% optimal rate across 9 tasks for GPT-4o and Claude 3.5 Sonnet, significantly outperforming the best baseline (direct planning with OpenAI o1-preview) with 37.6\% and 40.7\% improvements. We also validate components of LLMFP with ablation experiments and analyzed the underlying success and failure reasons.

Published as a conference paper at ICLR 2025
PLANNING ANYTHING WITH RIGOR: GENERAL- PURPOSE ZERO-SHOT PLANNING WITH LLM-BASED FORMALIZED PROGRAMMING
Yilun Hao
Yang Zhang
Chuchu Fan MIT chuchugmit. edu
MIT yilunhac@mit.ed
MIT-IBM Watson Al Lab Yang. Zhang28 iim. com
ABSTRACT
While large language models (LLMs) have recently demonstrated strong potential in solving planning problems, there trade-off between flexibility and com- plexity. LLMs, zero-shot planners themselves, are still not capable of directly generating valid plans for complex planning problems such as multi-constraint or long-horizon tasks. On the other hand, many frameworks aiming solve complex planning problems often rely on task-specific preparatory efforts, such as task- specific in-context examples and pre-defined critics/verifiers, which limits their cross-task generalization capability. In this paper, we tackle these challenges by observing that the core of many planning problems lies in optimization problems: searching for the optimal solution (best plan) with goals subject to constraints (preconditions and effects of decisions). With LLMs' commonsense, reasoning. and programming capabilities, this opens up the possibilities of a universal I.I.-- based approach planning problems. Inspired by this observation, we propose LLMFP, general-purpon framework that leverages LLMs to capture key infor- mation from plamning problems and formally formulate and solve them as opti- mization problems from seratch, with no task task-specific examples needed. We apply LLMFP to 9 planning problems, ranging from muulti-constraint decision making to multi-step planning problems, and demonstrate that LLMFP achieves average 83.7% and 86.8% optimal rate across 9 tasks for GPT-4o and Claude 3.5 Son- net, significantly outperforming the best baseline (direct planning with OpenAl ol ol-preveew) with 37.6% and 40.7% improvements. We also validate components of LLMFP with ablation experiments and analyzed the underlying success and failure reasons. Project page: http://sites..gogle.oom www.gol..oco//ee///mmii...
INTRODUCTION
Making complex plans subject multiple constraints is time- and labor-intensive process, but is critical in many aspects of our lives such as work arrangement, business management, logistics, and robotics. In the past, people used domain-specifie tools and languages make specific plans in their areas, which often required steep learning curve and were hard to adapt to other domains. When large language models (L..Ms) emerge with their versatile capabilities such as language understand- ing, reasoning, and tool-using, using LLMs for planning has gained significant traction.
For such planning systems be deployed complex, real- world applications, two desirable proper- ties need to be satisfied: 1). Zero-shot flexibility: Unlike many experimental settings where planning tasks usually come with labeled datasets, is very challenging to request such datasets from users in many realistic settings. Ideally, flexible plamning system should be able conduct planning with only a task description provided by users, and nothing clse. 2). High performance on compler tasks: Realistic planning problems usually require multi-step, long-horizon solutions, with many explicit and implicit constraints.
However, there is off between flexibility and task complexity, and thus existing LLM-based planning systems are typically unable to achieve both simultaneously. On one hand, planning sys- tems capable of performing zero-shot planning, utilizing the abundant knowledge and generalization

---

Published as a conference paper at ICLR 2025
Natural Language Task Description
Background Information
Query
Coffee company sources beans
suppliers with fixed capacity, roests them at two lacilities into either dark or light coffee. ships the roasted coffee three retail locations... The company nim to minimize the total cost while fulfill the demand at each retail
coffee peoded for cafe shipping, cost supplier to mastery roasting, cost -... shipping cost mastery to cafe
What
outcome if cafel experienced 23% rise in demand?
location.
Definer
Formulator
Code Generator
Goal
solver Optinizel)
solver initiation
Rumiine Errar
Minimize total cost, including
'variable 1
variable_ inieiation..
Decision Variables
"svr variable": true, "nunber of variables"' "data soureouure"" "variable__
"nams": "total_com costt",
total. cost Int Int 'tttal cost
other variables
beans shipped from.... ... to
dark coffee roasted
solver add[taoal cost Sum(-]) constraint assertion...
dark coflee shipped from
to variable
"value"; "sun of all costs"
All docision variables 20
Constraints
"mininize"
solver wwn.nineetttallll...o.......i cost)
goal
Beans shipped from each supplier s supplier capacity
Result Formatter
No, modified
No, modified
No, modified
X Self Assess & Modification
Definer Correct?
2. Formulator Correct?
3. Cude Generator Correct?
Plan
Consider goal and all constraints?
Consider constraints, goal, and query?
Create all needed variables?
Yes
An overview of LLMFP and how it is applied sections in yellow are inputs, and all sections in blue are steps accomplished by LLMs. With task description, background information, and query as inputs, L.MFF defines the goal, decision vari- ables, and constraints of this optimization problem, identifies all necessary variables and summarizes their key information into JSON representation, generates codes to solve the optimization problem, executes the codes and formats the execution results, and performs self-assessment for each step.
Figure 1:
coffee supply chain example. All
capabilities in LLMs, in many successful applications can only solve single-objective tasks such as household chores, with st-p-by-step interactive planning and grounding (Huang et al.] [022a]] |Ahn et al. 2022 Huang et al. 2022b). For complex, multi-constraint, and long-horizon tasks that involve iterative trials and errors even for humans, L.LSs still do not have the capabilities to generate valid plans by themselves (Kambhampati et al. (024). On the other hand, recent research efforts to em- power LLM-based planners to solve complex tasks are often based on well-designed task-specific in-context examples and extensive task-specific pre-defined efforts (Liu al.) 2023 Xie al. [023] ei al. 2023 Song et al., 2023) Gundawar et al., 2024), impacting their zero-shot flexibility. In short, few existing planning systems can flexibly resolve generic complex tasks with only task de- scriptions in natural language. Hence we ask: Can we build a universal LLM-based planning system that can solve complex planning problems without task-specific efforts?
In this paper, we observe that, although planning problems come with drastic variations, many of them can be recast constrained optimization problems - The optimization problems aim find the optimal solution, which is equivalent to locating the best plan that satisfies the goal for planning problems; the decision's 's precondition and effect are equivalent to constraints of optimization prob- lems. Furthermore, although solving complex planning tasks is generally beyond the capabilities of LLMs, converting any planning tasks into optimization problems much more tractable problem, and can be within the zero-shot capabilities of LLMs..
Motivated by this, we propose I.MM-Based Formalized Programming (LLMFP, illustrated in Fig. [] general-purpose zero-shot planning framework that leverages LLM'' strong common reasoning, and programming capabilities to encode planning problems into optimization problems without any task-specific examples or designs, combined with formal planner to solve the opti- mization problem. LLMFP takes in natural language domain description, natural language query under this domain, and available background information APIs inputs, and solves the plan- ning problem in five steps. First, LLMFP prompts LLMs to reason and propose the goal, decision variables, and key constraints necessary for the task. Second, based on the response, L.MFFP asks
2

---

Published as a conference paper at ICLR 2025
LLMs to formulate representation that includes all variables needed construct and their detailed information and requirements. Third, with the representation, LLMs write codes to formally encode the problem into optimization problem. Fourth, LLMFP executes the generated codes, converts the execution results into plans. Finally, LLMFP performs overall self-assessment and automatic modification to fix the broken parts of the previous steps. Currently, LLMFP uses the satisfiability modulo theory (SMT) to encode the optimization problems but can be adapted to any planners or solvers by updating the requirements and representation format in the prompts.
We evaluate our framework with 9 diverse planning problems, ranging from single-step supply chain problem to multi-step robot block stacking and moving. Experiment results demonstrate that LLMFP achieves strong performance across all tasks with an average of 83.7% and 86.8% opti- mal rates for GPT-4o and Claude 3.5 Sonnet, which greatly outperforms the baselines, including direct plan generation with OpenAl ol -preview.. We conduct ablation experiments to validate the key components of LLMFP and investigate the underlying reasons why more effective than baselines. In addition, although our framework does not require task-specific examples, we show the ease of adding task-specific examples to one stage of LLMFP, and how it could help to clarify unclear queries and therefore can further improve the performance within the same domain.
In summary, our key contributions are:
We offer novel perspective using LLMs to solve planning problems by rigorously constructing optimization problems from scratch, alike how human experts use optimization tools for planning.
We propose LLMFP, a general-purper purpose planning framework with zero-shot generalization capa- bility. To our knowledge, LLMFP the first enable LLMs to build and solve diverse types of planning problems as optimization problems with no task-specific examples or external critics.
LLMFP notably achieves 83.7% and 86.8% optimal rates for GPT-4o and Claude 3.5 Sonnet, outperforming the best baseline (direct planning with OpenAl ol ol-previww) by 37.6% and 40.7%. We examine the effectiveness of our framework and analyze the success and failure reasons.
RELATED WORKS
2.1
LLMS FOR PLANNING
The remarkable capabilities of LLMs in reasoning (Wei et al. 2022: Kojima et al. 2022 Yao et al.,
2022 2024) Raman et al. 2024) and tool-use (Qin et al. 2023 Schick et al. 2024) brings up inter- ests of utilizing L.MSs for planning problems. Based on L.LMs's 8 zero-shot generalization capability, many works are proposed to perform zero-shot planning (Huang al.] 2022a Ahn et al., 2022] Huang et al. 2022b). However, their planning scenarios are limited to single-objective tasks such as household cleaning and often require step-by-step interactive planning with grounding. To im- prove LLMs planning capabilities for complex problems, chain-of-thought (CoT) prompting LLMs to perform by-p-bb-stepp reasoning (Wei et al.
2022). Recent works also propose to aid the LLM
planning with external tools (Liu al. 2023 Guan et al. 2023 Chen et al. 2023 Li et al. 2023
Gundawar et al. 2024: Hao et al. 2024 Chen et al. 2024). For example, (Liu et al. 2023 Xie et al,
2023]Gundawar Gundawar al. (2024) leverages LLMs to translates problems into fixed formats and solve with external planners, (Li ct al. (223)) prompts LLM to add short codes to existing codes to account for follow-up what-if questions, and (Gundawar et al. 2024 Chen et al. 2024) empowers LLMs to iteratively refine plans or prompts based on feedback from external task- critics/verifiers/hu mans. However, achieve strong performance, these methods often need extensive task sescispecific pre-defined efforts. For example, CoT depends on task-specific examples to achieve strong per- formance, planning domain definition language (PDDL) domain files (Aeronautiques et al. [998] Haslum al., 2019)
required for (Liu et al. 2023), mixeddinteger linear program (MILP) codes of current domains are necessary for (Li ct al. 2023), ) and external constraint critics are needed for (Gundawaar et al. 2024 These requirements limit their generalization capabilities to new domains.
2.2 LLM SOLVER
As existing LLMs do not have the capability to perform long- hong-horizon reasoning for complex tasks (Achiam et al. 2023 Valmeekam et al., 2022 2023 Kambhampati et al.) (024). many works propose to take advantages of both L.LMs and external solvers by combining them for reasoning
3

---

Published as a conference paper at ICLR 2025
or planning. (Wu et al. 2022 He- Yueya al. 2023 Pan et al. 2023 Ye al. 2024 combines
LLM with symbolic solvers to solve logical reasoning problems. While most logical reasoning problems are single-step satisfaction problems with clear constraint descriptions. LLMFP aims to solve complex planning problems, which could include implicit constraints not described the task description and could be sequentially long-horizon tasks with defined actions. In addition, LLMFP
proposes general approach, which does not require task-specific examples or task-specific efforts. (Li al. 2023) teaches LLMs to add constraints to existing MILP codes. (Li et al. (022) asks the developer to express planning problems into automaton and plan based on it. (Manas et al. 2024) uses LLMs to translate problem into linear temporal logic and solves with optimization solver. (Liu)
et al. 2023: Guan et al. 2023 Zhou et al.) 2024 Xie et al. 2023) leverages PDDL planner to aid the planning processes. Except for the natural language task description, they require human efforts
to design solver-related specifications and task-specific examples, which is not needed for LLMFP.
3 LLMFP
LLMFP aims to resolve generic planning problems. For example, consider coffee supply chain problem, where a coffee company sources beans from three suppliers with fixed capacity, roasts them at two facilities into dark or light coffee and ships the roasted coffee to three retail locations to fulfill their demands. Then planning problem involves accomplishing the task at the cheapest cost.
To achieve this, LLMFP takes the following inputs from users, as shown Figure (top panels).
Natural Language Task Description d. A natural language description that details the problem settings and the planning objective, such as the above description of the coffee problem.
Background Information & API i. A list of background information about the tasks as well as information APIs that the planner use. An example of the background information for the coffee task is the variables containing specific numbers of supplier capacities, cafe demands, and costs for shipping and roasting.
User Query q. The question that either describes the detailed initial and/or goal states or adds/- modifies existing requirements of the tasks. In the coffee planning task, one example query is 'What would be the outcome if cafel experienced a 23% rise demand".
Example inputs for all tasks can be found in Appendix A.8 Note that LLMFP does not require any task-specific examples from the users. Considering the diversity of user requests, LLMFP needs to accommodate large variety of domains, planning problem setups, user queries, constraints, and complexity levels, which poses a great challenge.
3.1 OVERVIEW
Devising solutions for the vast diversity different complex planning problems seems prohibitive even for humans, let alone LLMs. However, despite the diversity of the planning problems, plan- ning problem can generally be cast as a constrained optimization problem. Formally, a constrained optimization, P {ax, f()), (-), h(-)), is defined min, (x) s.t., g(x)) < 0, h(z) 0. rep- resents the decision variables; f(-) represents the objective function; g(-) represents the (multiple) inequality constraints; hu-)) represents the (multiple) equality constraints. In the coffee supply prob- lem, æ includes the amount of coffee beans shipped from each supplier to each roastery, and from each roastery to each cafe, f describes the total cost of shipping, and g and h include the clearing conditions for each facility. A detailed description of the variables and constraints for the coffee supply problem can be found in Appendix A.922
Once a planning problem is formulated as the constrained optimization problem, it can be rigor- ously solved by solvers such as the SMT solver. Therefore, we propose an pipeline that solves the planning problem by converting them into constrained optimization problems and then solving them using the SMT solver. Our pipeline consists the following steps, as shown in Fig. 1 DEFINER: LLMFP first prompts LLM to define the constrained optimization problem from the user inputs, P = D(d, i) (Sec. 3.2 @ FORMULATOR: LLMFP asks LLM to think about the necessary vari- ables and steps build when programming, and formulate representation to summarize all key information of these variables (S3c. @ CODE GENERATOR: Given this representation, LLMs generate codes that initialize all necessary variables, assert constraints, and add goals (Sec. 3.4
4

---

Published as a conference paper at ICLR 2025
4 RESULT FORMAITER: After LLMFP executes the generated codes, it prompts LLMs to vert the execution result into a fixed format and provide a short assessment of the execution results (Sec. 3.5). @ SELF ASSESSMENT AND MODIFICATION: LLMFP assesses each step based on the execution result, and modifies the first incorrect step (Sec. (Sec.3.6). The generated plan is delivered when it passes self-assessments of all steps. Please refer to Appendix [.. and A.10 for example outputs and prompts of all steps in LLMFP.
3.2 DEFINER
The first step of building an optimization problem is to identify the goal, decision variables, and constraints of the problem from the user-supplied task description d and background information i, ie.. P = D(d, a). The definer function ' is accomplished by prompting the LLM to express in a natural language format (See F for an example), where the prompt skeleton includes O description of what goal, decision variables, and constraints mean and @ an instruction to output the aforementioned information, which is invariant across tasks. The detailed prompt listed in Appendix(AA101.
While generating the goals and decision variables are straightforward, generating the constraints is challenging, because certain constraints are not explicitly stated and can only be inferred by commonsense reasoning. We refer to these as the implicit constraints.
For example, in the coffee supply chain task example, the implicit constraints include 'the roasted coffee in each roastery does not exceed the beans it receives', 'the shipped coffee from each roastery does not exceed the coffee it roasts', and importantly but easily overlooked, 'all numbers of shipped and roasted beans and coffee need to be non-negative integer'.
To facilitate uncovering the implicit constraints, include in the prompts (under the description of constraints) a three-step instruction to derive the constraints: Identify all decision variables in this task, @ for each pair of decision variables, consider relations (explicit, implicit, underlying assumption, unmentioned commonsense) between them to make sure all variables are connected, and @ provide a constraint reasoning first before answering. This effectively helps LLMs to better identify implicit constraints for multi-constrain planning problems. Since for multi-step planning problems the task description needs to explicitly define the preconditions and effects of each action, there will be no implicit constraint so this step is omitted.
3.3 FORMULATOR
Before turning the optimization problem P into an executable code to run the SMT solver, the FORMULATOR is called to supplement P with additional information regarding each variable in æ that is necessary to ensure the correctness of the generated code. Formally, the FORMULATOR is defined as R = F(P, d. i, 4). where the output R is a JSON representation that describes N fields of information for each variable in æ, i.e. R = U,f
(z:).. fieldy (x:)). Examples
of the JSON representation are shown in Fig. 2
Each field describes the information that governs the declaration and instantiation of each variable in the code, such as whether the variable is continuous, binary, or integer, what data structures they need to be arranged into, etc. The definition of fields is different for single-step and multi-step problems. In what follows, we will describe the fields in each problem type.
Single-Step Cunti-Constraint Problem As shown in Fig. and Fig. 2 for each variable, R includes 6 fields to summarize the information related to this variable. The name field indicates the variable name. Since we are using SMT as the optimization solver, the SMT sirl_arrias whether the variable is an SMT or a normal variable. SMT variables are different from normal vari- ables in that they don't hold specific values upfront, rather, they are symbolic variables represent unknown values. The number_of vari sarriaalee field represents the length of the variable. The aata_source field denotes the dependencies of the variable. The value field further specifies the value of the variable. It could either be a real number list, dictionary descriptions, any de- scription of operations to do with the data source. The specific recuirement field is where point to the constraints or goals related to this variable. For the coffee supply chain task, the total cost has a length of 1, the dat a a___suuucee ce is all shipping costs and roasting costs, the value is string "sum of all costs", and the specific requirement is "minimize". With this intermediate
field indicates
5

---

Published as a conference paper at ICLR 2025
Example Fomulator output for multi-construint
Example Fomulator output for multi-step Blucksworld
"variable_1",
'objects": : {
"name": "beans_pae beans_shippped',
"variable_1"
"SHZ varioblei": true,
"nams": "abjacts",
"number_of_ ofw.oof_rrrrbeessreeeoee
"sMT variable": false, "sumber of var.aaabbl" 1, source's "query',
:ata_source": "capacity_ ier
shipping cost. from supplier so zoastery', "value": "amount of coffee beans shipped from
"value"; "all objects in the problen."", apeoitic_requiremene null
each supplier to each roastezy",
"nuat exceed
suppliar" capacitty
predicates':
"variable_2"::
varriabl__""
"namee" "light_ roastedd_
"name"1 "on",
"SHI _ur_variablee: true, "number_of of viraiibbles's
"sMI variable"; false,
"sumber of of_ovofavblee""" 1,
"data satta_o "roassing_coat, tight,,
"data_ dattasouuee": "query, variable_1',
"value": "anount of Light coffee roasted at each roastary",
"value' ; "a dictionary of boolesn variables
representing whether a block is on another black at tinestep.",,
"specific requirenent"; "total smount of coffee beans received by each roastery must equal total anount of coffee roasted'
"specifie roquirenent": initialize stop
"vrriable_""
'actionss !
"name" i "totsl_com cost',
"variable_""'
"SHT v
"name" "pickup",
"nunber variables":
"swe varisble' false,
"variable 1, variable 2,
"number_of_ variialeearrible""
variable variable 4, variable_51 5", "value"; "sun shipping, roasting, shipping roasted coffee costs",
"data dataa_oouree's "variable_""
"value": "a dictionary
variables
representing whether pickup is perforned on a block."", appeifii_rrquirenent null
"ninimize"
Figure 2: Example FORMULATOR output for multi-constraint Coffee and multi-step Blocksworld.
step between DEFINER and CODE GENERATOR, LLMFP is capable of obtaining a more detailed, well-formulated, and overall coding plan. We teach LLMs to formulate variables into this represen- tation by including two simple examples in the prompt. These two examples are not task specifcc examples of any of our testing tasks and we do not modify these two examples across tasks.
Multi-Step Planning Problem Since multiple steps are involved in multi-step planning problems, the FORMULATOR not only needs to deal with relationships between variables, but also needs to update the states of variables across different timesteps. All the variables are divided into five sec- tions, representing the five stages for defining the variables: ob jbjets,, predicat cidicattss act ions, update, goal. The variables within each section are appended with the same six fields as intro- duced above. The objects stage declares all objects in the scenario. The predicates stage defines the predicates, which represent the properties of objects and the relationships between them. The act i ons stage initializes variables represent all actions. The update stage adds assertions to existing action variables to account for the preconditions and effects of actions. The goal stage adds constraint to existing predicate variables encode the goal. As shown in Fig. 2 the example on the right includes different stages and variables within each stage. The variable in predi cates stage initializes a dictionary to represent whether a block on another block certain timestep. The variable in act i onstage initializes dictionary to represent if action pickup is performed. For multi-step planning tasks, we replace the examples with a multi-step task example, and similarly, is not a task-specific example and we do not modify it across tasks.
The formulator function F is enabled by prompting an LLM, where the prompt includes 1 brief instruction for the FORMULATOR, @ Example input-output pairs of FORMULATOR demonstra- tions, and @ The user-provided task information and DEFINER'S output. The detailed prompt is listed in Appendix A..0. Note that although example input-output pairs are used, they are task- agnostic examples fixed for all the planning tasks. No task-fpe examples are needed.
3.4 CODE GENERATOR
With R. now we have all the information needed to build an optimization problem with codes. In the CODE GENERATOR's prompt, we explain the meanings of different stages and fields in R.

---

Published as a conference paper at ICLR 2025
and ask LLMs to follow Python and Z3 SMT syntax (De Moura & Bjömer 2008 By including user inputs and results from DEFINER and FORMULATOR, with no examples, LLM could reliably generate reasonable, executable, and correct Python codes. Then, LLMFP executes the codes and returns to re-generation there are runtime errors. We set maximum re- re-generation times to be 5.
3.5 RESULT FORMATTER
Since the variable names are decided by LLMs and have chances to be very different across queries, after code generation, we use RESULT FORMATTER to ask LLM to convert the execution result to a fixed output format. For example, the output for the coffee task would be a JSON includes: O the number of coffee beans shipped from each supplier to each roastery, @ the number of light and dark coffee roasted in each roastery, and @ the number of light and dark coffee shipped from each roastery to each cafe. After filling in this result, we prompt the LLM to provide brief evaluation of the result based on whether the result achieves the goal, satisfies constraints, and makes sense in common sense. Taking commonsense into consideration is important because sometimes if necessary constraint is missing from the DEFINER step. it could result in unreasonable execution result that is unrealistic in commonsense. For example, for the coffee task, if the DEFINER does not include the non-negative constraint, to minimize the cost, the solver could propose negative units of shipped coffee. Detecting these unrealistic plans helpful for SELF ASSESS MODIFICATION.
3.6 SELF ASSESS & MODIEICATION
With the execution result and evaluation, LLMFP perform self-assessment to reason about the cor- rectness and provide ratings for the DEFINER, FORMULATOOR, and CODE GENERATOR. If the assessment marks all three steps to be correct, this plan will be delivered as the final output. Oth- erwise, the assessment will reason about how to modify this step, and provide a modification by itself. This modification will replace the output of the incorrect step and LLMFP will loop back to continue the next steps from there again. That is, if the SELF ASSESSOR thinks the FORMULATOR output is incorrect, it will generate a JSON representation R' by itself, and the framework will use this R' to enter CODE GENERATOR again. We set the maximum number of loops to be 5.
3.7 CHOICE OF SOLVER
As a framework that formulates and solves planning problems as optimization problems, LLMFP can be adapted to use any planner or solver by modifying the requirements prompt to follow the syntax of new solvers. In this work, we compare with popular PDDL and MILP solvers and choose SMT solver with following reasons: SMT allows explicit goal and constraint assertion from scratch, which can solve both single-step multi-constraint problems and multi-step problems, while PDDL. solvers require rigidly formatted PDDL domain and problem files, which limits applicability for non-PDDL problems. SMT is complete and sound, guaranteeing optimal plans, while PDDL planners lack completeness guarantees. Additionly, for all optimization solvers like SMT and MILP, building optimization problems involves the same steps: defining the goal, constraints, and decision variables, and writing codes to encode relationships between decision variables. Thus, utilizing any optimization planner has a similar process. We show how easily our framework could adapt to use MILP by including prompt differences and output examples in Appendix(A.I 10.3 We selected SMT over MILP because the SMT 73 solver is publicly available and more accessible to all users than the Gurobi MILP solver, which requires licenses and limits the number of devices per license.
4 EXPERIMENTAL RESULTS
4.1 DOMAINS
We test on 9 planning problems, which includes multi-constraint decision making tasks, Coffee, Workforce, Facility, Task Allocation, and Warehouse, and 4 multi-step tasks, Blocksworid, Mys- tery Blocksworld, Movie, and Gripper (Li et al., 2023; Valmeekam et al.] 2024 Stein & Koller 2023]. Task descriptions and complexity analysis either if questions that change/add constraints to the existing scenarios or different task initial and goal conditions. Task inputs including example queries are given in Appendix A.8
included in Appendix A. The queries are
7

---

Published as a conference paper at ICLR 2025
Table Optimal rate (%) comparison of L.MMP with baselines 5 multi-constraint problems.
Method
Coffee
Workforce
Facility
Task Allocation
Warchouse
Average
DirecloPT-4o
0.8
2.6
0.0
0.0
0.0
0.7
Direclet-ravvew
25.9
47.6
4.0
29.7
CGTGP1-40
0.0
5.6
0.0
0.0
16.0
4.3
CodeGPT-4u
17.7
75.8
53.9
0.0
8.0
31.1
Code_SMTaPran
0.0
10.8
0.6
0.0
2.0
2.7
L.MFP GPT-40
64.7
92.2
70.7
96.0
72.0
79.1
Directe AUDE SOWNET
0.0
0,0
0.0
0.0
0.0
0.0
CLAULAD 3.5 SONNET
7.1
0.0
0.0
CodECLAUDE SONNET
59.8
71.9
47.3
0.0
42.0
Code SMT CLAUDE
75.6
36.8
49.7
86.0
64.0
62.4
LLMFP CLADDE SONNET
80.5
88.7
48.2
96.0
90.0
80.7
Table 2: Optimal rate (%) comparison of LLMFP with baselines on multi-step problems.
Method
Blocksworld
Mystery Blocksworld
Movic
Gripper
Average
DirectarT-40
41.5
0.8
85.7
32.0
Directot-raovINN
88.4
31.9
100.0
52.0
68.1
CoTCPT-10
39,9
2.7
81.0
0.0
30.9
CodCoPT-to
0.0
0.3
0.0
0.0
0.1
Code SMTGPT-tO
0.0
0.0
0.0
4.0
1.0
L.MFP GPT-4n
96,2
77.7
100.0
87.5
DireCTCLAUDE 3.5 SONNET
43.2
0.5
100.0
12.0
38.9
CoT CLAUDE 3.5 SONNET
COTCCLADE
52.8
2.8
100.0
28.0
45.9
CodeCLATOE
0.0
0.0
0.0
0.0
0.0
Code SMT CLADDE SONNET
0.0
0.0
0.0
0.0
0.0
LLMFP CLAUDE 3.5 SONNET
93.0
98.0
100.0
76.0
91.8
4.2 LLMFP PERFORMANCE
We evaluate LLMFP 9 tasks with GPT-4o (gpt) and Claude 3.5 Sonnet (cla) with temperature 0. Each task comes with natural language task descriptions, queries, background information, APIs. LLMFP takes these inputs and outputs plans with task-specific examples. We optimal rate as the evaluation metric, measuring whether plans are optimal for given the task and query. We also include success rate as another metric and results A.3
Baselines We compare LLMFP against 1) Direct: LLM direct plan generation, 2) CoT: chain-of- thought prompting ( (Wei al. 2022) by asking L.LMs reason before generating the final answer, 3) Code: prompts LLM to generate Python codes to solve the problem, allowing the use of any package or solver, and 4) Mode_SMT: prompts LLM to generate Python codes using Z3 SMT solver, the same tool we use in LLMFP. For all baselines, we use both GPT-4o and Claude 3.5 Sonnet and also include a direct plan generation baseline with OpenAl ol orevveww (olp). All baselines are zero-shot with no task-specift examples. All baselines have the same input information as LLMFP, including task description, task background information or info collection API, and query. We also provide all baselines with formatters convert their generated plans to fixed formats for better evaluation. Please refer to Sec. A.10 for prompts of baselines.
Results and Analysis We include the optimal rate comparison of LLMFP and baselines 5 multi- constraint problems and 4 multi-step problems Table and 2 There are four key takeaways:
First, LLMFP achieves strong performance across all 9 tasks, significantly outperforming all base- lines. For GPT-4o, LLMFP achieves an average of 83.7% optimal rate across 9 tasks (79.1% for multi-constraint problems and 87.5% for multi-step problems). For Claude 3.5 Sonnet, LLMFP achieve an 86.8% optimal rate across 9 tasks (80.7% % for multi-constraint problems and 91.8% for multi-step problems). For 5 multi-constrrint problems, L.MFP 4PT-40 and L.MMFP CLAUDE 3.5 SONNET
8

---

Published as a conference paper at ICLR 2025
outperform best baselines Codeopt-4o and Sasie_SMccAuu 3.5 SONNET by large margin of 48.0% and 18.3%. For 4 multi-step problems, LLMFP GPT-40 and LLMFP CLAUDE SONNET outperform Directai PREVIEE and CoTCLAuDe 3.5 SONNET by average of 19.4% and 45.9%. This highlights both the effectiveness and the generalization capability of LLMFP.
Second, among baselines, Code works better for multi-constraint problems, while Direct and CoT work better for multi-step problems. This validates that the skills required for solving different tasks are different. For multi-constraint problems, heavy calculations are required test every possible solution, it is hard for LLMs to directly plan, even with the strongest o lpprevvew model. For multi-step problems, since Code tries to use PDDL planner, which requires LLM generate fixed- format PDDL domain and problem files, it almost always fails to generate and call them correctly. While it is easier for LLMMs to directly devise plans as the preconditions and effects of each action are easier to reason about than calculations. This further proves that L.MMP can tackle problems that are fundamentally different because it uses a universal and formal approach for all tasks.
Third, for Direct and CoT, Mystery Blocksworid's performance degrades largely compared to Blocksworld, though they are fundamentally same problems. Changing predicate and action names to illogical names makes LLMs hard to understand the problem and generate reasonable plans. However, LLMFP still can obtain an overall strong optimal rate of 77.7% and 98.0% on Mystery Blocksworld for GPT-4o and Claude 3.5 Sonnet. This shows LLMFP is robust to obfuscated prob- lems, as can encode the problem as long as the problem is clearly defined regardless of the names.
Fourth, for multi-constraint problems. Code Mode_SMTcLAUDE 3.5 SONNET improves 18.2% compared to COdECLAEDE SONNET: though Code_ Mode_SMTarr-4 performs poorly. This showcases the strong cod- ing capability of Claude 3.5 Sonnet, especially the capability to understand and utilize the SMT solver. At the same time, it showcases the instability for different LLMs to reach strong perfor- mance, motivating the need of frameworks like LLMFP to overcome existing limitations of LLMs.
To summarize, LLMFP is capable of solving all 9 tasks with strong performance and is robust to fundamentally different and obfuscated problems. We show the performance of LLMFP across iterations, time and cost statistics, and failure analysis in Appendix A.2 A.4 and A.6
4.3 EFFECTIVENESS OF LLMFP COMPONENTS
We then validate each component of LLMFP with ablation experiments on 9 tasks. We examine the effectiveness of DEFINER, FORMULATOR, and SELF ASSESS & MODIFICATION by removing these components from our framework one at a time and comparing with LLMFP. We do not re- move CODE GENERATOR and RESULT FORMATTER because they are the necessary components of LLMFP to deliver outputs. We use GPT-4o as the LLMs and optimal rate as the evaluation metric.
Results and Analysis We include the optimal rate performance comparison of LLMFP and baselines on 9 problems in Table From Table there are two key takeaways:
First, removing any of the components from LLMFP negatively affects the performance. For multi-constraint problems, removing DEFINER, FORMULATORR and SEL. ASSESS & MODIFICA- TION lowers the optimal rate by 15.4%, 22.2%, and 21.9%. For multi-step problems, removing FORMULATOR and SELF ASSESS & MODIFICATION reduces the optimal rate by 87.4% and 12.4%.
Second, for different problems, the most effective components are different. Coffee degrades the most for No DEFINER; Warehouse and all multi-step problems drop the most for No FORMULATOR; and Workforce decreases the most for No SELF ASSESS & MODIFICATION, This again validates the diversity of the 9 problems and how they require different efforts to be successfully solved. Thus, I.IMFP an overall framework that could aid the process of planning from all aspects.
To summarize, all three components in LLMFP are effective and could account for diverse problems by providing comprehensive aids to solve planning problems.
4.4 LLMFP WITH TASK-SPECIFIC EXAMPLE
Although LLMFP is capable of achieving strong performance wide range of problems with no task-specific example, we test LLMFP by only replacing the two examples in FORMULATOR to one task-specific example Coffee task to how much the task-specific example could further

---

Published as a conference paper at ICLR 2025
Table 3: Optimal (%) comparison when removing key components of LLMFP all tasks. LLMs used are GPT-4o.
Domain
No Definer
No Formulator
No Self Assess Modification
LLMFP
Coffee
8.6
56,4
55.3
64.7
Workforce
84.4
80.5
27.3
92.2
Facility
61.6
53.7
53.7
70.7
Task Allocation
74.0
92.0
96.0
96.0
Warehouse
90
2.0
54.0
72.0
Average
63.7
56,9
57.2
79.1
Blocksworld
95.3
96.2
Mystery Blocksworld
N/A
0.0
74.4
77.7
Movie
N/A
0.0
66.7
100.0
Gripper
N/A
0.0
64.0
76.0
Average
N/A
0.1
75.1
87.5
Table 4: Optimal rate (%) comparison of LLMFP and LLMFP with one task-specific example in Formulator on Coffee task. Sets represent different types of what-if questions. LLMs are GPT-4o.
Method
Set
Set 2
Sct
Sct
Set 5
Set 6
Set
Average
LLMFP
58.3
70.9
42.4
80.0
83.3
81.5
61.2
LLMFP TASK-SPUCIIIE
78.3
72.7
70.6
84.8
91.4
100.0
100.0
85.4
improve LLMFP. Queries of Coffee tasks are what-if questions and are categorized into 7 sets. Each set is type of question. For example, every query in Set asks about what the demand some cafes increases some amount. For each set, include one task task-specific example in FORMULATOR prompt, test LLMFP TASK-SPECIFIC over this set, and compare with LLMFP over this set. We use GPT- 4o for LLMs. From the results in Table 4 we observe that average, LLMFP TASK-SPECiRIC improves the performance of LLMFP by 24.2%. The performance of Set 3 increases the most. This set asks queries like "What led the decision use supplier3 for the roasting facility at roasteryl?", It is both plausible to test "using supplier3" or to test "not using supplier3" to answer the question. However, the ground truth answer for this type of question is "not using supplier 3". They are confusing queries even for humans to understand, let alone LLMs. Thus, for these queries, adding task-specific examples significantly improves the performance. To summarize, LLMFP can achieve strong performance with no task-specific example, but easily adding task-specific examples only to FORMULATOR improves the performance, especially when task or query not clearly presented.
5 CONCLUSION
To account for the challenging trade-off between flexibility and task complexity for LLM planning, we observe that the core of many planning problems lies in optimization problems and propose universal approach for LLMs solve planning problems. We propose LLMFP, a general-purpose LLM-based framework that captures key information from planning problems and formally formu- lates and solves them as optimization problems, with no task specific examples needed. We test LLMFP 9 diverse planning tasks with two LLMs,, proving LLMFP can achieve strong perfor- mance over fundamentally different tasks and showing the effectiveness of components in L.MMF..
Limitations LLMFP needs clear and detailed task descriptions and queries. It is hard for LLMFP to define the problems' goals and constraints if the task description is ambiguous or missing some important information. In addition, since LLMFP solves encoded planning problems with optimiza- tion solvers, the capability of LLMFP depends on the strength of the solvers. For massive databases with numerous feasible plans, the speed for solvers to search for optimal plans is slow. Ways to mitigate this is to introduce heuristics to prioritize a portion of the choices or to switch from solving optimization problems satisfaction problems for planning tasks that do require optimality.
10

---

Published as a conference paper at ICLR 2025
6 ACKNOWLEDGMENTS
This work was supported by ONR under Award 400014-22-1-2478 and MIT-IBM Watson Al Lab. However, this article solely reflects the opinions and conclusions of its authors.
REFERENCES
Claude 3.5 sonnet. cessed: 2024-06-20.
httes:/www.aatrrroo..cco//////sc 5-sonneet
Ac-
Hello gpt-4o.
http:///ppeas..ccmmm/deexx//llo/
Accessed: 2024-05-13.
Introducing openai ol-preview.
https://openai: cwm.indemiesnne.ee.. Accessed: 2024-09-12.
Latrroddciing-opeaaa-----------e
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, llge Akkaya, Florencia Leoni Ale-
man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXivv2303.08777 2023.
Constructions Acronautiques, Adele Howe, Craig Knoblock, ISI Drew McDermott, Ashwin Ram, Manuela Veloso, Daniel Weld, David Wilkins Sri, Anthony Barrett, Dave Christianson, et al. Pddl the planning domain definition language. Technical Report, Tech. Rep., 1998.
Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al. Do as can, not as say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.0166 2022.
Yongchao Chen, Jacob Arkin, Yang Zhang, Nicholas Roy, and Chuchu Fan. Autotamp: Au- toregressive task and motion planning with llms as translators and checkers. arXiv preprint 06530,, 2023.
Yongchao Chen, Jacob Arkin, Yilun Hao, Yang Zhang, Nicholas Roy, and Chuchu Fan. Prompt optimization in multi-step tasks (promst): Integrating human feedback and preference alignment. arXiv preprint MXXi:2402.087000 2024.
Leonardo Moura and Nikolaj Bjørner. 73: efficient smt solver. In International conference Tools and Algorithms for the Construction and Analysis of Systems, pp. 337-340, Springer, 2008.
Lin Guan, Karthik Valmeekam, Sarath Sreedharan, and Subbarao Kambhampati. Leveraging pre- trained large language models to construct and utilize world models for model-based task plan- ning. Advances in Neural Information Processing Systems, 36:79081-79000 2023.
Atharva Gundawar, Mudit Verma, Lin Guan, Karthik Valmeekam, Siddhant Bhambri, and Subbarao Kambhampati. Robust planning with Ilm-modulo framework: Case study in travel planning. arXiv preprini 2005v:2405.2062. 2024.
Naresh Gupta and Dana S Nau. On the complexity of blocks-world planning. Artificial intelligence, 56(2-3):223-255 1992.
Yilun Hao, Yongchao Chen, Yang Zhang, and Chuchu Fan. Large language models can plan your travels rigorously with formal verification tools. arXiv preprint arXiv:2404. 11911 2024.
Patrik Haslum, Nir Lipovetzky, Daniele Magazzeni, Christian Muise, Ronald Brachman, Francesca Rossi, and Peter Stone. An introduction to the planning domain definition language, volume 13. Springer, 2019.
Joy He- Yueya, Gabriel Poesia, Rose E Wang, and Noah Goodman. Solving math word problems by combining language models with symbolic solvers. arXiv preprint arXiv:2304.09102 2023.
Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language models zero-shot planners: Extracting actionable knowledge for embodied agents. International Conference on Machine Learning, pp. 9118-911977 PMLR, 2022a.

---

Published as a conference paper at ICLR 2025
Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, et al. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint FrXiv:2207.05608 2022b.
Subbarao Kambhampati, Karthik Valmeekam, Lin Guan, Kaya Stechly, Mudit Verma, Siddhant Bhambri, Lucas Saldyt, and Anil Murthy, Lims can't 't plan, but can help planning in llm-modulo frameworks. arXiv preprint urXiv:2402.01817, 2024.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199-222133 2022.
Beibin Li, Konstantina Mellou, Bo Zhang. Jeevan Pathuri, and Ishai Menache. Large language models for supply chain optimization. arXiv preprint arXiv:2307.0388 2023.
Zelong Li, Wenyue Hua, Hao Wang, He Zhu, and Yongfeng Zhang. Formal-llm: Integrat- ing formal language and natural language for controllable llmebased agents. arXiv preprint 2402:24022.0798, 2024.
Bo Liu, Yuqian Jiang, Xiaohan Zhang, Qiang Liu, Shiqi Zhang, Joydeep Biswas, and Peter Stone. Lim+ p: Empowering large language models with optimal planning proficiency. arXiv preprint arXiv:2304. 14477, 2023.
Kumar Manas, Stefan Zwicklbauer, and Adrian Paschke. Cot-tl: Low-resource temporal knowl- edge representation of planning instructions using chain-of-thought reasoning. arXiv preprint arXiv:24I0. 16207, 2024.
Liangming Pan, Alon Albalak, Xinyi Wang, and William Yang Wang. Logic-Im: Empower- ing large language models with symbolic solvers for faithful logical reasoning. arXiv preprint wwXiv:230.112295 2023.
Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, al. Toolllm: Facilitating large language models master 16000+ weal-world apis. arXiv preprint 1ixxi:2307.16788) 2023.
Shreyas Sundara Raman, Vanya Cohen, Ifrah Idrees, Eric Rosen, Raymond Mooney, Stefanie Tellex, and David Paulius. Cape: Corrective actions from precondition errors using large language mod- cls. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 14070- 14077. IEEE, 2024.
Timo Schick, Jane Dwivedi- Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36, 2024.
Chan Hee Song. Jiaman Wu, Clayton Washington, Brian M Sadler, Wei-Lun Chao, and Yu Su. Lim-planner: Few-shot grounded planning for embodied agents with large language models. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 2998-3009, 2023.
Katharina Stein and Alexander Koller. Autoplanbench: Automatically generating benchmarks for IIm planners from pddl. arXiv preprint 1tKi::::1/10088::1 2023.
Karthik Valmeckam, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. Large lan- guage models still can't 't plan (a benchmark for Ilms plamning and reasoning about change). In NewrlPS 2022 Foundation Models for Decision Making Workshop, 2022.
Karthik Valmeekam, Matthew Marquez. Sarath Sreedharan, and Subbarao Kambhampati. On the planning abilities of large language models-a critical investigation. Advances in Neural Informa- fion Processing Systems, 36:75993-76005, 2023.
Karthik Valmeekam, Matthew Marquez, Alberto Olmo, Sarath Sreedharan, and Subbarao Kamb- hampati. Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change. Advances in Neural Information Processing Systems, 36, 2024.
12

---

Published as a conference paper at ICLR 2025
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. th-thutt prompting elicits reasoning in large language models. Advances in neural information processing systems, 011244444248331 2022.
Yuhuai Wu, Albert Qiaochu Jiang, Wenda Li, Markus Rabe, Charles Staats, Mateja Jamnik, and Christian Szegedy. Autoformalization with large language models. Advances in Neural Informa- tion Processing Systems, 35:32353-32368 2022.
Yaqi Xie, Chen Yu, Tongyao Zhu, Jinbin Bai, Ze Gong, and Harold Soh. Translating natural lan- guage to planning goals with large-language models. arXiv preprint vrXiv:2302.05/28, 2023.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Ad- vances Neural Information Processing Systems, 36, 2024.
Xi Ye, Qiaochu Chen, Isil Dillig, and Greg Durrett. Satim: Satisfiability-aideed language models using declarative prompting. Advances in Neural Information Processing Systems, 36, 2024.
Zhehua Zhou, Jiayang Song, Kumpeng Yao, Zhan Shu, and Lei Ma. Isr-llm: Iterative self-refine large language model for long-horizon sequential task planning. In 2024 IEEE International Conference Robotics and Automation (ICRA), pp. 2081-2088. IEEE, 2024.
13

---

Published as a conference paper at ICLR 2025
PLANNING ANYTHING WITH RIGOR: GENERAL-PURPPSE ZERO-SHOT PLANNING WITH LLM-BASED FORMALIZED PROGRAMMING
Introduction
Related Works
2.1 LLMs for Planning
2.2 LLM + Solver
3
LLMFP
Overview
Definer
Formulator
3.4
Code Generator
3,5
Result Formatter
3.6
Self Assess Modification
3.7
Choice of Solver
Experimental Results
Domains
LLMFP Performance
4.3
Effectiveness of LLMFP Components
4.4
LLMFP with Task-Specific Example
5
Conclusion
10
6
Acknowledgments
11
Appendix
15
A.1
Domains and Complexity Analysis
15
LLMFP Performance over Iterations
18
A.3
Additional Metric Performance: Success rate
19
A.4
Time and Cost Statistics and Analysis
20
A.5
Baselines Failure Case Analysis
22
A.6
LLMFP Failure Case Analysis
24
A.7
Baselines with Explicit Optimal Requirements
26
A.8
Inputs on 9 tasks
27
A.9
Example Outputs on Coffee tasks
35
A.I0 Prompts
43
14

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1
DOMAINS AND COMPLEXITY ANALYSIS
We test on 9 planning problems, including 5 multi-constraint decision making tasks and multi-step
tasks Li et al. 2023 Valmeekam et al. 2024 Stein & Koller 2023)
Coffee Coffee company sources beans from three suppliers with fixed capacity, roasts them at two facilities into dark or light coffee, and ships the roasted coffee to three retail locations. The company aims to minimize the total shipping and roasting cost while fulfilling the demand at each retail location. There are 266 different queries of types in the dataset.
Workforce Assign workers to shifts; each worker may or may not be available on a particular day. The goal is to minimize the total payments workers while fulfilling the shift requirements for two weeks. There are 231 different queries of types in the dataset.
Facility company currently ships its product from plants to 4 warehouses. is considering closing some plants to reduce costs. The goal is to decide which plant(s) to close to minimize transportation and fixed costs. There are 165 different queries of types in the dataset.
Task Allocation Given tasks and three robots skilled in different tasks, the goal is to assign tasks to robots to minimize finish time. The finish time counts when the last robot stops working. There are 50 different queries describing the number of different tasks. This task and its data and queries are created by us.
Warehouse The robots need to finish tasks by visiting stations that are capable of accomplishing corresponding tasks. The goal is to find the list of stations while minimizing the total distance traveled. There are 50 different queries that include the random-length list of tasks to finish. This task and its data and queries are created by us.
Blocksworld The robot has four actions: pickup, putdown, stack, and unstack. The goal is to stack the blocks the scene from their initial setup to specific order with minimum steps. There are 602 different queries that describe blocks' initial conditions and goal states.
Mystery Blocksworld An obfuscated version of Blocksworld. The action and predicate names are replaced with names that logically make no sense. There are 602 different queries that describe objects' initial conditions and goal states.
Movie The goal is get the required snacks, watch the movie, and recover the movie and counter to the original state with minimum steps. There are 21 different queries that describe objects' initial conditions and goal states.
Gripper There are robots and balls in different rooms. Each robot, with two grippers, can pick, drop, and move balls between rooms. The goal is place balls specific rooms with minimum steps. There are 25 different queries describing objects' initial conditions and goal states.
The queries for Coffee, Workforce, and Facility are what-if questions that change or add constraints to the existing scenarios. The queries of the rest tasks are different task initial and goal conditions. Task inputs including example queries are given Appendix A.8
Mathematical Representation We use the benchmark from (Li et al. 2023 for the first 3 problems (Coffee, Workforce, and Facility), in which these 3 problems are built as Mixed-integer linear pro- gramming (MILP) problems. As an example, here is the problem definition of Coffee as a MILP problem (Defined in Page 6-7 from (Li et al. 2023)):
Jer is the number of units purchased from supplier for roasting facility r, and vf: and y is the amount of light and dark roast sent to retail location E from roasting facility r. C, is the capacity for each supplier s, and D and DP are demand for light and dark roast for each retail location é. There is cost c,r for each unit purchased from supplier for roasting facility shipping cost of Ge.e for each unit sent retail location e from roasting facility r, and roasting cost be and h per
15

---

Published as a conference paper at ICLR 2025
unit of light roast and dark roast respectively in facility
minimize
Σ 8,T
Jair Ca. +
whe
Σ
hip +
Σ r,t
+ +
gre
r.l
r,e
subject to
Σ Jsr S Cs:
4s (Supplier capacity constraint)
Σ Far = Σ (whe + vP,)), Vr
(Conservation of flow constraint)
Σ whe 2 D/, Ve (Light coffee demand constraint)
Σ
Wri De, vl (Dark coffee demand constraint)
D
Tsrr Vorg. E zt,
rs (Integrality constraint)
Complexity Analysis
The Coffee problem can be framed as a max-flow problem, which can be solved polynomial time. Specifically, some algorithms solve the max-flow problem with O E) O(V²E) E)
The Workforce problem, with no additional constraint, can also be framed as a max-flow problem. However, different types of constraints added by to form different instances. Some types of queries can increase the complexity. For example, "What if Gu and Bob cannot work on the same day?". Adding constraints to introduce conflicting workers turns the problem be as hard as a maximum independent set problem(also NP-Hard), where we add an edge between conflicted workers, and finding the maximum indentpendent set.
The facility problem is a NP- Hard problem Capacited Facility Location Problem(CFLP), The formal definition of CFLP as below:
min
Giddza Vig
+
Σ Mkj =1 for all j 1, 1,...,m
s.t.
1=1
Σ d,bii S u,;; for all = 1,...,
1=1
Vij 2 0 for all i 1,..., and I,...,
I E {0,1} for all = 1.....
where 1 if facility i is open, and v; = 0 otherwise. Yij for i 1,..., n. and j 1,..., which represents the fraction of the demand d, filled by facility i.
For the Task Allocation problem, since it is equivalent to multi-agent traveling salesman prob- lemtagent=robbos, tasks=cities), it reduces from the classic traveling salesman problem (TSP) and thus is also NP-hard.
For the Warehouse problem, as TSP special case when one station can be used to finish one specific task and there extra stations, the Warchouse problem is at least complex as TSP, thus is also NP-hard.
For multi-step problems, Blocksworld is proved to be NP-hard problem (Gupta & Nau 1992). so same for Mystery Blocksworld as it is the same problem with obfuscated names. Although there is no existing proof, Movie has 13 predicates and possible actions, and Gripper has 4 types of objects (rooms, objects, robots, grippers), predicates, and 3 possible actions. These show that they are not simple straightforward tasks.
16

---

Published as a conference paper at ICLR 2025
LLMFP Performance Sokoban To further test capability of LLMFP tasks, we tested LLMFP on the Sokoban environment, a NP-Hard problem with large maps thus needs more variables. We created evaluation set containing 15 queries describing the game setup and goals with different map sizes and number of boxes. We have five queries with 5x5 maps and 1 box, five queries with 6x6 maps and box, and five queries with 5x5 maps and 2 boxes. The evaluation results are presented in the following table.
challenging
Table 5: Optimal rates (%) comparison of LLMFP with baselines on Sokoban task.
Directorr-40
Directoi
CoTGPT-40
CodeGrт40
Code_SMToPr-4
LLMFP GPT-40
0.0
26.7
0.0
0.0
0.0
80.0
As can be observed, LLMFP achieves a success rate of 80%, outperforming the baselines. The new results, along with other problems, showcase the potential of LLMFP to solve complex problems.
The major failure mode is: when the generated codes initialize the adjacent predicate, only ini- tializes adjacent positions to be True but fails to initialize unmentioned positions to be False (since the query only mentions position_x xx and position_y y are adjacent), so the solver would set the non- adjacent positions adjacent to get solution with fewer steps. In addition, although LLMFP is demonstrated to be capable of correctly encoding and solving the Sokoban problem, it is true that there are many more variables in the Sokoban problem than in other tasks because the problem is represented with a map with large number of different positions. This slows down the speed of the SMT solver. To mitigate this problem, some potential solutions include 1) introducing methods to estimate the lower and upper bounds of step numbers needed and start from there, 2) developing heuristics to prioritize some possible options first, and 3) developing methods that put attention on part of the map and ignore the unnecessary positions in the map. We would love to extend our work to explore these directions to make our framework more efficient.
17

---

Published as a conference paper at ICLR 2025
A.2 LLMFP PERFORMANCE OVER ITERATIONS
Fig. 3 shows the performance of LLMFP over 5 iterations. The key observation is: number of it- erations of Self Assess & Midification stage enables LLMFP further improve the optimal rates, although we can observe that LLMFP does not need extensive iterations to achieve an overall satis- fying performance.
LLMEP GPT-4a
LLMFP_Claude Claude 3.5 S-Snnnet
Coffee
Workforce
Facility
1.0
0.9
0.8
Optmall Rate
Optimal Rate Rate
0.7
0.7
0.7
Optmal
0.6
0.6
0.6
0.5
0.5
0.5
0.4
0.4
0.3
Iteration Allocation
Iteration
Iteration
Warehouse
Blocksworid
1.0
0.9
Optimal Rate
Optimal Rate
Ogtimal Rate
0.7
0.6
0.6
0.6
0.5
0.5
0.5
0.4
0.4
0.4
0.3
0.3
Iteration
Iteration
Iteration
Mystery Blocks wworld
Movic
Gripper
1.0
1.0
1.0
0.9
0.8
Optmall Rate
Optimal Rate
Optmal
0.6
0.5
0.5
0.5
0.4
0.4
0.4
0.3
0.3
0.3
1
2
Figure 3: Optimal rates models across LLMFP Iterations
18

---

Published as a conference paper at ICLR 2025
A.3 ADDITIONAL METRIC PERFORMANCE: SUCCESS RATE
In addition to the optimal rate, we also include another metric, success rate to evaluate baselines and LLMFP. We include the result in Table and Table
Note that although for multi-constraint problems the optimization goal described in the task de- scription, we exclude the optimization goal when calculating the success rate and only evaluate whether the plan fulfills the task setup and the query. This would largely decrease the difficulty of multi-constraint problems. For example, even assigning all tasks one robot is considered success for the task allocation task. Thus, the success rates of all baselines for multi-constraint problems are significantly higher than the optimal rates. However, although the success rates of LLMFP almost remains the same as optimal rates since the SMT solver guarantees output the optimal result with correct encoding, the performance LLMFP still outperforms other baselines, with an average of 86.4%, 18.1 1 % higher than the best baseline.
While for the multi-step problems, considering all initial conditions, predicate and action definitions, and goals are the same, developing reasonable and correct plan is not significantly easier than developing an optimal plan with the least number of steps. Thus, the success rates baselines are improved, but not significantly, compared to the optimal rates.
Table 6: Success rate (%) comparison of LLMFP with baselines multi-constraint problems.
Method
Coffee
Workforce
Facility
Task Allocation
Warchouse
Average
DirectaPT-50
5.6
54.5
31.7
100.0
42.0
46.8
Directo1 PREVIEW
26.3
92.6
41.5
94.0
86.0
68.1
CaTaPT-o
17.7
72.3
31.7
100.0
82.0
60.7
CodecpT-40
18.8
76.2
64.6
92.0
90.0
68.3
Code_SMe SMTaPT-40
0.0
10.8
1.2
0.0
34.0
9.2
LLMFP GPT-40
64.7
92.2
79.3
100.0
96.0
86.4
Direcle 3.5 SONNET
5.3
91.3
36.0
100.0
76.0
61.7
CoT CLAUDE 3.5 SONNET
10.9
60.6
1.2
100.0
96.0
S3.7
CodeCLAuDE 3.5 SONNET
61.3
59.1
100.0
60.0
73.9
CodecLAuDE 3.5 SONNET
77.1
39.0
59.1
90.0
74.0
67.8
LLMFP
80.5
88.7
61.6
100.0
92.0
84.6
Table Success rate (%) comparison of LLMFP with baselines on multi-step problems.
Method
Blocksworld
Mystery Blocksworld
Movie
Gripper
Average
Directarr-40
56.1
1.0
90.5
16.0
40.9
Directoi- PREVIEE
90.9
37.9
100.0
76.2
CoTGPT-40
62.0
3.0
95.2
10.0
42.5
CodeGpT-to
0.0
0.3
0.0
0.0
0.1
Code SMTGPT-40
0.2
0.0
0.0
4.0
1.0
LLMFP GPT-40
96.2
77.7
100.0
76.0
87.5
DireCTCLAUDE 3.5 SONNET
54.5
0.5
100.0
56.0
52.7
CoT COTCLAUDEE 3.5 SONNET
76.1
3.2
100.0
72.0
62.8
CodecI
0.0
0.0
0.0
0.0
0.0
Mode_SMTc. SMMCLAAIE
0.0
0.0
4.0
0.0
1.0
LLMFP CLAUDE 3.5 SONNET
93.4
98.0
100.0
76.0
91.8
19

---

Published as a conference paper at ICLR 2025
A.4
TIME AND COST STATISTICS AND ANALYSIS
Table Bande Tabel 9 show the wall time comparison of all methods for GPT-4o on 9 tasks. From the results, could observe that the time taken for LLMFP, although longer than most of the baselines, is within reasonable ranges. Especially, for multi-constraint problems, it shorter than Direct with ol-preview because of the inherent difficulty for LLMs solve these combinatorial optimization problems.
Table 10 shows the detailed time statisties of all components of LLMFP for GPT-4o 9 tasks. We could observe that for both LLM querying time and solver running time, all stages of LLMFP requires reasonable runtime. The longest runtime is prompting FORMULATOR it time-consuming to reason about all needed variables and information to form representation formulation.
Table 8: Average wall time (s) per query comparison for 5 multi-constraint problems with GPT-4o.
Method
Coffee
Workforce
Facility
Task Allocation
Warehouse
Average
DirectaPr-40
8.8
2.2
2.1
1.8
0.9
3,2
Directol- PREVIEw
104.2
63.9
77.7
70.5
63.7
76.0
CoTGrT-to
16.9
12.0
6.0
9.6
7.4
10.4
CodecpT-40
30.6
10.0
8.2
12.3
Code SMTGPT-4N
30.0
15.3
10.3
15.0
8.3
15.8
LLMFP 87.1
55.1
29.9
62.3
28.9
52.7
Table Average wall time (s) per query comparison for multi-step problems with GPT-4o.
Method
Blocksworld
Mystery Blocksworld
Movie
Gripper
Average
DirectaPT-40
0.7
0.5
8.8
2.7
Directat-rawvww
26.3
87.9
25.7
23.8
40,9
CoTarт-4
2.1
4.0
1.0
10.2
4.3
Codeart-40
19.7
8.9
7.3
8.2
11.0
Code SMTaPT-0
9.1
8.5
10.6
12.9
10.3
LLMFP GPT-40
43.3
48.3
58.6
141.6
73.0
Table 10: Average time (s) spent per query for all components of LLMFP GPT-40 all 9 tasks.
Domain
Definer
Formulator
Solver
Formatter
Code Gen.
Self' Assess & Mod.
Coffee
5.6
10.8
17.1
0.1
14.2
11.3
Workforce
3.4
5.1
8.3
11.0
3.1
7.8
Facility
3.8
0.7
Task Allocation
8.6
23.8
5.2
0,2
6.5
5.9
Warchouse
3.9
3.1
6.2
0.2
4.1
3.3
Blocksworld
N/A
21.0
14.6
0.6
1.9
3.4
Mys. Blocksworld
N/A
24.3
14.6
0.6
2.3
4.1
Movie
N/A
21.2
9.9
0.3
2.1
10.6
Gripper
N/A
18.3
16.0
6.9
11.2
7.0
20

---

Published as a conference paper at ICLR 2025
Table 11 shows the average cost comparison of all methods the coffee task, and Table 12 and 13 shows the cost statistics of LLMFP over all 9 tasks. We observe that although LLMFP is more costly than most of the baselines, it is cheaper than Direct with ol -previww with better performance. In addition, the average cost per query for all tasks around 0.1 dollar, indicating LLMFP is not very costly.
Table 11: Average ($) per query comparison of LLMFP GPT-4o the Coffee task.
DirectaPT-4n
Direclnt-reswrww
CoTaPT-In
CodeGPt-0
Code SMTGPT-40
LLMFP GPT-4n
0.008
0.536
0.013
0.023
0.024
0.139
Table 12: Average cost ($) per query of LLMFP GPT-40 on 5 multi-constraint problems.
Coffee
Workforce
Facility
Task Allocation
Warehouse
0.139
0.140
0.083
0.081
0.085
Table 13: Average cost ($) per query LLMEP GPT-40 on multi-step problems.
Blocksworld
Mystery Blocksworld
Movie
Gripper
0.122
0.105
0.131
0.128
21

---

Published as a conference paper at ICLR 2025
A.5 BASELINES FAILURE CASE ANALYSIS
Here we describe the major failure cases for the baselines. Please refer to Appendix A.9 for example outputs.
A.5.1 DIRECT, DIRECTOI AND COT
For multi-constrain tasks, since they involve various constraints, intensive calculations, and numer- ous possible solutions, LLMs still do not have the capability to directly solve the optimal solution considering all constraints. They either fail to understand consider some important constraints or fail to optimize the goal. Although utilizing stronger ol ol-preview or taking advantages of prompting techniques like CoT could result in less mistakes, the major underlying failure reasons are similar.
For multi-step tasks, the major failure cases are the failure to deliver reasonable plans considering preconditions and effects of all actions accurately.
A.5.2 CODE
For multi-constraint tasks, the major failure cases 1) failing to consider all necessary constraints, 2) failing to consider or understand the query, and 3) overwriting the given API.
For multi-step tasks, the major failure cases are 1) failing correctly represent the problem, includ- ing the problem setup, predicates, and actions, 2) failing to write codes with correct logic or syntax.
A.5.3 CODE_SMT
For multi-constraint tasks, the major failure cases are same Code.
For multi-step tasks, the major failure cases are 1) Poor SMT Utilization: including failing to dis- tinguish And and Implies, to correctly use SMT Array or Function, or to write correct SMT syntax (or Python syntax sometimes), and 2) Poor Problem Understanding: failing to initialize the initial value of unmentioned predicates (eg. when the query says blocks b, are clear, codes also need to initialize e to be not clear), to assert one action per step, or to update unchanged variables for next step
A.5.4 THEORETICAL INSIGHTT
Overall, LLMs are good at understanding the syntax and semantics of planning problems as opti- mization problems but are not good at solving optimization problems directly. Specifically, next- token prediction is fundamentally different from deterministic algorithms for optimization. There's a growing belief that next token prediction cannot truly model human thought and cannot support human-like capabilities of understanding a problem, imagining, curating, and backtracking plans before executing [1-3]. Specifically, the claim that next token predictions are "ill-suited for planning tasks" is supported by works [4-7], which tested the planning capabilities of L.LMs on various plan- ning tasks. These works empirically show that in addition to identifying patterns in language and predicting the next word in sequence, LLMs still can not truly understand a problem and thus do not have the capability to perform intense calculations to optimize for any objectives. Thus, this is major reason why baselines are not capable of solving the complex planning problems our paper. However, since LLMFP teaches LLMs to build the optimization problem step by step and calls the external solver to solve for plan, this bypasses the need to devise a plan by LLMs themselves.
To support this claim that LLMs cannot understand and solve an optimization problem, we conduct an experiment on the Coffee task that, instead of using natural language task descriptions as inputs, directly map this Coffee task to optimization problem and use the formal mathematical def- inition of this problem as the inputs to LLMs. Thus, LLMs do not need to understand the problem formal definition is given and could be directly solved.
and find the underlying constraints,
We tested Direct with the most powerful LLM OpenAl ol o1-preveew model on all queries of Coffee, which only achieves optimal rate of 34.2%. Compared to its 25.9% optimal rate with natural language task description, this is not significant improvement, given all goals and constraints are clearly formally specified in the new setting. This is consistent with the conclusion that LLMss still
22

---

Published as a conference paper at ICLR 2025
cannot solve optimization problems by themselves, given formal representation. LLMFP enables LLMs to formalize planning problems as optimization problems. Since SMT solvers can guarantee to return correct answers given correct input, the high optimal and success rate of LLMFP indicates that LLMFP allows L.LM to parse the correct syntax and semantics information of a plan- ning problem from its natural language description to formal mathematical description. Such translation is also non-trivial when no task-specific examples are provided. As shown in our newly added baseline approach Code_SMT as shown in Table I and when we directly ask LLMs to translate and encode the natural language task description in an SMT format, the optimal rate is low, with an average of 2.7% and 62.4% for multi-constraint tasks, and 1.0% and 0.0% for multi-step tasks across two LLMs GPT-4o and Claude 3.5 Sonnet.
23

---

Published as a conference paper at ICLR 2025
A.6
LLMFP FAILURE CASE ANALYSIS
Here we analyze the major failure cases for all 9 tasks.
A.6.1 COFFEE
There are two major failure cases for Coffee tasks:
First, some queries are not clearly presented, indicating ambiguous information. Queries of Coffee tasks are what-if questions and are categorized into 7 sets. Each set is type of question. We notice that the type "supply-roastery" asks queries like "What led to the decision to use supplier3 for the roasting facilliy roastery!?" ?". To answer this question, is both plausible to test "using supplier3" or to test "not using supplier3" to see the performance. However, the ground truth answer for this type of questions to "nol using supplier 3". As confusing queries even for human, they are hard for LLMs to understand. Thus, for these queries, LLMFP sometimes generate codes with opposite meanings as what is expected.
Second, sometimes LLMFP DEFINER fails to consider all implicit constraints. The most easily neglectable implicit constraints are 1) the beans roasted each roastery do not exceed the beans receives, and 2) the beans ship from each roastery do not exceed the coffee it roasts. When any of the two constraints are missing, to minimize the cost, the model will automatically set the shipped beans or roasted coffee to be O, assuming the company delivers coffee without sourcing beans or roasting coffee.
A.6.2 WORKFORCE
There are two major failure cases for Coffee tasks:
First, sometimes LLMFP fails to understand the queries. Some of the queries asks questions like 'Can Gu transition from Sun14 to Sun7 for work purposes?'. The meaning is to force Gu to work on Sun7 and take rest on Sun14. However, sometimes LLMFP builds variables to test both taking and not taking this transition, and returns solutions with less costs.
Second, sometimes when the solution space is large, it is hard to find the optimal solution within maximum runtime set for solver. We set the maximum solver runtime to 15 minutes, which is exceeded when solving some hard queries.
A.6.3 FACILITY
Similarly as the first failure case of Coffee, some queries are not clearly presented. The queries are like "What justifies the opening of plant 0?". which is confusing even for humans. Both opening plant and closing plant 0 to report the costs make sense to answer this query. However, the ground truth meaning of this query close plant
A.6.4 TASK ALLOCATION
LLMFP only fails one query in Task Allocation. The reason is the FORMULATOR generates wrong values for robot finish time.
A.6.5
WAREHOUSE
The major failure case for Warchouse is CODE GENERATOR overwrites the provided API ret_distance and provide as the output during Code Generation. Thus, the distance between each station mistakenly set to be 1.
A.6.6 BLOCKSWORLD
One major failure case for Blocksworld is CODE GENERATOR fails to initialize the states of predi- cates correctly and thoroughly. Since the query will only meantion the predicates that are true, for example, block is block 2, but when initializing, LLMFP needs to initialize both mentioned states but also unmentioned states that are false. For example, block is not on block 1. However, CODE GENERATOR sometimes fails consider all unmentioned states.
24

---

Published as a conference paper at ICLR 2025
A.6.7 MYSTERY BLOCKSWORLD
Similarly Blocksworld, Mystery Blocksworld has same failure case. For Mystery Blocksworid, since the predicate and action names are not meaningful, more this kind of errors are made by GPT- 4o. However, Claude seems to have better reasoning capability support from making more these errors.
A.6.8 MOVIE
There is no failure case for Movie.
A.6.9 GRIPPER
The major failure case for Gripper is when the solver fails to find the solution because there are some code generation emrors, the SELF ASSESS & MODIFICATION sometimes would think it is because the timestep is not enough, thus adding another loop within the original loop. However, this would result in the program to execute forever.
25

---

Published as a conference paper at ICLR 2025
A.7 BASELINES WITH EXPLICIT OPTIMAL REQUIREMENTS
For all methods including LLMFP and baselines, we implicitly mention the goal of each multi- constraint task in the task description. For example, for the Coffee task, the task description "...The company's ' objective is to minimize the total cost, including shipping beans, roasting, and shipping roasted coffee, while ensuring that all coffee produced meets or exceeds the demand each retail location" implicitly shows the goal is to find the plan that minimizes the total cost.
While for multi-step problems, the methods not explicitly instructed to provide optimal solutions. Since SMT solver guarantees to find the solution if there exists one, it can rigorously show the solution does not exist for smaller timesteps and increase timestep, thus can always find the optimal solution if the formulation and generated codes are correct. This is an advantage of incorporating complete and sound solver like SMT in our framework.
However, to better understand the capabilities of baselines, we modify the prompts to explicitly instruct them to find the optimal solution and re-evaluate them on the multi-step problems. We add _Opt to the name of the baselines represent the baselines with explicit optimal instructions.
Table 14 shows the optimal rate of baselines with explicit instruction on finding the optimal plan. Compared with Table 2 could observe that some baselines achieve better performance (from average 0.1% % to 16.4% for Code-Optopt-40- and from average 30.9% to 36.7% for CoT.OptoPT-4) while some achieve slightly worse performance(average 68.1% to 37.0% for Direct Opt.1 pReView). However, despite the changes due to the explicit instruction to find the optimal plan, LLMFP still could largely outperform all baselines.
Table 14: Optimal rate (%) comparison of LLMFP with baselines that explicitly instructed gen- erate optimal plans on multi-step problems.
Method
Blocksworld
Mystery Blocksworld
Movie
Gripper
Average
Direct_Or Optect.p
35.2
0.8
100.0
0.0
34.0
Direct.Opi. Opfot-
80.9
39,0
100.0
48.0
67.0
CoT OoT_Opto
33.4
2.3
95.2
16.0
36.7
Code_ Optoeroo
0.0
3.8
61.9
0.0
16.4
Code_SMT. SMT..Otarre
0.0
0.0
0.0
0.0
LLMFP GPT-40
96.2
77.7
100.0
76.0
87.5
Direct.Optclane SONNEY
40.9
1.5
100
20.0
40.6
COT COT_OOOCL
52.5
4.5
100
20.0
44.2
Code_OpicLADE 3.5 SONNET
0.0
0.0
0.0
0.0
0.0
Code SMT OPICLAUDE 3.5 SONNET
0.0
0.0
0.0
0.0
0.0
I.MEP CLAUDE 3.5 SONNET
93.0
98.0
100.0
76.0
26

---

Published as a conference paper at ICLR 2025
A.8 INPUTS ON TASKS
We include the inputs, which includes task description, background information or API, and example queries, for all 9 tasks in Fig. 4 Fig.
Coffee
Task Description:
coffee production company sources beans from three suppliers, roasts them at one of two facilities into either dark or light coffee, and ships the roasted coffee to three retail locations. Each supplier has limited capacity. Each roastery, with no existing inventory, can roast one unit coffee bean into one unit of dark or light coffee. The retail locations have specific demands for dark and light coffee, with no existing inventory. The company's objective is minimize the total cost, including shipping beans, roasting, and shipping roasted coffee, while ensuring that all coffee produced meets or exccods the demand at each retail location.
Background Information or API:
capacity in supplier = !'supplierl': 150, 'supplier2"; 50, 'supplier3': 100)
light_ _atttteooeeee___eee needed for cafe = ('cafel': 20, 'cafe2': 30, 'cafe3': 40)
dark coffee needed for cafe ('cafel': 20, 'cafe2'; 20, 'cafe3': 100)
shipping cost from from_supplier_ to vo_roastery
('supplierl', 'roasteryl'): 5,
('supplierl', 'roastery2"): 4,
('supplier2", 'roasteryl'): 6,
('supplicr2', 'roastcry2"): 3,
('supplier3', "roastryy'); 2,
('supplier3), 'roastery2"): 7
roasting cost light frrastcryy': 'roastery2;: 5} roasting Ceaettnng_sttt____1eeee1 dark = ''roasteryl's 5, 'roastory2': 6} shipping_com cost from roastery_ to to_caae-
(rraastryyl', "cafel'): 5,
("roastery 'cafe2"):
(rroastery I', 'cafe3"):
("roastery2', "cafel"): 4,
("roastery2', "cafe2"):
("roastery2), 'cafe3'): 2
math package: function math.ceil() to round UP float to int and math.floor) to round DOWN float to int Expect output format
Example Query:
What the potential impact of 29% increase in demand cafe cafe2?
Set 2: What if demand for light coffee at cafe cafel incrcased by 23%?
Sct 3: Why we using supplier supplicr2 for roasting facility roastery2?
Set 4: Assume cafe cafe2 can exclusively buy coffee from roasting facility roastery2, and conversely,
only sell its coffee cafe cafe2. How does that affect the outcome? Set 5: What if roasting facility roastery2 can only be used for cafe cafe2?
roasting facility roastery2
Set 6: What if supplier supplier3 now provide only half of the quantity?
Set 7: The per-unit cost from supplier supplier3 to roasting facility roastery2 is now 1. How does that
affect the total cost?
Figure 4: Task description, background information API, and example queries for Coffee
27

---

Published as a conference paper at ICLR 2025
Workforce
Task Description:
Assign workers to work day for two weeks (dayl 14); cach work day (shift) requires different number of workers, and each worker may or may be available on particular day. The goal find the work arrangement of every day that fulfills both worker availability and the shift requirement while minimizing the total payment to workers.
Background Information or API:
# Number workers required for each day, each key is a different day include both day in a week and the real date. Monl means Monday and 1st day for this month. shift requirement=1
"Monl": ": I, "Tuc2": I, "Wed3": 2, "Thu4": 2, "Fris": 2, "Sat6": 3, "Sun7": 2, "Mon8": I, "Tuc9": 1, "Wed10": "Thul1": "Fril2":3, "Sat 3, "Sunl44
Worker availability
availability {
"Amy": ["Tue2", "Wed3", "Fri5", "Sun7", "Tue9", ""ed10", "Fril",, "Sat13", "Sunl4"),
"Bob": ["Monl", "Tue2", "Fri5", "Sat6", "Mon8", "Thull", "Sat13"], (Cathy""["Wod3""" "Thu4", "Fri5", "Sun7", "Mon8", "Tue9", "Wed10", "Thull", "Fri12", "Sat13",
"Sun14"].
"Dan": ["Tue2", "Wed3", "Fri5", "Sat6", "Mon8", "Tue9", "Wed10", "Thull", "Fri12", "Sat13",
"Sun14"],
"Ed": ["Monl", "Tuc2", "Wed3", "Thu4", "Fri5", "Sun7", "Mon8", "Tuc9", "Thul1", "Sat13",
"Sun "Sun14"],
"Fred": ["Monl", "Tue2", "Wed3", "Sat6", "Mon8", "Tue9", "Fril2", "SatI3", "Sun14"],
"Gu": ["Monl", "Tue2", "Wed3", "Fri5", "Sat6", "Sun7", "Mon8", "Tue9", "Wed10", "Thul1", "Fril2", "Sat13", "Sun14"],
# Amount each worker is paid work day worker pay- = "Amy": 10, "Bob": 12, "Cathy": 10, "Dan": "Ed": 8, "Fred": 9, "Gu": 11,
Expected output format
Example Query:
Set 1: Can Gu's work schedule be adjusted from Sun7 to Tue2?
Set 2: What about the scenario where Gu promoted and starts earning 15 dollars an hour?
Set What if Gu's shift capacity is capped at 6?
Sct 4: need more people specifically on Mondays, how can accommodate that?
Set 5: What oceurs if Gu and Bob prevented from working the same day?
Figure 5: Task description, background information or API, and example queries for Workforce
28

---

Published as a conference paper at ICLR 2025
Facility
Task Description:
company currently ships its product from plants (Names: Plant O, Plant 1, Plant Plant Plant to 4 warchouses (Names: Warchouse O, Warchouse 1, Warchouse 2, Warchouse 3). Each plant has capacity and each warehouse has demand. It is considering closing some plants reduce costs. The goal is to find out which plant(s) should the company close and optimal transportation units from each plant to warchouse in order to minimize total cost, which includes transportation and fixed costs.
Background Information or API:
# Warehouse demand in thousands of units demand [15, 18, 14, 20]
# Plant capacity in thousands of units capacity = [20, 22, 17, 19, 18]
# Fixed costs for each plant fixedCosts [12000, 15000, 17000, 13000, 16000]
# Transportation costs per thousand units
transCosts = [[4000, 2500, 1200, 2200],
[2000, 2600, 1800, 2600].
[3000, 3400, 2600, 3100].
[25500, 3000, 4100, 3700],
[4500, 4000, 3000, 3200]]
math package: function math.ceil() to round UP float to int and math.floor) to round DOWN float to int
Expected output format
Example Query:
Set I: If we were to close Plant 3, what might be the potential impact?
Set 2: Why is the edge from plant to warehouse 3 not considered for selection?
Set 3: What would happen if plant 3's opening cost is reduced by 50%?
Set 4: What would happen if the demand were to rise by 4?
Figure 6: Task description, background information API, and example queries for Facility
29

---

Published as a conference paper at ICLR 2025
Task Allocation
Task Description:
Given list of tasks (Number Task A, Number B Task B, Number C Task C) and three heterogeneous robots (Robot A, Robot B, Robot C) that are skilled at different tasks, the goal is to find the way assign different number of tasks to different robots and finish the tasks with minimized finish time. The three robots could work in parallel, but the finish time counts the time when last robot stops working.
Background Information or API:
# Finish time for each robot-task pair robot work time for tasks
('Robot A', 'Task A"): 24,
('Robot A', 'Task B"): 89,
('Robot A', 'Task C"): 38,
('oobot B', 'Task A²): 27,
('Robot B', 'Task B'): 58,
('oobot B', 'Task C"): 56,
('Robot C* 'Task A"): 18,
('Robot C., 'Task B"): 57,
('Robot C, 'Task C"): 49,
Mux(variable list) function takes a as input and outputs the max this variables. Expected output format
Example Query: Number of Task is 54; Number of Task B is 57; Number of Task is 74,
Figure 7: Task description, background information or API, and example queries for Task Allocation
Warehouse
Task Description:
The robots need to finish N tasks one by one by visiting stations (repeatable) that capable of accomplishing corresponding tasks. The robot to start at origin, finish N given tasks with given order, and return back to origin. The goal is to find the list of N stations while minimizing the total distance travelled.
Background Information or API:
# Each row is the stations that could be used to accomplish the task station task info
'Task 0': [2, 3, 4, 7, 9].
"Task 1t: [1, 2].
"Task 2': [1, 5],
"Task 3": [3, 4),
"Task 4': [5, 8),
"Task 5!: [0, 4, 5, 6),
"Task 6': [3, 6, 8, 9].
'Task 7': [0, 1].
"Task 8:: [2, 7, 8),
"Task 91: [7, 9]
get distance(station. Int(), station Int()) to calculate the distance: Real@ between two stations(use index 10 to represent origin)
Expected output format
Example Query:
Number of Tasks is 7. The Task ids needs to be accomplished are: [6, 0, 2, 3,
Figure 8: Task description, background information API, and example queries for Warehouse
30

---

Published as a conference paper at ICLR 2025
Blocksworld
Task Description:
The robot has four actions: pickup, putdown, stack, and unstack. The domain assumes world where there are of blocks that can be stacked on top of each other, an arm that can hold one block a time, and a table where blocks can be placed. The actions defined in this domain include: pickup: allows the arm to pick up a block if the block clear, the block is on table, and the arm is empty. After the pickup action, the arm will be holding the block thus not empty, and the block will longer on table or clear.
putdown: allows the arm to put down block if the arm is holding block. After the putdown action, the arm will be empty thus not holding the block, and the block will be on table and clear.
stack: allows the arm to stack block on top of another block if the arm is holding the top block and the bottom block is clear. After the stack action, the will be empty thus not holding the block, the top block will be clear and on top of the bottom block, and the bottom block will no longer be clear.
unstack: allows the arm unstack block from top of another block if the top block is the bottom block, the arm empty, and the top block clear. After the unstack action, the arm will be holding the top block thus not empty, the top block will no longer be top of the bottom block and not clear, and the bottom block will clear.
Background Information or API: update_dataasoleee data(solver) that helps to update the unchanged predicate variables
Example Query:
You have 4 blocks.
b on top of c. с top of d.
d on top of a.
a is on the table.
b is clear. Your arm empty.
Your goal is to move the blocks.
a should be on top cf d should be top a.
Figure 9: Task deseription, background information or API, and example queries for Blocksworld
31

---

Published as a conference paper at ICLR 2025
Mystery Blocksworld
Task Description:
I am playing with a set of objects. The objects can be province or not, planet or not, pain or not, and one object could craves another object. The world has harmony state. I have four actions: attack, succumb, overcome, and feast.
The actions defined in this domain include:
attack: allows to attack an object the object province, the object is planet, and harmony is true. After the attack action, the object is pain, the object will no longer be on province or planet, and harmony is true.
succumb: allows to succumb an object the object is pain. A After the succumb action, the object is no longer pain and harmony is true, and the object will be the province and planet.
overcome: allows to overcome an object from another object the first object is pain and the second object is province. After the overcome action, harmony become true, the first object will not pain, the first object be province and craves the second object, province.
second object will no longer be
feast: allows feast an object from another object if the first object is province, the first object craves the sccond object, and harmony is truc. After the fcast action, harmony becomes not truc, the first object will be pain, the first object no longer craves the second object and not province, and the second object will be province.
Background Information or API:
update_datee data(s)lver) that helps to update the unchanged predicate variables
Example Query:
You have 4 objects.
b craves c. craves d. craves a.
planet. is province. harmony is true. Your goal is to play with the objects achieve: a should craves c.
d should craves
Figure 10: Task description, background information or API, and example queries for Mystery Blocksworld
32

---

Published as a conference paper at ICLR 2025
Movie
Task Description:
You work to play movic. You want to get several objects as snacks (objects could be chips, dip, pop, cheese, crackers), and have movie rewound and set counter zero the end. You have nine actions: rewind-movie, reset-counter, start-movie, undo-rewind, get-chips, get-dip, get-pop, get-cheese, and get- crackers.
The actions defined in this domain include:
start-movie: allows start movie if counter at zero. After the start-movie action, counter zero is no longer true, and counter at other than zero true.
rewind-movic: allows to rewind movic if counter at other than zero, After the rewind-movie action, movic rewound is true.
undo-rewind: allows to undo movie rewind if movie rewound is true. After the undo-rewind action, movie rewound is no longer true.
resct-counter: allows reset counter counter at other tham zero. After the resef-counter action, counter at other than zero is no longer true, and counter zero is true. get-chips: allows to get an object if counter at zero true, rewoudd not true, and this object is chips. After the get-chips action, have chhpss is true.
get-dip: allows to get an object counter zero is true, movie rewound not true, and this object is dip. After the get-dip action, have __ip is true.
get-pop: allows to get object if counter zero is true, movie rewound is not true, and this object is pop. After the get-pop action, have Boteppppp___e_ee is true.
get-cheese: allows get object if counter at zero is truc, movie rewound is not true, and this object cheese, After get-cheese action, have cheese true.
have_cheese get-crackers: allows get an object if counter at zero is true, movie rewound is not true, and this object crackers. After the get-crackers action, have crackers truc.
Background Information or API:
update_data data(solver) that helps to update the unchanged predicate variables
Example Query:
You have 5 objects.
object 0 chips.
object 1 is dip.
object 2 pop.
object_3 3 is cheese.
object 4 crackers. counter-at-zen true. Your goal is to achieve: movie-rewound counter-at-zero
have-dip
Figure 11: Task description, background information or API, and example queries for Movie
33

---

Published as a conference paper at ICLR 2025
Gripper
Task Deseription:
You control robots, each with left and right gripper that can move balls between different rooms. There are three actions defined in this domain:
move: allows a robot to move from one room to another room if the robot room from. After the move action, the robot is no longer at room from, and the robot will be at room to.
pick: allows robot pick up ball with gripper in room if the robot is at this room, the ball is this room, and the gripper is free. After the pick action, the robot carry the ball, this gripper of the robot not free, and the ball will not at the room.
drop: allows a robot to drop ball with gripper room if the robot carry this ball, the robot the room. After the drop action, the robot will not carry the ball, this gripper of the robot will be free, and the ball will the room.
Background Information API: update_ data(solver) that helps to update the unchanged predicate variables
Example Query:
You have robot. The robots' grippers are free.
You have rooms.
You have balls.
robot! is at room I.
robotl is at rooml.
balll is at rooml.
ball2 is at room2,
ball3 is at room2.
ball4 is at room2.
ball5 is at room2.
ball6 is at room3,
ball7 is at room2,
Your goal is to achieve: balll should be at room3.
ball2 should be at room3.
ball3 should
room4.
ball4 should be at rooml. ball5 should be room4.
ball6 should be at room4. ball7 should be at room2.
Figure 12: Task description, background information or API, and example queries for Gripper
34

---

Published as a conference paper at ICLR 2025
A.9 EXAMPLE OUTPUTS ON COFFEE TASKS
In this section, we include the outputs of all models for the query "What is the potential impact of 29% increase in demand cafe cafe2?". The optimal solution has total cost to be 2612.
A.9.1 BASELINES
We include the outputs of four baselines for Coffee task in Fig. 13 reason in the figure descriptions:
Fig
15
and analyze the failure
Direct Direct_GPT-40 GPT-4o Output
Direct ol-preview Output
"json
"suppliers":
"suppliers":
"supplier ship roastery___iii
"supplier 1 ship to roastery 1": 100,
"supplicr 1 ship to roastery 20
"supplier _sshppppp to roastery__.... 50,
"supplier 2 ship roastery 1":
"supplier 2 ship to roastery__yrr 50,,
"supplicr 2_shippto to__ttoorootr_________
50,
"supplier 2. ship to roastery 2": 0,
"supplier 3 ship roastery I": 100,
"supplier 3 ship to roastery
100,
"supplicr_3 3_shpppp___ppp____ re_rotttte___2
2"r""""""000 2":0.
"supplier 3 shipptte ship_to__nt roastery_22 2": 0,
"supplicr_ Baclllr____tttt__spps _us__uipppdd beans": 95,
"supplier 1 total supplied beans": 150,
"supplier 2 total __ttoa_supppieedddd beans": 50,
"supplier 2 total _uup___epeeed_____ beans": 50,
"supplicr total cotal_supplied, bcans": 100
"supplier_ 3 total supplied beans": 100
}.
"roasterics":
"roasteries":
"roastery dark coffee": 1,
"roastery 1 dark coffee": 120,
"roastery 1 light light_coofce". 99,
"roastery_ light coffee": 130,
"roastery 2 dark coffee": 145,
"roastery 2 dark dark_coffee".. 20,
"roastery 2 light coctttt_oocc""""" 0,
"roastery 2 light coffee": 30,
"roastery 1 total _oasstroattddd bcans": 100,
"roastery total roasted beans": 250,
"roastery 2 total roasted beans": 145
"roastery_2 2 total roasted beans": 50
}.
"cafe":
"cafe": {
"roastery 1 1_ship dark co_cafe_
"roastery ship dark to cafe 1": 20,
"roastery 1 ship light to cafe 20,
"roastery 1 ship light cafe 1": 20,
"roastery 1 ship dark to cafe 2":
"roastery ship dark to cafe 2": 20,
"roastery ship light to cafe 29,
"roastery 1 ship light to cafe 2": 30,
"roastery 1 ship dark to co__ 3": 0,
"roastery_ 1 ship dark to__tottoo cafe 3": 80,
"roastery 1 ship_ light cafe 40,
"roastery |_ship light to cafe 3":80,
"roastery ship dark to cafe 10:
"roastery 2 ship dar__arrkk__ to cafe -a:00
"roastery 2 Ship_light_ to__tttf_f____eeeeeeeeeeeee
cafe _afe_11=000__ 0,
"roastery 2 2_ship light to cafe 0,
"roastery 2 ship dark cafe 2": 25,
"roastery 2 2_ship dark cafe 2": 20: 20,
"roastery 2 2hsp light to totttoo_cccc cafe 2": 0.
"roastery 2 ship light to cafe 30,
"roastery 2 ship dark to cafe 3": 100,
"roastery 2 ship dark cafe 0,
"roastery 2 ship light cafe 3.:0.
"roastery 2_ ship light to cafe 3"
"total dark coffee received cafe 10:
"total dark coffee reccivede cafe 20,
"total dark coffee received cafe 2": 26,
"total dark coffee received cafe 2": 40,
"total dark coffee received cafe 3": 100,
"total dark coffee recciveeceeeeeeeeeeeeeeeeeeeeeee cafe 3": 80,
"total light coffee received cafe 1": 20,
"total light coffee received cafe 20.
"total light coffee reccieedeeeeeeeeeeeeeeeeeeeeeeee cafe 2": 39,
"total light coffee_received received cafe 2": 60,
"total light coffee received cafe
"total light_coofee coffee caaiiee_rccivvedeeaeeeeeee
80
}.
"total cost": 2767
"total cotal_ 2150
Figure 13: Output of Directorr-40 and Directo1-PREVIEW for Coffee. LLMs fail to directly solve complex multi-constrain planning tasks that involve intensive calculations.
35

---

Published as a conference paper at ICLR 2025
Coll GPT-4o Output
To address the problem, noed to follow these sieps:
*CCaleslate the demand cafe2**:
Original light coffee deriand for cafe2:
Original dark coffee demand for cafe2: 20 units
New light coffee demand for cali2: 50 1.29 38.7 units (round umils)
units)
"DDtermmine the total demand all calos
light
(cafal) (cafe2)
TTtall dark codfee demand: 20 (cafel) 26 (cafe2) 100 (cafes) = 146 units
** CChck supply capacity
Totall supply capacity: 150 (supplier))- - (supplier2) 100 tsuppliec)) - 300 units
Total demand: 99 (light) 146 (dark) 245 units
Since supply capacity (300
greater
demand umiss),
"*Optimize the shipping and rossting costs**: We minimise the
includes shipping
suppliens rousting costs, and shipping rousted collie cafes.
5. **Fomullate the plan"*:
Allocare
suppliers ressteries hased lowest shipping
Alloccaee rousting tasks to minimize roastng costs.
Allocase shipping of rowsted coffee minimiiz shipping costs to cafies.
response possible plan:
"suppliess::
'supplier roastrry 1": 100.
"supplier_ 1_ 3o joastery_
"supplier ship. So rosstry. 1", 50,
"supplier
noastrry 2":
"supplier ship to roastery
"supplier 5. ship 3o ronetry 2": 100.
"supplier lotal supplied beans": 150.
"supplier total, supplizd bouns":
"supplier_3 3_tooal__ supplied beans": 100
"ronsteries::
"roaslery dark collee": 100,
"roastery_ light coffee": 70,
"roastery_ cark coffee":
"roastery light coffie"::
"ruastery_ lotal ruasted beans": 170.
"roastery_2 roasted beans":
"cafe":
"roastar___shhp __shi________________e darkd_aaaaa ____cae____ "adil_"::000
Proastery ship light
Truaserry ship dark, cafe 26,
"massary__shhppp light to_ 2":
Proastery ship dark, salee__e__
Trusseery ship light to cafee
"roastary 2 ship dark,
"ruastery
"roastery, 2 ship dark, to st__ooo_________
"rosstery 2 light cafe 2": 26, "ruasiery
dark
"roastery ship light to cafe
"tosal dark coffee receieedceddddoe eade 1": 20, "toaal dark coffee receivel
"roaa dark coffee received cafa 100. "tosal light coffie-reeeiieee
received cafe "rosal light coffee received eafe
"ooaa. light, coflee reveived cafe
"ooaal 2150
"EEpplanation*"
The hoons are allocated from suppliers to masteries hased on the lowest shipping costs.
Roasting done minimize the noasting
Shipping The iotal cost is coleslated hased the shipping, roasting, and final shipping costs.
optimized
shipping
Figure 14: Output of CoTGPT-4o for Coffee. Although the reasoning of constraints makes sense, LLMs still fail to locate optimal values to fulfill the task as it involves intensive calculations.
36

---

Published as a conference paper at ICLR 2025
Code GPI-40 Ourput
inport pulp
Data
capecity in suppli:: (aans=((upppliiii ('suppliect': 150, 'suppliert's 50, 'suppliers's 1003
shipping ceat ceat_oot____ frer supplier_ co_soastery
mmgeltrr) "nastery)
sugeliert"; "caastrry2 1:
"suppleer", "coastory]
"suppleer"; "coastrry
"supplier"' "coastozy2
'Suppliorr',
"2castery2
cossting light - ('roastaryl's "romstury2';
roasting_cost_.dl ('roastaryl's r
('zoastery] "cafcl"):
('couxtery)
('zomterry"'
('rasteery""
('raattery"',
('coattery2', 'catc3')
Light coffee needed for
('casel's 20, 'cafe2"s 30 1.22,, "cade2's
dark coffee noeded for ('cafel' 20, 'cate2's 1.29, "cade3's 1033
Problem
prob pulp-igrroblen/ Productioo,, polp.ightnimisse
variables
ship pupplirr, to roasteyy polp:Lpvariablle dieis (SSiippupplerrroooocclcoe
[(e, El for in capecity, in suppliis for cousting_cuwe light], Lovbound-0, cat- Continuocs "
roast 1ight polp LpVariable dieto 'BoastLight'' roasting cost, light, lossound "ottiooonn roast roast_dark dark polp. LpVoriable dietol "BoastDark', zoasting 00st, dark, losbound eat "ottCooonooos
mnip_roareery___co cafe light pulp Levariable dieta (ictta"Shiiplane ("iiiiiaaatarrrrooeaaaaaaa
((r, 5l for romeiigg coweiin__cott... sa__oo____ggge Hight for
sighttoooffee___o needed for cafe],
1owBound "at"'Contmmouss"
ship_roastery to_osfe_dee dark pplp-ippariabble.c dicte( Shippkasterrhhhh
dark coffee pooded cate),
objectivv
grob
1p0mm eupppiereeese cater____rssstrrre ro-sstttryll r) shipping cost, eug_eupplion harrtteerr_tteeeeeeeee to r]
for capacity. in supplier
zoasting cost, light)
pulp-198ux/ ( [xoast, light/s]
roasting cost_ [ight]i
rean:ing seeiiig_oott_11e (ost_1ight))-1 11)
pula- tseast, dackle) zoasting cest daskle] for roaating_com cost dack]]
pulp- pplmm1p8mm| jehip, | rowsteryytoo cafe _a 6] shigeing cost, fron, for
cattery_toccaeette e)]
roasting cost 1ight cost,
Hight, coffes needed for cafe))
195px1 Iship roastery, to cate dx,k] 6) shipping cost fron, roastary 1o_yy_oo.000 to, e))
for r in roasting_cost costt cost_dack dazk for in dark coffee cotfooo_neeee__oooetoo feeded_for_oafee
Constrsiae
for in capacity in suppliar: iar:
prob ie polp. Iptini (wbip_ suppliar to (*, eil for in comningg_cot, light)) os caparity inmpriiyii__oppleettte
for in roasting roastinggcooet, coet, Light:
prob +o pelp. 1pduni (sbig supplicr, to (s for capacisy supplier) ss roast, light(k] roast_dark[ss
for in light coffue needed for cafe:
prob to pelp 1ptuni jabio roustery cade (eght for coarsing_cout_laee light]) Light coffee neaded [or_eaf[[]
dark coffee, needed for_cafer
lptuni jabip cowstooottttteeee
dackf(e,
conating cartsiig_cost_dieeeeeee dock]d paedeed__eeddd_eeeeeeeeeee
Hight:
palp-1ptumi prob polp. [p ittptiotttttttryc.......o ttreeryy_____eoe (r
light for_cade]] roset_ [oigt_lightt[]]
prob polp. 1pdun( (ohip, wtt.tppoostttr_.uoo toctttrr_to_otteee to_oost_ddra((C 0)] dack coffee, needed neeeed_ for_oafe]i rosst_dack(s]
prob. polvvo))
coder Eocmet the remule unitted
Figure 15: Output of CodeGPT-40 for Coffee. LLM also tries to use optimization solver to solve for the problem, however, without proper analysis of constraints, it fails notice that the updated demand for cafes needs to be integers. Instead, to account for the 29% demand increase, it directly update with "30 1.29" and "20 * 1.29
37

---

Published as a conference paper at ICLR 2025
A.9.2 LLMFP
We include the outputs of all stages of LLMFP for Coffee task in Fig. 16 Fig. 20
Definer Output
GOAL: Minimize the total cost, including shipping beans from suppliers roasteries, roasting costs, and shipping roasted coffee to retail locations.
Decision Variables:
Amount of beans shipped from each supplier to each roastery.
Amount of light coffee roasted at cach roastery.
Amount of dark coffee roasted at each roastery.
Amount of light coffee shipped from each roastery to each cafe.
Amount of dark coffee shipped from each roastery to each cafe.
Constraints Reasoning:
1. Supplier capacity limits the amount of coffee beans they can supply.
2. Roasteries can only roast as much coffee as they receive in beans.
3. Roasteries must meet the demand for both light and dark coffee at each cafe.
The total amount of coffee shipped from suppliers roasteries must equal total amount of coffee roasted.
The total amount of light and dark coffee shipped from roasteries to cafes must meet the demand each cafe.
Constraints:
Total beans shipped from each supplier to all roasteries S Supplier capacity.
Total beans received by each roastery Total coffee roasted (light + dark) at that roastery.
Total light coffee roasted at each roastery Total light coffee shipped from that roastery to all cafes.
Total dark coffee roasted at roastery Total dark coffee shipped from that roastery all cafes.
Total light coffee shipped to cach cafe from all roasterics 2 Light coffee needed at that cafe.
Total dark coffee shipped cach cafe from all roasteries Dark coffee needed at that cafe.
All decision variables 2 0.
Output of LLMFP GPT-40 Definer for Coffee. LLMFP successfully defines the goal, decision variables, and constraints.
Figure 16:
38

---

Published as a conference paper at ICLR 2025
Formulator Output
"variable_
"I "beans shipped",
"SMP variable': true,
"number_ of of__vrriables's
"shigping_cont costt__tt__on fron_ffoo__iiee caan__spplerr_.. _to rosstary',
'value": "anount beans shipged from each supplier each roastery',
"total beans shipped from each supplier to all roasteries supplier capacity"
variable__"" 2":
"aaa"" "Light. coffee_roostaaadd
"SHT váriable': true,
"number_e of "__yariablees:
"data_ sotrce' "zoasting_com cerstiigg_sss___oee light',
"value" "anount of light coffee roasted at each roaasery",
"specifie_requiieaie roastary"
"total received roastery total coffee roasted (light dark)
"variable_3":-
"name"1
roasted',
SMT variable': true,
"mumberoff, variaabellss 2,
"data_souse "aata_ouuree: geeaating_ostt_dee
'value": "anount dark coffee roasted each roastary",
appeific_reeuiteneee "total beans received by each roastary total coffee roasted (light. dark) at
that roastery"
varriable_''
"nane" siight_offfe_ehhppee variable";
"number, of variabless 6,
ddata_source, "shigping_com cost fron roastary to cafe", light coffee shipped fron "specific requirement". "total Light that roastery to all cafes"
"value":
eact cafe', each zoastery 1ight coffee shipped
"arriable_'';
:: "dark
shipped",
SNT variable' true,
"number variablles:
sources "shisping_cost
roastery cafe",
'value": "anount of dark coffee shipped from each roastery to each cafe",
'specific rerrer_rrrrrreeee "total dark coffee roasted each total dark coffee shipped from roastary to all cafes"
"ariable__"
"name": "sotal_coas",
'NN variable'; true,
"nunbar_of, Sactur_of_vrraaee variaaaaabll''
"vaaailae,"variialllee "data_souree": ('varisble variables', "variable variables", "variable variables', "rriablee","variablee 'variable
"value: "sun of shipping costs, roasting costa, and shipping roaated coffee costs", "specifie_reeee" retiiiieerrruireeeeet"""" "mininize"
"ariaabe_T"' :
"name": "light_cooffen needed for cafe",
varisble'i
"nuxber cumber_of_vvree varriabbeerr 3,
'data Hource"" gaght__eoffo__eeededd for fareeat"
'value"
"cafei": 20,
"cafe2": math.ceil(30 1.29),
"cafe3": 40
'specific requirement": "total Light coffee shipped to each cafe fron all roasteries >- light coffee needed at that cafe"
'variable_""
"name" "dark coffee needed for cafe",
'SNT variable't
variables'
'data source" www.oooeee_eeeddeeeee cate""
"value": {
"cafe2": ceth. 1.29),
"cafe3"; 100
"specific requirement": "bosal ahipped
roasteriea coffee
neoded at that cafe"
Figure 17: Output of LLMFP 4P Formulator for Coffee. LLMFP successfully formulates the decision variables with variable variable 5, account for the goal with variable 6, and consider additional constraints posed with query with variable_7 and variable_8.
39

---

Published as a conference paper at ICLR 2025
Code Generator Ouiput
solvez Optinize(1
Deflnw 5M7 variablex
beans sbipped [(5, 1) Intif beans, (t]_pp_______()) for 5 ('expplierl' w 'supplier2', 'supplier3") for in
"roastoryl', , 'sosstery2 13
light ltt__oooooreeeeeeeeeeeee roasted = [zs Int(e"light, coffce, roasted_ for i in l'roasteryl', 'zoastery2" 1)
durk cuffee rousted (es 2ueef'dark, coffeu rowsted(()) (r)" for in ('eoaxtucyl", "rowstacy2211
[ight_coffee_. coffee I' l'cafel', shigpedd
Ins(E"Light coghteoottee.o.....oooooooooeee wot_oooott___aaeeeeeeeee ctttre__pippe__t1_1 (e))
in I'roasteryl", 'roastery2')
daxk wotteotttteeeeeoee chipped [12, ejt dark cottee, www.tttehiooooeeeeooooooooooeeee I' "caSel", , "cafu2", "case3") total cost Inz( ('totall_ee ) a narlne nornal varlahlme Light coffce_netdes for cate I'catol's 20, 'cate2's math cctl.o 1,29), "cate3't 40)
('rossterp1', "roastozy2") 1
durk coffeofff_eeee pwwiedd_dddeee for__oorrrrtr cafe ('cuful's 20, "cufu2': (ith.owil(20 1,29), "cudu3"s 100)
rotal beans shipped each supplter all roasteries supplier eapacity capeoity. sn_ooplierr ('oupplier''s 159, 'supplier2's 50, 'supplier3's 100) for a in d'supplerr teri', 'suppliez2', "supolier3"):
sulver aedr.add (bdet shipped[(s, +)] for e in l'ronteryl"' y 'ronstary2' 1II <- capacity in. xugpliig((]] y Total beans received eacb reastery Zotal coffee roasted (light dark) that roastery for ('roasteryl'' "roasteryy"]s ]:
for l'supplieri', 'supplier2', 'supplier3' light gegtt_cofteeroostddip add bbens, (h coffee_ roffee__ssadd]]]]
rucal light coffew coasted each coustery Jutal light coffew shipped
roastary to all cafes
for r ('rasstery!' 'roastery"')
sol9erradd/ligh add/liggtt__ooo coffee ootfeee roasted[e] = Bun([Light, coffce, shigped[ e)l for e in l'eafel', "cate2', 'eate3'1))
d rotal dark coffew roasted at wach roustery dutal dark cotfee shipped From that romsery to all cafer for "soasteryl', "soastary2"
solver, add cof to
Sun([dark catffrrk_ooffeeeeeeeeeeee coffee shipde for l'cafel", 'cafe2", 'eafed]]])) that
rotal light coffee shipped
all roastories Light coffee
l'catel', "cate2", "cate3'] solvee. solveraadd(Sual add]sun( coffee phipped (2, etl for in ('soasteryl', "coastezy2" [1]
light fee, needed for cafe[c])
d Tutal dark coffee ahipped to wach pade from all cousteriun aae Dark coffee needed at that cafe
for in l'cadel", "cate2", "cade3'
solver Badaspuddkk_ooffl tott_cofffee_eiipeettt.. eil for in ('roastozy3", "roastery2 11) >u dark coffee peeded for cafejel)
All decision veriables
add
for vaz 1ight coffeeo 2oasted posssod.valoaa(()
solver, addivar on
for var in dack dack_coff coffee icoasted.valoww
solver. add a= 0J
for var in light_coffee wwwfeetffeeeeee cicocs__hiipdddeeooee valuea()) vo
avar
dark_coffww shipped. values)):
solver. var
7 Caiculate cotal cost
thipping_cose from suppliur, co_roostery
( "supplierr',
"roasteryl"1 11 5,
'uupplieri', 'coastery2"
"uppliorr'' 'roastery""
"supplier2', "aupplierr', "soastazyy"]1 2,
"supplier3'. "roaxtwry2"]:
roasting cost, 1ight ('roasteryl's 3, roastary''' 5)
roasting cost, dark ("roasteryl' 5, 'roastory2's 6) whigping_oowe from
('rossteryl', Toatel')
'mmestrryl'
'roastery''' "cate3")s
sosstery'', "cafel") ]=
2oastery2', "cafe2' ): 5
masstarr'' 'cate3' }:
total shippedi shivping costt supplier to (oastazyi
Sun([
('Hugplierl', "mughlier'', "mipllierr]] 11 B
("roaxteryl', 'roaxtary3
Xight_cottee_rone coffee_roaeted(1) roasting costt/oot___t__e (igh(e] for x in ('roasteryl', "roastery? ]
11 Bun(
dark (all__offfe__eee rooffn]=oosdd((=)) rowwing cost darkjr] for in ('romttarl"', 'roastery2")
11 Sun(
Light_cotfee. (t, e) shipping sost_tron_roo wwt.trrrootttttte roastery ct,el
for ('soasteryl'
for ('cafol', 'eafe2", 'cafe3')
11 Sun[
dazk coffee shippedi (x, et] whipping cost, from, romatery to ta, e)1
for in [ ('osstery) "roastery?" ] for e in ('catel', 'cateZ', "cate3')
11
solver, additotal cost gotal_cost_ooppt
Opt lon gosl
Figure 18: Output of LLMFP GPT.40 Code Generator for Coffee. LLMFP successfully generates Python codes based the representation formulation.
40

---

Published as a conference paper at ICLR 2025
Formatter Output
II JSON:
"suppliers":
"supplier 1_ship______ yruustrr__""" _,
"supplier 1 [hipp to roastery 25:
"supplier 2 2_ship to roastery_ 1":0,
"supplier 2, 2_ship_Uu to roustery_22 2": 50,
"supplier 3 3_ship_
roustery
100.
"supplier 3 ship to roastery 2,
"supplier 1 |_lotal suppplueppdddd__e beuns"; 95,
"supplier 2_ttrrl____a________eeeea supplied
:uupplied_bbuus"::a_ beuns": 50,
"supplicr_3 totattall httpsssppppp_e_eeeeeeeeee bcans": 100
).
"roasteries":
"roastery dark coffee": 41,
"roastery 1 light coffee": 59,
"roastery 2 dark coffee": 105,
"roastery 2 light coffee": 40,
"roastery 1 total roasted beans": 100,
"roastery 2 total roasted beans": 145
).
"cafe":
"roastery 1 |_ship dark to cafe [": 15,
"roastery ship light to cafe 20,
"roastery ship dark to cafe
"roastery ship light
cafe
"roastery ship dark cafe 0,
"roastery ship light to to cafe 3" 3"
"roastery 2 ship dark Booioktto_eeeelleee
cafe _al_caff__="_ 1":
"roastery 2 ship light to cafe
"roastery 2. ship dark cafe 0,
"roastery 2 ship light toc_tto__uul cufe
"roastery 2 ship dark_to_caat cooeeotoo_eeee____e _a: 100,
"roastery 2 shp light cafe 40,
"total dark coffee received cafe 20,
"total dark coffee received cafe 26.
"total, dark coffee received cafe
"total light coffee received cafe _afe_1":000 20.
"total light jar_ccooroeeevveeeeeeeeeeeeeeeee received_ cafe 2":
"total light_coofeee____o receeeeeceeeeeeeeeeeeeeeeeeeeeee _a:e_3"ffe4_ 40
1.
_ostt!" 2612
Correctness reasoning:
GOAL: The task plan aims to minimize the total cost, which is reflected in the total cost of 2612.
Total beans shipped from cach supplier to roasteries z Supplier capacity: Yes, the total beans shipped from each supplier do not exceed their capacities.
Total beans received each roastery Total coffee roasted (light dark) at that roastery: Yes, roastery reccives 100 beans and roasts 100 beans, roastery 2 receives 145 beans and roasts 145 beans.
Total light coffee roasted at each roastery Total light coffee shipped from that roastery to all cafes: Yes, roastery roasts and ships 59 units of light coffee, roastery 2 roasts and ships 40 units of light coffee.
Total dark coffee roasted each roastery Total dark coffee shipped from that roastery to cafes: Yes, roastery roasts and ships 41 units dark coffee, roastery roasts and ships 105 units of dark coffee.
Total light coffee shipped each cafe from all roasteries light coffee needed that cafe: cafe2 needs units of light coffee but reccives 39 units.
Total dark coffee shipped to cach cafe from all roasteries 2 Dark coffee needed at that cafe: No, cafe2 needs 20 units of dark coffee receives units.
decision variables Yes, all decision variubles are non-negative.
Does the task plan make und achievable reality and commonsense?: Yes, the task plan achievable and makes sense in reality it meets the constraints and minimizes the total cost.
Figure 19: Output of LLMFP GPT-40 Formatter for Coffee. LLMFP format the execution result in fixed format, and analyze the correctness and reasonableness of the execution result.
41

---

Published as a conference paper at ICLR 2025
Self Assess & Modification Output
[[Step I:
Correctness Reasoning: The considers the correct goal and all needed constraints. The execution result makes sense and is achievable in reality and meets commonsense.
Rating:
How mofify Reasoning: NULL
Modified Step I(no explanation): NULL
END
[]
[[Step 2:
Correctness Reasoning: The variables explicitly consider query and encode all provided constraints and goals. The order makes sense, the execution result achievable in reality and meets commonsense.
Rating:
How mofify Reasoning: NULL
Modified Step 2(no explumation): NULL
END
[]
[[Step
Correctness Reasoning: The code creutes needed variables and does make up any ungiven information. It correctly addresses the query. and the execution result makes sense and achievable in reulity and meets commonsense.
Rating:
How to mofify Reasoning: NULL
Modified Step 3(no explanation): NULL
END
11
Figure 20: Output of LLMFP GPT-4o Self Assess & Modification for Coffee. LLMFP checks for all 3 steps and provide ratings.
42

---

Published as a conference paper at ICLR 2025
A.10 PROMPTS
A.10.1
BASELINE PROMPT
Direct Prompt
You have domain and query under this domain that you need to fulfill. The domain is: (task) Qucry: (question) You have the access to (info_api)
What is the plan to achieve my geal? Answer by fill in this JSON response directly with explanation:
(output_format]
CoT Prompt
You have domain and query under this domain that you need to fulfill. The domain is: (task)
Query: (question]
You have the access to (info_api)
What is the plan to achieve my geal? Let's think step by step, first reason about the problen and how to solve it, then answer by fill the JSON:
Reason:
JSON response:
purpuut_formal
Code Prompt
You have a demain and query under this domain that you need to fulfill. The domain is: (task)
Query: (question) You have the access to (info_api]
Please write Python code to help me find the plan to achieve my goal. You can import any package and use any solver. At the end, following format:
your found plan in variable named 'feedback' with the
formatl
Please respond with code only and wrap your answer with "python and "*;
A.10.2 LLMFP PROMPT
We use general templates for all tasks. The full prompts for tasks are available in https://sites, http://iteesgoogl..coooo/ woogle.com/view//lnng. Here we show the templates we have for GPT-4o. Since Claude naturally considers more constraints and more strict assessing, we edit the prompts a little to account for the different traits of Claude, and prompts are also included in the codes.
We include the prompt template we use for GPT-4o as below:
Definer Prompt
You are given task description natural language, and you want solve it by building an optinizaiion problem for this task. The task is:
You have the access to (info_api)
To get started of building the optimization problem, what is the goal, decisicn variables, and constraints to consider for this task? Specifically, consider:
Goal: define the objective trying to optimize
Decision variables: identify all the decision variables involved in the problem
43

---

Published as a conference paper at ICLR 2025
Constraints:
key requirement for decision variables; For every pair of decision variables, carefully consider relations (explicit, implicit, underlying assumption, unmentioned commonsense) between them and explicitly include as constraint to ensure all variables are connected with each other
Response with [[GOAL: }], [[Decision Variables: 11, [[Constraints Reasoning: ]], and ['Constraints: ]] only with no explanation and no math formulas. Iry to be thorough and include all needed information much as can.
Formulator Prompt for single-step multi-constraint problems
You are given Query under task description in natural language, and you want solve by building an optimization problem for this task. You already have considered the goal and constraints of this optimization problem. Your job is, given access to existing variables APIs and specific natural language query, think about other variables needed to encode and solve this problem with 23 SMT solver and describe the important attributes of variables as JSON format description. Here are some example task-output pairs to refer to: Example task 1: There are blocks of different colors and scores in the scene. You need select zequired number of non-repeat blocks with required color, while maximizing the score.
Query: previcusly want to select 20 blocks that are black or red, but now demand raises 98.
GOAL: Maximize the total score of selected blocks. Decision Variables: Indexes of blocks selected Constraint: The required number of selected blocks is met.
Constraint: The selected blocks are non-repeat.
Constraint: The selected blocks have required color.
Variable API:
You have the access function math.ceil()
round UP float to int and mator () to round DOWN float to int. Please ONLY use these to convert from float to int.
You have access to BlockSearch API.
gives 1.all possible block ids of color in "color" 1ist and 2. c.corresponaing score inřo. Blocksearch.get (score info, pLock_indes gives the score of certain block.
run
JSON description:
"variable
"name": "blocks",
"ST SWWT_vrraale" true,
math. ceil (20 1.09), "data_scuree"; "BlockScarch.run
"value"; "selecting math.ceil (20 1.09) blocks from black and red blocks",
rementt "selected blocks are black or red; non-repeat blocks"
"variable_2 2":
name": "score",
"SMT_ "MM_variable": true,
" nuuber_off_vrriblles math. ceil (20 1.09), "data_scurce"; broocksearch.get_inne
"value"; "depends on variable_1 variables", null
"variable_3"
"name": "total_score" "SMT_ wsrraaarabl""; true, "number_of_we of_variabeee" "data__ccrre "variable_2 variables", "value": "sum of variable variables",
44

---

Published as a conference paper at ICLR 2025
"specific requirement": "equal to of variable_2 variables, maximize"
Example task Given list of cities, you need start from origin city, non-repeatly visit each other city exactly once, and traval back to origin city, with minimized total distance travelled. Query: Total number of cities is 10.
GOAL: Minimize the total travel distance. visited city indexes Constraint: Start from and end with same city.
Decision Variables: list
Constraint: Each city is visited exactly once and non-repeat. Variable or API: You have access to DistanceSearch API.
DistanceSearch.run takes argument and gives the distance info between cities, and DistanceSerarch. geterchheeettttie (diittaneee city_2) gives the distance (a real numbor) between two citics. Based on below examples, your task is to generate JSON description to describe the problem.
city_l,
JSON description:
"arrable__";
"name": "cities",
" "MM variableii true,
"number variables": 10,
hhow_tc_p hoo_te_pick"; "selecting 10 cities from 10 cities",
"data_scurce"; null,
"non-repeat cities"
"variable_2""
"name": "distance", " "SM__variable": true,
"number_of oaribbrr____arrablleeeee 10, "how_te "how_to_pick"; "depends on constraint_ variables",
"data_sse dtata_scurce"; "pistanceSearch, ru)
DistanceSerarch.. (
sspecific requirement" "distance between each city pair, and the distance back to origin city"
"variable_3"
namee: "total_distance""
SMT SSMT_variablee true, 'number_of_varibbles 1, " "data_scurce": "variable_2 variables", "value": "sum of variable_2 variables",
"equal to sum of variable_2 variables, minimize"
Now, based on the examples, solve the Query under new task setting and respond with similar format, please explicitly specify the action/ requirement needed to fulfill query, and explicitly take into consideration every constraint mentioned: The task (task)
Query: (question)
(definer_response) Variable or API:
{info_api]
Think about variables needed to encode all constraints and goal, describe all important attributes of variables as a JSON format description. Make sure to explicitly consider and include needed to the query. Note that to the query "Why do
45

---

Published as a conference paper at ICLR 2025
you need examine the effect of "not doing provide reasons; and to answer query "Why not xxx", you need to examine the effect of "do xxx" to provide reasons. Response with JSON only with no explanation.
Formulator Prompt for multi-step problems
You are given Query under task description in natural language, and you want solve it by bui buing
oetimization problem for this task. Your job is, given access APIs and specitic natural language quezy, think about variables needed to encode and solve this problem with 23 SMT solver and describe the important attributes of variables JSON format description. Here is example task-output pairs to refer to:
Example task: You have to plan logistics to transport packages within cities via trucks and between cities via airplanes. Locations within a city are move between any two such locations),
directly connected (trucks and so are the cities. In each city there is exactly one truck and each city has one location that serves as an airport.
Here are the actions that can be performed and its preconditions and
effects:
load truck: head (package) inte (truck) a (location) only if the package and the truck are both at location. After the load truck action, the package is not at the location and is in the truck.
Load airplane: Load a (package) into both at location. After the Load
lairplane) at (locstion) only
if the package and the airplane airplane action, the package is not at the location and is in the airplane.
Unload truck: Unload a (package) frem (truck) at a (location) only if the truck is at location and the package is in truck. After the Unload truck action, the package is not in the truck and is the location.
Unload airplane: Unload a (package) from an (airplane) at a (location] only if the airplane is at location and the package is in airplane. After the Unload airpiane action, the package is in the airplane and at the Location.
Drive truck: Drive a truck from one (location_1) to another (location_2)
within a (city) only if the truck is at location_l and both location_l and location_2 are both in city. After the Drive truck action, the truck is not at location_1 and is location a city to another
Fly airplane: Fly an airplane from one [location_i] (location_2] in another city only if both locations are airport and the airplane is at location_1, After the Fly airplane action, the airplanc is not location_1 and is at location_2.
Query: You have 2 airplanes a0 and al, 2 trucks to and tl, 2 cities c0 and cl, city c0 has location 10-0 and 10-C is airport, city el has location 10-1 and 10-! is airport, and package p0. Initially, t0 is at location 10-0, t: is at location 11-0, p0 is Location 11-0, al and al are at 10-0. The goal is to have pu to be at 10-0.
API: You can assune you already knew as the input. You have access to update_data () API that helps to update the predicate variables. JSON description:
objects";
"variable_1";
"name": "objects",
"SMT : false, "number_of_variablee"" 1,
"data_source"; "query", "value": "a dictionary that summarizes all objects in the problem: key 'package', value ['po']; key 'airplane', value ['a0', 'al']; key 'truck', value ['to', 'tl']; key
46

---

Published as a conference paper at ICLR 2025
"iity' value ('c0', 'cl'1; key 'Location', value ['10 '10-1'1; key 'airport', value ['10-0', '10-1")", "apucificerrquuirrmnnd null
b,
"variable_2" "predicates":
"name": "at", "SMT_variaa" variable"; faise, "number "fuubber____ariablee"".." variables": "data_source": Mquery, variable_l", "value": "a dictionary of boolean variables representing whether an object is at location at timestep: keys are location, timestep)",
"specifii__equuuremnnt""
"specific requirement": "add constraint initislize timestep according to query, for unmentioned objects explicitly set it to be False"
"variable_3"; "name": "in", "SMT "SMT_variable": false, 1, "data_souroe"; "query, variable_1", "value": "a dicticnary of boolean variables representing whether an object is in airpiane or in truck: keys are package, airplane/truck, timestepl", eepecific_requiremettt "add coist to initialize all values to be False timestep 0"
"variable_
"name": "in-city", = false,
"number_off.rrrialee'::
"data_souroe"; "query, variable_1",
"value": "a dictionary of boolean variables representing
whether an location is in a city: keys are [location, city, timestep]",
"specific_requi rement": rementt: "add constraint initialize timestep according to query, for unmentioned objects explicitly set it to be False"
"actions";
"variable_5";
"name": [load_ttuck" [SMT_variable: false, Lnumber_of_variablee
"tata_ssorre"" "variable_1", "value": "a dictionary of boolean variables representing
whether Load_truck action is pertormed for package, truck location: keys are [package, truck, location, timestep ]"
rapecifii_requiremmmn null
"variable_6" 6";
"name": "load airplane", "SMI_variablo" false, "numor__of_varaalles"" 1,
"data_source": "variable_1", "value": "a dictionary of boolean variables representing whether load pirplane actior is performed for package, airplane, location: keys are [package, airplane, location timestep cerpriiiii_rruurrmmettt" null
47

---

Published as a conference paper at ICLR 2025
"variable 7
"name": "unload_truck", "SMT_variable": false, "aumbr_o__varioles"" "variable_l",
"value": "a dicticnary of boolean variables representing whether unioad_tree aarckk action is performed for package, truck, location: keys are Ipackage, truck, location, timestep ", requirement"; null
"variable_";
"name": "unload_airplane"
"SMT_varri" variable"; false,
"number_se of "aaaarroo__arialees" staat__soure"; "variable_1"
"valuc": "a dictionary of boolean variables representing whether unioad_airplane action is performed for package, airplane, location: keys are [package, airplane, location timestepl", "specific_requii rement": null
"variable_";;
"name": "drive "trckve "SMT variableri false, "nunber_of_varrialees:: 1, "data_source": "variable_1", "value"; "a dictionary of boolean variables representing whether drive_truck action is performed for truck, socation_from, location_ton city: keys are [truck, location, location, city, timestep]", "specific_requuireeei" null
"variable_li :
"name": "fly_airplane", "SMT [SMT_wariable": false, "number_of_vartablee"" 1
"data_source": "variable_1", "value": "a dicticnary of boolean variables representing whether fly siy_aim action is performed for airplane, location_ from, location to: keys are [airplane, location, location, timestep]", "gpecificerequuremenn""" null
"update":
"step_1";
"name"; "action load_truck precondition and effect", "SMI_variable": null,
"number_of_vee variables"; mull,
"data_source": "query, variable_l, variable_5", "value": "add constraints for preconditions and effects load_truck",
ret.rireerruiimmmett..... "for each timestep t until T, for all package, truck, and location, assert that load_truck[ package, truck, location, implies at/truck, location, tl, at [package, location, t], not (package, location, +1], inlpackage, truck, t+1]*
truck
"step_2";
"name": "action load_airplane precondition and effect", "SMT "SMT_variable": null, "number oat.ooffvarabbess"" null, "query, variable 1, variable_ 6",
48

---

Published as a conference paper at ICLR 2025
"value": "add constraints for preconditions and effects load_airplane",
aepecifieerrquiirmmmnt "for each timestep t until T, for all packaçe, airplane, and location, assert that load aatpppiaaaeepcccaaaaaa airplane, location, t] implies at airplane, location, t], at(package, location, t], not at package, location, t+1], in (package, airplans, t+1]"
b
sstep_3::
"name": "action mllcad_truck precondition and effect", "SMI_wariable": null,
"nunber_of_variables" mull, "data_souroe"; "query, variable_l, variable_7", "value": "add constraints for preconditions and effects of unload_truce
pnreecfiie_reuizeenn
"for each timestep t until T, for all package, truck, and location, assert that unload_truck[ package, truck, location, t] implics at [truck, location, t], in (packaae,, truck, t], not in[package, truck, t+1], at(pckaage, location, t+1]"
step_4":
"name": "action uilaad_irples precondition and effect", " "SMT variabler; null,
"number of of_ variables": null,
"data_source" "query, variable_l, variable_8", "value": "add constraints for preconditions and effects of unload_airplaane", pnpecifiie_requiremett "for each timestep until T, for all package, airplane, and location, assert that anload_airplane (package, airplane, location, t] implies at [airplane, location, t], (package,, airplane, t), not in iplpakkage, airplane, t+1l, at (package, location, t+1]"
1,
"step_5":
"name": "action drive_truck precondition and effect." "SMT "SMT_variable"; null,
"nomber_of_uaribbles"". null,
source_ "query, varisble 1, variable
"value": "add constraints for preconditions and effects of trive_truck
"specifie_requirement"" "for each timestep until for all truck, location_from, location_to, city, assert that drive_truck (truck, location implies at (truck, location_ from, tl, not at [truck,, location_from from, t+l], at [truck, location_to, t+1]"
location_to, city, t]
b,
@ste__6;
"name": "action airplane precondition and effect", SSWT_variables null, "number_of_varibbles" null, "data_source": "query, variable_l, variable_10", "value": "add constraints for preconditions and effects of fly aiy_airpl,""",
"for each timestep t until т, for all airplane, location from, location_ton assert that fly_airplanc aalaan/aaiipaae, location location_to, t] implies at jairplane, location_from tl, not at (airplane, location_from, t+1], at lairplanc, location_to, t+1]*
"step_7":
"name": "all_actions"
"SMT variable"; false,
"number_oe of v "list of all actions",
49

---

Published as a conference paper at ICLR 2025
"data source": "variable 1, variable 5, variable 6, variable_7, variable_8, variable_9, variable_10", "value": "for each timestep t until T, a list of all possible actions corresponding to different objects", rspecciiiereeuuirement "for each timestep t until T, explicitly assert ONLY ONE action per timestep"
s step__"; 8":
"nane": "unchanged predicate variables update",
" "SMI_ variable"; null, null,
"number_of_varrddb""
_oota_s "update_data
"value": "update at, in, in-city using update_data ()", rererific_requirmment" "update data with update d)ta
},
"goal":
"stcp_9";
"name"; null, [MM[_arrsble": mull, "nunber_of_variablee null,
"data_souree" null,
"value": null,
requirement": "assert for timestep I, package p0 is at location 10-0"
Now, based on the example, solve the Query under new task setting and respond with similar format, please explicitly specify the action/ requirement needed to fulfill query in your response: The task is: (task)
Query:
(question)
input, so do re-initialize anywhere. You have access to a data (solver) API that helps to update the unchanged predicate variables. Please ONLY use this API to update unchaged predicates. Response with JSON only with explanation. JSON description:
have access
Code Generator Prompt
You are given task description natural language, a specific natural. language query, available APIs and variables, and JSON description that summarizes zes important variables that guide you to ancode and solve the problem with SMT solver.
Your task to generate steps and corresponding Pythen codes that utilizes 23 SMT solver to solve the problem.
For the variables summarized in the JSON description:
(1) 'naee' represents the name of the variable
(2) SST__vrriabl'' indicates whether you should assign as normal
variable or an SMT variable
SMT_variable Example: price = Int ('price')
Flight_index = [Int fwigghggglhlit (I_ideex format (1) for i in range
pick Booll'pick ball') Boolean SMT variable
Normal variable Example: price = 100
flight_index [1,2,3]
(3) 'numeer hnumber_of_vartee _o__varibblle
represents the length of the variable
50

---

Published as a conference paper at ICLR 2025
(4) 'data source' is the source for the variable to get the data
(5) 'value' is, after you get needed data from any source, how you should assign these data to the variable
(6) ppecifierrequrrement is if there are any specific requirements that needs to be considered.
For the belew problem, can you generate steps and correspending Python codes to encode it? Do not include any explanations. You do not need to solve the problem or print the solutions. The task is: (task)
Query: (question) (definer_reeponse)
Variable or API:
(info_ap)) __p] JSON variable representation:
formulator_responee
Please use SMT variable named tetal_cost when calculating the total cost. Please put the imtimiationn goal at the end after all needed calculation and constraints additions.
Make sure your code add constraints
solver that considers and could answer the query. Note that to answer the query "Why do xxx", you need to examine the effect of "not doing xxx" to provide reasons; and to answer the query "Why not do xxx", you need to examine the effect of "do xxx" to provide reasons.
Initialize 23 optimizer solver Optimize () at the beginning the code.
Response with Python code only with no explanation.
Formatter Prompt
You are given task description in natural language, specific natural Language query, pre-detined variables, and execution feedback by running Python Code that tries to solve the task. The task is: (task)
Query: (question) Execution feedback: (feedback)
Variable API: (info_api)
If the execution feedback is runtime errors, please return RUNTIME ERROR for JSON: and NULL for Correctness reasoning:. If the exccution feedback is cannct find the solution, please return CANNOT FIND SCLUTION for JSON and NULL for Correctness reasoning:. the execut ion feedback is not runtime errors, the execution feedback is the sol velved solution for this task. Only using the information from Execution feedback (do not use predefined variables), transform the execution feedback into JSON format task plan by filling in the JSON below:
[output_formati format
In addition, for Correctness reascning, please explicitly answer one by one does the task plan satisfy these constraints? Include one sentece explanation for each constaint:
{I [[défiinrr_reppnne))
Then explicitly answer and explain in one sentence: Does the task plan make sense and achievable in reality and meet commonsense?:
Please include your response here with no explanations:
I1
JSON:
Correctness reasoning:
1]
Self Assess & Modification Prompt
51

---

Published as a conference paper at ICLR 2025
You are given task and steps that tries to solve it optimization problem. The steps include:
1) specifying the goal and constraints of the optimization problem.
2) JSON description that summarizes important variables that guide to encode and solve the problem with 23 SMT solver.
3) the Pythen code encode and sclve the problem with SMT solver. Your goal to, based on the task description, specific query, available
API or variables, and runtime execution feedback (it could either be an execution error or generated plan it there's no runtime error), whether any steps 1-3 correct. The task is: (task)
Query: (question)
Variable or API: linfo_aph) api)
Steps judge:
1) )
2) {formulator_responnl
3) [coc_genetttorreeoonee
Exccution feedback: {feedback)
Based on the previous information, evaluate whether steps 1-3 are correct
For Step 1: Does the step consider correct goal and all needed constraints? Are there unnecessary missing constraints? Does the execution result make sense and achievable reality and meet commonsense?
For Step 2: Do the variables explicitly consider the query? Do the variables explicitly consider and encode all provided constraints and goal? Does the order make sense? Does the execution result make sense and achievable in reality and meet commonsense?
For Step 3: Does the code create all needed variables? Does the code make any ungiven information? Does the code correctly address the query? Does the execution result make sense and achievable in reality and meet commonsense?
Please reason the correctness with task context, rate each step with a binary score: 1 is correct, is incorrect, think about how to modi fy in detail according to task and query, and modi fy the step if you think it incorrect.
For Step 2 modification, please write JSON format. For Step modification, please write in Python and do noy change the content after line 'if check == sat:
Your response format should be belew, put NULL to How to mofify Reasoning and Modified step if you think the step is correct, not include extra explanation:
[[Step 1:
Correctness Reasoning:
Rating:
How to mofity Reasoning:
Modifiea Step 1 (no explanation):
END
]]
[[Step 2:
Correctness Reasoning:
Rating:
How to mofify Reasoning:
Modified Step (no explanation)
END
1]
[[Step 3:
Correctness Reasoning:
Rating:
How to mofify Reasoning:
Modified Step 3 (no explanation)
END
52

---

Published as a conference paper at ICLR 2025
A.10.3 PROMPTS AND OUTPUT FOR LLMFP WITH MILP SOLVER FOR COFFEE EXAMPLE
DEFINER and FORMATTER prompt remain exactly the same as for SMT solver.
We include the comparison of prompts for FORMULATOR, CODE GENERATOR, and SELF ASSESS & MODIFICATION in Fig. 2 to 23 and labelled all the differences with red. We then include the output of FORMULATOR and CODE GENERATOR in Fig. 24 and Fig. 25 The key takeaway
is very easy to switch from one solver to another with LLMFP, as the inner logic is same: building an optimization problem.
Formalator Prompt- SMT
Formulator Prompt- - MILP
wihe building optimization pollen for this taik. You aleady have cassilered the goal and omerinms this opriitattion prokiem. Yier jub givn sccew to conting mmetlos or Allle and a speclfic ratural langogg quay, Biikk glout other varitbtes seeded cocode and solve prebum with
give undera todk deeription nanual
solve building opinizaion problan fo the uak. You ciresdy have considered the aod and conetrannc of this oprmizatin prebkan Your joh given aecose to ociding vanabiie or Alfss and a specific ustual largagge query, thisk about otee. vasstig saillle acoded encade and salve this pomblm wits curoh
You given inder dosoripiion ratural
sahverand dearihe the importass anriuios vanablis a some escmppe tal-oottoo pairs refor No:
auber and disenbe the importani atmbhrrss variubls socce examppe tok outyst palis
Rimaa desorption
ISON descrtgtion
190N desor
ISON dest destiption:
Vasiible 1
"curable 1%,
"esmr"" "Necka"
"rome" "Hlake"" "GRB vurishee" tue.
"SMI vaitille" lnc,
"sumieer vanialze' much crikito 1.09)
"wumber vorahie math-ozl(00 1.09).
"salie"; "BlockSearch.nr
source":
"dar, soure" "Bloc&Serrhrrutt"
colll1 blacks from black bicce""
from black and opeiifi_regureeeee "selected blocks beck blockss"
math.ccl120
ggooife_ogpireeee "silected blocks Mack nenvepoor Nocks"
"vrrittl
"oorabie
"sasce" "acoee'
"Hane': "scarc".
vartiler tuz,
conabic"
variatled auth coll:o 1,89)
"BlocSearchgg "valte" "Sepends varable_ vorisbles"
"Haa "vale": dopeads ihlle vanables"
'ppciffe roquirenena null
'spezific requrmmen mull
Vaiidll 1
"arale 3
"eamr"" "ustal wood"
"rome" "teeal soors".
varidlles buc.
vunable"
"Iaa "ourshii varisboa"
"valie" "san voribbblee variales.
"valu": variaat varibles'.
'ppaoife roquirenent": "squal sun variable 2 varabies, masimize"
'spezific requrement: eequal sum varshll 2 variablo, mucimise
dearption:
dosrripione
Siiibll 1":
"carabi
"axmre" "ikees"
"home" "cllee"
"SMI vartill's bsc.
"GitS varalie tnc.
"sumber of variattes:
"nunter worabees
"how sick": "sdectiu 10 citcs ezies",
"how pick: "sclecting 10 ciiss citie".
"ppcille isquirtticst
"specifi: sequeenen" "ooa-repout clies
"vaiiilee_""
"uurabie 2"
"axmre" "disance". "SMT varticle's bsc.
"hone" "dittaee"" "GitS vanable" Inc..
"sumber of variatte": 18,
"runter vorabres" 10
sick": "Sependa conatraist varisbles"
"how to pick:
deppasda contrint varibles..
get pai, the distance
pe exch dituses
Sppcille roquirctica". "Sisance
"speiifi: sequenent
back
back origin
Sariill 1
"vorabie 5%
"sumee" "wall dizance"
"hane": distare""
variele's buc,
varalic"
"data source": "variable vanables"
"das. "varible variisoo".
"valse" "am viribb vantallo".
"vole"" "eam vanatly vonshiie'
"apociffe soquirenent" "squal sim vuriable vanables, tininize"
"specifii jequrement equall sun variable vasailos, mininize
78A75-
SAME TEA7S-
Figure 21: The Formulator prompt difference when switching from using Z3 SMT solver to Gurobi MILP solver.
53

---

Published as a conference paper at ICLR 2025
Code Generator Prompt SMT
You given task description natural language, speeific natural language query, available APIs and variables, and JSON description that summarizes important variables that guide you encode and solve the problem with SMT solver. Your task is to generate steps and corresponding Python codes that utilizes Z33 SMT solver to solve the problem.
For the variables summarized in the JSON description:
(1) 'hume' represents the name of the variable
(2) 'SMT variable' indicutes whether you should assign it as normal variable or an SMT variable
SMT variable Example: price Int pu(ppicee)
flight index = lint(flight 1 index' (ormmt(() for i in range(3)]
pick ball - Bool("pick ball") # Booleun SMT variable
Normal variable Example: price 100
flight index = [1,2.3]
(3) humber of variable' represents the length of the variable
(4) 'data source' is the source for the variable to get the data
(5) 'value' is, after you get needed data from any source, how you should assign these data to the variable
(6) 'specific requirement' is if there are any specific requirements that needs to be considered.
SAME TEXTS-
Initialize Z3 optimizer solver = Optimize() at the beginning of the code.
Response with Python code only with explanation.
Code Generator Prompt- - MILP
You are given task description in natural language, specific natural language query, available A Alss and variables, and a JSON description that summarizes important variables that guide you to encode and solve the problem with Gurobi MILP solver.
Your task to generate steps and corresponding Python codes that urilizes Gurobi MILP solver to solve the problem. For the variables summarized in the JSON description:
(I) 'name' represents name the variable
(2) 'GRB variable' indicules whether you should assigm it normal variable GRB vuriable
GRB variable Example: model Ladd Var.aVVaavyr-GBBB.... INTERGENNERRR name="w") # Integer GRB variable
Meoo-GRB.NTEGGEE name="x") # List of Integer GRB variables of leangth z goooeel.addVarrvtyyee BRB.BBB---RBBBBBBRYYYY mume="z")> # Booleam GRB variuble
vars Jist = anddelladd
Normal variable Example: pricc = 100
flight index - [1,2,3]
(3) 'number_ variable' represents the length the variable
(4) 'data
source' 'data_source'
source for the variable to get data
(5) 'valuc' is, after you get needed data from any sourec, how you should assign these data to the variable
(6) 'specific requirement' if there any specific requirements that needs to be considered.
SAME TEXTS
Initialize Z3 optimizer model = Model("model") at the beginning of the code.
Response with Python code only with explanation.
Figure 22: The Code Genetator prompt difference when switching from using Z3 SMT solver to Gurobi MILP solver.
Code Generator Prompt SMT
You are given task and stops that tries to solve it as an optimization problem. The steps include:
1) specifying the goul and constraints of the optimization problem.
2) JSON description that summarizes important variables
guide encode and solve problem with Z3 SMT solver.
3) the Python code to encode and solve the problem with Z3 SMT solver.
Your goal is based on the task description, specific query, available API variables, and runtime exccution feedhack (it
could either be an execution error or generated plan if there's runtime error). assess whether any steps 1-3 are correct.
SAME TEXTS
Self Assess & Modification Prompt- MILP
You given task and steps that tries solve it an optimization problem. The steps include:
specifying the goal and constraints of the optimization problem.
2) JSON description that summarizes important variables that guide to encode and solve the problem with Gurobi MILP solver.
3) the Python code to encode and solve the prublem with Gurobi MILP solver.
Your goal is to, based on the task description, specific query, available API or variables, and runtime exccution foodback (it could either be an execution error or generated plan if there's no runtime error), assess whether any steps 1-3 are correct. SAME TEXTS-
Figure 23:
The Self Assess & Modification prompt difference when switching from using Z3 SMT solver to Gurobi MILP solver.
54

---

Published as a conference paper at ICLR 2025
Formulator Output MILP
vvariable_""
"nams" supplier, roastery",
"CRB variable": true,
"number of variebles";
"data "data_souree" source": "capacitty supplier, shipping, from supplier roastery", "amount of beans shipped from each supplier to each roastery'
variable_2":
"name": "y_light roastery",
"GRB veriaabial""
"number of oflooof_rrrraa variables": 2,
"data source'; "roasting_cost light",
sppeifiie_rouurrmmnnt "amount of light coffee roasted each roastery"
varrablee3";
"name": : "y_dark________
"CRB veriabiial"
"aumber of s__arraaales'
"data source"; "roasting costcoot___ee dark",
pheeiffic_reuiremne
"anount of dark coffee roasted at each roastery"
varieble_4',
"name"; "z_ "z_light_ roastery_cafe",
"GRB variable": true,
"number_ of s__vrriablls"
"data source"; "shipping_con" costcottt__eee from roastery to cafe",
appeiifierrequraanee
"anount light coffee shipped from each roastery cafe"
variable_5":
"aame"s ; "2_dark s___dar_roaateee roastery_cates
"GRB variable": true,
"aumber variables":
caata_source', "shipping_cos" wwtor/rrrotteeeeeeeeeeeeeeeeeeee roastery to to_cafe",
"specific requirement': "anount
shippad each roastery to each
sariablee66';
"name": "total_cost"
"GRB variable"; true, "numbor of variables"
"data clata_source'; "variable_1 variable_2 variable variable_1, variable_5;
"value"; "sum of shipping, roasting, and shipping costs", speeifiee_rguirrment "minimiza"
"variable_""
"name":
needed for for__afee cafe",
"GRB variable"i false,
"aumber of variables":
"data_soure" source": "light_ coffee needed for cafe", sneeiffic_reqirement
by
variablee_" 8":
"name": wwwfeeeneeeeeeeeeeeeeeeeoeeeeeee needed for__oo________e needed_for__efee for_cafe",
"GRB_variee variable": false,
"aumber variabaess"
Slata_source"; "dark_coffee" coffee coffee_nceded for for__cafee cafe",
speeifficrreqireeee "cafe2 demand increased by 298*
Figure 24: Output of Formulator for Coffee after switching the solver MIL., using GPT-4o.
55

---

Published as a conference paper at ICLR 2025
Code Generator Output -MIL.P
model Model ("rodel")
cupacity_ in supplier ('supplieri's 150, 'supplier2'= 50, "supplierá"; 2003 shigping cost, from sugglter,
fron-trron.uggllerttto bo_ rosstery
( ('uppplirr'',
"reastezyl'1
11 5,
supplieei',
"coastery2 [] 4,
"oastezyy "uupplier2',
'uippieer', 'roastery?""
"supplier3', "roastery"" 11
( ''uuplierr', "roastery2"
ronsting nowe,_ Hight ('roastary1's 3, rowstary2's cousting cost, dark dark shipoing_oost. oost_ ftootttoooooooooo wtt.:_toooootroooe www.rraattttooooo to_oate
("coaxtasyl"; 5, "oostwry2'
( "zoasteryl", "cafel"):
"roastrryl' 'axfe2"):
"aoasturyl'' 'cufe3'):
'rrasttery'' 'catel')
"soastery"', "cafo2')s
( "roastery2', "cate3' ):
1ight, coffee. needed for cafe + !'catel's 20, 'onte2's 30, ootes's 48)
dark coffee peoded_ for cafe ('cafel't 20, "cafe2's 20, "cate3's 168)
desand cafe?
light, serfee__eedd__ttee for safel'cated")
(1ight, dark coffee paedeed_eeddd__eeeee for por__aff[ "cafe2) ] nath cwillddrk cat peeded for _acee " 11.29)
paaeeaaaee J. 1.29)
f Decision Variables
s_suppleer_a roastery model. addVars chipping cecpoinnnessss_eeeeeeeeee cost_tronn fron supplier to conteryy.kyya)) Vtypa CONTIRUOUR,
w_1ight_roaster model addvara y
1ight, roastery") GRB CONTIISOUS, dark roasterv") 1_igh woastery cafe xodel httpar//siippiggeoo cost, froo, soastery so, cafe keysl vtype COSTINUOUS, nurme light coaxtsry_caff"
model addWarss
light, dark, leysll
dark rowstory cafe model. addiarssshippllai taciippnn___ooooo post, post, poet_from from roastery_to. to_oste.keyss) keyo(), vtype-GRs.
CONTINUOUS, rtype-9n6.comrx00055
name-" dark www_eer_oonetry.ccoo rosetarryyttat"
Comstraints
Sur
shipped each suppiter all roasceries aue supplier ospacity
sugel exgacity_ inppllrre
model, addi cupplier joastory roastory) for roastery roasting_coos cost, koys() cupacity suppliar [supliae)]
d Sun of beans reweived by mach roaxtrry from all sugeli - sun of light and dark coffee roasted an that roastery
for roastory in roasting cost 1ight.
model addoonstr(eunts supplier_ pempllie__raeteryllle roaeteryleupplier, roastery) for supplier eapacity pupplleer............... keys())
y Light soattrry
Y__ark__ooae (ottetrrr
Light coffee roasted at each roastory
light coffee shipped that roastery all cates
for roastery in roassing cost Hight k)y
model addconstrly light, coasterylsotstery] ss pants light eafeeeosoeae/rsstoryy cafe] for cafe in light coffee needed for cafe. keysl))
Sun dark coffee rossted each roastery of dark coffee shipped from that roastary all oxtes
in wtt.duuk__ooxecllllleo contscyytouuteery andel uddtamstrly_du bay__dee__rrttaae
dark, derk.
xun(s_durt caarkk_oosttry_etteo cam cafe]
dack coffee pouded for caf.eaaeeeeee baya(1)
y Bun of Light coffee shipped to each cafe frem all roastories 18 light coffee needed for that cafe
for oate in light_ooffes coffee needed for_cafei cafei
model ad8tonstr Light coastery_cafejrraaeee cafe] for zoastery is roaating_cos wttpooootttttttttt_eeeee Light, keya1)) >e 1ight coffee needed for [aafe]])
dark coffee shipped
roasteries coffee needed that
cafe dark coffee needed for cafe:
model jddd (uuns dark roaxtery comtery,,,, oxfe] for roastary in roasting dark. keys(11 keysk. dusk coffen saeded
_adeeee_____eeeeeddee for cafe]
# objoetive: Minimize the cotal cost
total model addVar sudel.
"ate-"totaaeece cost")
con(ehipping cost_ fron
fron_rroelilleer supplier rossterylnmppiee roastery) ___nppleer_ronttrryeenng
roastery)
supplies, zeastery in shipping_coss fron, supplier to roastery keys())
controasting cost hos__igh(rrosstrr)
Hight coattery/rosseery for roastory is reasting cost (ight:keys() for roastasy in romstng coss kayaa)
suntrousting coss
coss_dark dack) constey]
sun(shipping in shipping cowt__frnn fron romstryy to pa.aafe.
caciippin__oos___o. coss_teon, www.tooootterrreeee to to cste]
rosster,,,, cafe] for roastery, oate
suntshigoing shipping fwwe_rrooetteeeeeeee rta_rrrratttyyeeee tareaayttt__eeeeeeeeee tttptttteeeeeeeeeeeeeeeeeeeeeeee key
reastery cal_cate//oo rrastory,,
_eddr___ratttss pww.arrry_ofeeerrsttses. rooorrssser cate] for roastery,
model setde cost, cost, GRB. HINIMTEE)
# imiin model
sodw). optiniss!!
Figure 25: Output of Code Generator for Coffee after switching the solver to MILP, using GPT-4o.
56

---

Published as a conference paper at ICLR 2025
A.10.4 PARAPHRASED PROMPTS AND PERFORMANCE
To prove the flexibility of our prompt, also paraphrase our NL task description and re-test the framework with paraphrased description. The paraphrasing is performed by LLMs.
One example paraphrased description is:
In this blocksworld problem, robet can perform four actions: pickup putdown, stack, and unstack. The environment consists of blocks that can be stacked, a single-block capacity arm, and a table. Pickup: The arm can lift a block it it's clear, on the table, and the arm is empty. This results in the arm holding the block, which is no longer on the table or clear.
Putdown: If the arm is holding a bleck, it can place it on the table.
This leaves the arm empty and the block on the table and clear.
Stack: The arm can place block it's holding onto ancther clear block. This empties the arm, makes the top block clear and the bottom block, while the botton block becomes unclear.
Unstack: If a clear block is on ancther block and the arm is empty, it
can lift the top block. This results in the arm holding the top block (no longer clear or on the botton block), while the bottom block becomes clear.
With LLM- paraphrased random task descriptions, we test on 50 queries in Blockworld with Claude 3.5 Sonnet and shows LLMFP is still able to correctly generate 46/50 plans, reaching high optimal rate of 92%, significantly outperforming baselines. This shows our framework is not sensitive to specific wordings of the task description, as long as they have adequate information. We can add more paraphrasing examples to show the robustness of LLMFP to different user inputs, if the reviewer finds it helpful to show the generalizability of LLMFP.
Table 15: Optimal rate (%) comparison of LLMFP with baselines with paraphrased prompts on Blocksworld with Claude 3.5 Sonnet
DirectorT40
CoTaPT-40
CodecPt-4o
SMTe-SMTo
LLMFP GPT-40
32.0
0.0
92.0
57


Reviews:
[{"review_id": "mhaMSvVr6p", "reviewer": "Reviewer_qLBs", "summary": "This paper addresses the problem of solving planning problems that are given in natural language. The proposed algorithm they propose – LLMFP -  is a workflow of multiple LLMs, including an LLM to extract variables and constraints from the text, an LLM to formulate the extracted variables and constraints as an SMT problem in a specific format,  an LLM to convert this format to code that can be run by an SMT solver, and an LLM to verify and correct mistakes by the other LLMs. This LLM workflow is evaluated against the other LLM-based methods to solve planning problems, including one that is similar to the LLMFP but creates PDDL instead of an SMT problem. The authors also examine how results can be better by adding some task-specific expertise. The results over a set of benchmark problems show that LLMFP is, in general, much better than the baselines.", "questions": "1.  As the authors noted, LLMs have been used to translate natural language to planning problems. Similarly, the mapping from planning to SMT is well known in the planning literature. So, is the novelty is limited to combining the two ideas together??\n2. In page 3, just above the first paragraph, you seem to say that encoding to PDDL requires more human effort than encoding to SMT. Can you elaborate why?\n3. How do you encode the length of the plan? When compiling planning to SAT or SMT, this is an issue because the solver (SAT/SMT) requires to set an upper bound while in PDDL it does not have too.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Strength\n- The paper is in general clear (even if it is sometimes hand-wavey)\n- The problem is interesting and the related work seems to cover all bases\n- The results are impressive, and much better than the baselines. \n- The proposed workflow makes sense and works well.", "weaknesses": "- I’m not sure if the novelty of the proposed work over the PDDL-based approach is sufficiently novel for a top conference. \n- The appendix is huge (~40 pages!). This seems to me not reasonable, as the main paper should be self contained. \n- The presentation is too much hand-wavy. It would be great to try to capture more of it in a more formal manner", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JLUp1XZhHG", "reviewer": "Reviewer_8M6X", "summary": "LLMFP is proposed which leverages LLMs to tackle complex planning problems by formulating them as optimization tasks.", "questions": "1. What is the definition of a planning problem in this paper?\n2. Why are the baselines only LLMs when the proposed approach is a framework/architecture? LLM-PFC [1] approaches planning problems similarly and there are other baselines to consider like Plan-SOFAI [2].\n3. LLMs when used with API's are found to hallucinate new API functions or overuse a specific API call. Is such behavior observed here?\n4. When it is a planning problem, why not directly use a symbolic planner and why is this architecture beneficial?", "limitations": "", "rating": 1, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "LLMFP's ability to handle a wide variety of planning problems without task-specific examples is a significant strength.", "weaknesses": "1. The baselines for comparison do not seem to be a fair comparison to LLMFP. See questions.\n2. The related work does not cover relevant set of papers that should have been used a baseline to compare this work. Mentioning a few of them below - \n[1] Webb, T., Mondal, S. S., Wang, C., Krabach, B., & Momennejad, I. (2023). A Prefrontal Cortex-inspired Architecture for Planning in Large Language Models. arXiv preprint arXiv:2310.00194.\n[2] Fabiano, F., Pallagani, V., Ganapini, M. B., Horesh, L., Loreggia, A., Murugesan, K., ... & Srivastava, B. (2023, December). Plan-SOFAI: A Neuro-Symbolic Planning Architecture. In Neuro-Symbolic Learning and Reasoning in the era of Large Language Models.\n[3] Katz, M., Kokel, H., Srinivas, K., & Sohrabi, S. (2024). Thought of Search: Planning with Language Models Through The Lens of Efficiency. In The First Workshop on System-2 Reasoning at Scale, NeurIPS'24.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "d4lqmi6qrr", "reviewer": "Reviewer_dysu", "summary": "The paper presents a framework that pairs LLMs with optimization tools to solve planning tasks without using task-specific knowledge. The authors define consecutive stages of reasoning that, generally speaking, consist of understanding, coding, and refining. For each stage, they discuss the prompting, formatting, and other relevant decisions. Through experimental validation, they show that LLMFP outperforms baselines in 9 domains.", "questions": "1. What are the most common failure modes of the baselines?\n\n2. Are the baselines prompted vanilla models plus the components described in lines 410-416, or do they also include other components, e.g. formatter?\n\n3. What are the success rates of the tested methods? Do they all achieve 100% and the question is only whether the solution is optimal, or do some methods fail to solve some instances at all?\n\n4. What's the wall time of LLMFP compared to the baselines?\n\n5. Are the methods explicitly instructed to provide optimal solutions?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I like the general idea and the presented approach. One could argue that it is simply a combination of prompt engineering and the incorporation of external tools. However, showing an effective way of doing this can be a significant contribution.\n\nThe baselines and ablations are well-chosen for evaluating the performance of LLMFP.\n\nThe paper is written very clearly, making it easy to read. The figures are well-chosen (particularly Figure 1), they are helpful in understanding the pipeline. I like the section structure and the focus on key takeaways when discussing experimental results. Most of my questions that arose while reading the text were addressed in later sections.", "weaknesses": "The goal stated in the introduction is \"Can we build a universal LLM-based planning system that can solve complex planning problems without task-specific efforts?\". However, my main concern is whether the tasks used for experiments are indeed complex planning problems. Specifically, the 5 multi-constraint problems resemble simply optimization problems rather than planning problems. Hence it's quite clear that adding an external optimizer to LLM would be much better than just using LLM. On the other hand, the multi-step problems seem to be rather simple and the main difficulty is to understand what we have to do rather than finding a good solution. Hence, I suggest adding at least one multi-step domain with high underlying complexity (e.g. Sokoban). If I missed something and some of your environments are actually NP-hard (or hard in any other reasonable sense), it should be remarked in the paper.\n\nSince the method you propose is clearly subject to a tradeoff between performance and computation time, there should be a discussion of that. What's the wall time of LLMFP compared to the baselines? What's the cost of using SMT compared to querying LLM?\n\nThe description of baselines should be extended a bit. Are they prompted vanilla models plus the components described in lines 410-416, or do they also include other components, e.g. formatter? Also, the Code variant uses pure Python, but for a completely fair comparison you should also add variant which is forced to use SMT like LLMFP does. After reading the prompts used, it's also not clear to me whether they are explicitly instructed to provide optimal solutions, which is captured by the metrics. Also, I suggest discussing the failure modes of the baselines (in the Appendix).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FqZ0wHoWcR", "reviewer": "Reviewer_FxuD", "summary": "This paper provides the LLM prompting-based framework for planning tasks. The main contribution lies in its use of prompt and pipeline templates, which can be used across various planning tasks. Here, planning problems from various domains were considered, and planning is treated as an optimization problem. In summary, LLMs are used as an optimizer. They used the formal planner to achieve the goal of planning as already shown in the previous works that LLM still lacks the coherent reasoning needed for planning. The main contribution is an end framework that deploys a zero-shot learning approach for both single and multi-stage planning tasks. Additionally, the author claims that their framework can handle self-critique to assess the problem in planning code to change and achieve the goal. Effectiveness of framework components is supported via the ablations.", "questions": "1. How LLMFP handles generalization across different planning tasks. Please correct me, but it seems that we need a very Elaborate prompt with a high level of detail for each task. \n2. In section 3.4 (code generator), readers can Benefit from prior work such as \"CAPE: Corrective Actions from Precondition Errors using Large Language Models\" and \"CoT-TL: Temporal Knowledge Representation of Natural Language Planning Task for Autonomous Agents using Chain-Of-Thought.\" Or is LLMFP doing differently compared to the above works; if yes, explain or not, and make sure to provide proper background.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. LLMFP introduces a new perspective on using LLMs for formal optimization-based planning, a method that significantly expands the generalizability of planning tasks.\n2. Experimental results are solid, with clear evidence of performance gains across diverse tasks and models. The ablation studies reinforce the utility of the framework components, which I really liked.\n3. The ability to solve multi-step, multi-constraint problems without task-specific examples or extensive prior efforts is a major step forward in the area of LLM-based planning.", "weaknesses": "1. Complexity in FORMULATOR: Some parts, particularly the JSON representation and the code generation steps, could be simplified. While important, the handling of different variable types and constraints might be a bit dense for readers unfamiliar with optimization theory.\n2. Regrading Multi-Step Planning Problem: The predicate, object, and update structure are not clear in multi-step planning. Also image shown for this is not utilized in conveying the idea. Overall, Figure. 2 examples are not clear and make things confusing. \n3. The author claims that their framework is a \"general approach, which does not require task-specific examples or task-specific efforts\"; however, in the paper, this statement is not supported in terms of explanations and prompt structure.\n4. Some theoretical insights regarding performance would make this work more strong, right now its presented more like a experimental results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)