A Systematic Evaluation of the Planning and Scheduling Abilities of the Reasoning Model o1
Authors: Karthik Valmeekam
Kaya Stechly
Atharva Gundawar
Subbarao Kambhampati
Venue: Accepted by TMLR ()

Abstract: OpenAI claims that their recent o1 (Strawberry) model has been specifically constructed and trained to escape the normal limitations of autoregressive Large Language Models (LLMs)–making it a new kind of model: a Large Reasoning Model (LRM)–and be generally capable of tackling procedural reasoning tasks. We present the first comprehensive evaluation of these models on the fundamental tasks of planning and scheduling. Previous research attempted to use LLMs’ expressive generation capabilities to solve these problems, but met with only limited success. We fill in the gaps in this literature by testing a larger suite of state-of-the-art LLMs on a set of large benchmarks, and then use this as a baseline to evaluate o1-preview and o1-mini. We see that while they can offer significant accuracy improvements over LLMs, this single metric is misleading and incomplete, as LRM queries demand large and unpredictable costs and take significant amounts of time to complete. We provide a case study demonstrating that, at those same price points, other methods of inference time scaling can do just as well. We also show that, contrary to OpenAI’s injunctions, o1’s performance can be improved further by embedding it in compound systems that separately, but complementarily, scale inference time further. Finally, while the paper is focused on o1, we provide similar evaluations of a more recent (and open-weight) LRM -- DeepSeek R1.

Published in Transactions on Machine Learning Research (April/2025)
A Systematic Evaluation of the Planning and Scheduling Abilities of the Reasoning Model ol
Karthik Valmeekam"
kwalmeek@asu..ek edu
School of Computing and AI Arizona State University
Kaya Stechly"
khstechluasuu. edu
School of Computing and AI Arizona State University
Atharva Gundawar
agundowo@asu.cmm edu
School of Computing and A1 Arizona State University
Subbarao Kambhampati
roo edu
School of Computing and AI Arizona State University
Reviewed on OpenReview: https: // openreview. net/ forum? id= FIKKBpOFFR
Abstract
OpenAl claims that their recent ol (Strawberry) model has been specifically constructed and trained to escape the normal limitations of autoregressive Large Language Models new kind model: a Large Reasoning Model (LRM) and generally capable of lackling procedural reasoning tasks. We present the first comprehensive evaluation of these models on the fundamental tasks of planning and scheduling. Previous research attempted to use LLMs exprossive generation capabilitics to solve these problems, but met with only limited success. We fill in the gaps in this literature by testing a larger suite of state-of-the-art LLMs on a set of large benchmarks, and then use this as a baseline to evaluate ol-preview and ol-mini. We see that while they can offer significant accuracy improvements over LLMs, this single metric misleading and incomplete, as LRM queries demand large and unpredictable costs and take significant amounts of time to complete. We provide case study demonstrating that, those same price points, other methods of inference time scaling do just well. We also show that, contrary to OpenAl's injunctions, ol's performance can be improved further by embedding it in compound systems that separately, but complementarily, scale inference time further. Finally, while the paper is focused on ol, we provide similar evaluations of a more recent (and oeen-weight) LRM DeepSeek R1.
(LLMs) making
1 Introduction
Large Language Models (LLMs) have captured the imagination of the AI research community by demonstrating linguistic behaviors that no one expected text completion systems to possess. Their seemiing versatility
initially led to claims that they are equally capable of deliberative behaviors such as planning and scheduling, but they have been called into question by a steady stream more systematic investigations (Valmeekam
et al. 2023; Silver al. 2022 Xie al. Zheng et. al. 2024 Mirzadeh et al. 2024). In some sense, this
"Equal Contribution

---

Published in Transactions on Machine Learning Research (April/2025)
shouldn't be too surprising the autoregressive 'next token prediction in constant time' nature of LLMs, which makes them so popular, leaves little scope for runtime deliberation.
There have been several attempts to elicit reasoning from LLMs while keeping their essential auto-regressive nature intact. Early attempts- such as Chain of Thought and ane-tuning were not particularly successful in robustly teasing planning behaviors out of L.MSs (Stechly et al.] 2024a] Valmeekam et al. 2024; Dziri let al. 2024). Compound systems which combine multiple LLM calls with verifiers, solvers, or other symbolic components have shown more promise Yao et al., 2023b Kambhampati et al.) 2024) and demonstrate the potential of scaling the amount of LLM compute used at inference time to the problem at hand.
Against this background comes the recent release of OpenAl's ol (Strawberry) (OpenAl, 2024a which OpenAl claims to have trained specifically for tackling complex reasoning lasks (OpenAl 2024a bj. Unlike vanilla LLMs whish can roughly be viewed as approximate retrievers ol seems to have been trained to be an approximate reasoner, capable of scaling the amount of compute it uses depending on the query[ Following OpenA P we draw a distinetion between previous Large Language Models (LLMs) and ol, Large Reasoning Model (or LRM), as its new (unknown) architecture, operation, and capabilities all seem to be fundamentally different from those of vanilla LLMs during both the pre-training phase and inference time.
Properly evaluating this new kind of model will require new tools and evaluation metrics. In this paper, we comprehensively evaluate the performance of ol models (specifically ol-preview and ol-mini) on established planning and scheduling benchmarks. As a step towards better metries and a clearer understanding of the utility and capabilities of LRMs, we build extensions of these benchmarks. We analyze not just the raw accuracy but the cost-efficiency trade-offs between these models and baseline- compound system that utilizes off-the-shelf LLM and sound verifier (as in the LLM-modulo framework (Kambhampati et al. 2024)) in order to scale inference-time compute. Our paper covers three major areas:
Extending Planning and Scheduling Benchmarks: We start with existing benchmarks to support baseline comparisons with autoregressive LLMs. For planning, we start with PlanBench (Valmeekam et al. 2024), which consists of both specific test sets and a suite of tools intended for evaluating language models on arbitrary planning domains that were part of International Planning Competitions (IPC) 1998). To evaluate scheduling capabilities, we test TravelPlanner (Xie et al.),, on the the domains from Natural Plan (Zheng
ct al. 2024), and on graph coloring problems (Stechly et al., 2024b). We also built extensions of these domains to more closely analyze the generalization capabilities of ol-preview and ol-mini as problem difficulty increases.
Analyzing Inference Cost vs. Accuracy Tradeoffs: Current benchmarks designed to test the planning and scheduling capabilitics of autoregrossive LLMs do not explicitly take efficiency into consideration. As the time taken by a vanilla. LLM to produce some output is only dependent the length of that output, rather than the semantic content or difficulty of the instance, this has not previously mattered. However, when a system adaptively varies time and dollar cost in response to the input, it is crucial to check if the accuracy gains it makes are worth the cost and whether they can be replicated with other techniques when given equivalent resources. In general, we argue that new approaches to measuring LRM reasoning capabilities must take into account efficiency, cost, and guarantees. While OpenAl's announceeent does admit that the inference costs for ol will be higher than the prior autoregrossive models, it doesn't 't give clear picture of how high the costs can be, nor does it mention the remarkable and unpredictable variance in that cost. As third-party independent evaluators, we quantify the (rather steep) inference cost of these models in dollar terms, and more clearly quantify the trade-offs between LRMs and other ways to scale inference time compute. We find that, in some cases, our bascline (an LLd-Mooull (Kambhampati et al.) 2024) approach with a much cheaper base LLM) can beat ol performance when comparably scaled up.
We speculate that the complete system learns to improve its ability to make appropriate Mlliinoof-Thuusee (CoT) moves useful for reasoning in pretraining RL step with synthetic data, and does inference time prompt-specific rollouts; see Appendix B for the details. In other words, it may be an Rl-trained system in the same vein as AlphaGo, but where the 'moves' being generated and evaluated
Chains of Thought. Note that this is very different of RL from that in RLHF, which can be seen as a fine tuning stage for an LLM that keeps the inference step unchanged.
PPer the blogpost announcing the model: "A new serics of reasoning models [...] for complex reasoning tasks this significant advancement and represents new level of AI capability. Given this, are resetting the counter back to and naming this series OpenAT ol." (OpenAi 2024c
2

---

Published in Transactions on Machine Learning Research (April/2025)
Complementing ol with Other Inference-time Scaling Techniques: OpenAI released ol ol-preview with an injunction to keep prompts "simple and direct., claiming that the system worked best when fed questions directly without external augmentation (OpenAl, 2024d). We test whether this is accurate. In particular, we investigate whether inference time scaling techniques that preceded ol can be used in tandom with the idea being lo scale further in different ways at inference time. Specifically, we adapt the generate-test style LLM-Modulo framework (Kambhampati et al. 2024) to LRMs. Our results show that such an LRM-Modulo approach can not only provide correctness guarantees, but also further improve LRM performance. In essence, LRMs can replace LLMs as significantly better-but still fallible- generators in the L.M-Modulo framework. This also shows that there is potential in combining multiple inference-time scaling approaches at once when pursuing higher performance.
Evaluations on DeepSeek R.I: The bulk of this study was done in Fall 2024, right after the release of the "Strawberry" ol models. Since then, there have been other large reasoning models (see (Kambhampati ct al. 2025 for an overview). Of these, DeepSeek's R1 is of particular interest as it is an open weight model with an accompanying full technical report about its internal operations (Guo et al. 2025). We have recently
completed our PlanBench evaluations on RI too, and share those results in Section 6
2 Domains
2.1
Planning
We focus on classical planning problems, or STRIPS planning problems, which are a formalism for automatod planning in discrete, deterministic spaces. To define a planning problem, we specify an initial state, a domain, and a goal. The domain contains all relevant information about the types of objects that may exist and the allowable actions from any given state, specified by defining the preconditions and effects of each named action. Problems and domains are represented in the flexible PDDL (Planning Domain and Definition Language) framework (MeDermott et al. 1998). Solutions to PDDL problems are correct plans sequences of actions executable from the initial state which arrive at a goal-satisfying final state. These are problems in which the the planner already knows all relevant facts about the world and the possible actions; thus only deliberation is required.
The STRIPS planning evaluation pipeline first introduced in Valmeekam et al. 2024) provides extensible suite of domains and tools for evaluating language models, a collection of static test sets across these domains, and ways of easily generating additional problems. We draw on these static test sets to provide our initial ol planning results on Blocksworld and Logisties commonsense domains from the International Planning Competition IPC.
on Mystery Blocksworld-ar obfuscated version of the same.
However, these domains are insufficient for a thorough evaluation of the limitations of ol. We construct new obfuscations of this benchmark. This can be seen as private test set, generated post-training as light test of pre-training contamination. Randomized Mystery Blocksworld consists of the same problems as the original Blocksworkd test set,, but with action and predicate names replaced by randomly generated strings. Randomized Mystery Logistics is constructed in the same way. We also extend the Blocksworld test set by adding 110 harder instances. Problems in this set range from 6 to 20 blocks in length and require 20 to 40 step optimal plans.
While STRIPS planning in principle PSPACE-complete, the problems presented above are actually of a lower computational complexity. Plan existence for both Blocksworld and Logistics is polynomial (Holfmann) et al. 2006). We add Sokoban, non-ergodic domain which is known to be PSPACE-complete (Culberson, 1997.. In this domain, an agent moves around a constrained grid, pushing boxes by running into them, where the goal is to move every box to one of a set of final locations. As it can be represented in PDDL form, it is amenable to classical planning techniques.
Using a generator from the 2008 International Planning Competition IPC 1998). we generate 55 Sokoban instances with grid sizes ranging from 4 x 4 10 10, 1 to 4 boxes, and 4 walls. We then create corresponding prompts in PDDL and test ol models them.
3

---

Published in Transactions on Machine Learning Research (April/2025)
Table 1: Performance on 600 instances from the Blocksworld and Mystery Blocksworld domains across large language models from different familied using both zero-shot and one-shot prompts. Best-in-class accuracies are bolded.
Claude Models
OpenAI GPT-4 Models
LLaMA Models
Gemini Models
Domain Shots
3.5
3
4o
40
4
4
3.1
3
1.5
1
Sonnet
Opus
mini
Turbo
405B
70B
Pro
Pro
Blocks
One Shot
344/600
289/600
170/600
49/600
206/600
138/60M
284/600
76/600
101/600
68/600
world
(57.6%)
(48.1%)
(28.3%)
(8.1%)
(34.3%)
(23%)
(47.3%)
(12.6%)
(16.8%)
(11.3%)
Zero
329/600
356/600
213/600
53/600
210/600
241/600
376/600
205/600
143/600
3/600
Shot
(54.8%)
(59.3%)
(35.5%)
(8.8%)
(34.6%)
(40.1%)
(62.6%)
(34.16%)
(23.8%)
(0.5%)
Mystery
One
19/600
8/600
5/600
0/600
26/600
5/600
21/600
15/600
2/500
Blocks
Shot
(3.1%)
(1.3%)
(0.83%)
(0%)
(4.3%)
(0.83%)
(3.5%)
(2.5%)
(0.4%)
world
Zero
0/600
0/600
0/600
0/600
1/600
1/600
5/600
0/600
0/500
Shot
(0%)
(0%)
(0%)
(0%)
(0.16%)
(0.16%)
(0.8%)
(0%)
(0%)
2.2 Scheduling
We also evaluate ol schoduling problems. Zheng et al. (2024) Natural Plan benchmark consists of three domains: trip planning, calendar scheduling, and meeting planming. Xie et al.ss Travel Planning benchmark consists of large dataset of travel information (flights, accommodations, restaurants, etc.) with prompts that ask the model to create a three to seven day itinerary based on natural language instructions. Note that, despite the word "planning" in these titles, these benchmarks are better characterized as testing scheduling abilities (Ghallab et al.] 2016), which are generally much easier than planning and, at worst, only NP-Hard (Carlier 1982). They are also equivalent to constraint satisfaction problems, such as graph coloring. Stechly et al. (2024b) translate graph coloring into natural language prompts and evaluate GPT-4's accuracy them. We use their test set to evaluate ol.
We also extend this set to more difficult instances. Using the Erdös-Rény: procedure with p = 0.4, = 20, we generated 50 more graphs, and did not rejection sample for planarity. As in that paper, we precalculate the chromatic number and provide it in the prompt, asking that the model produce a coloring that uses exactly that number of colors. These harder graphs have vertices and around 60 odges each.
3 Preliminaries: Auto-regressive LLM's Still Can't Plan
Previous work has already shown that autoregressive LLMs have considerable difficulty on planning (Valmeekam et al. 2024, Silver al. (022) and scheduling (Xie al. Zheng et al. 2024) benchmarks. However, each piece of previous work evaluated previous, no longer relevant models. To provide a clean baseline, we fill in the missing results.
different subset of available state-of-the-art LLMs or
Table D shows the results of running current and previous generation LLMs on static test set of 600 three to five block Blocksworld problems, as well as on a sot of 600 semantically identical but syntactically obfuscated instances which Valmeekam et al. (2024) calls Mystery Blocksworld. Across the models tested, the best performance on regular Blocksworld achieved by LLaMA 3.1 405B with 62.6% accuracy. Despite containing identical underlying problems, Mystery Blocksworid performance lags far behind no LLM achieves even 5% version of the domain does not clearly predict performance the
test set-and performance other.
Vanilla language models typically perform better when tested on natural language versions of prompts rather than PDDL (Valmeekam et al. 2024). even though natural language can introduce uncertainty due to polysemanticity and syntactic ambiguity. To make our comparisons "fair" for the models being tested, the
We do not provide Mystery Blocksworld data for Gemini 1.5 Pro only because we haven't been able to generate it. The model refuses produce any output, instead claiming that responding these queries would be harmful. We include this output in Appendix
4

---

Published in Transactions on Machine Learning Research (April/2025)
Optimal plan Length Correct Predictions on Mystery Blocksworid Plan Generation Zero Shot
Optimal pian Length vs Correct Predictions for all Models Mystery Blocksworid Plan Generation One Shot
100
80
Model
Model
Fast Downword
UAMA 2.1 4050
LLAMA 4058
Fost Downward
satisticng
80
GP1-4o Mini
sostalfcing
GPT-40
40
GPT-4o
%
GP1-4
GPT-4
Cloude 3.5 Sonnet
Cloude Opus
Cloude 3.5 Sonnet
40
Medel Group
Cloude
Model Group
Models (URMs)
Fost Downword
LLMS
LiMs
8
14
10
12
Optimal plan Length
Optimal pian Langth
Figure 1: These examples are Mystery Blocksworld. Fast. Downward, domain-inndeeendenn plan- ner (Helmert) 2006) solves all given instances pear-instantly with guaranteed perfect accuracy. LLMs struggle even the smallest instances. The two LRMs we tested, ol-preview and ol-mini, are surprisingly effective, but this performance still not robust, and degrades quickly with length.
results we report are the higher percent accuracy natural language prompting numbers, and can be considered an upper bound on LLM performance on PDDI.speccfied problems.
The lackluster performance of LLMs on even the easiest static test set leads to continue to believe that planning cannot be generally and robustly solved by approximate retrieval alone. We relegate additional discussion of the particulars of LLM performance to appendix A
There have also been works that have at tempted to improve the planning performance of LLMs through methods like prompt engineering (Wei et al. 2022 Yao et al.) 2023b), self reflection (Huang et al. 2022
Shinn et al., 2024), repeated sampling combined with self-reflection (Hao et al. 2023 Yao et al.) 2023a Hu
et al.) among others. However, systematic evaluations (Stechly al. 2024b Dziri al. 2024
Prabhakar
et al. 2024 Verma al.) have shown that these methods brittle and fail enable LLMss to perform procedural reasoning generally and effectively.
Rather than relying on LLMs to generate solutions, some works have integrated LLMs with external planners.
In these settings, LLMs handle tasks like translation (Tiu et al. 2023 Dagan et al. 2023 Gestrin et al.) or knowledge acquisition (Guan et al.) 2023 Caglar et al., 2024
Tang et al. 2024 while the solvers uses the generated specification to provide solutions. Given that L.Mss provide completions without guarantees for almost any problem presented to them, works have explored using LLMs to not. only generate plans or models but also search components that can be used part of planning (Katz et al.
2024 or domain specific plan
programs that would produce plans for tasks within the domain
Silver al.
2024
4 From Approximate Retrieval to Approximate Reasoning
Exact details about ol are currently sparse, and so we can only speculate about its exact mechanisms. Our best guess is that there are two major differences between this model and vanilla LLMs: an additional reinforcement learning pro-training phase (perhaps learn the q-values (Watkins) 1989) of different CoTs from massive amounts of synthetic data) and new adaptively scaling inference procedure (maybe it further refines learned values by something like rollout before selecting particular CoT; see Appendix B).
5

---

Published in Transactions on Machine Learning Research (April/2025)
Table 2: Performance and average time taken on 600 instances from the Blocksworld, Mystery Blocksworld and Randomized Mystery Blocksworld domains and 200 instances from the Logistics and Randomized Logistics domains by OpenAl's ol family of large reasoning models and satisficing Fast Downward (F.D.)
Average Time
Total Instances
Instances correct
Domain
Shots
Taken (in secs)
ol
ol mini
o1 preview
ol mini
F.D.
F.D.
preview
600
Blocksworld
Zero Shot
97.8%
56.6%
100%
40.43
10.84
0.12
One Shot
41.6%
100%
82.03
0.12
600
Mystery Blocksworld
Zero Shot
52.8%
19.1%
100%
83.37
35.54
0.12
Randomized
600
Zero Shot
37 .7%%
3.5%
100%
111.11
55.40
0.12
Mystery Blocksworld
200
Logistics
Zero Shot
94%
100%
84.07
0.13
Randomized
200
Zero Shol
52%
100%
167.41
0.13
Mystery Logistics
55
Sokoban
Zero Shot
12.73%
10.9%
100%
147.98
32.31
89.65
Regardless, what is clear from the detail available is that this model is fundamentally different in nature from previons LLMs. Rather than doing merely approximate retrieval- exxrival-exeeuttngg some kind of miidll-generaaizing paltern-matching over mixtures of pre-training data- it seems to be doing something closer to approximate reasoning, which necessarily requires the ability to scale inference time. We systematically evaluate the performance difference this provides on planning and scheduling tasks.
4.1 Planning
Evaluating LRMs on PlanBench: We test ol-preview and ol-mini on the original 600-instance PlanBench
test set H The full results can be seen in Table 2 These 600 Blocksworld instances range from 3 to 5 blocks, and require plans of between 2 to 16 steps to solve. Far surpassing any LLM, ol correctly answers 97.8% of these instances. On Mystery Blocksworld, the model docs not maintain this level of performance, but it does lar surpass previous models (which barely managed a few percent), answering 52.8% of the questions correctly. To test whether the exact obfuscation might be compromised because of data contamination, we also generated
obfuscation using completely random strings, and presented these problems in new, semantically equivalent prompt format with fully specified and unanbiguuus PDDL descriptions of both the domain and problem. This is presented in the table as Randomized Mystery Blocksworld. Fxact prompts can be seen in the appendix. While performance did dip further, 37.3% of instances are answered correctly, sharply contrasting the flat zerocs of previous models. The same pattern can be seen when evaluating Logistics and a freshly generated obfuscation of that domain. Despite the higher branching factor of the domain, ol- ol-preview solves 94% of all 200 problems tested and achieves 52% on the obfuscated variant..
Increasing Problem Size: On the harder Blocksworld set constructed for this paper, we see performance quickly degrade from the 97.8% reported earlier to 23.63%. Most of this accuracy comes from correctly solving problems which require fewer than 28 steps. While these models are overall impressive, this shows that their performance is still far from robust, especially in comparison to classical solvers as we shall see. These results are collated together with a representative sampling of smaller instances in Figures and showing how performance begins to fall on problems requiring plans of more than 10 steps.
Whiil for previous models, the model itself enforced the desired plan format, some modifications had be made to accurately test ol's 's abilitics. In its current form, ol-preview does always conform to explicit formatting restrictions. In order extract the generated plans, we used GPT-4o-mini translate them into PDDL, and wrote small Python parser strip any remaining extraneous symbols before evaluating each proposed plan.

---

Published in Transactions on Machine Learning Research (April/2025)
Optimal plan length vs Satisficing plans by O1 Preview on Blocksworld Plan Generation Zero Shot
100
satisficing Avg Satisficing
80
satisficing
60
40
20
18
22
26
32
36
40
Optimal plan length
Figure 2: Extending even the (regular, not obfuscated) Blocksworld dataset to problems requiring greater numbers of steps worsens the performance of ol-preview. When tested on 110 instances which cach require least 20 steps to solve, only manages 23.63%.
A More Difficult Domain: Sokoban We also test on the 55 Sokoban instances that we generated part of our extension of Planbench. ol-preview answers 12.7% of these instances correctly, while ol-mini is not far behind with 10.9%. However, the only problems on which these models succeed are those in which there is only one box on the grid. For comparison, when evaluated on these same instances, Llama3. 1-405B, despite doing the best of all LLMs on the Blocksword scts, does not answer a single question correctly.
ol's Creative Justifications While main focus has been providing quantitative evaluation of ol-mini and ol-preview's performance PlanBench, we have also noticed idiosyncrasy that is worth commenting on. When the model gives an incorrect answer, it also sometimes provides creative, but nonsensical, justification for its decision. It is almost as if ol has gone from hallucinating to gaslighting! In one casc, it docided that a problem was solved because a goal condition, while not present in the final state, had been true at some point during the execution, and thus should continue to count. another, it declared that (a, was true because, as it explained in a brief parenthetical, a was on which was on c, and thus was somewhere above c, which should count being "on top' of it. To avoid this, changed prompts from natural language to PDDL in order to make it extremely clear that divergences from exact deninitions are disallowed.
Table 3: Error analysis of ol ol-preview
Total Instances
Valid plans
Inexecutable plans
Non goal-reaching plans
Domain
600
Blocksworld
97.83%
2%
0.17%
600
Mystery Blocksworld
52.83%
33.67%
13.50%
600
Randomized Mystery Blocksworld
37.33%
41.50%
21.17%
200
Logistics
94%
6%
0%
200
Randomized Mystery Logistics
52%
15%
33%
55
Sokoban
12.73%
67.27%
20%

---

Published in Transactions on Machine Learning Research (April/2025)
Table Error analysis of ol-mini
Total Instances
Valid plans
Inexecutable plans
Non goal-reaching plans
Domain
600
Blocksworld
56.67%
39.50%
3.83%
600
Mystery Blocksworld
19.17%
57.67%
23.33%
600
Randomized Mystery Blocksworld
3.50%
19%
77.50%
55
Sokoban
10.9%
74.54%
14.5%
Table 5: Cost per 100 instances (in USD). LRMs are significantly more expensive than LLMs.
Large Reasoning Models
Large Language Models
Claude 3.5
Claude 3 (Opus)
GPT 4o
Gemini
Gemini
o1 preview
o1
GPT 4o
GPT 4
(Sonnet)
mini
1.5 Pro
1 Pro
mini
$0.44
$1.70
$0.65
$0.02
$1.80
$0.33
$0.03
$42.12
$3.69
Qualitative Analysis Of Errors For closer look at the kind of errors the models make, we also examined what kinds of invalid plans ol-preview and ol-mini generate (see Tables 3 and 4 The invalidity of a plan could be either due to an action having unsatisfied preconditions or the plan not reaching the desired goal state after execution. As shown in the tables, the majority of invalid plans generated by ol-preview and ol-mini are invalid bocause they are incxccutable.
4.2 Scheduling
We also evaluate ol-mini and ol-preview set of scheduling problems that have previously been used in testing LLM capabilities.
Graph Coloring: We evaluated ol-mini the set of 220 problems included in the codebase associated with Stechly et al. (2024b)'s paper, and found that it solved 96%, surpassing the 16% reported by that paper for GPT-4. On the extended graph coloring test set we created for this paper, consisting of 50 graphs with 20 vertices each, ol-mini solves 50%, while ol-preview solves 64%.
Travel Planning: We test ol models on the 180 instance validation set of the sole-planning mode. In this mode, the model is provided upfront with all relevant information required to construct the requested itincrary. The previous stato-of-the-art direct prompting was 4.4%, achieved by GPT-4-turbo. ol-previcw surpasses this solving 10% of all instances. ol-mini does not beat GPT-4-turbo, remaining at 1.67%.
Natural Plan: This benchmark consists of three domains: calendar schoduling, trip planning, and meeting planning. On calendar scheduling, ol-mini solves 94% of all instances correctly. Given cost constraints, we did not test ol-preview on this domain, as ol-mini tends be a lower bound on its performance. Neither model performs well on trip plamning-ol-previerr reaches 4% and ol-mini only 1%, both lower than the previous sate--of-the-art set by Gemini 1.5 Pro. When provided with prompts from the meeting planning domain, both ol models refuse to respond and flag the input as potential terms violation (see Appendix G
4.3 Accuracy/Cost Tradeoffs and Guarantees
With LRMs showing better performance on planning and scheduling problems, our evaluations must explicitly take into account the trade-offs that come from choosing general models over established deep and narrow systems. While in most of our domains ol ol-preview provides higher accuracy than L.Mss, it still fails to
8

---

Published in Transactions on Machine Learning Research (April/2025)
provide any correctness guarantees, and it is unclear that it is at all cost-effective. Unlike previous models, whose APIs only charge based on the number of input tokens and the number of output tokens (usually at a rate that is five times higher for the latter), ol's 's price-per-call includes a surcharge based on the number of 'reasoning tokens'-tokens generated as part of inference and not revealed to the user-it used, which are charged at significantly higher rate. Currently, end users have no control over the number of these lokens generated, a number which is expanded or limited by the model in its own opaque way.
Without exposing the ability to scale inference time to particular specifications, influence the internal 'thinking' process in task-specifie ways, ensure that intermediate steps evaluated by trusted sound verifiers, the ol models are a goars--grained choice in the space of cost, inference time, guarantees, and performance trade-offs. They aren't, however, the only choices in that space, and reasonable LRM evaluations must take this into account (see similar arguments in Katz et al. (2024) and Kapoor et al. (0244)
Classical planners like Fast Downward (Helmert 2006) achieve 100% on our dataset in a fraction of the time, compute, and cost, while providing guarantees that their answers are correct. Running Fast Downward on a personal computer was essentially free in dollar terms and averaged 0.12 seconds per instance, which is many orders of magnitude faster than the average ol clock times listed in table 2 It is also generally predictable, and can be scaled to harder instances very directly. Vanilla LLMs are typically very good at translating problems between formats, and could be used to do so in concert with a classical planner at a significantly lower cost than querying an LRM (e.g. Olmo et al. (2021); Liu et al. (2023) ) For problems which don't it have simple PDDL domain and instance specifications, LLM-Modulo systems may be a safer and cheaper approach: run a smaller, faster LLM in a loop with a sound verifier, so that the combined system will only
output guaranteed correct solutions (e.g. Kambhampati et al. (2024) Romera-Paredes et al. (0024) Trinh
et al. 2024
The correctness guarantees provided by these latter two methods are sorely lacking in LRMs like ol. A general reasoning system cannot be deployed in safety critical and non-ergodic domains if it continues to confidently make incorrect plans. Even more than previous models, ol is a fully black box system, and AT's decision to not only keep the architecture under wraps and hide the reasoning traces, but to warn away and even ban anyone who attempts to understand what is going on inside (Edwards 2024). makes interpretability nearly impossible, and reduces trust in the system overall [
5 LRM-Modulo: Inference Scaling Techniques Can Be Combined
Contrary to OpenAT's claims that ol models work best when prompted simply and without external augmentation (OpenAl 2024d), we find that the (unknown) inference time scaling ol executes automatically can in fact be combined with other ways of adaptively incrcasing LLM compute. The particular method that we investigate is an adaption of Kambhampali et al. (2024)'s 's LLM-Modulo to LRMs, which we call LRM-Modulo.
We augment ol-preview and ol-mini with external verifiers to endow the combined system with soundness guarantees- this compound system also allows for some degree of control over the total compute used, by just letting the user set limits enforceable between backprompts. While ol a stride in the direction of general-purpose, expressive planning systems, our results have shown that it cannot plan robustly when faced with harder instances (see Figure 2]. In other words, ol models, just like LLMs, do not come with guarantees for their outputs. Prior to these models, the best way to coax planning capabilities out of LLMs was to pair them with sound external verifiers in generate-test frameworks, in what are known as LLM-Modulo systems (Kambhanpati et al.) 2024 TTrinh el al. 2024]. This framework is broadly applicable even beyond LLMs, and given a sound verifier for some domain requires only a generator expressive enough to provide guesses for that domain. Morcover, because of the built-in verification, it guarantees that any answer actually output is correct. For safety-critical systems, this is essential. High accuracies are not sufficient, especially when the underlying system- as is the case for LLMs and even more so for LRMs-is an opaque black box.
TThe current model also set to default temperature of 1.0, which further reduces replicability and interpretability- any given problem, it never clear whether the result merely the result of stochasticity. This compounds problem with OpenAl models that has existed since least. GPT3. Temperature never gave deterministic outputs, and worse, the logprahs provided by the OPenAT API for any given prompt have long been known fluctuate wildly (Xuan 2023).
9

---

Published in Transactions on Machine Learning Research (April/2025)
Table 6: Performance of LRM-modulo with ol-preview as the underlying LRM on our hardest test sets: Blocksworld (20+ length plans), Mystery Blocksworld, Sokoban, Graph Coloring (20 vertex), Travel Planning, and Trip Planning (10 cities). Due to cost constraints, we run each problem set for up ten iterations, stopping carly if the improvement per iteration levels off. For all the planning domains prompts were in PDDL.
Total Instances
Direct ol-preview
LRM (ol-preview) Modulo
Domain
% correct
Cost (in $)
% correct
# of iters
Cost (in $)
110
Blocksworld (hard)
23.65%
52.12
98.2%
7
139.36
600
Mystery Blocksworld
64.3%
241.2
96%
6
489.21
55
Sokoban 12.73%
42.89
43.6%
7
203.38
50
Graph Coloring (hard)
64%
26.65
94%
10
56.16
180
Travel Planning
10%
68.21
65%
10
431.75
200
Trip Planning (10 cities)
4%
131.26
15.50%
4
535.49
Table 7: Performance of LRM-modulo with ol-mini as the underlying LRM hardest test sets:
Blocksworld (20+ length optinal plans), Sokoban, Graph Coloring (20 vertex), Travel Planning, and Trip Planning (10 cities).
Total Instances
Direct ol-mini
LRM (ol-mini) Modulo
Domain
correct
Cost (in $)
% correct
# of iters
Cost (in $)
110
Blocksworld (hard)
0.90%
16.30
10%
4
40.80
55
Sokoban
10.90%
5.28
12.70%
4
17.49
50
Graph Coloring (hard)
50%
3.33
84%
15
14.18
180
Travel Planning
1.67%
12.11
41.11%
10
97.17
200
Trip Planning (10 citics)
1%
20.01
3.50%
4
77.71
Generate-tost systems are limited by how good the generator is. poor but complete generator, such as one that produces completely random strings, may be capable event producing the correct answer, but be so unlikely do so that is functionally useless, while an incomplete generator may be incapable of ever outputting the correct answer. LLMs and LRMs can be backprompted that is, we can take feedback from the sound verifier and send it back to the model or modify the next prompt in some other way to increase the diversity of the responses generated- which may steer their next output towards the correct answer. Based on our results, ol models are much better generators than anything that came before them.
We test LRM-modulo setups on our five hardest test sets: 20+ length plan Blocksworld, Sokoban, 20 vertex graph coloring, OSU's Travel Planning, and 10 city trip planning. Due to cost constraints, we limit the number of iterations to a maximum of ten, but we stop the system carly once the performance increase from round round has become mostly flat. Even with few iterations, we see significant jumps in performance almost all of our domains. ol-preview's performance on harder Blocksworld saturates within 7 iterations, with the combined system achieving 98.2%. Harder graph coloring shows similar results, going up to 94%. Perhaps most surprising, our most difficult domain, Sokoban, shows a significant jump from 12.7% 7% to 43.6%. ol-mini-Modulo performance, while also impressive, only comes close on graph coloring and travel planning.
We present these results in Table D
Our results imply that these models are not only better generators, but also benefit more from the sound verification signal and provided feedback. However, we can't really know how they the critique provided-
10

---

Published in Transactions on Machine Learning Research (April/2025)
Table 8: Performance and average time taken on (extended) PlanBench by OpenAl's ol-preview and DeepSeek- R1. For all the domains prompts were in PDDL.
Avg. Time
Total Instances
Instances correct
Domain
(in secs)
ol ol-preview DeepSeek-RI
ol-preview
DeepSeek-RI
600
Mystery Blocksworld
64.3%
54.4%
68.14
230.87
Randomized
600
37.3%
47.5%
111.11
343.09
Mystery Blocksworld
200
Logisties
94%
88%
84.07
153.39
200
Randomized Mystery Logisties
52%
45.26%
167.41
223.56
55
Sokoban
12.73%
14.55%
147.98
240.57
Table 9: Performance of LRM-modulo with DeepSeek-RI as the underlying LRM on Blocksworld (20+ length plans), Mystery Blocksworld and Sokoban. For all the domains prompts were PDDL.
Total Instances
Direct DeepSeck-RI
LRM (DeepSeck-R1) Modulo
Domain
% correct
Cost (in 8)
% correct
# of iters
Cost (in $)
110
Blocksworid (hard)
53.64%
1.89
87.27%
3.29
600
Mystery Blocksworld
54.5%
10.6
76.17%
28.2
55
Sokoban
14.55%
1.33
30.91%
4.29
this question is crucial but umanswerable, given OpenAl's current stance against revealing the internal workings of the model or the intermediate reasoning tokens it generates.
LLM-Modulo vs. LRM: With the high cost of ol model queries, it is also essential to examine in which situations these models are not just the best option, but the cost-effective one. Returning to LLM-modulo, where we use a smaller, cheaper, and faster LLM in loop with a verifier, can provide similar or better performance in some domains. For example, in calendar scheduling, ol-mini costs $2.70 to run over the entire test set, and has final accuracy of 94%. GPT-4o-min can be run for 50 iterations in modulo loop for only $2.48, beating that performance to get 97%.
6 PlanBench Evaluations on DeepSeek R1
While ol seemed like quantum jump in LLM research when it came out in Fall 2024, there have since been other commercial reasoning models. In Kambhampati et al. (2025) we provide an overview of these developments. Of these models, special mention must be made of DeepSeek RI, an open-weight model whose training regime and architecture are documented in Guo et al. 22055..
We've run the same PlanBench evaluations on this model. As we can see in Table the results are qualitatively very similar. RI slightly better on some domains, and slightly worse on others the major distinction is that it is also significantly better than vanilla LLMs. The bigger difference between the two models (at least when accessed through their respective APIs at the time of writing) is cost and time: R1 is significantly cheaper but noticeably slower, providing another point on the frontier we discussed in subsection 4.3
We also ran the same LRM-modulo experiments with R. in the role of LRM. These results are presented in Table g showing that significant performance gains can also be made with this newer model, and that combining inference-ting time scaling techniques is still very much a viable method of further improving performance.
11

---

Published in Transactions on Machine Learning Research (April/2025)
7 Conclusion
In this paper, we investigated the performance of ol-preview and ol-mini the new so-called LRMss on a variety of planning and scheduling benchmarks. While LLMs have thus far failed to make much progress obfuscated (or 'Mystery') versions of PlanBench domains, these new models show the first bit of real progress. As ol models are quite different from the previous LLMs, there is a lack of understanding about ol's performance and limitations on reasoning problems. Our systematic suuyy-wichh starts with existing planning and scheduling benchmarks, but then constructs extensions to better evaluate ol-preview and ol-mini in places where they saturate the existing benchmarks, and carcfully analyzes the cost/ ssceccuurayy tradeofls, will be valuable contribution in understanding this new breed of models.
In general, ol-preview has made impressive headway benchmarks that were previously unassailable. However, when evaluated on longer and harder problems the model's accuracy gains are neither general nor robust. While these models made some gains on scheduling problems, performing much better on graph coloring than previous models, these were not evenly distributed and were somewhat lackluster on OSU's Travel Plan domain and Google's Natural Plan benchmark suite.
We also discussed the critical accuracy /efficiency tradeoffs brought up by the fact that ol models use (and charge for) significant inference-time compute, as well as how it compares to other inference time scaling approaches (such as LLM-Modulo (Kambhampati et al. 2024)) and dedicated solvers. Future evaluations will have to maintain a focus on these factors if they are to remain meaningful or relevant. We showed that multiple inference time scaling approaches can be combined, and provided case study of constructing an LRd-Mo system that further improved performance and, crucially, provided much-ncoded guarantees. Finally, while the bulk of the paper focused on ol, we provided similar evaluations on DeepSeek RI, a more recent (and (pen-weight)) LRM. The results for R1 are qualitatively very similar to our ol results, including the LRM-Modulo experiments.
Acknowledgments
We acknowledge support from and spirited discussions with fellow Yochan lab members. Our research is supported in part by DARPA grant HRR00122500166, ONR grant N0001423-12409 and gift from Qualcomm.
References
Mark S Boddy, Johnathan Gohde, Thomas Haigh, and Steven A Harp. Course of action generation for cyber security using classical planning. In ICAPS, pp. 12 21, 2005.
Noam Brown. ol gets it right, almost always, 2024. URL httpp::/e..ca////ooooooaaa/ttttt 1834280720493422724 Accessed: 2024-09-14.
Turgay Caglar, Sirine Belhaj, Tathagata Chakraborty, Michael Katz, and Sarath Sreedharan. Can 1lms fix issues with reasoning models? towards more likely models for ai planning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 20061-20069, 2024.
Jacques Carlier. The one-machine sequencing problem. European Journal of Operational Research, 11(1): 47, 1982.
Joseph Culberson. Sokoban is pspace-complete. 1997.
Gautier Dagan, Frank Keller, and Alex Lascarides. Dynamic planning with 1mm. arXio preprinl arXiv:2308. 00630,,,0 2023.
Nouha Dziri, Ximing Lu, Mclanic Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Scan Wellock, Peter West, Chandra Bhagavatula, Ronan Le Bras, et al. Faith and fate: Limits of transformers on compasitionality. Advances in Neural Information Processing Systems, 36, 2024.
12

---

Published in Transactions on Machine Learning Research (April/2025)
Benj Edwards. Ban warnings fly users dare to probe the "thoughts" of openai's latest model.
Ars Technicu, September 2024. URL
Elliot Gestrin, Marco Kuhlmann, and Jendrik Scipp. N12plan: Robust llm-driven planning from minimal text descriptions. In ICAPS 2024 Workshop on Human-Aware Explainable Plamning.
Malik Ghallab, Dana S. Nau, and Paolo Traverso.
Automated Planning and Acting. Cambridge
Univorsity Press, 2016. ISBN 978-1-107-037277- URL
Lin Guan, Karthik Valmeckam, Sarath Sreedharan, and Subbarao Kambhampati, Leveraging pre-trained large language models to construct and utilize world models for model-based task planning. Advances in Neural Information Processing Systems, 36:79081-79094, 2023.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. ar preprint arXiv:2501. 12948, 2025.
Shibo Hao, Yi Gu, Haodi Ma, Joshua Hong, Zhen Wang, Daisy Wang, and Zhiting Hu. Reasoning with language model is planning with world model. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 8154-8173, 2023.
Malte Helmert. The fast downward planning system. Journal of Arlificial Intelligence Research, 26:191 246, 2006.
Jörg Hoffmann, Stefan Edelkamp, Sylvie Thiébaux, Roman Englert, Frederico Liporace, and Sebastian Trüg. Engineering benchmarks for planning: the domains used in the deterministic part of ipc-4. Journal of Artificial Intelligence Research, 26:453-25 44111-41, 2006.
Mengkang Hu, Yao Mu, Xinmiao Chelsey Yu, Mingyu Ding, Shiguang Wu, Wenqi Shao, Qiguang Chen, Bin Wang, Yu Qiao, and Ping Luo. Tree-planner: Efficient close-loop task planning with large language models. In The Twelfth International Conference Learning Representations.
Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, et al. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint srXim:2207.05608, 2022.
IPC. International planning competition. https://www.ccggs-coffrooco.. competitins,s 1998.
Subbarao Kambhampati, Karthik Valmeekam, Lin Guan, Kaya Stechly, Mudit. Verma, Siddhant Bhambri, Lucas Saldyt, and Anil Murthy. Lims can't plan, but can help planning in llm-modulo frameworks. arXiv preprint vrXiu:2402.08817 2024.
Subbarao Kambhampali, Kaya Stechly, and Karthik Valmeekam. (How) Do Reasoning Models Reason? Annals of the New York Academy of Sciences, 2025. DOI: 10.1111/myas.15339.
Sayash Kapoor, Benedikt Stroebl, Zachary S. Siegol, Nitya Nadgir, and Arvind Narayanan. Ai agents that matter, 2024. URL nittps://arxa httpa///aaaivvvooaaa/////a/ 01502
Michael Katz, Harsha Kokel, Kavitha Srinivas, and Shirin Sohrabi. Thought of search: Planning with language models through the lens of efficiency. In The Thirty-eighth Anaual Conference on Neural Information Processing Systems, 2024.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large Language Models are Shro-Shot Reasoners. Xix preprint arXin:2205.11991 2022.
13

---

Published in Transactions on Machine Learning Research (April/2025)
Bo Liu, Yuqian Jiang, Xiaohan Zhang, Qiang Liu, Shiqi Zhang, Joydeep Biswas, and Peter Stone. Lim+ pb Empowering large language models with optimal planning proficiency. aiv preprint or Xix::2344.11477, 2023.
Drew McDermott, Malik Ghallab, Adele E. Howe, Craig A. Knoblock, Ashwin Ram, Manuela M. Veloso, Daniel S. Weld, and David E. Wilkins. Pddl-the planning domain definition language. 1998.
Iman Mirzadch, Keivan Alizadch, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mchrdad Farajtabar. Gsm-symbolie: Understanding the limitations of mathematical reasoning in large language models. arXiv preprint arXit:2410.05299, 2024.
Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. GPT3-to-plam: Extracting plans from text using GPT-3. arXiv preprint arXiv:9106.0713 2021.
OpenAL. Openai ol and new lools for developers, 2024a.
URL
attps://openai.commiince
o1 and- for-ttoo----rroeeellpoo------
OpenAT. Learning to reason with 1lms, 2024b.
URL
https://openaii.cmmmnn ionwe/
Shamingg-o-reas00 wwthh- 11mm
OpenAT. Introducing openai ol-preview, 2024c.
URL
https:://peeaa..com/
introducing pirroduccingopeenaa------rrsss Accessed: 2024-09-19.
OpenAL. Guides: Reasoning, 2024d. URL http:///aaaaoom.oopaaaaaamc////s
Ac-
cessed: 2024-09-14.
Akshara Prabhakar, Thomas I. Griffiths, and R McCoy. Deciphering the factors influencing the efficacy of chain-of-thought:: Probability, memorization, and noisy reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 3710-3722 3724, 2024.
Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. Mathematical discoveries from program search with large language models. Nature, 46577995) 2024.
Julian Schrittwicser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, L. Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, Timothy P. Lillicrap, and David Silver. Mastering atari, go, chess and shogi by planning with learned model. Nature, 588:604 609, 2019. URL
https://api. www.ntiischooliiee..e www.ooolarrrrrg//gg/eeoe org/CorrpsII:20000 20085552225
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024.
Tom Silver, Varun Hariprasad, Reece S Shuttleworth, Nishanth Kumar, Tomás Lozano-Pérez, and Leslie Pack Kaelbling, PDDL planning with pretrained large language models. In NewrIPS 2022 Foundation Models for Decision Making Workshop, 2022. URL ftttp:://operrvieee net net/fforum2id=11QMUBBZZZ
Tom Silver, Soham Dan, Kavitha Srinivas, Joshua B Tenenbaum, Leslie Kaelbling, and Michael Katz. Generalized planning in pddl domains with pretrained large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.
2024.
Kaya Stechly, Karthik Valmeekam, and Subbarao Kambhampati. Chain of thoughtlessness? analysis of cot in planning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024a. URL
Kaya Stechly, Karthik Valmeckam, and Subbarao Kambhampati. On the self-verification limitations of large language models on reasoning and planning tasks. arXiv preprinl erXiv:2402.0811 15, 2024b.
Hao Tang, Darren Key, and Kevin Ellis. Worldcoder, a model-based 1lm agent: Building world models by writing code and interacting with the environment. arXiv preprint arXiv:2402. 120.2422212.7.5.. 2024.
14

---

Published in Transactions on Machine Learning Research (April/2025)
Trien H Trinh, Yuhuai Wu, Quoc V Le, He He, and Thang Luong. Solving olympiad geometry without human demonstrations. Nature, 625(7995):476-44 482, 2024.
Karthik Valmoekam, Matthew Marquez, Sarath Sreedharan, and Subbarao Kambhampati. On the planning abilities of large language models-a critical investigation. Advances in Neural Information Processing Systems, 36:75993-76005, 2023.
Karthik Valmeekam, Matthew Marquez, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change. Advances in Neural Information Processing Systems, 36, 2024.
Mudit Verma, Siddhant Bhambri, and Subbarao Kambhampati. Do think tags really help 1lms plan? a critical evaluation of react-style prompting. In Adaptive Foundation Models: Evolving Al for Personalized and Efficient Learning.
Christopher John Cornish Hellaby Watkins. Learning from delayed rewards. 1989.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc Le, Denny Zhou, et al. Chain-of-thought prompting clicits reasoning in large language models. Advances in neural information processing systems, 35:24824 24837, 2022.
Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. Travelplanner: A benchmark for real-world planning with language agents. In Forty-forst International Conference Machine Learning.
Tan Zhi Xuan. log, probs returned by openai's api are "incredibly* unstable, 2023. URL https://x.com com/ Accessed: 2024-09-14.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas I. Griffiths, Yuan Cao, and Karthik R Narasimhan. Deliberate problem solving with large language models. In Thirty-seventh Conference on Neural Information Processing Systems, 2023a. URL / htt.://iww....eeeooooad//aaaaaaa
Tree of thoughts:
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In The Fleventh International Conference Learning Representations, 2023b. URL https://openrevieu net/forum?id-WE, vit
Huaixiu Steven Zheng, Swaroop Mishra, Hugh Zhang, Xinyun Chen, Minmin Chen, Azade Nova, Le Hou, Heng-Tzo Cheng, Quoc V Le, Ed H Chi, et al. Natural plan: Benchmarking 1lms natural language planning. XrXiv preprint ar arXiv:2406. 04520,, 2024.
15

---

Published in Transactions on Machine Learning Research (April/2255)
Appendix
A
Further Discussion of LLM Planning Performance
LLMs are highly capable at providing translations betwoen equivalent representations (Olmo et al. 2021 This fact, combined with their significantly higher performance on the unobfuscated version of the Blocksworld domain, predicts that-if they are capable of composing reasoning operations the performance gap wetween Mystery Blocksworld and classic Blocksworld should shrink substantially if the translation from Mystery Blocksworld back into Blocksworld is explicitly provided. However, when we provide this in the prompt (see Appendix D. performance only improves very small amount: GPT-4 achieves 10%.
We also find that, contrary to previous claims, one-shot prompting is not a strict improvement over zero-shot. In fact, for many models seems to do significantly worse. (This is most notable in our tests of LLaMA family models.) While the reverse is generally true for Mystery Blocksworld problems, it's important to note that the performance of vanilla LLMs on Mystery Blocksworld has consistently and uniformly been poor (the same it was when this benchmark was first released), so those results do not provide too clear a picture. Most models do not solve even single instance in zero-shot mode, and only one (LLaMA 3.1 405B) manages more than one.
B Speculations about ol internal operation
While our evaluation of ol did not depend any specific assumption about its operation, we did have working model of ol based on the very skimpy description that was provided in the blog post that accompanies ol's release [T 2024ch. Verifying our model is unfortunately made infeasible by the fact that ol doesn't. 't actually provide any trace of its operations (even during the costly inference stage), and OpenAI warns that API access will be revoked any attempts are made to surface its reasoning tokens.
There are two things-"reinforeee "reinfoccemennt learning' and "Private Chain-of- Thoughh (CoT)" that are mentioned in the writeup. So imagine you are trying to transplant "generalizod AlphaGo"-ler -let's call it GPTGo-onto the underlying LLM token prediction substrate.
To do this, you need to know
1. What are the GPTGo moves? For AlphaGo, we had GO moves). What would be the right moves when the task is just "complete the prompt the right way'?
2. Where is it getting its external success/faill failure signal from? For AlphaGo, we had simulators / verifiers giving the success /failess/ailuree signal. The most interesting question in transplanting the self-play idea to a general AI agent where is it getting this signal?
Our guess is that the moves are anto-generated CoTs (thus the moves have a very high branching factor) Let's assune for simplification that we have a CoT-generating LLM, that generates these CoTs conditioned the prompl. (It is not clear if the CoT's are domaín independent of the "think step by step" variety (Kojima et al. 2022) domain/tass specific,
combination.)
The success signal is likely from massive amounts of synthetic training data with correct answers. When the completed prompt is seen to contain the correct answer (presumably judged by the LLM itself), then the episode is considered a success, and a failure otherwise.
The task for the reinforcement learner then is: Given the original problem prompt, generate and select a CoT, and use it to continue to extend the prompt (possibly generating subgoal CoTs after every few stages). Get the final success/ failure signal for the example (for which you do have answer).
The RL stage may involve training on a huge number of training examples with answers. The training examples with answers can either be coming from benchmarks, or from synthetic data with problems and
Note that the moves here "generic" language prompt extcusion moves. In this sense, they closer to pi [Schrittwieser 2019 that used generic "shift the piece on the board" moves applicable all board games than Go or Chess specific moves.
et.
16

---

Published in Transactions on Machine Learning Research (April/2025)
their solutions-usiin external solvers. In this phase the RL part attempts to learn the q-values of the CoT moves (much like AlphaGo learns the q-values of the moves of the Go). (The q-values learning may be incorporated into the internal weights of the CoT generator LLM). At this point, we have a CoT move generator that is better than the random one before the RL stage
During the inference stage- which OpenAT says can be indefinitely long (although it is currently capped internally by them, with no external control), like AlphaGo, ol might be further improving its evaluation of the v-values of the CoT moves in the context of the current prompt. While AlphaGo used MCT-based rollouts, obviously don't know the mechanism ol uses. The announcement only says that at inference stage a long chain of thought is added to the original prompt (and ol does charge the end users for its "reasoning tokens," " which are never seen by the end user, at the same high rate as the output tokens). In this sense, our speculations seem to be consistent, even though it is not clear whether the reasoning tokens are proportional to the entire inferenco-stage computation, or just represent the final sequence of CoT moves that get selected after the rollout-like inference stago.
Some corollaries of our speculation are:
1. Note that this use of RL is very different from that in RLHF, which can be seen as fine tuning stage for an LLM that keeps the inference step unchanged. It is also different from techniques, including Ap'nAT's, that advocated fine tuning both on synthetic data accompanied with derivational traces- these too will be a form of finetuning that leave inference stage unchanged. Here ol could, in thoory, be getting significantly more leverage out of the data by learning move (auto CoT) generators.
2. There still are guarantees that the answers provided are "correct' they may be probabilistically little more correct (subject to the training data). If you want guarantees, you still will need some sort of L.M--Mouulo approach even on top of this.
3. It is certainly not clear that anyone will be willing to really wait for long periods of time during inference (it is already painful to wait for 10 see for 10 word last letter concatenationl). The kind of people who will wait for longer periods would certainly want guarantecs-ane and there are deep and narrow System 2's a plenty that can be used for many such cases.
4. There a bit of a Ship of Theseus feel to calling ol an LuM-ccniidering how far it is from the other LLMs (all of which essentially have teacher-forced training and sub-real-time next token prediction. That said, this is certainly an interesting way to build a generalized system 2'ish component on top of LLM rates but without guarantees.
B.1 Comparison to DeepSeek R1
Our speculations about ol's operations were written in the fall of 2024. While OpenAl ol's internal operations have not boen made public to-date, DeepSeck R1 is an open weight model whose internal operations were described in a technical report (Guo et al.) 2025). Although we don't. know how representative R1 is of ol, R1 does use reinforcement learning to improve the CoTss (intermediate tokens) on synthetic data, with the reward signal coming from the verifiers for that data. Different from our speculation on ol, R1 uses the base model itself to generate the initial CoTs (intermediate tokens). R1 also does not use any test-time inference/adaye
computation (as we speculated for ol); its operation at inference time is thus like that of a standard LLM, albeit one trained to produce intermediate tokens on the way to a solution. We invite the reader to see (Kambhampati et al., 2025) for more detailed and up-to-date perspective on how, and whether, large reasoning models reason.
C Performance on Unsolvable Instances:
While planning problems normally require the agent to formulate a course of action to achieve a goal, an oqually valid use of planning abilitics is to recognize that a given goal cannot be accomplished by any plan. A real-world example of this is network vulnerability analysis, where an agent may wish to certify that no plan of attack exists for a specified system (Boddy et al. 2005). So far, LLMs have struggled to recognize that some
17

---

Published in Transactions on Machine Learning Research (April/2025)
Table 10: Rate of claiming that a problem impossible by OpenAl's ol-preview on 100 unsolvable and 600 solvable instances the Blocksworld and Randomized Mystery Blocksworld domains. The True Negative rate is the percent of unsolvable instances that were correctly marked as unsolvable. The False Negative rate is the percent of solvable instances that were incorrectly marked unsolvable. Previous models not shown in this table as their true negative and false negative rales were generally 0% across the board.
ol-preview
Domain
Shots
True Negatives
False Negatives
Blocksworid
0-Shot
27%
0%
Randomized Mystery
1-Shot
16%
11.5%
Blocksworld
problems cannot be solved, instead confidently confabulating nonsensical answers. ol launched with the claim that it has started to overcome this issue, and can now accurately identify unsolvable problems (Brown) 2024). To test this systematically, we modified 100 instances from the easier three to five block test set by adding one
conjunct to each instance's goal state, making the goal unsatisfiable. We ensured our instances were unambiguous by giving the full PDDL representation of both the domain and the instance, avoid quibbles such "A on B because is on C and on B' where the model redefines the meanings of potentially ambiguous natural language statement. The results are in Table I0 On Blocksworld, only 27% of all instances were correctly and explicitly identified by ol unsolvable. In 19% of all cases, the model returned dot somee kind of "Jemply plan]" marker, without any explanation indication of unsolvability. We consider these incorrect, as 'empty plan" is only the correct answer if the goal is already satisfied. In the remaining 54% of cases, the model generated a full (and therefore impossible and incorrect!) plan.
On Randomized Mystery Blocksworld, these numbers are worse: 16% of cases were correctly identified as unsolvable, 5% returned an empty plan, and the remaining 79% were answered with a full (impossible or goal-unsatisfying) plan. Therefore, unsolvable instances continue be problem for LRMs. Furthermore, this ability to sometimes note impossible plans correctly comes cost: the model sometimes falsely claims that solvable problems are actually unsolvable. On Randomized Mystery Blocksworld, 11.5 11.5% of instances are incorrectly claimed to be impossible. These results can be seen in Table I0
D NL vs PDDL prompts
Table 11: Performance of ol- ol-preview natural language (NL) and PDDL prompts.
Instances correct
Total Instances
Domain
NL
PDDL
600
Mystery Blocksworld
52.83%
64.3%
230
Randomized Obfuscated Blocksworld
36.52%
47.82%
In (Valmeekam et al. 2023), was shown that LLMs performed better on natural language prompts opposed to PDDL prompts for planning. With ol models (specifically ol-preview), reversed trend. As shown in Table [1 ol-preview performs better on PDDL prompts than natural language.
E
LRM-Modulo performance over iterations
See Figure
18

---

Published in Transactions on Machine Learning Research (April/2025)
Correctness vs Iterations
100
90
80
Domain
Correctness (%)
70
Blocksworld (Hard)
Sokoban
60
Travel Planning
Trip Planning
50
Graph Coloring (Hard)
40
Model
30
O1 Preview
Mini
20
10
2
4
5
7
8
10
11
Iterations
Figure 3: LRM-Modulo significantly improves performance over direct prompting as we increase the number of itcrations.
F Gemini 1.5 Pro Response Mystery Blocksworld
When using the default settings, Gemini 1.5 Pro refuses provide output our Mystery Blocksworld queries and instead returns the following:
kinish_reason, SAFETY
safety_ratinn ratings {
category: HARM CATEGORY SETEULLYSSEXUALLL,CCCL SEXUALLY_EEXEE EXPLICIT
probability: NEGLIGIBLE
safety_ratings {
category: HARM_ CARM__AATEORR_LAC BAADHH__HHTTEEEEEEAAAS
probability: NEGLIGIBLE
eafety_ratinng {
category: HARM_ SETEGORRY_ARASSSEEE
probability: NEGLIGIBLE
safety_rating ratings {
category: CRR_GCAY DANGEROUSS
probability: MEDIUM
G ol-preview and ol-mini Response to Meeting Planning
openal BadRequestError: b'{
Perrorr:
"message"; "Invalid prompt: your prompt flagged potentially violating our usage policy. Please try again with a different prompt.",
"type": "invalid "invalid_request reriiid_reuuet_errrrrr
19

---

Published in Transactions on Machine Learning Research (April/2025) 2025)
"param": null,
"code": "invalid_proon
}
H ol Token Use Versus Problem Difficulty
Nodes Expanded vs Reasoning Tokens for 01 Preview on Blocksworid Pian Generation Zero Shot
12000
Average Reasoning Tokens
10000
Nodes Expanded
le6
Figure 4: The mumber reasoning tokens used by ol-preview when solving Blocksworld instances does not track the number of nodes that need to be expanded solve the problem.
Prompt to Translate From Mystery Back to Blocksworld
am playing with a set of objects. Here are the actions can do
Attack object
Feast object from another object Succumb object Dvercome object from another object
have the following restrictions on my actions:
To perform Attack action, the following facts need to be true: Province object, Planet object, Harmony.
Oace Attack action is performed the following facts will be true: Pain object. Dace Attack action is performed the following facts will be false: Province object, Planet object, Harmony.
To perform Succumb action, the following facts need to be true: Pain object.
Oace Succumb action is performed the following facts will true: Province object, Harmony.
Planet object,
Once Succumb action performed the following facts will be false: Pain object.
To perform Overcome action, the following needs to be true: Province other object, Pain object. Once Overcome action performed the following will be true: Harmony, Province object, Object Craves other object.
Once Overcome action performed the following will be false: Province other object, Pain object. To perform Feast action, the following needs to be true: Object Craves other object, Province object, Harmony.
Dnce Feast action is performed the following will be true: Pain object, Province other object. Dace Feast action is performed the following will be false:, Object Craves other object, Province object, Harmony.
You will be given a set of initial conditions and a goal condition. To solve the problem, you will have tell which actions take and which order order achieve the goal.
20

---

Published in Transactions on Machine Learning Research (April/2025)
Please provide your anawers using the above terminology. However, you may find it helpful to translate the above description into common-sense format while working your solution. Just remember to translate it back later!
Instead thinking in
different alphabet blocks (block A, block B, etc.) which you are stacking (using just one hand) in towers on a table.
"objects", think in
Then the "facts" that are true false at given time are really facts about the blocks and the hand:
"Province object a" just means that "block A is clear" or, equivalently, "nothing is top of block
"Planet object a" is another way of saying "block A is the table"
"Harmony" is a codeword for "my hand isn't holding anything"
"Pain object a" "the hand is holding block A" "object a Craves object b" translates to "block A is on top of block B"
And the "actions" can be seen as stacking and unstacking of blocks (where the restrictions stop us from picking up the bottom block in tower holding more one block in the hand at time):
"Attack object a" translates to "pick up block directly from the table"
"Feast object a from object b" translates to "pick up block A from directly on top of block D"
"Succumb object a" tranelates to "put block directly on the table" "Overcome object from object b" translates to "put block directly on top of block D"
If you do this framing, make to remember translate back into the original terms.
Prompts for Blocksworld
J.1 (Solvable) Blocksworld Instances Zero-Shot in Natural Language
playing with of blocks where need to arrange the blocks into stacks. Here the actions I can do
Pick up a block Unstack block from top of another block Put down block Stack block on top another block
have the following restrictions on my actions: can only pick up unstack one block at time. can only pick up or unstack a block if my hand is empty.
can only pick up block if the block is the table and the block is clear. A block is clear if
the block has no other blocks on top of it and if the block is not picked up.
only unstack block from on top of another block if the block unstacking really top of the other block.
only unstack block from top another block the block unstacking clear. Once I pick or unstack a block, I am holding the block.
can only put can only stack block on top of another block if I am holding the block being stacked. can only stack block on top of another block if the block onto which am stacking the block is clear.
a block that
holding.
Dnce put down or stack block, my hand becomes empty. Dnce you stack block top of second block, the second block is no longer clear.
[STATEMENT]
As initial conditions I have that, the red block is clear, the blue block is clear, the yellow block is clear, the hand is empty, the blue block is on the table, the orange block is on the table and the yellow block is on the table. My goal have that the orange block is top of the blue block.
top of the orange block, the red block
21

---

Published in Transactions on Machine Learning Research (02))
What is the plan to achieve my goal? Just give the actions in the plan.
pleevemnsslllrrrieer RESPONSE
To achieve your goal of stacking the orange block on top of the blue block, can follow these steps:
1. *Unstack the blue block from on top of the orange block.
Since the blue block is clear and your hand is empty, you unstack it from the orange block.
2. **Put down the blue block the table.**
With the blue block in your hand, you can place it onto the table.
3. *+Pick up the orange block from the table.**
The orange block is now clear (no blocks on top of it), and your hand is empty after putting down the blue block, you
pick it up.
4. **Stack the orange block on top of the blue block.**
The blue block clear on table, so you can stack the orange block you're re holding onto it.
**tions in the plan:**
**nntacck** the blue block from the orange block.
**Put down** the blue block on the table.
**Pick up** the orange block from the table.
#+Stack** the orange block on top of the blue block.
J.2 (Solvable) Harder Blocksworld Instances Zero-Shot in PDDL
The following is PDDL specification of a planning problem. The first part, under the heading DOMAIN], is the domain file. The second part, under the heading [QUERY PROBLEM), , is the problem file. Using this information, which is correct, and no further assumptions, find a plan which, when from the specified initial state, satisfies the specified goal. Provide your as sequence of actions in PDDL format. An action ACTION which acts on tuo objects OBJ1 and 0BJ2 would written (ACTION OBJ1 0BJ2). [DOMAIN]
provide anything else in your anawer.
(def ine (domain blocksworld-4ops)
(:requirements :stripa)
(:predicates (clear ?x)
(ontable ?x)
(handempty)
(holding ?x)
(on ?x ?y))
(:action pick-up
:parameters (?ob)
:precondition (and (clear ?ob) (ontable ?ob) (handempty))
:effect (and (holding ?ob) (not (clear ?ob)) (not (ontable ?ob))
(not (handempty))))
(:action put-down
pprrameters (?ob)
:precondition (holding ?ob)
:effect (and (clear ?ob) (handempty) (ontable ?ob)
(not (holding ?ob))))
(:action stack
:parameters (?ob ?underob)
:precondition (and (clear ?underob) (holding ?ob))
22

---

Published in Transactions on Machine Learning Research April/20255)
:effect (and (handempty) (clear ?ob) (on ?ob ?underob) (not (clear ?underob)) (not (holding ?ob))))
(:action unstack
:parametera (?ob ?underob)
pprecondition (and (on ?ob ?underob) (clear ?ob) (handempty))
:effect (and (holding ?ob) (clear ?underob)
(not (on ?ob ?underob)) (not (clear ?ob)) (not (handempty)))) )))
[QUERY PROBLEM]
(define (problem BW-rand-6)
(:domain blocksworld-4ops)
(:objects a b d (:init
(handempty)
(ontable a)
(on b f)
(on a)
(on d b)
(on c)
(on e)
(clear d)
(ggoal
(and
(on a f)
(on c)
(on c a)
(on b)
(on e))
[PLAN]
oeeesssssl--previiee RESPONSE
(unstack b)
(put-down d)
(unstack f)
(stack b c)
(pick-up d)
(stack d b)
(unstack e)
(put-down f)
(unstack c)
(put-down e)
(pick-up f)
(stack f e)
(unstack a)
(put-down c)
(pick-up a)
(stack f)
(pick-up c)
(stack a)
(pick-up b)
(stack c)
(pick-up d)
(stack b)
23

---

Published in Transactions on Machine Learning Research (02))
J.3 (Solvable) Harder Blocksworld Instances Backprompt
The above plan is invalid. The following action step has precondition: stack b c
The unmet precondition is:
(clear c)
Provide your revised plan as sequence of actions in PDDL format. An action ACTION which acts on two objects OBJ1 and OBJ2 would be written (ACTION 0BJ1 0BJ2). Do not provide anything else in your anawer.
[PLAN]
J.4 Unsolvable Blocksworld Instances Zero-Shot in PDDL
The following PDDL specification of a planning problem. The first part, under the heading DOMAIN], is the domain file. The second part, under the heading [QUERY PROBLEM) is the problem file. Using this information, which is correct, and no further assumptions, find a plan which, when from the specified initial state, satisfies the specified goal. Provide your answer as sequence of actions in PDDL format. An action ACTION which acts on two objects OBJ1 and OBJ2 would be written (ACTION OBJ1 0BJ2). Do not provide anything else in your anawer. [DOMAIN]
(def ine (domain blocksworld-4ops)
(:requirements :strips)
(:predicates (clear ?x)
(ontable ?x)
(handempty)
(holding ?x)
(on ?x ?y))
(:action pick-up
:parameters (7ob)
:precondition (and (clear ?ob) (ontable ?ob) (handempty))
:effect (and (holding ?ob) (not (clear ?ob)) (not (ontable ?ob))
(not (handempty)))
(:action put-down
:parameters (?ob)
:precondition (holding ?ob)
:effect (and (clear ?ob) (handempty) (ontable ?ob)
(not (holding ?ob))))
(:action stack
-arammeters (7ob ?underob)
:precondition (and (clear ?underob) (holding ?ob))
:effect (and (handempty) (clear ?ob) (on ?ob ?underob)
(not (clear ?underob)) (not (holding ?ob))))
(:action unstack
:parameters (?ob ?underob)
:precondition (and (on ?ob ?underob) (clear ?ob) (handempty))
:effect (and (holding ?ob) (clear ?underob)
(not (on ?ob ?underob)) (not (clear ?ob)) (not (handempty))))
[QUERY PROBLEM]
(define (problem BW-rand-4)
(:domain blocksworld-4ops)
(:objects b
(: (iini (handempty)
24

---

Published in Transactions on Machine Learning Research (April/2255)
(on b)
(on b c)
(on d)
(ontable d)
(clear a)
(:goal
(and
(on c)
(on d)
[PLAN]
ool-rrveeeeee RESPONSE
**No plan exists that satisfies the goal with the given initial state and domain.
K Prompts for Mystery Blocksworld
K.1 Mystery Blocksworld Instances Zero-Shot in Natural Language
am playing with a set of objects. Here are the actions I can do
Attack object
Feast object from another object Succumb object
Overcome object from another object
have the following restrictions on my actions:
To perform Åttack action, the following facts need be true: Province object, Planet object, Harmony.
Once Attack action performed the following facts will be true: Pain object. Once Attack action is performed the following facts will be false: Province object, Planet
object, Harmony.
To perform Succumb action, the following facts need to be true: Pain object.
Once Succumb action is performed the following facts will be true: Province object, Planet
object, Harmony.
Once Succumb action is performed the following facts will be false: Pain object.
To perform Overcome action, the following needs to be true: Province other object, Pain object. Dnce Overcome action is performed the following will true: Harmony, Province object, Object
Craves other object.
Once Overcome action is performed the following will be false: Province other object, Pain object.
To perform Feast action, the folloving needs to be true: Object Craves other object, Province object, Harmony.
Once Feast action is performed the following will be true: Pain object, Province other object. Once Feast action performed the following will be false:, Object Craves other object, Province object, Harmony.
[STATEMENT]
initial conditions I have that, object craves object c, harmony, planet object a, planet object c, planet object d, province object a, province object and province object d. My goal is to have that object c craves object
What is the plan to achieve my goal? Just give the actions in the plan.
To achieve the goal where object a craves object b, you can perform the following sequence of actions:
25

---

Published in Transactions on Machine Learning Research (April/2055)
1. **Feast object b from object c**
Preconditions:
'Object craves object e' (initial fact)
Province object bf (initial fact)
'Harmony' (initial fact)
Effects:
Adds: 'Pain object b', 'Province object c'
Deletes: 'Object craves object c', 'Province object b', 'Harmony'
2. **Succumb object b**
Preconditions:
'Pain object b' (from previous action)
Effects:
Adda: 'Province object b', 'Planet object b', 'Harmony'
Deletes: 'Pain object b'
3. **ttack object
Preconditions:
'Province object c' (from step 1)
'Planet object e' (initial fact)
'Harmony' (restored step
Effects:
Adds: 'Pain object c'
Deletes:
'Province object ct, 'Planet object ct, 'Harmony'
4. **vvercome object from object b**
Preconditions:
'Province object b' (restored in step 2)
'Pain object c' (from step 3)
Effects:
Adds: 'Harmony', 'Province object c', 'Object craves object b' (goal achieved)
Deletes:
'Province object b', 'Pain object
By following this plan, you manipulate the states of the objects through the actions provided, ultimately achieving the desired state where 'Object craves object b'.
Answer:**
To achieve the goal, perform these actions in order:
1. Feast object b from object
2. Succumb object b
3. Attack object
4. Overcome object from object
L Prompts for Randomized Mystery Blocksworld
L.1 Randomized Mystery Blocksworld Instances Zero-Shot in Natural Language
am playing with a set of objects. Here are the actions can do
J4gv801gmu2it0y1 object_0 object_1.
U64y1a9apusms1xb object_0.
Kip9us781pr62um object_O. Vio5amhg7814n006 object_0 object_1.
have the following restrictions my actions:
26

---

Published in Transactions on Machine Learning Research (April/2025)
To perform j4gv801gmu2ityy) action, the following facts need to be tv30k33pzoulqq6r object_1, and a46zhwylm6jvbabk object_
Once j4gy801gmu2it0yf is performed the following facts will true: cq14062p1yeke30k,
tv30k33pzoulq16x object_0, and b6e9q4r60gagvden object_0 object_1
Once j4gva01gmu2it0yy is performed the folloving facts will be false: tv30k33pzoulq16x object_
and a46zhwykn6jvbabb object_0
perform u64yla9apusms1xb action, the following facts need to true: tv80k33pzoulq16r object_O,, afftse23ljclsqad object_0, and cq14062plyeke3ok
Once u64yla8apusmsIxxb is performed the following facts will true: a46zhwybn6jvbabi object_0
Once ub4y1a99pu is performed the following facts will be false: tv30k33pzoulq16e object_0,
af9tse231 jcleqqad object_O,, and cq14o62plyekešot To perform xip9um781pv62um action, the following facts need to be true:ad6zhuykn6 vbab object_0
Once kipsuu781pp662um is performed the following facts will be true: tv30k33pzoulq166 object_O, cq14o62p1yeke3ok, and af9tse231jclsqat object_0
Once kip9un781pv62umn is performed the following facts will be false: a46zhuykn6jrbabi object_0
To perform vio5ambq7814n006 action, the following facts need to be b6e9g4r60gagvden object_0 object_1, tr30k33pzoulq16x object_0, and cq14o62plyekeãok
Once wi05amhq7814n006 is performed following facts will true: a46zhuykn6jvbabe object_0, and
tv30k33pzoulqq6w object_1
Once vi05amhg7814:006 is performed the following facts false: b6e9q4r60gagvden object_
object_1, vx30433pzoulq16* object_0, and cq14662p1yekeãok
[STATEMENT]
As initial conditions I have that, af9tse231jclsqad object_O, af9tse231jclsqad object_2, af9tse231jclsqad object_3, b669g4r60gagvden object_1 object_2, cq14o62p1yeke3ok, tv30k33pzoulqI6x object_0, tv30133pzoulq16x object_1 and tr30k33pzoulq166 object_3. My goal to have that b6ggg4r60gagvden object_ object_1.
What is the plan achieve my goal? Just give the actions in the plan.
a -prevview RESPONSE
achieve the goal 'b6e9q4r60gagvden object_2 object_1', you need to perform the following sequence of actions:
1. ##p1o06amh77814m006 object_1 object_28
This action reverses the relationship between 'object_1" and 'object_2", setting up the necessary conditions for the next steps.
2.
##kipuuuu71pp062mm object_1**
By performing this action 'object_1', you restore certain facts (' ('cg4466211yexe30k3)) required for subsequent actions and adjust the state of 'object_1'.
3. #tub4ylalapusmslxb object_2**
This action modifies 'object_2 2' to have the required property '246zhwyk66jvbaba object_2', which is precondition for the final action.
4. +*j4gr801gnu2it0y object_2 object_1**
Performing this action with 'object_e 2' 'object_1' establishes the goal fact b6egq4r60gagvvcn object_2 2 object_1'. 1'.
**Plan Summary:**
**Action 1:** 'v1o5amhq77814n006 object_1 object_2'
**Action 2:** 'kip9uu781pv62umn object_1'
*Action 3:** "u64yla@aapusmslxb object_2'
#*ction 4:** 'j4gr801gmu2it0r object_2 object_1'
This sequence of actions satisfies all the necessary preconditions step-by-step and ultimately achieves your goal.
L.2 Unsolvable Randomized Mystery Blocksworld Instances Zero-Shot in PDDL
27

---

Published in Transactions on Machine Learning Research (April/2025)
The following PDDL specification of planning problem. The first part, under the heading DOMAIN], is the domain file. The second part, under the heading [QUERY PROBLEM), is the problem file. Using this information, which is correct, and further assumptions, find plan which, when run from the specified initial state, satisfies the specified goal. Provide your answer sequence of actions in PDDL format. An action ACTION which acts OBJ2 would be written (ACTION OBJ1 0BJ2). Do not provide anything else in your answer. [DOMAIN]
objecta OBJ1 and
(define (domain xaji0y)
(:requirements :equality)
(:predicates
(tv30k33pzoulq16G ?x1 object)
(af9tse231jclsaad ?x1 object)
(cq14o62p1yekesox )
(a46zhwykn6j vbabk ?x1 object)
(b6esq4r60gagvvcn ?x1 object ?x2 object)
(:action uB4yla9apusmsI.b
parameters (?ob1 object)
precondition (and (tv30033pzoulql16 ?ob1) (af9tse23ljclsqad ?ob1) (<q14062p1yeke3ok )
reffect (and
(#46zhaykn6jvaabx ?ob1)
(not (tv30k33pzoulqq16 ?ob1))
(not (a19ts0231jclsqad ?ob1))
(not (cq14062plyeke00k )))
)
(:action kip9us781pv62umm
paraametera (?ob1 object)
precondition (a46zhwykn6jvbabb ?ob1)
:effect (and
(tv30133pzoulq166 ?ob1)
(cq14o62p1yekeeok )
(af9tse231jclsqadd ?ob1)
(not (#46zhuyln6jvbabb ?ob1)))
(:action j4gv801gmu2it0yj
parametera (?ob1 object ?ob2 object)
precondition (and (tv30k33pzoulq16w ?ob2) (a46zhwykn6) vbabk ?ob1))
seffect (and
(cq14o62p1yekeBox
(tr30k33pzoulq16> ?ob1)
(b6e9q4r60gaavvdnn ?ob1 Pob2)
(not (6v30k33pzoulqq16 ?ob2))
(not (446zhuyln6jvbabk ?ob1)))
(:action vio5amhq7814n006
parameters (?ab1 object ?ob2 object)
preconnitioo (and (b6e9q4r60gagvden ?ob1 ?ob2) (tr30k33pzoulq16G ?ob1) (cq14o62ptyekečok ))
:effect (and
(a46zhwykn6jvbbabz ?ob1)
(tv30k33pzoulq166 ?ob2)
(not (b6e9ggr60gagvvden ?ob1 ?ob2))
(not (tr30k33pzoulq11w ?ob1)) )
28

---

Published in Transactions on Machine Learning Research Aprril22225)
(not (cql4062plyekeook )))
[QUERY PROBLEM]
(define (problem instance-1-4)
(:domain xaji0y)
(1objects a с d
(:init
(cq]4o62p1yekee3kk
(b6e9qg4r0gaavvdnn d)
(#19t:se23jcisqqa b)
(bgevyere0gaavven
60ggvgn d b)
(tw30k33pzoulq16r
(: goal
(and
(b6e9q4r60gagvden a
(b6e9g4r60gagvden b a)
(bGe9gtr60gagvden
[PLAN]
o RESPONSE- RESPONSE-
There sequence of actions that, when from the specified initial state, satisfies the apecified goal. Thus, no plan can be provided.
M Prompts for Logistics
M.1 Logistics Zero-Shot in PDDL
The following is PDDL specification of a plamning problem. The first part, under the heading [ DOMAIN], the domain file. second part, under heading [QUERY PROBLEM), is the problem file. Using this information, which is correct, and further assumptions, find plan which, when run from the specified initial state, satisfies the specified goal. Provide your answer sequence of actions in PDDL format. An action ACTION which acts tuo objects OBJ1 and 0BJ2 would written (ACTIDN OBJ1 (BJ2). not provide anything else in your answer. [DOMAIN]
(def ine (domain logistic--strip)
(:requirements :strips)
(:predicates (OBJ ?obj)
(TRUCK ?truck)
(LOCATION ?loc)
(AIRPLANE ?airplane)
(CITY ?city) (AIRPORT ?airport)
(at ?obj ?loc)
(in 7obj1 ?obj2) (in-city ?obj ?city))
(:types)
default object
(raction LOAD-TRUCK :parameters (?abj
29

---

Published in Transactions on Machine Learning Research (April/2055)
?truck
?loc)
ppeconddiion
(and (OBJ ?obj) (TRUCK ?truck) (LOCATION ?loc)
(at ?truck ?loc) (at ?obj ?loc))
:effect
(and (not (at ?obj ?loc)) (in ?obj ?truck)))
(:action LOAD-AIRPLANE
:parameters
(?obj
?airplane
?loc)
pprccnndiion
(and (OBJ ?obj) (AIRPLANE ?airplane) (LOCATION ?loc)
(at ?obj ?loc) (at ?airplane ?loc))
:effect
(and (not (at ?obj ?loc)) (in ?obj Pairplane)))
(:action UNLDAD-TRUCK
:parameters
(?obj
?truck
?loc)
:precondition
(and (OBJ ?obj) (TRUCK ?truck) (LOCATION ?loc)
(at ?truck ?loc) (in ?obj ?truck))
:effect
(and (not ?obj ?truck)) (at ?obj ?loc)))
(:action UNLOAD-AIRPLAME
:parameters
(?obj
?airplane
?loc)
:precondition
(and (OBJ ?obj) (AIRPLANE ?airplane) (LOCATION ?loc)
(in ?obj ?airplane) (at ?airplane ?loc))
:effect
(and (not (in ?obj Pairplane)) (at ?obj ?loc)))
(:action DRIVE-TRUCK
:parameters
(?truck
?loc-from
?loc-to
?city)
precondition
(and (TRUCK ?truck) (LOCATION ?loc-from) (LOCATIOM ?loc-to) (CITY ?city)
(at Ptruck ?loc-from)
(in-city ?loc-from ?city)
(in-city ?loc-to ?city))
:effect
(and (not (at ?truck ?loc-from)) (at ?truck ?loc-to)))
(:action FLY-AIRPLANE
:parameters
30

---

Published in Transactions on Machine Learning Research (April/2025) 2025)
(?airplane
?loc-from
?loc-to)
:precondition
(and (AIRPLANE ?airplane) (AIRPORT ?loc-from) (AIRPORT ?loc-to)
(at Pairplane ?loc-from))
:effect
(and (not (at ?airplane ?loc-from)) (at ?airplane ?loc-to)))
[QUERY PROBLEM]
(define (problem logistice-c22-s-pp-a)) (:domain logistics-strips) (:objects a0 a1 c0 c1 t0
10-0 11-0
p0
)
(iinit
(AIRPLANE a0)
(AIRPLANE a1)
(CITY c0)
(CITY c1)
(TRUCK t0)
(TRUCK t1)
(LDCATION 10-0)
(in-city 10-0 c0)
(LDCATION 11-0)
(in-city 11-0 c1)
(AIRPORT 10-0)
(AIRPORT 11-0)
(DBJ p0)
(at to 10-0)
(at t1 11-0)
(at p0 11-0)
(at a0 10-0)
(at a1 10-0)
(: (:goal (and
(at p0 10-0)
[PLAN]
o1 RESPONSE AFLY-AIRPLANE a0 10-0 11-0)
(LOAD-AIRPLAME p0 a0 11-0)
(FLY-AIRPLANE a0 11-0 10-0)
MUNLOAD-AIRPLAN p0 a0 10-0)
M.2 Randomized Logistics Zero-Shot in PDDL
31

---

Published in Transactions on Machine Learning Research (April/202)
The following is PDDL specification planning problem. The first part, under the heading [DOMAI]] is the domain file. The second part, under the heading [QUERY PROBLEM), is the problem file. Using this information, which correct, and further assumptions, find plan which, when run from the specified initial state, satisfies the specified goal. Provide your
sequence of actions in PDDL format. An action ACTION which two objects OBJ1 and OBJ2 would be written (ACTION OBJ1 OBJ2). Do not provide anything else in your answer.
[DOMAIN]
(define (domain dxpflc3up53obvi)
(:requirements :strips)
(:predicates (#46zhuykn6jvvabb ?airport) (abwaon6nhyf9799g ?obj1 ?obj2) (b6e9q1r60aagvden ?city)
(cq14662plyekesok ?airplane) (dvr5gjofrygv9zlx ?obj ?loc) (fe8cgp3y92p61185 ?loc) (
j3bxest71100cyme ?obj ?city) (Iwiiiuk75003u286 ?obj) (rygogthogul9yspo ?truck))
(:action afftsee3ljclsqad
:parameters (?ob_1 ?ob_2 ?ob ?ob_4)
:precondition (and (yvgggthogul9yapo ?ob_ (fe8cgp3y92p61b85 ?ob (fe8cgg3y92p61185 ?ob_3
) (b6e9q4r60gagvden ?ob_4) (dvr5gjofrygveelx ?ob_1 ?ob_2) (Jaxxzst71100cyne ?ob_2 ?ob_4)
(J3bx2st71100cyne ?ob_3 ?ob_4))
:effect (and (not (dvr5qjofrygv9il) ?ob_1 1 ?ob_ ?ob_2)) (dvr5gjofrygv9z1x 7ob_1 ?ob_3))
(:action j4gv801gnu2it0y1
:parameters (?ob_1 ?ob_2 ?ob_3, 3)
:precondition (and (1viiiuk750o3u2B) ?ob_1) (ragggfhogu19yypo ?ob 2) (fe8cgp3y92p61b85 7ob_3
) (dvr5gjoftygv921x rob_2 ?ob_3) (abwaom5nhyt9799g ?ob_1 vob_2))
:effect (and (not (abwaom5mhy1979wg ?ob_1 ?ob_2)) (dvr5gjofrygv92lk ?ob_1 ?ob_3)
(:action tip9ux781pv62umn :parameters (?ob_1 ?ob_2 2 ?ob_3. 3)
:precondition (and (Iviisuk750o3u2Be ?ob_ (cq14062pyyeke3ok ?ob_2) 2) (fe8cgp3y92p61b85 ?ob_3
) (dvr5qjofrygv9zix ?ob_1 1 ?ob_3) (dvr5qjofrygv9aL) Rob_2 ?ob_3))
:effect (and (not (dvr5gjofrygv9zix ?ob_1 ?ob_3)) (abwaomsnhy1979ng ?ob_1 ?ob_2))
(:action tv30k33pzoulq16w
:parameters (?ob_1 ?ob_2 ?ob 3)
:precondition (and (cq14062plyeke3ok ?ob_1) (a46zheyin6jvbabk ?ob_2) (a46zhaykn6jybabk 7ob_3
(dvr5gjofrygv921x ?ob_1 ?ob_2))
:effect (and (not (dvr5gjofrygv921x ?ob_1 ?ob_2)) (dvr5gjofrygv921x ?ob_1 ?ob_3))
(:action u64y1a9apusmsIxb
:parameters (?ob_1 ?ob 2 ?ob 3)
:precondition (and (Iviiiuk75033h28e ?ob_1) 1) (rygggfhogu19yapo ?ob 2) (fe8cgp3y92p61b85 ?ob_3
(dvr5gjofrygv92lx ?ob_2 ?ob_3) (dvz5gjofrygv921x ?ob_1 ?ob_3))
:effect (and (not (dvr5qj ofryggiiryy ?ob_1 ?ob_3)) (abwaom5mhyt9799g ?ob_1 ?ob_2))
(:action wi05amhg7814n006
pparameters (?ob_1 ?ob_2 ?ob_ 3)
precondition (and (1viisuk750o3u28e ?ob_1) (cq14062p1yeke3ok ?ob_ 2) (fe8cgp3y22p61b85 7ob_3
(abvacmmmhyf9979gg Pob_1 ?ob_2) (dvz5gjofrygu921z ?ob_2 ?ob_3))
:effect (and (not (abwaon5mhyt979ng ?ob_1 ?ob_2)) (dvr5gjofrygv921x ?ob_1 ?ob_3))
[QUERY PROBLEM]
(def ine (problem vd24g3dffuion155)
(:domain dxpflc3up53obvi)
(:objects o1 o2 o3 o4 o5 o6 o7 o8 09)
(:init (a462hwykn6jvbbbb) o4) (a462huykn6jvbbbb o7) (b6e9g4r60gagvden o1) (b6e9q4r60gagvden o9)
(cq14o62plyeke3ok o2) (cq14062plyekesok o6) (dvrBgjofrygv92lz o2 o7) (dvr5gjofrygveexx o3
o4) (dvr5qjofrygv921x o4) (dur5gjofrygv921x o7) (dvr5qjofrygv921z o8 o7)
32

---

Published in Transactions on Machine Learning Research (April/2025)
feBcgp3y92p6lles5 o4) (fe8cgp3y32p66b85 o7) (j3bzzst71100cyne 04 o9) (jbbzzat71100cyne o7 (Iwiiiuk75003u28 o5) (rygagohogullyapo o3) (rvgggfhogu1@yypo o8))
(:goal (drr5qjoftygr921x o5 o7))
[PLAN]
o1 RESPONSE
(tv30k33pzzouqq166 o2 o7 o4)
(xip9ur781pr62umn o2 o4)
(tv30k33pzoulq166 o2 04 o7)
(vio6amhq78144006 o2 o7)
N Prompts for Sokoban
N.1 Sokoban Zero-Shor Pero-Shotiin in PDDL
The following PDDL specification of a planning problem. The first part, under the heading DOMAIN] is the domain file. The second part, under the heading [QUERY PROBLEM), is the problem file. Using this information, which is correct, and no further assumptions, find a plan which, when from the specified initial state, satisfies the apecified goal. Provide your as sequence of actions in PDDL format. An action ACTION which acts on two objects 0BJ1 and OBJ2 would written (ACTION OBJ1 0BJ2). not provide anything else your answer. [DOMAIN]
(define (domain typed-sokobbnn (:requirements :typing)
(itypes LOC DIR BOX) (:predicates
(at-robot (at ?o BOX ?1 LOC)
LOC)
(adjacent ?11 LOC ?12 LOC ?d DIR)
(clear ?1 LOC)
(:action
sparameters (?from LOC ?to LOC ?dir DIR) specondition (and (clear ?to) (at-robot ?from) (adjacent ?from ?dir))
reffect (and (at-robot ?to) (not (at-robot ?from)))
(:action push
:parameters (?rloc LOC ?bloc LOC ?floc LOC ?dir DIR ?b BOX)
:precondition (and (at-robot ?rloc) (at ?b ?bloc) (clear ?floc) (adjacent ?rloc ?bloc ?dir) (adjacent ?bloc ?floc ?dir))
:effect (and (at-robot ?bloc) (at ?b ?floc) (clear ?bloc) (not (at-robot ?rloc)) (not (at ?b ?bloc)) (not (clear ?floc)))
[QUERY PROBLEM]
(define (problem sppel==occcbamnrrif///oreee--ll:
(: (:domain typed-sokoban)
(:objects
down left right DIR box0 BOX
f0-0f 40-1f 40-2t 40-3f 40-4f 40-5f f0-6f
f1-0f f1-1f f1-2f f1-3f f1-4f f1-5f f1-6f
33

---

Published in Transactions on Machine Learning Research (ppril/2255)
f2-0f f2-1f f2-2f
12-3f
f2-4f
f2-5f
f2-6f
f3-0f f3-1f f3-2t
f3-3t
f3-4f
f3-5f
f3-6f
f4-0f 14-1f 14-2f
14-3f
14-4f
14-5f
f4-6f
f5-0f f5-1f f5-2f
15-3f
f5-4f
15-5f
f5-6f
f6-0f f6-1f f6-2f f6-3f f6-4f
f6-5f
f6-6f
LOC
(:init
(adjacent f0-0f 10-1f right)
(adjacent f0-0f f1-0f down)
(adjacent f0-1f f0-0f left)
(adjacent f0-1f f0-2f right)
(adjacent f0-1f f1-1f down)
(adjacent f0-2f f0-1f left)
(adjacent f0-2f f0-3f right)
(adjacent f0-2f f1-2f down)
(adjacent f0-3f f0-2f left)
(adjacent f0-3f f0-4f right)
(adjacent f0-3f f1-3f down)
(adjacent f0-4f f0-3f left)
(adjacent f0-4f f0-5f right)
(adjacent f0-4f f1-4f down)
(adjacent f0-5f f0-4f left)
(adjacent f0-5f f0-6f right)
(adjacent f0-5f f1-5f down)
(adjacent f0-6f f0-5f left)
(adjacent f0-6f f1-6f down)
(adjacent f1-0f f1-1f right)
(adjacent f1-0f f0-0f up) (adjacent f1-0f f2-0f down)
(adjacent f1-1f f1-0f left)
(adjacent f1-1f f1-2f right) (adjacent f1-1f f0-1f up) (adjacent f1-1f f2-1f down) (adjacent f1-2f f1-1f left)
(adjacent f1-2f f1-3f right) (adjacent f1-2f f0-2f up) (adjacent f1-2f f2-2f down)
(adjacent f1-3f f1-2f left) (adjacent f1-3f f1-4f right)
(adjacent f1-3f 10-3f up)
(adjacent f1-3f f2-3f down)
(adjacent f1-4f f1-3f left)
(adjacent f1-4f f1-5f right)
(adjacent f1-4f 40-4f up)
(adjacent f1-4f f2-4f down)
(adjacent f1-5f f1-4f left) (adjacent f1-5f f1-6f right) (adjacent f1-5f f0-5f up) (adjacent f1-5f f2-5f down)
(adjacent f1-6f f1-5f left)
(adjacent f1-6f f0-6f up)
(adjacent f1-6f f2-6f down)
(adjacent f2-0f f2-1f right)
(adjacent f2-0f f1-0f up) (adjacent f2-0f f3-0f down)
(adjacent f2-1f f2-0f left) (adjacent f2-1f 12-2f right) (adjacent f2-1f f1-1f up) (adjacent f2-1f f3-1f down)
34

---

Published in Transactions on Machine Learning Research Apr2l/20255
(adjacent f2-2f f2-1f left)
(adjacent f2-2f f2-3f right)
(adjacent f2-2f f1-2f up) (adjacent f2-2f f3-2f down)
(adjacent f2-3f f2-2f left)
(adjacent f2-3f f2-4f right)
(adjacent f2-3f f1-3f up)
(adjacent f2-3f f3-3f down)
(adjacent f2-4f f2-3f left)
(adjacent f2-4f f2-5f right)
(adjacent f2-4f f1-4f up)
(adjacent f2-4f f3-4f down)
(adjacent f2-5f f2-4f left)
(adjacent f2-5f f2-6f right)
(adjacent f2-5f f1-5f up)
(adjacent f2-5f f3-5f down)
(adjacent f2-6f f2-5f left)
(adjacent f2-6f f1-6f up)
(adjacent f2-6f f3-6f down)
(adjacent f3-0f f3-1f right)
(adjacent f3-0f f2-0f up) (adjacent f3-0f 14-0f down)
(adjacent f3-1f f3-0f left)
(adjacent f3-1f f3-2f right)
(adjacent f3-1f f2-1f up)
(adjacent f3-1f f4-1f down)
(adjacent f3-2f f3-1f left)
(adjacent f3-2f f3-3f right)
(adjacent f3-2f f2-2f up) (adjacent f3-2f f4-2f down)
(adjacent f3-3f f3-2f left)
(adjacent f3-3f f3-4f right)
(adjacent f3-3f f2-3f up) (adjacent f3-3f f4-3f down)
(adjacent f3-4f f3-3f left)
(adjacent f3-4f f3-5f right)
(adjacent f3-4f f2-4f up)
(adjacent f3-4f f4-4f down)
(adjacent f3-5f f3-4f left)
(adjacent f3-5f f3-6f right)
(adjacent f3-5f f2-5f up)
(adjacent 13-5f 14-5f down)
(adjacent f3-6f f3-5f left)
(adjacent f3-6f f2-6f up)
(adjacent f3-6f f4-6f down)
(adjacent f4-0f 14-1f right)
(adjacent f4-0f f3-0f up)
(adjacent f4-0f 15-0f down)
(adjacent 14-1f 14-0f left)
(adjacent 14-1f 14-2f right)
(adjacent f4-1f f3-1f up)
(adjacent f4-1f f5-1f down)
(adjacent f4-2f 14-1f left)
(adjacent f4-2f f4-3f right)
(adjacent f4-2f f3-2f up)
(adjacent f4-2f f5-2f down)
(adjacent 14-3f 14-2f left)
(adjacent f4-3f 14-4f right)
(adjacent f4-3f f3-3f up)
35

---

Published in Transactions on Machine Learning Research Aprrl/20255
(adjacent f4-3f 15-3f down)
(adjacent #4-4f 14-3f left)
(adjacent f4-4f f4-5f right)
(adjacent 14-4f f3-4f up)
(adjacent f4-4f f5-4f down)
(adjacent 14-5f 14-4f left)
(adjacent 14-5f 14-6f right)
(adjacent f4-5f f3-5f up)
(adjacent f4-5f f5-5f down)
(adjacent f4-6f f4-5f left)
(adjacent f4-6f f3-6f up)
(adjacent f4-6f f5-6f down)
(adjacent f5-0f f5-1f right)
(adjacent f5-0f f4-0f up)
(adjacent f5-0f f6-0f down)
(adjacent f5-1f f5-0f left)
(adjacent f5-1f f5-2f right)
(adjacent f5-1f 14-1f up)
(adjacent f5-1f f6-1f down)
(adjacent f5-2f f5-1f left)
(adjacent f6-2f f5-3f right)
(adjacent f5-2f 14-2f up) (adjacent f5-2f f6-2f down)
(adjacent f5-3f f5-2f left)
(adjacent f5-3f f5-4f right)
(adjacent f5-3f f4-3f up)
(adjacent f5-3f f6-3f down)
(adjacent f5-4f f5-3f left)
(adjacent f5-4f f5-5f right)
(adjacent f5-4f f4-4f up)
(adjacent f5-4f 16-4f down)
(adjacent f5-5f f5-4f left)
(adjacent f5-5f 15-6f right)
(adjacent f5-5f f4-5f up)
(adjacent f5-5f f6-5f down)
(adjacent f5-6f f5-5f left)
(adjacent f5-6f 14-6f up)
(adjacent f5-6f f6-6f down)
(adjacent f6-0f f6-1f right)
(adjacent f6-0r f5-0f up)
(adjacent f6-1f f6-0f left)
(adjacent 16-1f 16-2f right)
(adjacent f6-1f 15-1f up)
(adjacent f6-2f f6-1f left)
(adjacent f6-2f f6-3f right)
(adjacent f6-2t f5-2f up)
(adjacent f6-3f f6-2f left)
(adjacent f6-3t 16-4f right)
(adjacent f6-31 f5-3f up)
(adjacent f6-4f f6-3f left)
(adjacent f6-4f f6-5f right)
(adjacent f6-4f f5-4f up)
(adjacent f6-5t 16-4f left)
(adjacent f6-5f f6-6f right)
(adjacent f6-5t f5-5f up)
(adjacent f6-6f f6-5f left)
(adjacent 16-6f 15-6f up) (at box0 f1-2f) (clear f0-0f)
36

---

Published in Transactions on Machine Learning Research Aprri/20255)
(clear f0-2f)
(clear f0-3f)
(clear f0-4f)
(clear f0-5f)
(clear f0-6f)
(clear f1-0f)
(clear f1-1f)
(clear f1-3f)
(clear f1-4f)
(clear f1-5f)
(clear f1-6f)
(clear f2-0f)
(clear f2-1f)
(clear f2-2f)
(clear f2-3f)
(clear f2-4f)
(clear f2-5f)
(clear f2-6f)
(clear f3-0f)
(clear f3-3f)
(clear f3-4f)
(clear f3-5f)
(clear f3-6f)
(clear f4-0f)
(clear f4-1f)
(clear f4-2f)
(clear f4-3f)
(clear f4-4f)
(clear f4-5f)
(clear f4-6f)
(clear f5-0f)
(clear f5-1f)
(clear f5-2f)
(clear f5-3f)
(clear f5-4f)
(clear f5-5f)
(clear f5-6f)
(at-robot f6-0f)
(clear f6-0f)
(clear f6-1f)
(clear f6-2f)
(clear f6-3f)
(clear f6-4f)
(clear f6-6f)
(:goal
(and
(at box0 f6-6f)
[PLAN]
passereessss--rrrriier RESPONSE
(move f6-0f f5-0f up)
(move f5-0f 14-0f up)
(move f4-0f 13-0f up)
(move f3-0f f2-0f up)
(move f2-0f f1-0f up)
37

---

Published in Transactions on Machine Learning Research (April/2255)
(move f1-0f f1-1f right)
(push f1-1f f1-2f f1-3f right box0)
(push f1-2f f1-3f f1-4f right box0)
(push f1-3f f1-4f f1-5f right box0)
(push f1-4f f1-5f f1-6f right box0)
(move f1-5f f0-5f up)
(move 10-5f 10-6f right)
(push f0-6f f1-6f f2-6f down box0)
(push f1-6f f2-6f f3-6f down box0)
(push f2-6f f3-6f f4-6f down box0)
(push f3-6f 14-6f f5-6f down box0)
(push f4-6f f5-6f f6-6f down box0)
N.2 Sokoban Backprompt
The above plan invalid. The following action at step has an unmet precondition:
push f8-5f f8-6f f8-7f right box1
The unmet precondition is: (clear f8-7f)
Provide your revised plan as sequence of actions in PDDL format. An action ACTION which acts on two objects OBJ1 and OBJ2 would be written (ACTION DBJ1 0BJ2). Do not provide anything else in your answer.
[PLAN]
O Prompts for Graph Coloring
0.1 Graph Coloring Hard
Color the following graph, described of edges, such that vertices the edge share a color.
You may
most 5 colors.
Vertex 0 is connected to vertex 3.
Vertex is connected vertex Vertex 0 is connected to vertex 8.
Vertex is connected vertex Vertex 0 is connected to vertex 15.
Vertex is connected to vertex
Vertex 0 is connected to vertex 19.
Vertex is connected to vertex 4.
Vertex 1 is connected to vertex 5.
Vertex 1 is connected to vertex 8,
Vertex 1 is connected to vertex 11.
Vertex 1 is connected to vertex 12.
Vertex is connected vertex Vertex 1 is connected to vertex 14.
Vertex is connected vertex
Vertex 1 is connected to vertex 18.
Vertex is connected to vertex 19.
Vertex 2 is connected to vertex 3.
Vertex is connected to vertex 4.
Vertex 2 is connected to vertex 6.
Vertex 2 is connected to vertex 8.
Vertex 2 is connected to vertex 13.
Vertex 2 is connected to vertez 14.
Vertex is connected vertex 15.
Vertex 2 is connected to vertez 16.
Vertex is connected
17.
38

---

Published in Transactions on Machine Learning Research (April/2025)
Vertex 3 is connected vertex 5.
Vertex 3 is connected to vertex 8.
Vertex 3 is connected to vertex 11.
Vertex 3 is connected to vertex 12.
Vertex is connected vertex 14.
Vertex 3 is connected to vertex 16.
Vertex is connected to vertex
Vertex 4 is connected to vertex 7.
Vertex is connected to vertex 9.
Vertex 4 is connected to vertex 11.
Vertex is connected to vertex 17.
Vertex 4 is connected to vertex 18.
Vertex 5 is connected to vertex 11.
Vertex 5 is connected to vertex 14.
Vertex 5 is connected to vertex 15.
Vertex is connected vertex 11.
Vertex 6 is connected to vertex 16.
Vertex is connected vertex
Vertex 7 is connected to vertex 9.
Vertex is connected to vertex
Vertex 7 is connected to vertex 13.
Vertex is connected to vertex 14.
Vertex 7 is connected to vertex 16.
Vertex 7 is connected to vertex 17.
Vertex 8 is connected to vertex 10.
Vertex 8 is connected to vertex 12.
Vertex 8 is connected vertex 13.
Vertex 8 is connected to vertex 16.
Vertex is connected
Vertex 9 is connected to vertex 11.
Vertex is connected to vertex Vertex 9 is connected to vertex 17.
Vertex 10 connected to vertex 11.
Vertex 10 is connected to vertex 13.
Vertex 10 connected to vertex 16.
Vertex 10 is connected to vertex 17.
Vertex 10 connected to vertex 18.
Vertex 10 connected vertex 19.
Vertex 11 is connected to vertex 13.
Vertex 11 connected vertex 17.
Vertex 11 is connected to vertex 18.
Vertex 11 connected to vertex 19.
Vertex 12 is connected to vertex 13.
Vertex 12 connected to vertex 15.
Vertex 12 is connected to vertex 16.
Vertex 12 connected to vertex 18.
Vertex 12 is connected to vertex 19.
Vertex 13 connected to vertex 14.
Vertex 13 is connected to vertex 18.
Vertex 14 is connected to vertex 15.
Vertex 14 connected to vertex 18.
Vertex 14 is connected to vertex 19.
There total of 20 vertices. Please label every vertex, if disconnected
reat of the graph. Please provide each vertex's color. Do not skip any vertices. Each color must be provided
line in the response and should be formatted "(VERTEX NUMBER): {VERTEX COLOR ASSIGNMENT)". Please do not provide anything else in your response, and end your response with [ANSWER END] wtw.:esss=oo--peeeee.i RESPONSE==2E
0:
39

---

Published in Transactions on Machine Learning Research AAprl/22255
1: 3
2: 1
3: 2
4: 4
5
6: 3
7:
8: 4
9: 3
10:
11:
12:
13: 2
14:
15: 2
16:
17: 2
18:
19: 2
[ANSWER END]
P Graph Coloring Backprompt - Hard
This is incorrect. Feedback:
Vertex was not given value the coloring. Vertex 1 was not given value in the coloring. Vertex was not given value the coloring. Vertex 2 was not given value in the coloring. Vertex 0 was not given value in the coloring.
Vertex 3 not given value the coloring.
Vertex 0 was not given value in the coloring. Vertex 4 not given value the coloring.
Vertex 0 was not given value in the coloring. given value in the coloring. Vertex 0 was not given value in the coloring. coloring.
Vertex 10
Vertex 12 was not given a value in Vertex 0 was not given value in the coloring. Vertex 13 was not given a value in the coloring. Vertex 0 was not given value in the coloring. Vertex 15 was not given a value in the coloring. Vertex 0 was not given value in the coloring. Vertex 16 was not given a value in the coloring. Vertex 0 not given value in the coloring. Vertex 17 was not given a value in the coloring. Vertex
given value the coloring.
Vertex 18 was not given a value in the coloring. Vertex not given value the coloring. Vertex 2 was not given value in the coloring. Vertex was not given value the coloring. Vertex 3 was not given value in the coloring. Vertex was not given value the coloring. Vertex 8 was not given value in the coloring. Vertex 1 was not given value in the coloring.
Vertex 9 was not given value in the coloring.
Vertex 1 was not given value in the coloring. Vertex 10 not given value in the coloring. Vertex 1 was not given value in the coloring. given value in the coloring.
Vertex 15
40

---

Published in Transactions on Machine Learning Research (April/2255)
Vertex not given value the coloring. Vertex 18 was not given a value in the coloring. Vertex 2 not given value in the coloring. Vertex 3 was not given value in the coloring. Vertex
given value the coloring. Vertex 5 was not given value in the coloring. Vertex was not given value the coloring. Vertex 6 was not given value in the coloring. Vertex was not given value the coloring.
Vertex 8 was not given value in the coloring.
Vertex 2 was not given value the coloring.
Vertex 11 was not given a value in the coloring.
Vertex 2 was not given a value in the coloring. given value in the coloring. Vertex 2 was not given a value in the coloring. given value in the coloring. Vertex 2 was not given a value in the coloring. Vertex 15 was not given a value in
Vertex 12 was
Vertex 14
coloring. Vertex 2 was not given value in the coloring. Vertex 19 was not given a value in the coloring. Vertex 3 was not given value in the coloring. Vertex 18 was not given a value in the coloring.
Vertex 3 was not given value in the coloring.
Vertex 19 was not given a value in the coloring.
Vertex 4 was not given value in the coloring.
Vertex 6 was not given value in the coloring.
Vertex 4 not given value the coloring.
Vertez 7 was not given value in the coloring. given value the coloring. Vertex 9 was not given a value in the coloring. Vertex was not given value the coloring. Vertex 11 was not given a value in the coloring. Vertex was not given value the coloring. Vertex 13 was not given a value in the coloring. Vertex was not given value in the coloring. Vertex 18 was not given a value in the coloring. Vertex was not given value the coloring. Vertex 19 not given value in the coloring. Vertex 5 was not given a value in the coloring.
Vertex
Vertex 6 not given value the coloring.
Vertex 5 was not given value in the coloring. Vertex was not given value the coloring. Vertex 5 was not given value in the coloring. Vertex was not given value the coloring. Vertex 5 was not given value in the coloring. Vertex 10 was not given a value in the coloring. Vertex 5 was not given value in the coloring. Vertex 14 was not given a value in the coloring. Vertex 6 was not given value in the coloring. Vertex 7 was not given value in the coloring. Vertex not given value the coloring. Vertex 9 was not given value in the coloring. Vertex not given value the coloring. Vertex 10 was not given a value in the coloring. Vertex
given value the coloring.
Vertex 12 was not given a value in the coloring. Vertex was not given value the coloring. Vertex 13 was not given a value in the coloring. Vertex was not given value the coloring.
41

---

Published in Transactions on Machine Learning Research (April/202)) 2025)
Vertex 14 not given value in the coloring. Vertex 6 was not given a value in the coloring. Vertex 15 not given value in the coloring. Vertex 6 was not given a value in the coloring. Vertex 16
given value in the coloring. Vertex 6 was not given value in the coloring. Vertex 17 was not given a value in the coloring. Vertex 6 was not given value in the coloring. Vertex 19 was not given a value in the coloring.
Vertex 7 was not given value in the coloring.
Vertex 8 was not given value in the coloring. Vertex 7 was not given value in the coloring. Vertex 9 was not given a value in the coloring. Vertex 7 was not given value in the coloring. Vertex 11 was not given a value in the coloring. given value the coloring. Vertex 12 was not given a value in the coloring. Vertex 7 was not given value the coloring. Vertex 13 was not given a value in the coloring. Vertex was not given value the coloring. Vertex 14 was not given a value in the coloring. Vertex was not given value the coloring. Vertex 15 was not given a value in the coloring. Vertex 7 was not given a value in the coloring. Vertex 18 was not given value in the coloring. Vertex 8 was not given a value in the coloring. Vertex 11 not given value in the coloring. Vertex 8 was not given a value in the coloring. given value in the coloring. Vertex 8 was not given a value in the coloring. Vertex 14 was not given a value in the coloring. Vertex 8 was not given value in the coloring. Vertex 19 was not given a value in the coloring. Vertex 9 was not given value in the coloring. Vertex 10 was not given a value in the coloring. Vertex 9 was not given value in the coloring. Vertex 11 was not given a value in the coloring. Vertex 9 not given value in the coloring. Vertex 13 was not given a value in the coloring. Vertex
Vertex 7
Vertex 13
given value the coloring. Vertex 14 was not given a value in the coloring. Vertex was not given value the coloring. Vertex 15 was not given a value in the coloring.
Vertex 10 was not given a value in the coloring.
Vertex 12 was not given a value in the coloring.
Vertex 10 was not given a value in the coloring.
Vertex 16 was not given value in the coloring.
Vertex 10 was not given a value in the coloring.
Vertex 17 was not given value in the coloring.
Vertex 11 was not given a value in the coloring.
Vertex 12 not given value in the coloring.
Vertex 11 was not given a value in the coloring.
Vertex 13 was not given value in the coloring.
Vertex 11 was not given a value in the coloring. given value in the coloring.
Vertex 14 was
Vertex 11 was not given a value in the coloring. Vertex 15 was not given a value in the coloring. Vertex 11 was not given value in the coloring. Vertex 17 was not given a value in the coloring.
42

---

Published in Transactions on Machine Learning Research (April/202)) 2025)
given value in the coloring. Vertex 19 was not given a value in the coloring.
Vertex 11
Vertex 12 not given value in the coloring.
Vertex 13 was not given a value in the coloring. given value in the coloring. Vertex 18 was not given a value in the coloring.
Vertex 12
Vertex 13 was not given a value in coloring.
Vertex 17 was not given value in the coloring.
Vertex 14 was not given value in the coloring.
Vertex 19 was not given value in the coloring.
Vertex 15 was not given a value in the coloring.
Vertex 19 was not given value in the coloring.
Vertex 16 was not given a value in the coloring.
Vertex 17 not given value in the coloring.
Vertex 17 was not given a value in the coloring. given value in the coloring.
Vertex 18
Using this feedback, please try again. Please provide each vertex's color. Do not skip any vertices. Each color must be provided on
line the response should formatted as "(VERTEX NUMBER): (VERTEX COLOR ASSIGNMENT)". Please do not provide anything else in your response, and end your response with [[ANSWER END]
Q Prompts for OSU Travel Planning
Q.1 First Iteration
You are proficient planner. Based on the provided information and query, please give detailed plan, including specifics such as flight numbers (e.g., F0123456),
restaurant names, and accommodation names. Note that all the information in your plan should be derived from the provided data. You should give a travel plan in JSON format as shown in the example below. Additionally, all details should align with that information is unnecessary. For example, in the provided sample, you do not need to plan after returning the departure city. When you travel two cities in day, you should note it in the 'current_city section as in the example (i.e., from to E).
The symbol -- indicates
***** Example *****
Query: Could you create a travel plan for people from Ithaca to Charlotte spanning days, from March 8th to March 14th, 2022, with budget of $30,200?
Travel Plan:
[
"day": 1,
"people_ number": 7,
"current_city" "from Ithaca Charlotte",
"transportation" "Flight Number: F3633413, from Ithaca to Charlotte, Departure Time: 05:38, Arrival Time: 07:46",
"breakfast"; "Nagaland's Kitchen, Charlotte", "attraction": "The Charlotte Museum of History, Charlotte", "lunch": "Cafe Maple Street, Charlotte",
"dinner"; "Bombay Vada Pav, Charlotte", "accommodation": "Affordable Spacious Refurbished Room in Bushwick!, Charlotte"
{
"day": : 2,
"people_mumber""
number": 7,
"current, "current_city"" "Charlotte",
"transportation": "_",
"breakfast"; "Dlive Tree Cafe, Charlotte",
"attraction": "The Mint Museum, MharlotteRRomar Bearden Park, Charlotte",
43

---

Published in Transactions on Machine Learning Research (April/2025)
"lunch"; "Birbal Ji Dhaba, Charlotte", "dinner": "Pind Balluchi, Charlotte", "accommodation": "Affordable Spacious Refurbished Room in Bushwick!, Charlotte"
{
"day": 3,
"people_mume number": 7,
"current_ :ity": "from Charlotte to Ithaca",
"transportation": "Flight Number: F3786167, from Charlotte to Ithaca, Departure Time: 21:42, Arrival Time: 23:26",
"breakfast": "Books Monument, Charlotte", "attraction": "Books Monument, Charlotte", "Lunch"; "Olive Tree Cafe, Charlotte", "dinner"; "Kylin Skybar, Charlotte", "accommodation":
}
Example Ends
Given information: [[ DeDeecriptio'' 'Attractions in Nashville', 'Cnttnt'; Address
Name Latitude Longitude
Phone
Website
CitylnCountry Music Hall of Fame and Museum 36. 158263 -86. 776126 222 Rep. John Lewis Way S, Nashville, TN 37203, USA (615) 416-2001
org/ Nashville'n
Nashville Zoo at Grassmere 36, 089705
-86. 742096
3777 Nolensville Pk, Nashville, TN 37211, USA (615) 833-1534
http:
org/ Nashville'n Belle Meade Historic Site Winery 36. 104916 -86 86.864695
5025 Harding Pike, Nashville, TN 37205, USA (615) 356-0501
https://viiittbellemeeddd com/ Mashville\n
Johnny Cash Museum 360939 -86. -75777777
119 3rd Ave S, Nashville, TN 37201, USA (615) 256-1777
Centennial Park 148946 -86.812750 2500 West End Ave, Nashville, TN 37203, USA (615) 862-8400
com/ Nashville'n
oatrao///arrrrc/Ceeeeamaaaaaaa aepp 600 Opry Mills Dr, Nashville, TN
aspx Nashville\n
Grand 0le Opry 36.206857 -866692100 37214, USA (615) 871-6779
https://uuu.or com/ utm bouddeeencetoouuuugooggooeeebeee Nashville'n TM 37203, USA (615) 244-3340
Baan//utm_camppee
www.tett-bbrnndtttrrggogggeeeeee sateee_yy__uusse syy_buusiesss businees_websiteee wewsite_beeeeeeee
Frist Art Museum 157897 1866783855
919 Broadway, Nashville,
https://tristarttmast org/ Naahville'n
The Parthenon 36. 149674 883347747
2500
Weat End Ave, Nashville, TK 37203, USA (615) 862-8431
https://uuv www..ashwilleparthennn.... com/ Nashville'n
Nashville Shores 36. 158209 -06.604915
4001 Bell Rd, Hermitage, TN 37076, USA (615) 889-7050
r com/lodgig Nashville'n Musicians Hall Fame and Museum 36. 167668 401 Gay St, Nashville, TN 37219, USA (615) 244-3263
www.naashiileehores... wtt.aorres.ommoooiiiin
86. 782399
http:://www.iiiiiiccccaalal www.:mmmiiiiaaccaloofaa.... com/ Washville'n Bicentennial Capitol Mall State Park 170887
-86. 787589 600 James Robertson Pkay, Nashville, TN 37243, USA (888) 867-2757
Nashville'n 734580 702 Murfreesboro Pike, Mashville, TN 37210, USA (615) 742-7445
Lane Motor Museum 36. 140197
http://wwv. http://www.lenemttomuueeeee org/ Nashville'n
Honky Tonk Highway 36. 160393 -8..778433
501 Broadway, Nashville, 37203, USA (800) 657-6910
http:///oooorisstoos/ooooio/eeee htth//ooooooooohaaaayy Nashville\m Adventure Science Center 36. 146614 775471 Fort Negley Blvd, Nashville, TN 37203, USA (615)
862-5160
http://www.. adventuresci. org/ Mashville\n
Patsy Cline Museum 160904 -86.776030
119 3rd Ave S, Nashville, TN 37201, USA (615) 454-4722 com/ Nashville\n Andrew Jackson's Hermitage 213757 -86. 615310 4580 Rachels Ln, Hermitage, TN 37076, USA (615) 889-2941
44

---

Published in Transactions on Machine Learning Research (April/2025)
http:///ww... http:///ww..hhhhrmittaag com/ Nashville'n
NashTrash Tours 36. 169474 -86. 788254 900
Rosa Parks Blvd, Nashville, IN 37208, USA (615) 226-7300
www.nnahhhhhhhcccmmm com/ Nashville'n
Fort Nashborough 36. 164268 -86 775430
170 1st Ave N, Nashville, TK 37201, USA (615) 862-8400
https://www.nashvlll www.:/aaarillee.o//artte-----eee
aapx Mashville\n
Ryman Auditorium 36. 161248 -86. 778471
116 5th Ave N,
Nashville, TN 37219, USA (615) 889-3060
https://www..e rywan. com/tutm http://tww.ccmmppaaparnaaantttte
utm htg.aaaeeeeettugooooeoeeeeoaaana
_gwogleeomyy.o _ business www.neesswebbiteeee.eeeeeeeeeeee wtt__eebteeeeeeeeeee
Nashville'n Mashville Public Square Park 36. 166611 778126 Union St & 3rd Ave N, Nashville
TM 37201, USA (615) 743-3090
https://ww http:://ww..sshbrlleeerrrs hwtpi/tttonnncc/////e/eeo
in Nashville', 'Content': wwtp./////pbblliccuuuueeeeeeeeee
Nashville'), ('Description' 'Restaurants
Name Average Cost Bangkok 1
Cuisines Aggregate Rating City\n Cafe, Seafood
44
3.3 Nashville'n
Bablu Fast
Food
13
Desserts, Pizza, Mexican, BBQ, Fast Food
0.0 Nashville'n
Full Dabba
77 Pizza, Mexican, Fast Food, Cafe, American
0.0 Nashville'n
Twigly
14 Pizza, French, Fast Food, Chinese,
Seafood
4.5 Nashville'n
Veg Hut
35
Tea, Mexican, Seafood
0.0 Nashville'n Desserts, Fast Food
The Toddy Shop
92
3.4 Nashville'n
GoGourmet
34
Tea, Cafe, Indian, Mediterranean 15 Kitchen King
3.7 Nashville'n Tea, Bakery, Mediterranean, Fast Food
Govinda's Confectionery
3.1 Nashville'n Mediterranean
13 Pizza, Italian, BBQ, Cafe,
0.0 NashvilleAn
Town Hall
55
Tea, Seafood
3.8 Nashrille'n
Smoke House Deli
59
Cafe
Mediterranean, Seafood
4.0 Nashville'n
Dialogue
Lounge & Caf
83 Tea, Indian, BBQ, Cafe, American, Seafood
3.0 Nashville'n
Meenakshi Bhawan
24
Tea,
Desserts
3.1 NashvilleIn
Oh! Calcutta
83
Tea,
Mexican, BBQ, Cafe, Indian
4.4 Nashville'n Tea, French, Pizza, BBQ
Chicago
Pizza
20
3.2 Nashville
Malhotra Restaurant
76
Tea, Desserts, Seafood
2.9 Nashville\n
Madras Cafe
88
Pizza, Desserts, Seafood
0.0 Mashville\nDakehii
Sheraton New Delhi Hotel
63
Cafe, Pizza, Indian, Bakery
4.0 NashvilleIn
1911 Bar The Imperial
76
Desserta, Pizza, Italian, American, Seafood
3.2 Nashville'n
Punjabi Virsa
35 Sagar Dhaba
Desserts, Italian, BBQ, Cafe, Seafood
0.0 Nashville\a
55
Tea, Bakery, BBQ, Cafe,
Mediterranean
0.0 Nashville\n
Kettle Kegs
60
Tea, Seafood
0.0 Nashville'n
Kargo Description':
Tea, Pizza, Desserts
4.4 Nashville"), {
'Accommodations in Nashville', 'Content': NAME price room type review rate number city\nHuuee 2 Bedroom, Great Location, Express Metro
house_rules minimum nights maximum occupancy
745.0 Entire home/apt No smoking children under 10 pets 4.0
4
5.0 Nashville\n Clean and large bedroom in private house 474.0
Private room
No smoking
1.0
4.0 NashvilleAn
Brooklyn Heights gem 993.0 Entire
home/apt
No pets
2.0
5.0
Mashwille
Lovely in heart Williamsburg 61.0 Private
No pets
2.0
4.0 Nashville'n *Light Love* vibrant,
historic, sleeps 679.0 Entire home/apt
No pets
5.0
2
2.0 Nashville'n
FiDi Cozy room overlooking East River 870.0
Private 5.0 Nashville'n Charming 1BR with sun-nook in Brooklyn 256.0 Entire home/apt No children under 10 pets & No smoking 3.0
No parties
1.0
2
4.0 NashvilleAn
Cozy bedroom close to Manhattan 576.0 Private room
No
children under 1.0
3.0
45

---

Published in Transactions on Machine Learning Research (April/2025)
Nashville'), ('Description'; 'Attractions Knoxville', 'Content';
Name Latitude Longitude
Address
Phone
Website
Cityln
World's Fair Park 36.96777 93.922192
525 Henley St, Knoxville, TN 37902, USA (865) 215-1158
http://oordsstairpakk.ogg
Knoxville'nn
Knoxville Museum of Art 35, 962426 -83. 035229 1050 Worlds Fair Park Dr, Knoxville, TN 37916, USA (865) 525-6101
http://sww.knoxee org/ Knoxville'n
Sunsphere 35.961701 923353355
810 Clinch Ave, Knoxville, 37902, USA (865) 314-0660
com/ Knoxville'n
Ijams Nature Center 35. 956454
-83.866775
2915 Island Home Ave, Knoxville, IN 37920, USA (865) 577-4717
http://ww..1jam org/ Knoxville'n
Knorville Walking Tours 35. 966448 -83.919167
301 S Gay St, Knoxville, TM 37902, USA (865) 309-4522
http://mmorrillewakinngge com/ Knoxville'n
Muse Knoxville 35.997617
-83.885467
516 N Beaman St, Knoxville, TN 37914, USA (865) 594-1494
http:://wwttemmmeehooxeee thwmaseknozville.coomo
Knoxville'n Knoxville Botanical Garden and Arboretum 35.982160
-83.881077 2743 Wimpole Ave, Knoxville, TN 37914, USA (865) 862-8717
http://wwu kww.gararr..n.nn org/ Knozville'n
Haunted Knexville Ghost Tours 35.965963 -83. 919553
36 Market Square #1404, Knoxville, TN 37902, USA (865) 377-9677
http://wnt http://sww.hanttennorriii... hwwntedknoxville.cokkkk
Knoxville\n
Three Rivers Rambler 35.952810
-83. 940376 2560 University Commons Way, Knoxville, TN 37919, USA (865) 524-9411
/ttp:///uu... www..threeriveee. wereeerrivrrrrmbeer... com/ Knoxville'n
Charles Krutch Park 35. 964414 504 Market St, Knoxville, TN 37902, USA (865) 215-4248
83.918895
attps://wver wwwpkkkoxvilltt. hww..kaww.lllete.og//rreeaaaaeaa pyooverment/ccity_.aparttmaa
offices/parks _andd____________ ret.aercccrettii///prea//////
krutch_ mautch_park Knoxville\a McClung Museum of Natural History Culture 35. 952005 83.927209
1327 Cir Park Dr, Knoxville, TN 37996, USA (865) 974-2144
http://accllngmuseum.uccc
Knozville\n
Knoxville Sightseeing 35.992799 904128
2519 Mitchell St, Knexville, 37917, (865)
566-0634
Unknown Knoxville\n
Chilhowee Park & Exposition Center 35. 996299 -83.884033
3301
Magnolia Ave, Knoxville, TW 37914, USA (865) 215-1450
https://chiihooeepakkk com/ KnoxvilleAn
Augusta Quarry 35. 944772 -83. 33.911886
3000 Fort Dickerson Rd SW, Knoxville, TK 37920, USA Unknown Unknown Knoxville\n
Zoo Knoxville 35.9 999812 -83, 888250
3500 Knoxville
Zoo Dr, Knoxville, IN 37914, USA (865) 637-5331
http///www.ooooovilleeeee org/ Knoxville(n Outdoor Knoxville Adventure Center 361962
-83. 012.912663 900 Volunteer Landing Ln, Knoxville, TN 37915, USA (865) 228-8424
http://w...ttiookooortea www.ouutoorkknoovill. com Knoxville(n
James White Fort Association 35. 962962
-83.912169
205
Ave, Knoxville, TN 37915, USA (865) 525-6514
http://jannsshhtessttrtt org/ Knoxville'a
Rowing Man Statue in Knoxville, TN 35. 963611 -83.917519
W. Church Avenue &, S Gay St, Knoxville, 37902, USA Unknown
Unknown Knoxville'n
UT Gardens Knoxville 35. 944000000 -83.938260
2518
Jacob Dr, Knexville, 37996, (865) 974-7324
tennessee.
KnoxvillelnEast Tennessee Historical Society and 601 Gay St, Knoxville, 37902, USA (865)
Museum 35.9 35.964117 -83. 917794
215-8830
org/ Knoxville"), ('Description': 'Restaurants in Knoxville', 'Content': Cuisines Aggregate Rating Cityln French, BBQ, Desserts, Seafood Desserts, Pizza, Italian, Cafe, Indian Tea,
Name Average Cost
Cafe Arabelle
29
3.6 Knoxville\n
Les Brasseurs
24
4.6 Knoxville'n
Sky On 57 3.4 KnoxvilleAn
150
Cafe, Pizza, Chinese, Seafood
Cafe El Chico
Tea, French, Indian, Seafood Tea, Pizza, Mexican, Fast Food, Cafe Tea, Cafe, Pizza, Desserts
3.6 KnoxvilleAn Knoxville
TcozY
Vaishnav Bholaalaya
46
46

---

Published in Transactions on Machine Learning Research (April/2025)
3.0 Knozville'n
Burger Planet
45
82 Tea, Bakery, American, Cafe Fast
3.2 Knoxville'n
Cafe Coffee Day
Food, American, BBQ, Italian
2.8 Knoxville'nn
Mamagoto
14
Indian, Mediterramean, Deaserts, Seafood
4.1 Knoxville'n Pizza, Desserts, Fast Food
Ali Baba & 3.5 Knoxville\n
41 Dishes
70
Punjabi Flavour
41
Tea, Bakery, Cafe, Indian, Seafood
0.0 Knoxville'n Tandoori Tadka
23
Chinese, Pizza, Desserts, Fast Food 35 Desserts,
0.0 Knoxville'n
Coalition Cafe
Tea, BBQ, Cafe, Mediterranean
3.4 Knoxville'n
Chaat Corner
Cafe, Pizza
3.2 Knoxville\n
Posto
Tea, Cafe, Pizza, BBQ
3.5 Knoxville\n
La-Nawaab
17
French, Bakery, Cafe, Seafood Chinese, Pizza, Cafe, Italian
3.0 Knoxvillekn
Peppers Pipes
86
3.1 Knoxville'n
Chit Chat
15 Anand Sweets
Pizza, Fast Food
3.1 Knoxville'n
45
Fast Food,
Pizza, Indian, Seafood
0.0 Knoxville'n
Open Kitchen
13 Pizza, Bakery, BBQ, Cafe, Indian, Mediterranean
3.0 Knoxville'n
Biryani By Kilo
Chinese, Pizza, Fast Food
4.1 Indian, Desserts, Fast Food
Knoxville'n Chinese Food Corner
30
Knoxville\nThe Indian Kaffe Express
Tea, Desserts
3.8 Knoxville'), ('Description': "Accommdaations in Knoxville', 'Content'; NAME price room type
house
house_rules rules minimum nights maximum occupancy review rate number city Cozy Private Room in Chinatown/ Lower East Side 132.0 Private room
No pets &
No visitors
7.0
2
4.0 Knoxville'n
The Diamond Room 1008.0 Private room
No parties Light-filled
No visitors
1.0
5.0 Knoxville'a
Room Renovated Apt 310.0 Private
No pets
2.0
2
2.0 Knoxville'n Private room
Private Room 922.0
No
visitors
1.0
4.0 Knoxville'nn
Beautiful
Private Manhattan Room 721.0 Private room
No parties & No
smoking
1.0
Lg Quiet Artist No visitors No pets & No
2.0 KnoxvilleAn
Home -Ditmas Park 561.0 0 Entire home/apt
smoking
24.0
2.0 Knoxville\n Beautiful 1- bdrm apt in
tranquil Inwood building 1091.0 . Entire home/apt
No smoking &
No pets
4.0 Knoxville'n 1,600sq ft modern duplex in harlem brownstone 1159.0 Entire home/apt No amoking No children under 10 &
2.0
4
No pets
2.0
2.0 Knoxville'n visitors children
Brooklyn Sunny room
min subway 793.0 Private room
under 10
2.0
2
5.0 KnoxvilleAn Amazing Large Sunny Studio in parties
Greenwich Village 144.0 Entire home/apt
smoking
30.0
3
4.0 Knozville\n Private Bdrm Suite in No
Historic Brownstone 479.0 Private room
visitors
2.0
2
2.0 KnoxvilleAn Charming bedroom with huge
terrace in Greenpoint 712.0 Private room No parties
& No children under & No
visitors
2.0
1
3.0 Knoxville'n
Huge
quiet modern casis 1165.0 Eatire home/apt
No smoking & No
parties
2.0
2.0 KnoxyilleNnPrivate large
LGA
airport with queen bed 552.0 Private room
No pets
2.0 Knoxville\n
east 86th Steet Upper No
East Side Building 632.0 Entire home/apt
smoking
30.0
2.0 KnoxvilleAn Spacious Garden Apt in
Clinton Hill/Ft Greene 1006.0 Entire home/apt No children under 10 & No visitors & No
smoking
3.0
3.0 Knoxville\n
LES Penthouse 993.0 0 Entire home/apt
No children under
10 No
47

---

Published in Transactions on Machine Learning Research (April/2025)
parties
2.0
4.0 Knoxville'), ('Deacription': 'Attractions
in Chattanooga', 'Content';
Name Latitude Longitude
Address
Phone
Website City\n
The Chattanooga Zoo at Warner Park 35.042632 -85.282000 301 Holtzclav Ave, Chattanooga, 37404, USA (423) 697-1322
http://wwn h///t/ccaattooooooo chattzoo... org/ Chattanooga'n
Rock City Gardens
34.9788 9733999 -50177777
1400 Patten Rd, Lookout Mountain, 30750, (706)
820-2531
http://seerockkcity com/ com/Putm_ssurces source=
gabbutm_medeeeee mediun-organic Chattanooga'n
Tennessee Aquarium 15.055823 311065
Broad St, Chattanooga, TN 37402, USA (423)
265-0695
http://wwu..e tnaqua. org/ Chattanooga'n
http://ww.tnqqa..ooom
Lookout Mountain Incline Railway 3917 St Elmo Ave, Chattanooga, TN 37409, USA (423)
35. 009546 328564
821-4224
http://www.
ridetheincline.. com/ Chattanooga'n
Coolidge Park 35. 060506 -85 306857
150 River St, Chattanooga, TN 37405, USA (423)
643-6311
Chattanooga
Creative Discovery Museum 35. 052700 -85.312280 321 Chestnut St, Chattanooga, TN 37402, USA (423) 756-2738
http://wwv.
cdmfun org/ Chattanooga
Chattanooga Ghost Tours Inc 35.051231 -85.309055
57 E 5th St, Chattanooga, TN 37402, USA (423) 800-5998
httpe://chattanoggggotttte comtt//ussccomo///oo. Chattanooga'n
Ross's
Landing 35.056741 -8..310303 201 Riverfront Pkwy, Chattanooga, TK 37402, USA (423) 643-6311
http://wwet http://ww..aaaaaaoo..///p///ppb/
Chattanoogaln Hunter Museum of American Art 35.055894 306405
pf
10 Bluff View Ave, Chattanooga, 37403,
USA (423) 267-0963
http///wwhwntrrussuu.....m org/ Chattanooga
Walmut Street
Bridge 35.058333 307.367296 (423) 643-6096
1 Walnut St, Chattanooga, TN 37403, USA
http://www.chattanoog. gow/pabllic-oork//prrrr
darrettor-dd--pa---fciiitt Chattanooga
Tennessee Riverpark 35. 095938 -85. 245311
4301 Amnicola Huy, Chattanooga, TN 37402, USA (423)
209-5370
http://parke. hwwiltontn. gov/ Chattanooga'n
Moccasin Bend National Archeological District 35.053022 328626 301 Manufacturers Rd, Chattanooga, TN 37405, USA (423) 648-5623
http:/parrkk.hammiitnnnnnn
attps://www.pp mps. wwwpe//ee//eeeeer/aooottraatttee htm Chattanooga'n
Ruby Falls 35. 35.019084 -85.3333 339355
1720 Scenic Hwy, Chattanooga, TN 37409, USA (423) 821-2544
http://www.. http://w.uufllllllae// com/ Chattanooga\n
Cravens House
35.0133 013781 -85. 341495 Point Park Visitor Ctr, Lookout Mountain, TN 37350, USA
(423) 821-7786
https://ww.nnt http::////nnpps.o////// 430 E St, Chattanooga, 37403,
places/
placess/cravenchousel.ooo www.e/eooseeeoo...omooooo htz Chattanoogalu
Williams Echo Dome 35. 054215 -85. 304836 Unknown
Unknown Chattanooga
Reflection Riding Arboretum & Nature Center 35.010130
885.664585
400 Garden Rd, Chattanooga, TN 37419, USA (423) 821-1160
httt:://uuw.refleett..oo www..rornecttiiridingg org/ Chattanooga'n
Audubon
Acres 34. 999350 -85. 180340
900 Sanctuary Rd, Chattanooga, 37421,
USA (423) 892-1499
http
://www.chate www.ttanoogaauuuuonnnoommm org/ Chattanooga\n
Sculpture Fields at Montague Park 35.026175 -85. 293624
1800 Polk St, Chattanooga, TN 37408,
USA (423) 266-7288
http://www.scuppturee
org/ Chattanooga inibtannooga/BBessiie Aneessie Smith Cultural Center and Chattanooga African King Blvd, Chattanooga, 37403,
American Museum 044634 306467 200
USA (423) 266-8658 http:/ ///::///www.we/ www.iesmithcc.ccmmmmmomo
/ttp:/utnnooooogooggoeeeeete mediumm= wix_google butieessueieessseeeeeee setiiness_roofieettta httpa//cc..cccaaoooo//////aaaaaa org.aaaa.................oaaaaaa Chattanooga\n
48

---

Published in Transactions on Machine Learning Research (April/2025)
Wooden Horse 055836 -85, -85.306713
10 Bluff View Ave, Chattanooga, TN 37403, USA
Unknown
Unknown Chattanooga"), ('Description' Restaurants in Chattanooga' 'Content': Cuisines Aggregate
Name
Average Cost
Rating Cityln
P.F. Chang's
33
Cafe, Bakery, BBQ,
Seafood
4.2 Chattanooga'n
L' L'amandier
28
Cafe, BBQ, Mediterranean, Seafood
4.3 Chattanooga Desserts, American, BBQ
Anupam Sweet Chattanooga'nn
3.4
64 2.6 Chattanooga Anupam Sweets & Restaurant
Nirula's
French, Desserta,
Seafood
Tea, Cafe, Desserts, Seafood
3.0 Chattanooga'n
Bisque Bakery
90
French, Desserts, Seafood Tea, Kexican, Bakery, Seafood
3.9 Chattanooga'n
Nooba
93
3.8 Chattanooga
The Royal
39
French, Pizza,
Bakery, Fast Food
3.3
Chattanoogaln
Curry Phulka
40
Tea, Chinese, Fast Food
2.8 Chattanooga DCK-
Dana Choga's Kitchen Chattanooga'n
67
Cafe, Pizza, Mediterranean
3.5
Chaayos
54 3.3 Chattanooga'n
Desserts, Cafe, BBQ, Chinese, Seafood
Truffles
53
Cafe, Bakery, BBQ, Fast Food, Chinese, American Liquid
3.2
Chattanoogaln
Tea, Pizza, Bakery, Mediterranean,
Seafood Desserta, Tea, Pizza, Mexican, Cafe, Indian 24
4.0 Chattanooga
Tpot
14
0.0
Chattanooga
Sardar Pure Meat Shop
Bakery, Pizza, American
3.4 Chattanoogaln
Warehouse Cafe
90
Fast Food, Pizza,
Seafood
3.7 Chattanooga'n
The Beer Cafe
62
Tea, Pizza, Fast Food
3.8 Chattanooga Tea, Cafe, Bakery, Desserts
Subway
66
2.4 Chattanooga
Indi-QUE
57
Tea, Italian, BBQ, Fast Food, Cafe
3.3 Chattanooga
nEssex Collections Patisserie Chinese, BBQ, Fast Food
3.4 Chattanooga'n
Food Adda
100
Desserts, Pizza, Fast Food, Mediterranean, Seafood
Chattanooga
Bikanervala
65
Cafe, Bakery
3.2
Chattanoogaln
Havmor Ice Cream
Pizza, Bakery, Fast
Food, Chinese, Seafood
3.6 Chattanooga
Moti Mahal Delux
52
Tea, Pizza, BBQ, Cafe, Mediterranean
2.7 Chattanooga Tea, Desserts
Aggarval Sweet India
30
2.8
Chattanooga'n
Jack Poltato's
Tea, Muradabadi
Bakery, Xediterranean, Seafood
3.3 Chattanoogalu
16
Cafe, Pizza, Bakery, BBQ
3.0
Chattanooga'n
Vishal Restaurant
27
Tea, Pizza, BBQ,
Seafood Desserts, Tea, BBQ, Cafe, Mediterranean Tea, French, Bakery, Fast Food, Chinese
3.1 Chattanooga'n
Raj Sveeta
69
2.8 Chattanooga
Punjabee's 's Darbar
41
3.0
Chattanoogaln
Basil Tree
20
Bakery, Indian, American,
Desserts
2.4 Chattanooga
Domino's Pizza
77
Bakery, Pizza, American, BBQ
2.8 Chattanoogaln
Pizza Hut Delivery
Tea, Cafe, American, Indian
3.9 Chattanooga
Ichiban
Mexican, Bakery, Desserts
3.9 Chattanooga
Habibi
95
Cafe, BBQ, Seafood
3.7 Chattanooga'n Food, Cafe, Indian, Mediterranean, Seafood
Cook Du Kdu 3.1 Chattanoogaln French, BBQ, Fast Food Tea, Fast Burger's King
69 Pizza, Fast
Go Krazy
64
3.5 Chattanooga
Bentoya
Food, Bakery, Seafood
3.8 8 Chattanooga Tea, Bakery, Desserts
16
4.1 Chattanooga"),
{ ('Description': 'Accommodations in Chattanooga', 'Content': price room type
NAME
house house_rules minimum
nights maximum occupancy review rate number Brooklyn 790.0 Private room
city
Affordable Private Spacious Room in No parties
49

---

Published in Transactions on Machine Learning Research (April/2025)
2.0
3.0 Chattanooga'n No pets & No children under 10
Sunny One Bedroom
722.0 Entire home/apt
1.0
3.0 ChattamoogaNn NuUpeer West / Morningside Heights Apt, Near Subway 290.0 Entire No visitors
home/apt
3.0
3
3.0 Chattanooga/n
Sunny room+Pvte office in huge loft 728.0 0 Private
No parties Extra Cozy Room Center Williamsburg 1033.0 Private room
4.0
5.0
Chattanooga'n
No pets
1.0
1
1.0 No pets
Chattanoogaln Luxury Charm. Steps from Christopher Park! 301.0 Entire home/apt
visitors & No parties
5.0
2
4.0
Chattanooga'n
Fort Greene Room 722.0
Private room
No visitors
No children under 10
2.0
2.0
Chattanoogaln
[Unicode Sequence Edited Out] 973.0 0 Entire home/ape
No pets
10.0
6
4.0 Chattanoogaln
bdrm/2 bath apt. Central Pk, Columbia U. 564.0 Entire home/apt
No viaitors & pets
30.0
4
3.0
Chattanooga'n
Cozy room in Bushwick- 15 min to the city 933.0 Private room parties pets
2.0
Chattanooga'n
Modern apartment w/ gorgeous view 921.0 Private room
pets Artsy Private BR in Fort Greene Cumberland 727.0 Private room
3.0
1.0
Chattanooga'n
visitors
30.0
3.0
Chattanooga'n
Studio Deluxe Wyndham Midtown 45 948.0 Private room No pets No smoking No children under 10
3.0
1
2.0
Chattanooga\n Trendy and Stylish Dountown apartment !!! 110.0 0 Entire home/apt
No pets
3.0
2
2.0
Chattanooga'n Sonder | 180 Water Luxurious 1BR Rooftop 676.0 Entire home/apt
No pets & No smoking Chattanooga'), ('Description': 'Flight from Houston Mashville 2022-03-21', 'Content': 'Flight
29.0
5
3.0
Number Price DepTime ArrTime ActualElapsedTine FlightDate OriginCityName DestCityName
Distance'n F3827247
241
14:22
16:10 hours minutes 2022-03-21 Houston Nashville
65m.0%n F3827724
147
18:35 20:33 1 hours 58 minutes 2022-03-21
Houston Nashville 657.0%m
F3827820
157 12:07
14:02 1 hours 55 minutes 2022-03-21
Houston Nashville
657.0'n
F3827930
305
07:36
09:41 2 hours 5
minutes 2022-03-21
Houston Nashville 657.0\n
F3956407
264
11:38
13:15
hours 37
minutes 2022-03-21 Houston Nashville
670.0'n
F3956408
246
08:08 09:47 hours 39 minutes 2022-03-21
Houston Nashville
670.0'n
F3956409
145
17:36
19:12 1 hours 36 minutes 2022-03-21
Houston
Nashville
670.0%z
F3956532
225
:4 08:52 hours 38 minutes 2022-03-21
Houston Nashville
657.0%n
F3956533
292
11:42 13:19 hours minutes 2022-03-21 Houston
Nashville
657.00 .0m F3956534
248 18:20 20:06 1 hours 46 minutes 2022-03-21
Houston
Nashville
657.0%n F4038756
10:11 11:52
41 minutes
2022-03-21
Houston Nashville 657-0%n F4038966 149 16:33 18:13 hours 40 minutes
2022-03-21 Houston Nashville 657 567,0"}, ('Description's 'Self-driving
from Houston to Nashville', 'Content'; 'self-driving, from Houston to Nashville, duration: 11 hours
mins, distance: 1,253 km, cost: 62'), ('Description': 'Taxi from Houston to Nashville', 'Content': 'taxi, from Houston to Nashville, duration: 11 hours 50 mins, distance:
1,253 km, cost: 1253'}, {'Description': 'Flight from Nashville to Knoxville on 2022-03-23', 'Content': 'There is flight from Nashville to Knoxville 2022-03-23.1 '), ("Description': *Self-
driving from Nashville to Knoxville', 'Content': 'self-driving, from Nashville to
Knoxville, duration: hours 42 mins, distance: 290 km, cost: 14'}, {'Description': 'Taxi from Nashville to Knoxville', 'Content': 'taxi, from Nashville to Knoxville, duration: 2 hours 42 mins, distance: 290 km, cost: 290'), ('Description': 'Flight from Knoxville to Chattanooga on 2022-03-25', 'Content': 'There is no flight from Knoxville to Chattanooga on 2022-03-25. { ('Description': 'Self-driving from Knoxville Chattanooga', 'Content': 'self-driving, from
Knoxville to Chattanooga, duration: 1 hour 41 mins, distance: 180 km, cost: 9*}, ('Description
Taax from Knoxville to Chattanooga', 'Content'; 'taxi, from Knoxville to Chattanooga, duration: 1
hour 41 mins, distance: 180 km, cost: 180'}, ('Description': 'Flight from Chattanooga
50

---

Published in Transactions on Machine Learning Research (April/2025)
Houston 2022-03-27', 'Content'; 'There is flight from Chattanooga Houston 2022-03-27. '), ('Description': "Self-driving from Chattanooga to Houston', 'Content'; 'self-
driving, from Chattanooga to Houston, duration: 11 hours 47 mins, distance: 1,309 km, cost: 65',, {
Deacription': 'Taxi from Chattanooga Houston', 'Content': 'taxi, from Chattanooga Houston
duration: hours mins, distance: 1,309 km, cost: 1309')]
Query: Could you design one-week travel itinerary for two, departing from Houston and touring three cities Tennessee from March 21st to March 27th, 2022? Our budget is now $8,200. We require accommodations that allow smoking and should ideally be private rooms. As for traneportation, we would prefer not self-drive.
Travel Plan (please only output the JSON string without explanatory information):
RESPONSE OF ol-preview
[
"day": 1,
"psople number": 2,
"current "current_city": "from Houston to Nashville", "transportation": "Flight Number: F3827820, from Houston to Nashville, Departure Time: 12:07, Arrival Time: 14:02",
"breakfast": "-",
"attraction": "Country Music Hall of Fame and Museum, Nashville", "lunch": "Cafe Arabelle, Nashville", "dinner": "Les 3 Brasseure, Nashville", "accommodation": "FiDi Cozy overlooking East River, Mashville"
},
{
"day": 2,
number"; 2, "current_ courrent_city": "Nashville",
"transportation"; "_",
"breakfast"; "Burger Planet, Nashville", "attraction": "Johnny Cash Museum; Centennial Park, Nashville",
"lunch": "Kamagoto, Nashville",
"dinner": "Oh! Calcutta, Nashville", "accommddation"; "FiDi Cozy overlooking East River, Nashville"
{
"day": : 3,
"people_numa"" number": 2,
"current_city": "from Nashville to Knoxville",
"transportation"; "Taxi, from Nashville to Knoxville, Departure Time: 09:00, Arrival Time: 11:42, Duration: 2 hours 42 minutes, Cost: $290", "breakfast"; "Cafe Chico, Nashville",
"attraction": "World's Fair Park, Knoxville", "lunch": "Ali Baba 41 Dishes, Knoxville",
"dinner": "Sky On 57, Knoxville",
"accommodation": "Light-filled Room in Renovated Apt, Knoxville"
{
"day": 4,
"people_ad number": 2,
"people_mumber":
ccurrent_city"! "Knoxville",
"transportation". "-",
"breakfast"; "Chaat Corner, Knoxville",
"attraction": "Ijams Nature Center; Sunsphere, Knoxville", "lunch": "Anand Sweeta, Knoxville",
51

---

Published in Transactions on Machine Learning Research (April/2025)
"dinner"; "La-Nawaab, Knoxville",
"accommodation"" "Light-filled Room in Renovated Apt, Knoxville"
"day": 5,
:people_number":
number": 2,
"iurrent_city"; "from Knoxville to Chattanooga", "transportation". "Taxi, from Knoxville to Chattanooga, Departure Time: 09:00, Arrival Time: 10:41, Duration: hour minutes, Cost: $180", "breakfast": "Anand Sweets, Knoxville",
"attraction": "Tennessee Aquarium, Chattanooga", "lunch": "Anupam Sweet, Chattanooga",
"dinner": "Bisque Bakery, Chattanooga", "accommodation"; "Trendy and Stylish Downtown apartment !!!, Chattanooga" "
},
{
"day": 6,
ppeople_nubber" 2,
"current city": "Chattamooga",
"transportation". "_",
"breakfast"; "Anupam Sweets & Restaurant, Chattanooga", "attraction": "Rock City Gardens; Ruby Falls, Chattanooga", "lunch": "Chaayos, Chattanooga",
"dinner": "L'amandier, Chattanooga", "accommodation": "Trendy and Stylish Dountown apartment !!! Chattanooga"
},
{
"day": : 7,
"people_nunae" number": 2,
caarreet_city"; "from Chattanooga to Houston",
"transportation"; "Taxi, from Chattanooga to Houston, Departure Time: 08:00, Arrival Time: 19:47, Duration: 11 hours 47 minutes, Cost: $1309", "breakfast"; "Bisque Bakery, Chattanooga", "attraction": "Coolidge Park, Chattanooga", "lunch": "Nooba, Chattanooga", "dinner": n-",
"accommodation": n_"
FEEDBACK FROM METACONTROLLER
By inspecting your plan we find the following isaue. Please refine your plan according to the feedback below:
1. The room type should be private room :>> Trendy and Stylish Downtown apartment !!!
2. The restaurant in day breakfast repeated.
3. The accommodation Trendy and Stylish Dountown apartment !!!! Chattanooga do not obey the minumum
nights rule.
4. The lunch in day 1 is invalid not in the data provided.
Q.2 Back Prompt
You are proficient planner. Based on the provided information and query, please give detailed plan, including specifics such as flight numbers (e.g., F0123456), restaurant names, and accommodation names. Note that all the information in your plan should be derived from the provided data. You should give travel plan in JSON format shoun in the example below. Additionally, all details should align with commonsense. The symbol is unnecessary. For example, in the provided sample, you do not need plan after returning to the departure city. When you travel to two cities in one day, you should note it in the 'current_city" section the example (i.e., from B).
indicates that information
52

---

Published in Transactions on Machine Learning Research (April/202)
***** Example *****
Query: Could you create travel plan for people from Ithaca to Charlotte spanning 3 days, from March 8th to March 14th, 2022, with a budget of $30,200?
Travel Plan:
[
"day": 1, "people_numbal" number": 7,
ciurrrentccity"" "from Ithaca Charlotte",
"transportation". "Flight Number: F3633413, from Ithaca to Charlotte, Departure Time: 05:38, Arrival Time: 07:46",
"breakfast"; "Nagaland's Kitchen, Charlotte",
"attraction": "The Charlotte Museum of History, Charlotte",
"lunch": "Cafe Maple Street, Charlotte",
"dinner": "Bombay Vada Pav, Charlotte", "accommodation": "Affordable Spacious Refurbished Room in Bushwick!, Charlotte"
{
"day": 2,
"people, n 7, ecurrent_cityy "Charlotte",
"transportation". "-",
"breakfast"; "Olive Tree Cafe, Charlotte",
"attraction": "The Mint Museum, Romaree Bearden Park, Charlotte", "lunch": "Birbal Ji Dhaba, Charlotte", "dinner": "Pind Balluchi, Charlotte", "accommodation". "Affordable Spacious Refurbished Room in Bushwick!, Charlotte"
"day": 3,
:
number": 7,
"current, sitrrent__ttt:: "from Charlotte to Ithaca",
"transportation" "Flight Number: F3796167, from Charlotte to Ithaca, Departure Time: 21:42, Arrival Time: 23:26",
"breakfast": "Books Monument, Charlotte", "attraction": "Books Monument, Charlotte", "lunch": "Olive Tree Cafe, Charlotte", "dinner": "Kylin Skybar, Charlotte", "accommodation": n_"
}
*****
Example Ends *****
Given information: [[ DDDecriiptin' 'Attractions in Nashville', 'Content'; Address
Name Latitude Longitude
Phone
Website
City/nCountry Music Hall of Fame and Museum 36. 158263 776126 222 Rep. John Lewis Way Nashville, TN 37203, USA (615) 416-2001
org/ Nashville'n 3777 Nolensville Pk, Nashville, TN 37211, USA (615) 833-1534
Nashville Zoo at Grassmere 36, 089705
-86. 742096
http:// www.nashrillezo..om org/ Mashville\n Belle Meade Historic Site Winery 36, 104916 864665
5025 Harding Pike, Nashville, 37205, USA (615) 356-0501
httpe://vieittelllemmae com/ Mashville\a
Johnny Cash Museum 36. 160939 -86 -86. 775757
119 3rd Ave S, Nashville, 37201, USA (615) 256-1777
wom/ Nashville'n
Centennial Park 36, 148946 862710
2500 West End Ave, Nashville, 37203, (615) 862-8400
https://wwr.mshhilll..oo www.nnassiile.eg.. www..nwwilee.go//aassaaaadaaeeee htw.aaddeeeeeeeeeeeeeeeereeeeeee
aspx Nashvilleln Grand 0le Opry 36. 206857 692108 600 Opry Mills Dr, Nashville,
53

---

Published in Transactions on Machine Learning Research (April/2025)
37214, USA (615) 871-6779
https://wwv.. opry. wttp:oppy.ccom/uttttcaaaaaaaaaaa
htg.://ccccccctte.ooooooooooooer _ _link
utm audience=toou auddemorrtofu_goglleebbaaaai
ottp://wwwaaaaaaaoooooooootttooa wwrollemg_aaeee my businessiees..m website
wowteen--banndstoryyyeeoeeoe google.
Washville\n
Frist Art Museum 36. 357897 -86.78385.
919 Broadway, Nashville,
TM 37203, USA (615) 244-3340
https: http:::/ttrsstartmseuur org/ Nashville'n
The Parthenon 36. 149674 -66. 06.311347
2500
West End Ave, Nashville, TN 37203, (615) 862-8431
http://www.naarrlepprrteeeee
com/ Nashville'n
Nashville Shores 36. 158209
4001 Bell Rd, Hermitage, TN 37076, USA (615) 889-7050
http::/w/re www.nnaasvvlllsreeeoo com.so/gooomo/ggggcc/cc Nashville'n Musicians Hall of Fame and Museum 367668
-86. 782399
401 Gay St, Nashville, TN 37219, USA (615) 244-3263
sttpp://wuu.i www.mmssiiinnn.
Mashville'n Bicentennial Capitol Mall State Park 36. 170887
-86.787588 600 James Robertson Pkwy, Nashville, TN 37243, USA (888) 867-2757
https: https://tastateppakks httpa//tppppaaseoooopopcs/aaaaoa con/pparas/bicnntenniaaal
Nashville'n 734580 702 Murfreesboro Pike, Naahville, TN 37210, USA (615) 742-7445
Lane Motor Museum 36. 140197
http://ww..anmmmtormuuuauass org/ Mashville\n
Honky Tonk Highway 36. 160393 7784322
501 Broadway, Nashville, TN 37203, USA (800) 657-6910
https://uuw.vistttmuss www.wvisittiiicmmmmmm ccity..ccit......... www.vwiiitrs//ovvooooooooo htsastorrr//nikytoottiihhaaaa Nashville'n Adventure Science Center 36. 146614 775481 800 Fort Negley Blvd, Nashville, TN 37203, USA (615)
862-5160
http://wwv www.:aadvetueescc.ooo Nashville'n
Patsy Cline Museum 36 160904 -86.776033
119 3rd Ave S, Nashville, 37201, (615) 454-4722
http////ppttmmuuuu..oo/ Mashville'n
Andrew Jackson's 's Hermitage 36. 213757 -86. 186.615310
4580 Rachels Ln, Hermitage, TN 37076, USA (615) 889-2941
tt com/ Nashville'n
NashTrash Tours 36. 169474 788254 900
Rosa Parks Blvd, Nashville, TN 37208, USA (615) 226-7300
http:// www.naaah.hhhcooommmmmmmm com/ Nashville'n
Fort Nashborough 36. 164268
170 1st Ave N, Nashville, TK 37201, USA (615) 862-8400
https://wuw www.wwwwwwwilee...e.m.e Bwwpoilleego//aaareeeeeee
aapx Mashville\n
Ryman Auditorium 36. 161248 -86.778477
116 5th Ave N,
Nashville, TN 37219, USA (615) 889-3060
www.n.rmmanooom wwwp//uttomeeeeeeeeeeeeeeeee http://uuttccmmpppgggrraaaaauut
htwp://wwwwao.eeoooooooooeaeatt. sorp source loy.ooogeebmssnsesssee..
utm ttn_audience-tofu, hog.r//ooouuggogggeoooussseeeeee
www.teebbbbbaaatttrreeeoooeeee my business website_ _innk
Nashville'n
Nashville Public Square Park 36. 166611 778126 Union St & 3rd Ave N, Nashville
TM 37201, USA (615) 743-3090
https://uw..maabbllleeee. mashvilledowmtonn. wwt.:/cccc/o///////eoo http./////pbbbccccuuurreeeeee/e
Nashville'), ('Description': "Restaurants
in Nashville', 'Content':
Name Average Cost Bangkok 1
Cuisines Aggregate Rating CityIn Cafe, Seafood
44
3.3 Nashville'n
Bablu Fast
Food
13 Desserta, Pizza, Mexican, BBQ, Fast Food
0.0 Mashville'n
Full Dabba
77 Pizza, Maxican, Fast Food, Cafe, American
0.0 NashvilleAn
Twigly
14 Pizza, French, Fast Food, Chinese, Veg Hut
Seafood
4.5 NashvilleAn
35
Tea, Mexican, Seafood
0.0 Nashvilleln Desserts, Fast Food
The Toddy Shop
92
3.4 Nashville'n
GoGourmet
34
Tea, Cafe, Indian, Mediterranean 15 Kitchen King
3.7 NashvilleAn Tea, Bakery, Mediterranean, Fast Food 13 Pizza, Italian, BBQ, Cafe,
Govinda's Confectionery
3.1 Washville\n
Mediterranean
0.0 Nashville\n
Town Hall
55
Tea, Seafood Mediterranean, Seafood
3.8 Nashville'n
Smoke House Deli
59
Cafe
4.0 NashvilleAn
Dialogue
Lounge & Caf
83 Tea, Indian, BBQ, Cafe, American, Seafood
3.0 Nashville\n
Meenakshi Bhawan
24
Tea,
Desserts
3.1 Nashville'a
Oh! Calcutta
83
Tea,
Mexican, BBQ, Cafe, Indian
4.4 Nashville\n Tea, French, Pizza, BBQ
Chicago
Pizza
20
3.2 Nashville\n
Malhotra Restaurant
76
Tea, Desserts, Seafood 88
2.9 NashvilleAn
Madras Cafe
Pizza, Desserte, Seafood
Mashville Dankhhin Sheraton New Delhi Hotel
54

---

Published in Transactions on Machine Learning Research (April/2025)
Cafe, Pizza, Indian, Bakery
4.0 Nashville'n
1911 Bar The Imperial
76
Desserts, Pizza, Italian, American, Seafood
3.2 Nashville'n
Punjabi Virsa
35
Desserts, Italian, BBQ, Cafe, Seafood
0.0 Nashville\n
Sagar Dhaba
55 Tea, Bakery, BBQ, Cafe,
Mediterranean
0.0 Nashville'n
Kettle Kegs
60
Tea, Seafood
0.0 Nashville'n Nashville"), {
Kargo Description':
Tea, Pizza, Desserts
"Accommodations in Nashville', 'Content':
NAME price room type
house_rules minimum nights maximum occupancy review rate number city\nHuge 2 Bedroom, Great Location, Express Metro
745.0 Entire home/apt No smoking children under 10 pets 4.0
4
5.0 Nashville\n Clean and large bedroom in private house 474.0 No smoking
Private roon
1.0
1
4.0 Nashville'n
Brooklyn Heights gem 993.0 Entire No peta
home/apt
2.0
5.0
Nashville'n
Lovely room in heart Williamsburg 61.0 Private
No pets
2.0
4.0 NashvilleIn *Light Love* vibrant,
historic, sleeps 679.0 Entire home/apt
No pets
5.0
2
2.0 Nashville'n FiDi Cozy room overlooking East River 870.0
Private room
No parties
1.0
2
5.0 Nashville'n Charming 1BR with sun-nook in Brooklyn 256.0 Entire home/apt No children under 10 & pets & No smoking 3.0
4.0
MashvilleAn
Cozy bedroom close to Manhattan 576.0 o Private room
No
children under 10 Nashville'), ('Description'; *Attractions in Knoxville', 'Content': Name Latitude Longitude
1.0
3.0
Address
Phone
Website
Cityln
World's s Fair Park 36.962577 -83.924199
525 Henley St, Knoxville, TN 37902, USA (865) 215-1158
http://worldefaairpack. /
Knoxville'nn
Knoxville Museum of Art 35.962426 -83. 92.2292 1050 Worlds Fair Park Dr, Knoxville, TN 37916, USA (865) 525-6101
http://sww. wwwp:knoxart.orro
Knoxville'n
Sunsphere 35. 36.961707 -83. 923353
810 Clinch Ave, Knoxville, TN 37902, USA (865) 314-0660
http:://www.uusppheeett.te
com/ Knoxville'n
Ijams Nature Center 35. 956455
866.766775
2915 Island Home Ave, Knoxville, 37920, USA (865) 577-4717
http://ww..jjmm .rrg/////////////////oo/o Knoxville'nn
Knorville Walking Tours 35 966448 -83.919167 301 S Gay St, Knoxville, TN 37902, USA (865) 309-4522
com/ Knoxville'n
Muse Knoxville 35.997617
$93.885467
516 N Beaman St, Knoxville, TN 37914, USA (865) 594-1494
http://wwv... themaseknoxville.co org/ Knoxville'n Knoxville Botanical Garden Arboretum 35.982160
-83. -33.881077 2743 Wimpole Ave, Knoxville, TN 37914, USA (865) 862-8717
http:// www.kkxggarden.. org/ Knozville\n
Haunted Knoxville Ghost Tours 35.965963 -83. 919553
36 Market Square #1404, Knoxville, TN 37902, USA (865) 377-9677
http://wwu.. www.whdnnwwvvllleeoooooo net/ Knoxville'n
Three Rivers Rambler 35.952810
-83. 940376 2560 University Commons Way, Knoxville, TN 37919, USA (865) 524-9411
http://ww..reeeiieeeere... threeriverrrambler. com/ Knoxville'n
Charles Krutch Park 35. 95.964414
-83.90 83..918695
504 Market St, Knoxville, TN 37902, USA (865) 215-4248
http:://ou..nooillleen
kwo.villetn. gotootggo//ooerrnnn///ttttyy sr__depprtmeetsss segpartments.offfcos///aee
offices/parks_nte hat.audrrcrratttoopprrs..
krutch_ barcch__arrkk_ Knoxvillekn McClung Museum of Natural History Culture 952005 -83. 927209
1327 Cir Park Dr, Knorville, TMl 37996, USA (865) 974-2144
http://nnclluggmusum...kk
Knoxville\n
Knoxville Sightseeing 35.992790 -83.9004 83.904122
2519 Mitchell St, Knoxville, TN 37917, USA (865)
566-0634 Unknown Knoxville'n
Chilhowee Park & Exposition Center 35.996296 -88.884033
Magnolia Ave, Knoxville, TK 37914, USA (865) 215-1450
55

---

Published in Transactions on Machine Learning Research (April/2025)
https://chiilooeepptt com/ Knozville'nn
Augusta Quarry 35. 05.944772 -33.911886
3000 Fort Dickerson Rd Sk, Knoxville, TN 37920, USA Unknown Unknown Knoxville\m
Zoo Knoxville 35. 25.999812 -83. 888250 3500 Knoxville
Zoo Dr, Xnoxville, TN 37914, USA (865) 637-5331
attps://wuu. www.ooooooolle..oooo org/ Knorville\n Outdoor Knoxville Adventure Center 35.961902
-83. 912663 900 Volunteer Landing Ln, Knoxville, TN 37915, USA (865) 228-8424
http//ww..ottoooroovvllllle com Knoxville'nn
James White Fort Association 35. 962962 205 Hill Ave, Knoxville, TN 37915, USA (865) 525-6514
-83.912169
http::/jaameet http:://aaneehhtteefort.oo org/ Knoxville'n
Rowing Man Statue in Knoxville, TN 35.963611 -83.917519
w. Church Avenue &, 3 Gay St, Knoxville, TN 37902, USA Unknown
Unknown Knoxville'n
UT Gardens Knoxville 35.944000 -83. 938260
2518
Jacob Dr, Knoxville, TN 37996, USA (865) 974-7324
http:
/uttar//ugardeess.. tennessee edu hetu/locations/kooxre ooooviile Knoxville\nEast Tennessee Historical Society and Museum 155964117 883.917796 601 Gay St, Knoxville, TN 37902, USA (865)
215-8830
http://wwr /ttp://wattttistrrrrrrr
org/ Knoxville"), {'Description': RRataurants Knoxville', 'Content':
Name Average Cost Cafe Arabelle
Cuisines Aggregate Rating Cityln
29
French, BBQ, Desserts, Seafood Desserts, Pizza, Italian, Cafe, Indian Tea,
3.6 Knorville'n
Les Brasseurs
24
4.6 Knoxville'nn
Sky On 3.4 Knoxville'n
150
Cafe, Pizza, Chinese, Seafood
Cafe E1 Chico
67
Tea, French, Indian, Seafood
3.6 Knoxville\n
TcozY
85
Tea, Pizza, Mexican, Fast Food, Cafe
0.0 Knoxville\nRama
Vaishnav jnalya 3.0 Knoxville'n
46
Tea, Cafe, Pizza, Desserts
Burger Planet
45
Tea, Bakery, American, Cafe Fast
3.2 Knoxville'n
Cafe Coffee Day 2.8 Knoxville\n
82
Food, American, BBQ, Italian
Mamagoto
14
Indian, Mediterranean, Desserts, Seafood
4.1 Knoxville'n Ali Baba & 3.5 Knoxville'n
41 Dishes Punjabi Flavour
70
Pizza, Desserts, Fast Food Tea, Bakery, Cafe, Indian, Seafood
41
0.0 Knoxville'n Tandoori Tadka
Chinese, Pizza, Desserts, Fast Food 35 Desserts,
0.0 Knoxville'n
Coalition Cafe
Tea, BBQ, Cafe, Mediterranean
3.4 KnoxvilleAn
Chaat Corner
Cafe, Pizza
3.2 Knorville\n
El Posto
63
Tea, Cafe, Pizza, BBQ French, Bakery, Cafe, Seafood Chinese, Pizza, Cafe, Italian
3.5 Knoxville\n
La-Nawaab
17
3.0 Knoxvillein
Peppers Pipes
86
3.1 Knozvilleln
Chit Chat
15 Anand Sweets
Pizza, Fast Food
3.1 Knoxville'n
45
Fast Food,
Pizza, Indian, Seafood
0.0 KnoxvilleAn
Open Kitchen
13 Pizza, Bakery, BBQ, Cafe, Indian, Mediterramean
3.0 Knoxville'n Biryani By Kilo
Chinese, Pizza, Fast Food
4.1 Indian, Desserts, Fast Food
Knoxvilleln Chinese Food Corner
30
Knoxville\nfhe Indian Kaffe Express
Tea, Desserts
3.8 Knoxville'}, ('Description': 'Accommodations in Knoxville', 'Content';
NAME price room type
house_rules minimum nights maximum occupancy review rate number city Cozy Private Room in pets &
Chinatown/ Lower East Side 132.0 Private room
No visitors
7.0
2
4.0 KnoxvilleAn
The Diamond Room 1008.0 Private room
No parties Light-filled
No visitors
1.0
1
5.0 Knozvilleka
Room Renovated Apt 310.0 Private room
No pets
2.0
2
2.0 Knoxville\n
Private Room 922.0 Private
No
visitors
1.0
4.0 Knoxville'n
Beautiful
Private Manhattan Room 721.0 Private room
No parties & No
56

---

Published in Transactions on Machine Learning Research (April/2025)
smoking
1.0
2.0 Knoxville\n
Lg Quiet Artist
Home -Ditmas Park 561.0 Entire home/apt
No visitors No pets & No
smoking
24.0
2.0 Knoxville'aa Beautiful 1- bdrm apt in
tranquil Inwood building 1091.0 Entire home/apt
No amoking &
No peta
2.0
4.0 Knoxville'n 1,600aq modern duplex
new harlem brownstone 1159.0 Entire home/apt No smoking & No children under 10 &
pets min to subway 793.0 Private room
2.0
Brooklyn Sunny room No visitors & No children
2.0 Knoxville'n
under
2.0
2
5.0 Knoxville'nn Amazing Large Sunny Studio in No parties No
Greenwich Village 144.0 Entire home/apt
smoking
30.0
4.0 Knoxville(n Private Bdrm Suite in No
Historic Brownstone 479.0 Private room
visitors
2.0 Knoxville`nn Charming bedroom with huge terrace in Greenpoint 712.0 Private room No parties & No children under 10 & No
2.0
2
visitors
2.0
1
3.0 Knoxville\n No smoking No
Huge
quiet modern oasis 1165.0 Entire home/apt
parties
2.0
2.0 Knoxville\nPrivate large room near LGA
airport with queen bed 552.0 Private room
No pets
1.0
2.0 Knoxville'n
12 east 86th Steet Upper No
East Side Building 632.0 Entire home/apt
smoking
2.0 KnozvilleAn Spacious Garden Apt in Clinton Hill/Ft Greene 1006.0 Entire home/apt children under 10 & visitors No
30.0
3
smoking
3.0
3.0 Knoxville'n
LES Penthouse
993.0 Entire home/apt
No children under
10 No
parties in Chattanooga', 'Content'; Latitude Longitude
2.0
4. O Knoxville'), ( ('Description': 'Attractions
Name Website City\n
Address
Phone
The Chattanooga Zoo at Warner Park 35. 042632 -85. 282099 301 и Holtzclav Ave, Chattanooga, TN 37404, USA (423) 697-1322
http://wwu..
http://www.caattzoo.o. chattzoo... org/ Chattanooga'n
Rock City Gardens
34.973392 85.355177 820-2531
1400 Patten Rd, Lookout Mountain, GA 30750, USA (706)
http://seerockcitty com/ com//uttssourcer source=
gmbbutm cecttt_mmiu=oorraii. Chattanoogaln
Tennessee Aquarium 055825 -85. 311065
Broad St, Chattanooga, TN 37402, USA (423)
265-0695
http://uuu.. http:://wwuttaaua...tt org/ Chattanooga'n
Lookout Mountain Incline Railway 35. 009546 328564 3917 St Elmo Ave, Chattanooga, TN 37409, USA (423)
821 221-4224
http://www.
ridetheincline com/ Chattanoogaln
Coolidge Park 35. 060506 -85.3068 643-6311
150 River St, Chattanooga, TN 37405, USA (423)
http: http:///uw... http://wwww.hattanogaaaaa
ot
Chattanooga
Creative Discovery Museum 052700 -85.312280 321 Chestnut St, Chattanooga, TN 37402, USA (423) 756-2738
http://www.
cdmfun.c /dmfun.org/
Chattanooga
Chattanooga Ghost Tours Inc 35.051231 309057
57 E 5th St, Chattanooga, TN 37402, USA (423) 800-5998
https://chhtaanoggghttttoo comett/ Chattanoogaln
Ross
Landing 35.056741 -85.310302 USA (423) 643-6311
201 Riverfront Pkay, Chattanooga, TK 37402,
http: htwp://www.cer whw.:enoccattnoaaaggoooo htt.raaaoog..ooppubli---ttttet
pati:///arcccccoy-----a---1eeeee Chattanooga'n
Hunter Museum of American Art 35.055894 -85. 306400
10 Bluff View Ave, Chattanooga, TH 37403,
USA (423) 267-0968
http::/www.huttrmmssse www.ermuseum...co.mmmm.uuuuuuuw. org/ Chattanooga In
Walnut St, Chattanooga, 37403, Walmut Street
Bridge 35.058330 307296 (423) 643-6096
http://www.hhattanoog wwwttanooga. wht.aaaaoog.....pp/llccoooooe//a
(f Chattanoogaln
57

---

Published in Transactions on Machine Learning Research (April/2025)
Tennessee Riverpark 35. 095938 245311
4301 Amnicola Hvy, Chattanooga, TN 37402, USA (423)
209-5370
http://parkss hamiltontn, gov/ Chattanooga'nn
Moccasin Bend National Archeological District 35.053022 -85. 301 Mamufacturers Rd, Chattanooga, TN 37405, USA (423) 648-5623
http://pprrkk.aamlltttnntt
https://www npa. wtt.g//wwwweeeeee//////////t//ea Chattanooga'n
Ruby Falls 019084 -39355
1720 Scenic Hwy, Chattanooga, 37409, USA (423) 821-2544
http://wwwrrbyfallllaa com/ Chattanooga'n
Cravens House
35. 013782 341495 Point Park Visitor Ctr, Lookout Mountain, 37350, https://wwt www.:/wwwpppp....mmm.mm..
(423) 821-7786
places/ cravenshouselom. om. htm Chattanooga'n
Williams Echo Dome 35. 054215 -34306
430 E 2nd St, Chattanooga, TN 37403, USA
Unknown Unknown Chattanooga/n
Reflection Riding Arboretum & Nature Center 35.010130 400 Carden Rd, Chattanooga, TN 37419, USA (423) 821-1160
85.364585
http//ww..elllcciiniiidi.ng org/ Chattanoogaln
Audubon
Acres 34.999350 -85. 180340
900 N Sanctuary Rd, Chattanooga, TN 37421,
USA (423) 892-1499
http
http://hhhatanngggauuuueea org/ Chattanooga
Sculpture Fields at Montague Park 35.026 -85. 293624
1800 Polk St, Chattanooga, TN 37408,
USA (423) 266-7288
http:///wu.sculpuurrffeeede.. org/ Chattanooga bhattanoogaanBeessie \nBeasie Smith Cultural Center and Chattanooga African
American Museum 35. 044634 -85. -85.306467 200 M King Blvd, Chattanooga, TN 37403,
USA (423) 266-8658 http://www. www.sesmittee.......o.e.....
wix wix_google, wtteee_ussnnssssssrsssese
camppiggnn3993778822222288388900 Chattanooga'n
Wooden Horse 35. 35.055836 305.30 Unknown Chattanooga"), ('Description': "Restaurants Chattanooga', 'Content':
10 Bluff View Ave, Chattanooga, TN 37403, USA
Unknown
Name
Average Cost
Cuisines Aggregate
Rating Cityln
P.F. Chang's
Cafe, Bakery, BBQ,
Seafood
4.2 Chattanoogaln
L' L'amandier
Cafe, BBQ, Mediterranean, Seafood 82
4.3 Chattanoogala Desserts, American, BBQ
Anupam Sweet 3.4 Chattanooga\n
Mirula's
French, Desserta,
Seafood Tea, Cafe, Desserts, Seafood
2.6 Chattamooga \n Anupam Sweets & Restaurant
62
3.0 Chattanooga'n
Bisque Bakery
90
French, Desserts, Seafood Tea, Mexican, Bakery, Seafood French, Pizza,
3.9 Chattanooga
Nooba
3.8 Chattanooga
The Royal
39
Bakery, Fast Food
3.3 Chattamoogaln
Curry n Phulka
40
Tea, Chinese, Faat Food
2.8 Chattanoogaln DCK-
Dana Choga's Kitchen
67
Cafe, Pizza, Mediterranean
3.5
ChattanoogaAn
Chaayos
Desserts, Cafe, BBQ, Chinese, Seafood Cafe, Bakery, BBQ, Fast Food, Chinese, American
3.3 Chattanoogaln
Truffles
53
3.2
Chattanooga'n
Liquid
15
Tea, Pizza, Bakery, Mediterranean,
4.0 Chattanooga Desserts, Tea, Pizza, Mexican, Cafe, Indian 24
Seafood
Tpot
0.0 Chattanooga Sardar A Pure Meat Shop
Bakery, Pizza, American
3.4 Chattanoogaln
Warehouse Cafe
90
Fast Food, Pizza,
Seafood
3.7 Chattanooga'n
The Beer Cafe
62
Tea, Pizza, Fast Food
3.8 Chattanooga Tea, Cafe, Bakery, Desserta Tea, Italian, BBQ, Fast Food, Cafe
Subway
66
2.4 Chattanooga An
Indi-QU
3.3 Chattanooga)
nEasex Collections Patisserie Chinese, BBQ, Fast Food Desserts, Pizza, Fast Food, Mediterranean, Seafood
48
3.4 Chattanooga
Food Adda
100
0.0 Chattanoogaln
Bikanervala
Cafe, Bakery
3.2
Chattanoogaln
Havmor Ice Cream
75
Pizza, Bakery, Fast
58

---

Published in Transactions on Machine Learning Research (April/2025)
Food, Chinese, Seafood
3.6 Chattanooga'n
Moti Mahal Delux
52
Tea, Pizza, BBQ, Cafe, Mediterramean
2.7 Chattanooga An Tea, Desserts Tea, Muradabadi
Aggarwal Sweet India
30
2.8
Chattanooga'n
Jack Poltato's
63
Bakery, Mediterranean, Seafood
3.3 Chattanooga'n
16
Cafe, Pizza, Bakery, BBQ
3.0
Chattanooga'n \n Seafood Desserts, Tea, BBQ, Cafe, Mediterranean
Vishal Restaurant
27
Tea, Pizza, BBQ,
3.1 Chattanooga'n
Raj Sweets
69
2.8 Chattanooga'n
Punjabee's Darbar
41
Tea, French, Bakery, Fast Food, Chinese
3.0
Chattanooga'n
Basil Tree
20
Bakery, Indian, American,
Desserts
2.4 Chattanooga'n
Domino's Pizza
77
Bakery, Pizza, American, BBQ
2.8 Chattanooga'n
Pizza Hut Delivery
25
Tea, Cafe, American, Indian
3.9 Chattanooga
Ichiban
87
Mexican, Bakery, Desserts
3.9 Chattanooga
Habibi
95
Cafe, BBQ, Seafood
3.7 Chattanooga
Cook Du Kdu 3.1 Chattanooga French, BBQ, Fast Food
69 Pizza, Fast
Food, Cafe, Indian, Mediterranean, Seafood
Go Krazy
64
3.5 Chattanooga
n Food, Bakery, Seafood
Bentoya
Tea, Fast Burger's King
3.8 Chattanooga/a Tea, Bakery, Desserts
16
4.1 Chattanooga"),
{ ('Description':
AAccomoddatoons in Chattanooga', 'Content': price room type
NAME
house_rules minimum
nights maximum occupancy review rate number Brooklyn 790.0 Private room
cityln Affordable Private Spacious Room in No parties
2.0
3.0 Chattanoogaln
Sunny One Bedroom
722.0 Entire home/apt
No pets & No children under 10
1.0
3.0 paaianooogannulpperr Weat / Morningside Heighta Apt, Near Subway 290.0 Entire
home/apt
No visitors
3.0
3
3.0 Chattanooga
Sunny room+Pvte office in huge loft 728.0 Private room
No parties
4.0
5.0
Chattanoogaln
Extra Cozy Room Center Williamaburg 1033.0 Private room
No pets
1.0
1
1.0 No pets No
Chattanooga'n Luxury Charm. Steps from Christopher Park! 301.0 Entire home/apt visitors & No parties
5.0
2
4.0
Chattanooga'n No children under 10
Fort Greene Room 722.0
Private room
No visitors
2.0
2.0
Chattanoogaln
[Unicode Sequence Edited Out] 973.0 Entire home/apt
pets bdr=/2 bath apt. Central Pk, Columbia U. 564.0 Entire home/apt
10.0
6
4.0 Chattanoogaln
visitors pets
30.0
3.0
Chattanooga'n
Cozy room in Bushwick- 15 min to the city 933.0 Private room
parties pets
1.0
2.0
Chattanooga'n
Modern apartment w/ gorgeous view 921.0 Private room
pets Artsy Private ER in Fort Greene Cumberland 727.0 Private room
3.0
1.0
Chattanooga'n
visitors
30.0
3.0
Chattanooga'n
Studio Deluxe Wyndham Midtown 45 848.0 Private room No pets No smoking No children under 10
3.0
1
2.0
Chattanooga\n Trendy and Stylish Dountown apartment !!! 110.0 Entire home/apt
No pets
3.0
2
2.0
Chattanooga'n Sonder | Water Luxurious 1BR Reoftop 676.0 Entire home/apt
No pets & No smoking Chattanooga'), ('Description': 'Flight from Houston Mashville 2022-03-21', 'Content': 'Flight
29.0
5
3.0
Number Price DepTime ArrTime ActualElapsedTine FlightDate OriginCityName DestCityName
Distance F3827247
241
14:22 16:10 hours minutes 2022-03-21 Houston Nashville
657.0kn
F3827724
147
18:35
20:33 1 hours 58 minutes 2022-03-21
59

---

Published in Transactions on Machine Learning Research (April/2025)
Houston Washville 6
F3827820
157
12:07
14:02 1 hours 55 minutes 2022-03-21
Houston Nashville 657.0 O' F3827930
305
07:36 09:41 2 hours 5
minutes 2022-03-21
Houston Nashville 857.0%m
F3956407
264
11:38
13:15
hours 37
minutes 2022-03-21 Houston Nashville
670.0|n
F3956408
246
08:08 09:47 houre minutes 2022-03-21
Houston Nashville
670.0An
F3956409
145
17:36
19:12 12 1 hours 36 minutes 2022-03-21
Houston
Nashville
670.0%n
F3956532
07:14
08:52 hours minutes 2022-03-21
Houston Nashville
657.0\n
F3956533
292
11:42 13:19 hours 37 minutes 2022-03-21 Houston
Nashville
657.0
F3956534
248
18:20 20:06 1 hours 46 minutes 2022-03-21
Houston
Nashville
65
F4038756
272 10:11 11:52 hours 41 minutes
2022-03-21
Houston Nashville 657.0'mn F4038966 149 16:33 18:13 hours 40 minutes
2022-03-21
Houston Nashville 657.0"}, {'Description': 'Self-driving
from Houston to Nashville', 'Content'; 'self-driving, from Houston to Nashville, duration: 11 hours
50 mins, distance: 1,253 km, cost: 62'), ('Description': 'Taxi from Houston to Nashville', 'Content': 'taxi, from Houston to Nashville, duration: 11 hours 50 mins, distance:
1,253 km, cost: 1253'}, {'Deacription': 'Flight from Nashville Knoxville 2022-03-23', 'Content': 'There is no flight from Nashville to Knoxville on 2022-03-23.'), ("Description': 'Self-
Nashville to Knoxville, duration: hours 42 mins, distance: 290 km, cost: 14'}, {'Description': 'Taxi from
driving from Nashville to Knoxville', 'Content': 'self-driving,
Nashville to Knoxville', 'Content'; 'taxi, from Nashville to Knoxville, duration: hours 42 mins, distance: 290 km, cost: 290'}, ('Description': 'Flight from Knoxville to Chattanooga 2022-03-25', 'Content': 'There is no flight from Knoxville to Chattanooga on 2022-03-25. '}, { ('Description': 'Self-driving from Knoxville to Chattanooga', 'Content': 'self-driving, from Knoxville to Chattanooga, duration: 1 hour 41 mins, distance: 180 km, cost: 9*), ("Description
'Taxi from Knoxville to Chattanooga', 'Content': 'taxi, from Knoxville to Chattanooga, duration: 1 hour 41 mins, distance: 180 km, cost: 180'}, ('Description': 'Flight from Chattanooga
Houston on 2022-03-27', 'Content'; 'There is no flight from Chattanooga to Houston on
2022-03-27 }}, (' {'Description': "Sef--rriving from Chattanooga Houston', 'Content': 'self- driving,
from Chattanooga to Houston, duration: hours mins, distance: 1,309 km, cost: 65'),
Description': 'Taxi from Chattanooga to Houston', 'Content'; taxi, from Chattanooga to Houston
duration: 11 hours 47 mins, distance: 1,309 km, cost: 1309'}]
Query: Could you design a one-week travel itinerary for two, departing from Houston and touring three cities in Tennessee from March 21st to March 27th, 2022? Our budget is now $8,200. We require accommodations that allow smoking and should ideally be private rooms. for transportation, would prefer not self-drive. Travel Plan:
[
"day": 1,
"people_number"; numberr 2,
"current "current_city": city": "from Houston to Nashville", "transportation": "Flight Number: F3827820, from Houston to Nashville, Departure Time: 12:07, Arrival Time: 14:02", "breakfast": n_"
"attraction": "Country Music Hall of Fame and Museum, Nashville", "lunch": "Cafe Arabelle, Nashville", "dinner": "Les Brasseurs, Nashville", "accommodation": "FiDi Cozy overlooking East River, Mashville"
},
"day": 2,
"people_nunde"" number": 2,
"current "current_city"" citt_ "Nashville", "transportation": "_",
"breakfast": "Burger Planet, Nashville", "attraction": "Johnny Cash Museum; Centennial Park, Nashville",
60

---

Published in Transactions on Machine Learning Research (April/2025)
"lunch"; "Mamagoto, Nashville",
"dinner": "Oh! Calcutta, Nashville",
"accommodation": "FiDi Cozy overlooking East River, Nashville"
},
{
"day": 3,
"people_number"" number": 2,
"current_
"current_city": city": "from Nashville to Knoxville",
"transportation": "Taxi, from Nashville to Knoxville, Departure Time: 09:00, Arrival Time: 11:42, Duration: 2 hours 42 minutes, Cost: $290", "breakfast"; "Cafe E1 Chico, Nashville",
"attraction": "World's Fair Park, Knoxville", "Lunch"; "Ali Baba 41 Dishes, Knoxville",
"dinner"; "Sky On 57, Knoxville", "accommodation". "Light-filled Room in Renovated Apt, Knoxville"
"day": 4,
"people number"; 2, "current "current_city"; "Knoxville", "transportation". "-",
"breakfast": "Chaat Corner, Knoxville",
"attraction": "Ijams Nature Center; Sunsphere, Knoxville", "Lunch"; "Anand Sweets, Knoxville", "dinner": "La-Nawaab, Knoxville", "accommodation": "Light-filled Room in Renovated Apt, Knoxville"
{
"day": 5,
number": 2,
"current_cityy "from Knoxville to Chattanooga", "transportation": "Taxi, from Knoxville to Chattanooga, Departure Time: 09:00, Arrival Time: 10:41, Duration: hour minutes, Cost: $180", "breakfast"; "Anand Sweets, Knoxville",
"attraction": "Tennessee Aquarium, Chattanooga", "lunch": "Anupam Saeet, Chattanooga",
"dinner": "Bisque Bakery, Chattanooga",
"accommoaation"; "Trendy and Stylish Downtown apartment !!! Chattanooga"
{
"day": : 6,
"people_nume"" number": 2,
courrent_city"". "Chattanooga",
"transportation"; "_"
"breakfast"; "Anupam Sweets & Restaurant, Chattanooga",
"attraction": "Rock City Gardens; Ruby Falls, Chattanooga",
"lunch": "Chaayos, Chattanooga",
"dinner": "L'amandier, Chattanooga", "accommodation": "Trendy and Stylish Downtown apartment !!! Chattanooga" "
{
"day": 7,
"heople_number". 2,
ciarreen_city"" "from Chattanooga to Houston", "transportationn "Taxi, from Chattanooga Houston, Departure Time: 08:00, Arrival Time: 19:47, Duration: 11 hours 47 minutes, Cost: $1309", "breakfast"; "Bisque Bakery, Chattanooga", "attraction": "Coolidge Park, Chattanooga", "lunch": "Nooba, Chattanooga",
61

---

Published in Transactions on Machine Learning Research (April/2025) 2025)
"dinner": "-", "accommodation"; "-"
}
By inspecting your plan we find the following issue. Please refine your plan according to the feedback below:
1. The room type should be private room :-> Trendy and Stylish Dountown apartment !!!
2. restaurant in day breakfast repeated.
3. The accommodation Trendy and Stylish Dountown apartment !!!, Chattanooga do not obey the minumum
nights rule.
4. The lunch in day 1 is invalid not in the data provided.
Travel Plan (please only output the JSON string without explanatory information):
RESPONSE OF ol-preview
[
{
"day": 1,
"people, number"; 2,
"current courrent_ctt""" "from Houston to Nashville",
"transportation". "Flight Number: F3827820, from Houston to Nashville, Departure Time: 12:07, Arrival Time: 14:02", "breakfast"; *-",
"attraction": "Country Music Hall of Fame and Museum, Nashville", "lunch": "Smoke House Deli, Nashville", "dinner": "Les 3 Brasseurs, Nashville", "accommddation": "FiDi Cozy overlooking East River, Mashville"
},
"day": 2,
"people_ "people_number"" number": 2, "current ceurrent_city";; "Nashville", "transportation". "-",
"breakfast": "Burger Planet, Nashville", "attraction": "Johnny Cash Museum; Centennial Park, Nashville",
"lunch"; "Kamagoto, Nashville",
"dinner": "Oh! Calcutta, Nashville", "accommodation": "FiDi Cozy overlooking East River, Mashville"
"day": 3,
"people "people_number"" number": 2,
courrent_city:: city": "from Nashville to Knoxville", "transportation"; "Taxi, from Nashville to Knoxville, Departure Time: 09:00, Arrival Time: 11:42, Duration: 2 hours 42 minutes, Cost: $290", "breakfast"; "Cafe El Chico, Nashville",
"current,
"attraction": "World's Fair Park, Knoxville", "lunch": "Ali Baba 41 Dishes, Knoxville",
"dinner": "Sky On 57, Knoxville",
"accommodation": "Light-filled Room in Renovated Apt, Knoxville"
{
"day": 4,
"people, number": 2, "current, ceurrent_city":: "Knoxville",
"transportation": "-", "breakfast"; "Chaat Corner, Knoxville",
62

---

Published in Transactions on Machine Learning Research (April/2025)
"attraction": "Ijams Nature Center; Sunsphere, Knoxville",
"lunch"; "Anand Sweets, Knoxville",
"dinner": "La-Nawaab, Knoxville", "accommodation": "Light-filled Room in Renovated Apt, Knoxville"
{
"day": 5,
"people number": 2,
"current_city"; city": "from Knoxville to Chattanooga", "transportation". "Taxi, from Knoxville to Chattanooga, Departure Time: 09:00, Arrival Time: 10:41, Duration: hour minutes, Cost: $180", "breakfast": "Burger Planet, Knoxville",
"current_
"attraction": "Tennessee Aquarium, Chattanooga",
"lunch"; "Anupam Sweet, Chattanooga",
"dinner": "Bisque Bakery, Chattanooga",
"accommodation": "Affordable Private Spacious Room in Brooklyn, Chattanooga"
},
"day": 6,
"people number": : 2,
"current couurenttcity";; "Chattanooga",
"traasportationn" "-",
"breakfast": "Anupam Sweets & Restaurant, Chattanooga",
"attraction": "Rock City Gardens; Ruby Falls, Chattanooga", "lunch": "Chaayos, Chattanooga",
"dinner": "L'a "L'amandier, Chattanooga", "accommodation": "Affordable Private Spacious Room in Brooklyn, Chattanooga"
},
{
"day": 7,
number": 2,
"current_ courrent_city":: "from Chattanooga to Houston",
"transportation": "Taxi, from Chattanooga to Houston, Departure Time: 08:00, Arrival Time: 19:47, Duration: 11 hours 47 minutes, Cost: $1309", "breakfast"; "Bisque Bakery, Chattanooga", "attraction": "Coolidge Park, Chattanooga", "lunch": "Nooba, Chattanooga", "dinner"; m_",
"accommodation"; "_"
}
R Prompts for Trip Planning
R.1 First Iteration
You are an expert at planning trips. You are given a few constraints regarding the cities to visit durations of staying at each city. You are also given the flight information between the cities('and) suggests flights are possible both ways and 'from to suggests one way flights). You can travel to any city only once.
Produce only one plan, strictly adhering to the format shown in the examples below, without any extra information. Your answer shold start with 'SOLUTION:'. Here few example tasks and solutions:
TASK: You plan visit 10 European cities for 21 days in total. You only take direct flights commute between cities. You plan to stay in Vienna for days. You want to spend 5 days in Frankfurt You want apend 2 days Delo. You going attend wedding in Oalo
63

---

Published in Transactions on Machine Learning Research (April/2025)
between day and day 21. You want to spend 3 days Prague. You would like to visit Valencia for days. You want to meet a friend in Valencia between day 17 and day 18. You plan to stay in Dubrovnik for days. You would like to visit Edinburgh for days. From day 1 to day 5, there is a anmual show you want to attend in Edinburgh. You plan to stay in London for 2 London between day 12 and day You would like visit
days. You plan visit relatives Munich for 3 days. You would like to meet your friends at Munich between day 18 and day to tour together. You would like to visit Budapest for 3 days.
Here are the cities that have direct flights:
Valencia and Munich, Vienna and Munich, Vienna and Valencia, London and Budapest, London and Oslo, Edinburgh and Budapest, Frankfurt and Budapest, Frankfurt and London, Prague and Oslo, Edinburgh and Oslo, Edinburgh and Munich, Prague and Munich, London and Prague, Edinburgh and London, Edinburgh and Frankfurt, Dubrovnik and Munich, Dubrovnik and Vienna, Munich and Oslo, Dubrovnik and Oslo, Budapest and Mumich, Frankfurt and Prague, Vienna and London, Frankfurt and Vienna, Frankfurt and Oslo, Frankfurt and Munich, Vienna and Dalo, Vienma and Prague, Budapest and Oalo, Budapest and Prague, London and Valencia, London and Munich, Frankfurt and Dubrovnik, Prague and Valencia, Frankfurt and Valencia, Edinburgh and Prague, Vienna and Budapest.
Find a trip plan of visiting the cities for 21 days by taking direct flights to commute between them.
SOLUTION:
Here is the trip plan for visiting the 10 European cities for 21 days:
**Day 1-5:** Arriving in Edinburgh and visit Edinburgh for days.
**aay 5:** Fly from Edinburgh to Frankfurt.
**Day 5-9:** Visit Frankfurt for 5 days.
+*aay 9:** Fly from Frankfurt to Dubrovnik.
**Day 9-10:** Visit Dubrovnik for 2 days.
+Daay 10:** Fly from Dubrovnik
Vienna.
*4aay 10-12:** Visit Vienna for 3 days.
+Day 12:** Fly from Vienna to London.
*Da 12-13:** Visit London for 2 days.
**aay 13:** Fly from London to Budapest.
**Day 13-15:#* Visit Budapest for 3 days.
*Daay 15:** Fly from Budapest to Prague.
*Daa 15-17:** Visit Prague for 3 days.
**aay 17:** Fly from Prague to Valencia.
+*aay 17-18:** Visit Valencia for days.
+*aay 18:** Fly from Valencia to Munich.
+Daay 18-20:** Visit Munich for days.
+*Day *0:** Fly from Munich to Oslo.
+*Day 20-21: Visit Oslo
days.
TASK: You plan to visit 10 European cities for 22 days in total. You only take direct flights to commute between cities. You want to spend 4 days in Helsinki. You plan to stay in Vilnius for 2 days. You plan to stay Istanbul for 5 days. From day to day 21, there is annual show you want to attend in Istanbul. You want to spend 2 days in Hamburg. You plan to stay in Porto for 2 days. During day and day 12, you have to attend a conference in Porto. You want spend days in Athens. You would like to visit Prague for 2 days. You plan to visit relatives in Prague between day 13 and day 14. You want to spend 2 days in Frankfurt. You want spend days in Krakow. You want to spend 5 days Munich. You would like to meet your friends at Munich between day 1 and day 5 to tour together.
Here are the cities that have direct flights:
from Krakow Vilnius, Helsinki and Hamburg, Hamburg and Athens, Munich and Frankfurt, Hamburg and Porto, Munich and Istanbul, Prague and Athens, Frankfurt and Athens, Munich and Athens, Munich and Prague, from Vilnius Munich, Hamburg and Istanbul, Frankfurt
64

---

Published in Transactions on Machine Learning Research (April/2025)
Istanbul, Mumich and Krakow, Munich and Hamburg, Munich and Helsinki, Prague and Istanbul, Frankfurt and Vilnius, Helsinki and Istanbul, Athens and Vilnius, Krakow and Frankfurt, Helsinki and Frankfurt, Porto and Frankfurt, Frankfurt and Prague, Istanbul and Vilnius, Krakow and Istanbul, Krakow and Prague, Munich and Porto, Helsinki and Vilnius, Helsinki and Prague, Porto and Istanbul, Hamburg and Frankfurt, Krakow and Helsinki, Athens and Istanbul.
Find a trip plan of visiting the cities for 22 days by taking direct flights to commute between them.
SOLUTION: Here is the trip plan for visiting the 10 European cities for 22 days:
**aay 1-5:** Arriving in Munich and visit Munich for days.
**Day 5:** Fly from Munich to Krakow.
**Day 5- Visit Krakow for 3 days.
**Day 7:** Fly from Krakow to Helsinki.
**Day *- Visit Helsinki for 4 days.
**Day 10:** Fly from Helsinki to Hamburg.
DDayy 10-11:** Visit Hamburg for 2 days.
+Day 11:** Fly from Hanburg
Porto.
*Dayy *1-12:** Visit Porto for 2 days.
*Day 12:** Fly from Porto to Frankfurt.
+Dyyy 1* Visit Frankfurt for 2 days.
**aay 13:** Fly from Frankfurt to Prague.
**aay 13-14:** Visit Prague for 2 days.
**aay 14:** Fly from Prague to Athens.
*Da 14-17:** Visit Athens for 4 days.
+Dayy 17:** Fly from Athens to Istanbul.
Day 17-21:** Visit Istanbul for days.
+ay 21:** Fly from Istanbul to Vilnius.
++aay 21-22:** Visit Vilnius for days.
TASK:
You plan to visit 10 European cities for 23 days in total. You only take direct flights to commute between cities. You would like to visit Stuttgart for 2 days. You would like to visit Split for 2 days. You are going to attend a wedding in Split between day 22 and day
23. You would like to visit Vienna for 5 days. You want to spend 4 days in Madrid. You plan to stay in Athens for 2 days. You would like to visit London for 3 days. During day 8 and day 10, you have to attend conference in London. You plan to stay in Paris for 3 days. You
want to meet friend in Paris between day 10 and day 12. You plan to stay Reykjavik for days. You have to attend workshop in Reykjavik between day 16 and day 17. You want spend days in Seville. You want to spend 5 days in Milan. You would like to meet your friends Milan between day and day 21
together.
Here are the cities that have direct flights:
Athens and Paris, Athens and Vienna, Madrid and Vienna, Madrid and Split, Vienna and Stuttgart, Paris and Milan, London and Vienna, London and Milan, Paris and Reykjavik, Athens and London, from Reykjavik to Stuttgart, Seville and Milan, from Reykjavik to Madrid, London and Stuttgart, Milan and Stuttgart, Vienna and Reykjavik, Athens and Split, Athens and Milan, Madrid and Athens, Madrid and London, Paris and Split, London and Paris, Seville and Vienna, Vienna Milan, Athens and Stuttgart, Madrid and Paris, Seville and Madrid, from Reykjavik Athens, Vienna and Split, London and Split, Stuttgart and Split, Seville and Paris, Paris and Stuttgart, Reykjavik and Milan, London and Reykjavik, Madrid and Milan, Paris and Vienna, Milan and Split.
Find trip plan of visiting the cities for days by taking direct flights commute between them.
SOLUTION: Here is the trip plan for visiting the 10 European cities for 23 days:
+*y 1-4:** Arriving in Seville visit Seville for days.
D*Day 4:** Fly from Seville to Madrid.
**aa 4-7:** Visit Madrid for days.
65

---

Published in Transactions on Machine Learning Research Aprri/20255)
+Da 7:** Fly from Madrid Athens.
**Day 7-8:** Visit Athens for 2 days.
**Day 8:** Fly from Athens London.
**Day 8-10:** Visit London for 3 days.
+aaay 10:** Fly from London to Paris.
**Day 10-12:** Visit Paris for 3 days.
**aay 12:** Fly from Paris Vienna.
**Day **-16:** Visit Vienna for 5 days.
**aay 16:** Fly from Vienna to Reykjavik.
**aay 16-17:** Visit Reykjavik for 2 days.
**aay 17:** Fly from Reykjavik to Milan.
**Day 17-21:** Visit Milan for 5 days.
+Daay 21:** Fly from Milan to Stuttgart.
+*aay 21-22:** Visit Stuttgart for 2 days.
**Day 22:** Fly from Stuttgart to Split.
**Day 22-23:** Visit Split for days.
TASK:
You plan to visit 10 European cities commute between cities. You would like to visit Berlin for 2 days. You would like to visit Riga for days. During day and day 6, you have to attend a conference in Riga. You want to spend 3 days in Barcelona, You would like to visit Lyon for 4 days. You would like to meet your friends at Lyon between day and day 11 to tour together. You plan to stay in Naples for 2 days. You plan to stay in Venice for days. You want to spend 5 days in Helsinki. You plan to visit relatives in Helsinki between day 21 and day 25. You plan to stay in Rome for 5 days. You want to spend 3 days in Vilnius. You want spend days in Amsterdam. You are going to attend wedding in Amsterdam between day 19 and day 21.
days in total. You only direct flights
Here are the cities that have direct flights:
Berlin and Amsterdam, Rome and Helainki, Rome and Lyon, Naples and Amsterdam, Riga and Barcelona, Rome and Venice, Riga and Amaterdam, from Riga to Vilnius, Barcelona and Berlin, Rome and Barcelona, Rome and Amsterdam, Barcelona and Venice, Berlin and Helsinki, Amsterdam and Helsinki, Vilnius and Helsinki, Rome and Berlin, from Rome to Riga, Barcelona and Amsterdam, Venice and Naples, Barcelona and Lyon, Naples Berlin, Barcelona and Helsinki, Venice and Helsinki Earcelona and Naples, Vilnius and Amsterdam, Vemice and Amsterdam, Lyon and
Venice, Naples and Helsinki, Riga and Berlin, Rome and Naples, Venice and Berlin, Riga and Helsinki Berlin and Vilnius, Lyon and Amsterdam.
Find trip plan of visiting the cities for 25 days by taking direct flights commute between them.
SOLUTION:
Here is the trip plan for visiting the 10 European cities for 25 days:
+*Day 1-5:** Arriving Rome and visit Rome days.
++Day 5:** Fly from Rome to Riga.
**Day 5-6:** Visit Riga for days.
**Day 6:** Fly from Riga to Barcelona.
**Day 6-8:** Visit Barcelona for days.
**Day 8:** Fly from Barcelona to Lyon.
+*aay 8-11:## Visit Lyon for days.
**aay 11:** Fly from Lyon to Venice.
**aay 11-15:** Visit Venice for 5 days.
+Day 15:** Fly from Vemice to Naples.
+*aay 15-16:** Visit Naples for 2 days.
+Day 16:** Fly from Naples to Berlin.
+Dayy 16-17:** Visit Berlin for 2 days.
+Day 17:** Fly from Berlin Vilnius.
Day 17-19:** Visit Vilnius for 3 days.
Day 19:** Fly from Vilnius Amsterdam.
**Day *9-21:** Visit Amsterdam for 3 days.
**ay 21:** Fly from Amsterdam Helsinki.
66

---

Published in Transactions on Machine Learning Research (April/2025)
+*Day 21-25:** Visit Helsinki for days.
TASK: You plan to visit 10 European cities for 27 days in total. You only take direct flights commute between cities. You would like to visit Prague for 5 days. You have to attend a workshop Prague between day 7 and day 11. You would like visit Helsinki for days. You are going to attend a wedding in Helsinki between day 15 and day 17. You plan to stay in Tallinn for days. You want to meet friend in Tallinn between day 6 and day You want to spend 4 days in Edinburgh. You want to spend days in Paris. You want to spend 4 days Vienna. You plan to stay in Lisbon for days. From day 11 to day 15, there want to attend in Lisbon. You want spend days in Budapest. You plan to stay in Lyon for 2 days. You plan to stay in Brussels for 2 days. You would like to meet your friends Brussels between day 1 and day to tour together.
annual show you
Here the cities that have direct flights:
Prague and Lyon, Brussels and Lisbon, Helsinki and Budapest, Vienna and Lyon, Paris and Tallinm, Brussels and Prague, Brussels and Helainki, Prague and Helainki, Brussels and Vienna, Brussels and Budapest, Lisbon and Budapest, Tallinn and Helsinki, Brussels and Paris,
Brussels and Tallinn, Lisbon and Lyon, Prague and Lisbon, Paris Prague, Helsinki and Edinburgh, Prague and Edinburgh, Tallinn and Prague, Brussels and Lyon, Paris and Lisbon, Helsinki and Vienna, Paris and Helsinki, Paris and Budapest, Edinburgh and Budapest, Brussels and Edinburgh, Lisbon and Vienna, Paris and Lyon, Lisbon and Helsinki, Prague and Vienna, Paris and Vienna, Prague and Budapest, Paris and Edinburgh, Budapest and Vienna.
Find a trip plan of visiting the cities for 27 days by taking direct flights to commute between them.
SOLUTION: Here is the trip plan for visiting the 10 European cities for 27 days:
**Day 1-2:** Arriving in Brussels and visit Brussels for 2 days.
++aay 2:** Fly from Brussels Paris.
*+Day 2-6:** Visit Paris for 5 days.
+*aay 6:** Fly from Paris to Tallinn.
**aay 6- Visit Tallinn for 2 days.
**Day 7:** Fly from Tallinn to Prague.
**Day 7-11:** Visit Prague for 5 days.
**aay 11:** Fly from Prague to Lisbon.
*Daa 11-15:** Visit Lisbon for 5 days.
**aay 15:** Fly from Lisbon to Helsinki.
+*ayy 15-17:** Visit Helsinki for days.
+Dayy 17:** Fly from Helsinki to Edinburgh.
+Daay 17 Visit Edinburgh for days.
**Day *0:** Fly from Edinburgh to Budapest.
+*aay 20-23:** Visit Budapest for days.
+*a 23:** Fly from Budapest to Vienna.
**aay 23-26:** Visit Vienna for days.
D*y 26:** Fly from Vienna to Lyon.
D*y 26-27: Visit Lyon for days.
Query:
You plan to visit 10 European cities for 25 days in total. You only take direct flights to commute between cities. You plan to stay in Berlin for days. You have attend workshop in Berlin between day 12 and day 15. You would like to visit Prague for 2 days. You plan to stay in Stuttgart for 5 days. You would like visit Manchester for days. You want spend 2 days in Nice. You would like to visit Reykjavik for 2 days. You would like to visit Florence for 3 days. You plan stay in Vilnius for 5 days. You would like
your
friends at Vilnius between day 15 and day 19 to tour together. You plan to stay in Oslo for days. You would like to visit Dubrovnik for days. You plan to visit relatives in Dubrovnik between day and day 4.
67

---

Published in Transactions on Machine Learning Research (02))
Here the cities that have direct flights:
from Reykjavik to Stuttgart, Manchester and Stuttgart, Nice and Berlin, Oslo and Prague, Stuttgart and Berlin, Manchester and Nice, Reykjavik and Oslo, Reykjavik and Prague, Manchester and Prague, Reykjavik and Berlin, Dubrovmik and Manchester, Manchester and Oslo, Manchester and Berlin, Prague and Florence, Berlin and Vilmius, Dubrovnik and Oslo, Nice and Oalo, Berlin and Oslo, Nice and Reykjavik, Vilnius and Oslo.
Find trip plan of visiting the cities for 25 days by taking direct flights to commute between them.
C. OF ol-previeeeannnnnnnn
SOLUTION: Here is the trip plan for visiting the 10 European cities for 25 days:
**aay 1-4:** Arriving in Dubroviik and visit Dubrovnik for days.
++yay 4:** Fly from Dubrovnik to Manchester.
+*yay 4-7:** Visit Manchester for 3 days.
+ay 7:** Fly from Manchester to Stuttgart.
**aay 7-12:** Visit Stuttgart for 5 days.
+*aay 12:** Fly from Stuttgart Berlin.
++Day 12-15:** Visit Berlin for 4 days. (Attend workshop between day 12 and day 15)
++aay 15:** Fly from Berlin Vilnius.
+*y 15-19:** Visit Vilnius for days. (Meet friends between day day 19)
**Day 19:** Fly from Vilnius to Oslo via Berlin.
+*Day 19-23:** Visit Oslo for days.
++Day 23:** Fly from Delo to Reykjavik.
++Day 23-25:** Visit Reykjavik for 2 days.
FEEDBACC FROM METACDNTROLLER
Number of cities in plan 7, expected
R.2 Back Prompt
You expert planning trips. Fix the below given trip schedule auch that meets many specified constraints as possible. You are given few constraints regarding the cities to visit and the durations of staying at each city. You are also given the flight information between the cities.
Produce only one plan, strictly adhering to the format shown the examples belox, without any extra information
The produced plan will be parsed using the following python code:
< 'python
response def parse_response (response: str): .""Parse the response.
Returns parsed plan list of (city, stay_days) tuples.
68

---

Published in Transactions on Machine Learning Research (April/2255)
Args:
response:
Raw response from the model.
Returns:
Structured plan after parsing.
pattern_
pattern_visit visit r' r'\d+=
pattern_flight r'. +Day ((d+). *from (A++) (A++)
battern_days r'European cities for (Ad+) days'
days, flights, flight_dayy [ D, total_dayy None
for piece in response. split(""))
days_ aare_maach-- = re. fi.danl (patterrpppttttntaaa days, piece) if days_matche match:
total_dayy int(dss__macch(0)
visit_ match re. www.ainnnllllll. (patter visit, piece)
visit aasit_match:
days.appent (visit_ (visit_matchh())
end_day int (eiisvt_matchh [o] [o] sppit(((()) [1])
# Reach the end of the plan, stop to avoid parsing alternative plans.
if end_day total_days:
break
Lright_match = re. fi.danl (aataalllppttttr___aa flight, piace) if flight_mated
flights. sights..ppenddCC.
visit_ visit_cities, parsed_plan = O, 0
for flight_day, begin_cite city, end_ aid_yity_o flights: flight_den fiight_dyysoo ayysssappendddnn (flight_day))
not
visit_cities. append (begin_cctt) visit_cities. append (end_city) else:
risit_cctien cpppee (end_citt))))
if not days or not flights or not visit_cities: return
last_day = int (daass[-1) im((dxyt-----eepli(((-))111 spppit(()))))
flight_days [1] flight_days:
for i, visit_city in enumerate (visit_citiessss))
city_stay flight_daal _ayss_dda____ flight_daa 1 parsed plaiipllaaaaaa append((visit......ooo(((((((p).
city_stay))
return parsed_plas _plan
<et
Here are a few example tasks and solutions:
You plan to visit 10 European cities for 21 days in total. You only take direct flights to between cities. You plan to stay in Vienna for days. You want spend days Frankfurt You want to spend 2 days in Delo. You are going to attend a wedding in Oslo between day and day 21. You want spend 3 days Prague. You would like to visit Valencia for days. You want to meet friend in Valencia between day 17 and day 18. You plan to stay in Dubrovnik days. You would like to visit Edinburgh for days. From day to
TASK:
69

---

Published in Transactions on Machine Learning Research (April/2025)
day 5, there is annual show you want attend in Edinburgh. You plan to stay in London for days. You plan to visit relatives in London between day 12 and day 13. You would like to visit Munich for 3 days. You would like meet your friends Munich between day 18 and day 20 to tour together. You would like to visit Budapest for 3 days.
Here are the cities that have direct flights:
Valencia and Munich, Vienna and Munich, Vienna Valencia, London Budapest, London Oslo, Edinburgh and Budapest, Frankfurt and Budapest, Frankfurt and London, Prague and Oslo, Edinburgh and Oslo, Edinburgh and Munich, Prague and Munich, London and Prague, Edinburgh and London, Edinburgh and Frankfurt, Dubrovnik and Munich, Dubrovnik and Vienna, Munich and Oslo, Dubrovnik and Oalo, Budapest and Munich, Frankfurt and Prague, Vienna and London, Frankfurt Vienna, Frankfurt and Oslo, Frankfurt and Munich, Vienna and Oslo, Vienna and Prague, Budapest and Oslo, Budapest and Prague, London and Valencia, London and Munich, Frankfurt and Dubrovnik, Prague and Valencia, Frankfurt and Valencia, Edinburgh and Prague, Vienna and Budapest.
Find a trip plan of visiting the cities for 21 days by taking direct flights to commute between them.
SOLUTION:
Here is the trip plan for visiting the 10 European cities for 21 days:
Day 1-5:** Arriving in Edinburgh and visit Edinburgh for 5 days.
**Day 5:** Fly from Edinburgh to Frankfurt.
**Day 5-9:** Visit Frankfurt for 5 days.
**Day 9:** Fly from Frankfurt to Dubrovnik.
**aay 9-10:** Visit Dubrovnik for days.
**Day 10:** Fly from Dubrovnik to Vienna.
+Daay 10-12:** Visit Vienna for 3 days.
+*aay 12:** Fly from Vienna to London.
+Dayy 12-13:** Visit London for days.
*4aay 13:** Fly from London to Budapest.
+Day 13-15:** Visit Budapest for days.
+Dy 15:** Fly from Budapest to Prague.
**Day 15-17:** Visit Prague for days.
**Day 17:** Fly from Prague to Valencia.
+*aay 17-18:** Visit Valencia for days.
**aay 18:** Fly from Valencia to Munich.
**Day 18-20:** Visit Munich for days.
++Day 20:** Fly from Munich to Oslo.
++Day 20-21:** ** Visit Oslo for 2 days.
TASK:
You plan to visit 10 European cities for 22 days in total. You only take direct flights to commute between cities. You want to spend 4 days Helsinki. You plan to stay in Vilnius for 2 days. You plan to stay in Istanbul for 5 days. From day 17 to day 21, there is annual show you want to attend Istanbul. You want to spend 2 days Hamburg. You plan to stay in Porto for 2 days. During day 11 and day 12, you have to attend conference in Porto. You want to spend days in Athens, You would like to visit Prague for 2 days. You plan to visit relatives in Prague between day 13 and day 14. You want to spend 2 days in Frankfurt. You want to spend days in Krakow. You want to spend 5 days Munich. You would like to meet your friends Munich between day 1 and day tour together.
Here the cities that have direct flights:
from Krakow to Vilnius, Helsinki and Hamburg, Hamburg and Athens, Munich and Frankfurt, Hamburg and Porto, Munich and Istanbul, Prague and Athens, Frankfurt and Athens, Munich and Athens, Munich and Prague, from Vilnius to Munich, Hamburg and Istanbul, Frankfurt and Istanbul, Munich and Krakow, Munich and Hamburg, Munich and Helsinki, Prague and Istanbul,
Frankfurt and Vilnius, Helsinki and Istanbul, Athens and Vilnius, Krakow and Frankfurt, Helsinki Frankfurt, Porto and Frankfurt, Frankfurt and Prague, Istanbul and Vilnius, Krakow and Istanbul, Krakov and Prague, Munich and Porto, Helsinki and Vilnius, Helsinki and Prague, Porto and Istanbul, Hamburg Frankfurt, Krakow and Helsinki, Athens and Istanbul.
70

---

Published in Transactions on Machine Learning Research (April/2025)
Find a trip plan of visiting the cities for 22 days by taking direct flights to commute between
them.
SOLUTION: Here is the trip plan for visiting the 10 European cities for 22 days:
**Day 1-5:** Arriving in Munich and visit Munich for 5 days.
**Day 5:** Fly from Munich Krakow.
**Day 5-7:** Visit Krakow for 3 days.
**Day 7:** Fly from Krakow to Helsinki.
**Day 7-10:** Visit Helsinki for 4 days.
**aay 10:** Fly from Helsinki to Hamburg.
**aay 10-11:#* Visit Hamburg for 2 days.
**aay 11:** Fly from Hamburg to Porto.
*Dayy 11-12:** Visit Porto for days.
**aay 12:** Fly from Porto to Frankfurt.
+aaay 12-13:** Visit Frankfurt for 2 days.
Day 13:** Fly from Frankfurt to Prague.
DDay 13-14:** Visit Prague for days.
**aay 14:** Fly from Prague to Athens.
*Dayy ** Visit Athens for days.
+ay 17:** Fly from Athens to Istanbul.
*aaay 17-21:** Visit Istanbul for days.
**Day 21:** Fly from Istanbul to Vilnius.
**Day 21-22:** ** Visit Vilnius for 2 days.
TASK:
You plan to visit 10 European cities for 23 days in total. You only take direct flights to commute between cities. You would like to visit Stuttgart for days. You would like to visit Split for 2 days. You are going to attend a wedding in Split between day 22 and day You would like visit Vienna for 5 days. You in Athens for 2 days. You would like to visit London for 3 days. During day 8 and day 10, you have to attend a conference want to meet friend in Paris between day 10 and day 12. You plan to stay in Reykjavik for days. You have to attend workshop in Reykjavik between day 16 and day 17. You want to spend days in Seville. You want to spend 5 days in Milan. You would like to meet your friends at Milan between day 17 and day 21 to tour together.
apend days in Madrid. You plan stay
London. You plan to stay in Paris for 3 days. You
Here are the cities that have direct flights:
Athens and Paris, Athens and Vienna, Madrid and Vienna, Madrid and Split, Vienna and Stuttgart, Paris and Milan, London and Vienna, London and Milam, Paris and Reykjavik, Athens and London, from Reykjavik Stuttgart, Seville and Milan, from Reykjavik Madrid, London and Stuttgart, Milan and Stuttgart, Vienna and Reykjavik, Athens and Split, Athens and Milan, Madrid Athens, Madrid and London, Paris and Split, London and Paris, Seville and Vienna, Vienna and Milan, Athens and Stuttgart, Madrid and Paris, Seville and Madrid, from Reykjavik to Athens, Vienna and Split, London and Split, Stuttgart and Split, Seville and Paris, Paris Stuttgart, Reykjavik and Milan, London and Reykjavik, Madrid and Milan, Paris and Vienna, Milan and Split.
Find a trip plan of visiting the cities for 23 days by taking direct flights to commute between them.
SOLUTION: Here is the trip plan for visiting the 10 European cities for 23 days:
D*y 1-4:** Arriving in Seville and visit Seville for 4 days.
+Day 4:** Fly from Seville to Madrid.
DDaay 4-7:** Visit Madrid for 4 days.
+Da 7:** Fly from Madrid Athens.
Day 7-8:** Visit Athens for 2 days.
Day 8:** Fly from Athens London.
D*aay 8-10:*# Visit London for 3 days.
**ay 10:** Fly from London to Paris.
71

---

Published in Transactions on Machine Learning Research (April/2025) 2025)
+aDay 10-12:** Visit Paris for days.
**Day 12:** Fly from Paris to Vienna.
**Day 12 Visit Vienna for 5 days.
**Day 16:** Fly from Vienna to Reykjavik.
+aaay 16-17:** Visit Reykjavik for 2 days.
**Day 17:** Fly from Reykjavik to Milan.
**aay 17-21:** Visit Milan for days.
**Day 21:** Fly from Milan to Stuttgart.
**Day 21-22:** Visit Stuttgart for 2 days.
+*Day 22:** Fly from Stuttgart to Split.
++Day 22-23: Visit Split for 2 days.
TASK:
You plan to visit 10 European cities for 25 days in total. You only take direct flights to commute between cities. You would like to visit Berlin for 2 days. You would like visit Riga for 2 days. During day 5 and day 6, you have to attend a conference in Riga. You want to spend 3 days Barcelona. You would like visit Lyon for 4 days. You would like meet your friends at Lyon between day 8 and day 11 to tour together. You plan to stay in Naples for 2 days. You plan stay Venice for days. You want to spend 5 days Helsinki. You plan to visit relatives in Helsinki between day 21 and day 25. You plan to stay in Rome for days. You want to spend days Vilnius. You want to spend days in Amsterdam You are going to attend wedding in Amsterdam between day 19 and day 21.
Here are the cities that have direct flighta:
Berlin and Amsterdam, Rome and Helsinki, Rome and Lyon, Naples and Amsterdam, Riga and Barcelona, Rome and Venice, Riga and Amsterdam, from Riga to Vilnius, Barcelona and Berlin, Rome and Barcelona, Rome and Amsterdam, Barcelona and Vemice, Berlin and Helsinki, Amsterdam and Helsinki, Vilnius and Helsinki, Rone and Berlin, from Rome to Riga, Barcelona and Amsterdam, Venice and Naples, Barcelona and Lyon, Naples and Berlin, Barcelona and Helsinki, Venice and Helsinki Earcelona and Naples, Vilmius and Amsterdam, Venice and Amsterdam, Lyon and Venice, Naples and Helsinki, Riga and Berlin, Rome and Naples, Venice and Berlin, Riga and Helsinki Berlin and Vilnius, Lyon Amsterdam.
Find a trip plan of visiting the cities for 25 days by taking direct flights to commute between them.
SOLUTION:
Here is the trip plan for visiting the 10 European cities for 25 days:
**Day 1-5:** Arriving in Rome and visit Rome for 5 days.
++Day 5:** Fly from Rome Riga.
+*Day 5-6:** Visit Riga for 2 days.
+*Day 6:** Fly from Riga Barcelona.
**Day 6-8:** Visit Barcelona for 3 days.
**Day 8:** Fly
Barcelona to Lyon.
**aay 8-11:** Visit Lyon for 4 days.
+Daay 11:** Fly from Lyon to Venice.
**aay 11-15:** Visit Venice for 5 days.
**aay 15:** Fly from Venice to Naples.
**Day 15-16:** Visit Naples for 2 days.
**aay 16:** Fly from Naples to Berlin.
**aay 16-17:** Visit Berlin for 2 days.
**aay 17:** Fly from Berlin to Vilnius.
+Daay 17-19:** Visit Vilnius for days.
+*a 19:** Fly from Vilmius to Amsterdam.
Daa 19-21:** Visit Amsterdam for days.
Day 21:** Fly from Amsterdam to Helsinki.
+Dayy 21-25:** Visit Helsinki for days.
TASK: You plan to visit 10 European cities commute between cities. You would like to visit Prague for 5 days. You have to attend workshop Prague between day 7 and day 11. You would like to visit Helsinki for days.
days in total. You only take direct flights to
72

---

Published in Transactions on Machine Learning Research (April/2025)
You are going attend wedding in Helsinki between day and day 17. You plan to stay in Tallinn for 2 days. You want to meet a friend in Tallinn between day 6 and day 7, You want to spend 4 days in Edinburgh. You want spend days in Paris. You want in Vienna. You plan to stay in Lisbon for days. From day 11 to day 15, there is a annual show you want attend in Lisbon. You want apend days in Budapest. You plan stay Lyon for 2 days. You plan to stay in Brussels for 2 days. You would like to meet your friends Brussels between day and day to tour together.
spend 4 days
Here are the cities that have direct flights:
Prague and Lyon, Brussels and Lisbon, Helsinki and Budapest, Vienna and Lyon, Paris and Tallinn, Brussels and Prague, Brussels and Helsinki, Prague and Helsinki, Brussels and Vienna, Brussels and Budapest, Lisbon and Budapest, Tallinn and Helsinki, Brussels and Paris,
Brussels and Tallinn, Lisbon and Lyon, Prague and Lisbon, Paris and Prague, Helsinki and Edinburgh, Prague and Edinburgh, Tallinn and Prague, Brussels and Lyon, Paris and Lisbon, Helsinki and Vienna, Paris and Helsinki, Paris and Budapest, Edinburgh and Budapest, Brussels and Edinburgh, Lisbon and Vienna, Paris and Lyon, Lisbon and Helsinki, Prague and Vienna, Paris and Vienna, Prague and Budapest, Paris and Edinburgh, Budapest and Vienna.
Find trip plan of visiting the cities for 27 days by taking direct flights to commute between them.
SOLUTION: Here is the trip plan for visiting the 10 European cities for 27 days:
**Day 1-2:** Arriving in Brussels and visit Brussels for 2 days.
**Day 2:** Fly from Brussels to Paris.
**aay 2* Visit Paris for 5 days.
**Day 6:** Fly from Paris to Tallinn.
+*aay 6-7:** Visit Tallinn for days.
**Day 7:** Fly from Tallinn to Prague.
+Daay 7-11:** Visit Prague for days.
**Day 11:** Fly from Prague to Lisbon.
+Day 11-15:** Visit Lisbon for days.
*Da 15:** Fly from Lisbon to Helsinki.
**Day 15-17:** Visit Helsinki for days.
**Day 17:** Fly from Helsinki to Edinburgh.
**aay 17-20:** ** Visit Edinburgh for 4 days.
**aay 20:** Fly from Edinburgh to Budapest.
**Day 20-23:** ** Visit Budapest for days.
+*Day 23:** Fly from Budapest to Vienna.
+*y *3-26:** Visit Vienna for 4 days.
+*aay 26:** Fly from Vienna to Lyon.
**Day 26-27 ** Visit Lyon for 2 days.
Query:
You plan to visit 10 European cities for 25 days in total. You only take direct flights to commute between cities. You plan to stay in Berlin for days. You have to attend workshop in Berlin between day 12 and day 15. You would like to visit Prague for 2 days. You plan to stay in Stuttgart for 5 days. You would like to visit Manchester for days. You want to spend 2 days in Nice. You would like to visit Reykjavik for 2 days. You would like to visit Florence for 3 days. You plan to stay in Vilnius for 5 days. You would like to meet your
friends Vilnius between day and day 19 tour together. You plan to stay in Oalo for days. You would like to visit Dubrovnik for days. You plan to visit relatives in Dubrovnik between day and day
Here the cities that have direct flights:
from Reykjavik to Stuttgart, Manchester and Stuttgart, Nice and Berlin, Oslo and Prague, Stuttgart Berlin, Manchester Nice, Reykjavik and Oslo, Reykjavik and Prague, Manchester and Prague, Reykjavik and Berlin, Dubrovnik and Manchester, Manchester and Oslo,
73

---

Published in Transactions on Machine Learning Research (April/2025)
Manchester and Berlin, Prague and Florence, Berlin and Vilmius, Dubrovnik and Oslo, Nice and Oslo, Berlin and Oslo, Nice and Reykjavik, Vilnius and Oalo.
Find a trip plan of visiting the cities for 25 days by taking direct flights to commute between them.
Incorrect plan natural language: SOLUTION: Here is the trip plan for visiting the 10 European cities for 25 days:
++Day 1-4:** Arriving in Dubrovnik and visit Dubrovnik for 4 days.
+*Day 4:** Fly from Dubrovnik to Manchester.
+*aay 4-7:** Visit Manchester for days.
+*Day 7:** Fly from Manchester Stuttgart.
+ay 7-12:** Visit Stuttgart days.
+*aay 12:** Fly from Stuttgart Berlin.
**Day 12-15:** Visit Berlin for days. (Attend workshop between day 12 and day 15)
+Daay 15:** Fly from Berlin to Vilnius.
+*aay *5-19:** Visit Vilmius for 5 days. (Meet friends between day 15 and day 19)
+*aay 19:** Fly from Vilmius to Oslo via Berlin.
+*aay 19-23:** Visit Dalo for 4 days.
+*y 23:** Fly from Oslo to Reykjavik.
**Day 23-25:** ** Visit Reykjavik for 2 days.
Incorrect plan in parsed format:
[['Dubrovnik', 4], ['Manchester', 4], ['Stuttgart', 6], ['Berlin', 4], ['iilnus', 5], ['Oslo', 5], ('Reykjavik', 3]]
Errors with the above plan: Number of cities in plan is 7, expected 10
Give the corrected plan in natural language such that it can be parsed by the above python code. Strictly adhere to the format shown the examples below, without any extra information.
Corrected plan:
OF ol-previee-
SOLUTION: Here is the trip plan for visiting the 10 European cities for 25 days:
**Day 1-4:** Arriving in Dubrovnik and visit Dubrovnik for days. (Visit relatives between day and day 4)
+*y 4:** Fly from Dubrovnik to Manchester.
* Day 4-6:** Visit Manchester for 3 days.
**Day 6:** Fly from Manchester to Nice.
74

---

Published in Transactions on Machine Learning Research (April/2255)
+*Day 6-7:** Visit Nice for days.
++Day 7:** Fly from Nice Reykjavik,
+aDay 7- Visit Reykjavik for days.
Day 8:** Fly from Reykjavik to Stuttgart.
+ay 8-10:** Visit Stuttgart for days.
+*Day 10:** Fly from Stuttgart to Prague.
+*aay 10-11:** Visit Prague for 2 days.
**Day 11:** Fly from Prague to Berlin.
Day 11-15:** Visit Berlin for 4 days. (Attend workshop between day 12 and day 15)
+*y 15:** Fly from Berlin to Vilnius.
Day *5 Visit Vilnius for 5 days. (Meet friends between day 15 and day 19)
+*aay 19:** Fly from Vilnius to Oslo.
++Day 19-22:** Visit Oslo for 4 days.
+*Day 22:** Fly from Oslo to Florence via Prague.
++Day 22-25:** Visit Florence for days.
S Prompts for Calendar Scheduling
S.1 First Iteration
You expert scheduling meetings. You are given few constraints the existing schedule of each participant, the meeting duration, and possibly some preferences on the meeting time. Note there exists solution that works with existing schedule every participant. Produce only one meeting time, strictly adhering to the format shown in the examples below, without providing any extra information. Here are few example tasks and solutions:
TASK: You need to schedule meeting for George, Steven, Aaron, Patrick Cynthia for half an hour between the work hours of 9:00 to 17:00 on Monday.
Here the existing schedules for everyone during the day:
Georgehas no meetings the whole day.
Steven is free the entire day.
Aaron has blocked their calendar on Monday during 9:00 to 10:00, 11:30 to 12:00, 15:30 to 17:00; Patrick has blocked their calendar Monday during 9:00 to 9:30, 10:00 to 11:00, 11:30 to 12:00, 12:30 to 14:00, 15:00 to 15:30;
Cynthia busy on Monday during 9:00 to 9:30, 10:30 to 11:30, 12:30 to 14:30, 15:00 16:30;
Aaron can not meet on Monday after 12:30. Find a time that works for everyone's schedule constraints.
SOLUTION: Here is the proposed time: Monday, 12:00 12:30
TASK:
You need to schedule a meeting for Elizabeth, Eugene, Nancy, Justin and Roy for half an hour between the work hours of 9:00 to 17:00 Monday.
Here the existing schedules for everyone during the day:
75

---

Published in Transactions on Machine Learning Research (April/2025) 2025)
Elizabeth's calendar is wide open the entire day.
Bugene has blocked their calendar on Monday during 12:00 to 12:30, 13:30 to 14:00, 15:00 to 16:00;
Nancy has meetings Monday during 10:30 to 11:00, 12:00 13:00, 14:00 to 14:30, 15:00 to 16:00;
Justin has meetings on Monday during 9:30 to 10:00, 10:30 to 11:30, 12:00 to 12:30, 13:00 to 13:30,
14:30 17:00;
Roy is busy on Monday during 10:00 to 11:00, 11:30 to 12:30, 13:30 to 14:00, 14:30 to 16:00;
Find time that works for everyone's schedule and constraints. SOLUTION: Here is the proposed time: Monday, 9:00 9:30
TASK: You need to schedule meeting for Mason, Linda, Barbara, Roger and Lisa for half an hour between the work hours of 9:00 to 17:00 Monday.
Here the existing schedules for everyone during the day: Masonhas no meetings the whole day.
Linda busy Monday during 9:30 to 11:00, 12:00 12:30, 13:00 13:30, 16:30 17:00;
Barbara has blocked their calendar on Monday during 9:00 to 9:30, 12:00 to 13:00, 15:30 to 16:00;
Roger has meetings on Monday during 9:00 9:30, 13:00 14:30, 15:00 to 17:00;
Lisa has blocked their calendar on Monday during 9:00 to 10:00, 10:30 to 11:30, 12:00 to 12:30, 13:00 to 14:30, 15:00 to 17:00;
Mason would rather not meet on Monday after 13:30. Find a time that worke for everyone's schedule and constraints.
SOLUTION:
Here is the proposed time: Monday, 11:30 12:00
TASK: You need to schedule a meeting for Thomas, 0livia, Doris, Amber and Ronald for half an hour between the work hours of 9:00 to 17:00 Monday.
Here the existing schedules for everyone during the day:
Thomas has meetings on Monday during 12:00 to 12:30, 16:30 to 17:00;
0livia has blocked their calendar on Monday during 10:30 to 11:30, 13:00 14:00, 16:00 to 17:00;
Doris is busy on Monday during 13:30 to 14:00, 16:30 to 17:00;
Amber busy on Monday during 10:00 to 10:30, 12:00 to 13:00, 14:00 to 15:00, 15:30 to 17:00;
Ronald has meetings on Monday during 9:30 to 11:30, 12:00 to 13:00, 13:30 to 14:00, 15:00 to 15:30;
Ronald would rather not meet Monday before 11:30. Find time that works for everyone's schedule and constraints.
SOLUTION: Here is the proposed time: Monday, 11:30 12:00
TASK: You need to schedule meeting for Patrick, Kathleen, Frank, Patricia and Carolyn for half hour between the work hours of 9:00 to 17:00 on Monday.
Here are the existing schedules for everyone during the day: Patrick free the entire day.
Kathleen has meetings on Monday during 9:30 to 10:00, 15:00 to 15:30;
Frank has blocked their calendar on Monday during 11:00 11:30, 15:30 to 17:00;
Patricia is busy on Monday during 9:00 to 12:00, 14:30 to 15:00, 16:00 to 16:30;
Carolyn has meetings on Monday during 9:00 to 9:30, 11:00 to 12:30, 13:00 to 17:00;
Find a time that works for everyone's schedule and constraints. SOLUTION: Here is the proposed time: Monday, 12:30 13:00
Query:
TASK: You need to schedule a meeting for Stephen, Elijah, William, Jeremy and Timothy for half an hour between the work hours of 9:00 17:00 Monday.
Here are the existing schedules for everyone during the day:
Stephen is free the entire day. Elijah is busy on Monday during 9:00 to 9:30, 12:30 to 13:00, 14:30 to 15:00, 16:00 to 16:30;
76

---

Published in Transactions on Machine Learning Research (April/2025)
William busy Monday during 9:30 10:00, 15:30 16:00;
Jeremy is busy on Monday during 9:00 to 9:30, 10:00 to 12:00, 13:00 to 15:00, 15:30 to 17:00;
Timothy busy Monday during 10:00 to 10:30, 11:30 to 14:30, 15:30 to 16:00;
Find time that works for everyone schedule and constraints. SOLUTION:
RESPONSE OF ol-mini=
SOLUTION: Here is the proposed time: Monday, 15:00 15:30
77


Reviews:
[{"review_id": "s9OlOJIrNt", "reviewer": "Reviewer_Bwjz", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "In summary, useful evaluations but inappropriate, unjustified or even misleading statements, which can all be fixed.\n\nSome of the results are not surprising, e.g., that o1 outperforms GPT-4 on most of the tasks, since this is coherent with OAI's own evaluations. It is also not surprising that results deteriorate with plan length, but it is good to quantify this on these tasks. \n\nSome of the results are much more significant, in particular on the importance of measuring and taking into account the accuracy-cost trade-off.\n\nIn general, the language used in the paper is highly depreciative of o1 and OpenAI, which is not appropriate for TMLR.\nSeveral value judgments are made, without justification, which is not acceptable, including calling the state-of-the-art LLMs 'approximate retrieval' systems doing 'mildly-generating pattern matching'. In other places, the authors seem to imply that OpenAI made claims about o1 which I have not seen previously, e.g., that it performs robust and general reasoning and planning. They also claim that Chain-of-Thought did not improve on planning tasks, but fail to mention that these systems do improve on other reasoning tasks.\n\nIn fact, my recollection from OAI's documentation is that they claimed advances in reasoning, but not notable advances in planning. \nIt is true that planning is a special form of reasoning, but it may require particular skills that were not particularly developed in the training of o1. OAI did not claim that o1 was a general reasoning system. Be careful to not give that impression and other similar inappropriately phrased statements.\n\nAlso, one should be careful when comparing a generic natural-language based system like o1 and a formal and somewhat task-specific search algorithm. In cases where one can use the latter with success, there is indeed no reason to use the former, but one should also point out where systems like o1 can be used but not the classical planning or reasoning systems.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors evaluate o1-preview and o1-mini on planning and scheduling tasks, with and without obfuscated names. They find the o1 series to outperform GPT-4 but to deteriorate with plan length or problem size.\n\nThey point out that it is important to evaluate the accuracy vs cost trade-off, and perform experiments with GPT-4 (to generate candidate solutions) coupled with a guaranteed external verifier as well as with o1-preview in this setup, finding that the advantage of o1 against GPT-4 vanishes at at least one accuracy-cost trade-off point.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The title is not appropriate. The paper is not about LLM planning (this has been discussed in other works). I would just keep the second part: \"A Systematic Evaluation of the Planning and Scheduling Abilities of o1\".\n\nIs OAI actually making the claim that o1 is \"generally capable of tackling procedural reasoning taks\"? I have not seen that.\nEither document your claim or remove it.\n\nMake a note in section 5 that the modulo approach can only be used when a verifier exists and is computationally efficient, whereas o1 can be used even without a verifier.\n\nModify the language to address the issues above, avoiding claims not justified in this paper.\n\nFor example rephrase \"it cannot plan robustly when faced with harder instances\" into something like \"it outperforms LLMs but results degrade when faced with longer or larger instances\".\n\nIt is also not very neutral to write \"o1 is still fallible and without guarantees\" because OAI did not claim that it was infallible or that it came with guarantees. Please soften the tone more generally.\n\nThe authors sometimes talk about o1 and sometimes about o1-preview and o1-mini. It may be good to clarify that they use the term o1 generically to talk about these two versions (since another version, o1 full, came out recently).\n\nDefine the IPC abbreviation before using it.\n\nDefine q-values.\n\nTypos:\n\"did not rejection\" missing \"do\" in the middle\n\"language typically models\" should be \"language models typically\"", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "GOrspuiBGL", "reviewer": "Reviewer_wCEF", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The presentation of this article is largely clear, and the results included could be considered comprehensive.\n\nHowever, my biggest concern about this article is its (lack of) contribution to the community. People have a consensus that o1 is much better at reasoning and planning. Although this article provides concrete numbers on some of the benchmarks, it is hard to imagine how these numbers can provide us with additional insights into the usage of such models other than those we are already aware of. The analysis, unfortunately, largely builds upon guesswork and therefore is not robust.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This article investigates the reasoning capabilities of GPT o1 preview and GPT o1 mini on some reasoning and planning benchmarks including PlanBench, Sokoban, and planning problems. The authors conclude that the GPT o1-series models surpass the performance of their predecessors by a large margin but at the cost of an increase in price and efficiency.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- It is not suggested that abbreviations be put in titles, especially when those are from new concepts.\n- I don't think it's a good practice to call GPT o1 \"large reasoning models\". First, I don't think LRM is a widely-recognized term. In addition, previous works have adopted the same term for completely different concepts. This may cause confusion.\n- Minor typos such as quotation marks (\"\" in LaTeX source), \"LLM models\" (page 14), etc.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ABlTf7O0nv", "reviewer": "Reviewer_tRV3", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThe task (planning/scheduling) itself is important for the ml community.\n\nIt is important to understand the limitations of a new model paradigm--reasoning model.\n\nThe paper is easy to follow.\n\nThe experiments are comprehensive and interesting.\n\nWeaknesses:\nThe literature review is a little bit insufficient. This paper seems to only be discussing a limited amount of papers from very similar research veins, while there are a lot of other LLM planning papers worth citing.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper evaluates and compares o1 models, as sota LRMs, with previous LLMs. It shows that while LRMs can generally outperform previous models, they require significantly more cost while still failing in extremely difficult instances.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The authors are encouraged to discuss more, diverse works in LLM planning/scheduling.", "broader_impact_concerns": "none", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Even though one of the reviewers (wCEF) questioned the value of analyzing a closed-source model, which is already outdated before the review process is complete, I think there is some value in publishing this work to shed some light on the planning ability of these models. I think TMLR is a perfect venue for such work.

Please incorporate all the changes you made during the rebuttal to the final version of the paper. Also, please try to be more precise in your claims and add a discussion on how this study is still valuable while there are new models coming out and making the previous generation models outdated in a matter of weeks or months.
Decision: Accept (poster)