Robo-Instruct: Simulator-Augmented Instruction Alignment For Finetuning Code LLMs
Authors: Zichao Hu
Junyi Jessy Li
Arjun Guha
Joydeep Biswas
Venue: COLM 2025 (2025)
Keywords: Self-Instruct
Fine-tuning Code LLMs for service robot tasks
Domain-Specific Program Generation
Code LLMs for Robotics
TL;DR: We propose a simulator-augmented approach for generating synthetic training data to fine-tune code LLMs on domain-specific robot tasks.

Abstract: Code LLMs have shown promising results with converting tasks in natural language to programs that can be executed by service robots. We are interested in finetuning small, specialized LLMs for this purpose, but collecting datasets of task-program pairs specific to each robot is time-consuming and expensive. While approaches such as SELF-INSTRUCT and EVOL-INSTRUCT are capable of generating novel tasks given a few examples, they are unable to provide the corresponding programs that correctly abide by physical-world and robot-constraints using the provided programming interface. Using a simulator is a natural potential solution to checking for such constraints, but building simulation environments that can handle arbitrary tasks and their necessary objects and locations, is challenging. To address these challenges, we introduce ROBO-INSTRUCT, which synthesizes task-specific simulation environments on the fly during program execution, by opportunistically inferring entity properties and enforcing corresponding constraints based on how the entities are used in the task program. Additionally, ROBO-INSTRUCT integrates an LLM-aided post-processing procedure to refine instructions for better alignment with robot programs. We demonstrate the effectiveness of ROBO-INSTRUCT across multiple LLMs, showing that our fine-tuned models outperform all baseline methods and even match or surpass the performance of several larger and proprietary models.

Published as a conference paper at COLM 2025
ROBO-INSTRUCT: Simulator-Augmented Instruction Align- ment For Finetuning Code LLMs
Zichao Hu
Junyi Jessy Li
Department of Computer Science
Department of Linguistics UT Austin
UT Austin
Austin, TX 78712 zichao@utexas.edu
Austin, 78712 jessy@utexas.edu
Arjun Guha
Joydeep Biswas
Khoury College of Computer Sciences
Department of Computer Science
Northeastern University
UT Austin Austin, 78712 joydeepbeutexas.cem edu
Boston, MA 02115
a. gwwa@nor edu
Abstract
Code LLMs have shown promising results with converting tasks in natural language programs that can be executed by service robots. We are inter- ested in finetuning small, specialized LLMs for this purpose, but collecting datasets of task-program pairs specific to each robot is time-consuming and expensive. While approaches such as SELF-INSTRUCT and EvOL- INSTRUCT are capable of generating novel tasks given few examples, they are unable to provide the corresponding programs that correctly abide by phosical-world and robot-constraints using the provided programming interface. Using simulator is natural potential solution to checking for such constraints, but building simulation environments that can handle arbitrary tasks and their necessary objects and locations, is challenging. To address these challenges, we introduce ROBO-INSTRUCT, which synthesizes lask-specific simulation environments on the fly during program execution, by opportunistically inferring entity properties and enforcing correspond- ing constraints based on how the entities are used in the task program. Additionally, ROBO-INSTRUCT integrates an LLM-aided post-processing procedure to refine instructions for better alignment with robot programs. We demonstrate the effectiveness of ROBO-INSTRUCT across multiple LLMs, showing that our fine-tuned models outperform all baseline methods and even match or surpass the performance of several larger and proprietary models.
Project page:
1 Introduction
Robot programs leverage robot skills, expressed as parameterized function calls, combined with common programming abstractions (loops, conditionals, etc) to perform complex open- world tasks. For example, by formulating robot manipulation and perception skills such as pick (object) and in__ioooooooeeetttteeeoo an LLM can generate a program for service mobile robot to complete the task: "Pick up apple if you see one here.". The
approaches robotics use large, proprietary LLMs (e.g., GPT) to generate such task-specific programs via in-context learning (Hu et al., 2024; Huang et al., 2023b; Biggie et al., 2023; Liu et al., 2023a; Wu et al., 2023; Liang et al., 2022; Singh et al., 2023; Huang et al., 2023a). While quite effective, such large models cannot be run locally connectivity to query remote LLM endpoints, increase response latency, and raise privacy concerns. Smaller models that can run locally robots, on the other hand, unfortunately
robots, require network

---

Published as a conference paper at COLM 2025
Verify Programs Against Domain-specific Constraints
Instruction Align with Program
Instruction Program Palr
Program
API Definition
Action API call
(RetVal Success/Errol mrr
Observation
X
Instruction
Simulatin
Program
Self-Instruct
Vertled7
Initialize
World
Instruction
Entiy Type Sttte
Entty Type State
LLM
Check
Program
Update
Aligned Instruction
80
Mii Evaluation Results
Robo-instruct
Evol- poll-nnstuct
70
Self-Instrucr
Basi Modill
62.5
62.5
68.75
66.25
69.75
62.5
60
paooa
40
42.5
DeagsnokeRt
OPT
Mnn
Sianadde22
CodeLlame--
Liama-3-88
Qwen-7B
Gemma2-98
Figure High-level overview of the ROBO-INSTRUCT framework. This figure also shows the pass@1 score performance of ROBO-INSTRUCT fine-tuned LLM compared to other LLMs on ROBOEVAL.
unable to match the performance of larger models which naturally raises the question of how to finetune small models for robot-specific code generation.
Obtaining high-quality training data is crucial for fine-tuning LLMs. Unlike some domains where training data exist (e.g., Meditron (Chen et al., 2023) for medical AI), robots vary in their capabilities, and manual data construction becomes unscal- able. Thus, methods like SELFF- INSTRUCT(Wang et al., 2022) and EVOL-INSTRUCT (Xu al., 2024) provide promising approaches for generating synthetic training data.
LLM Generated Program def task bisk_program(); is ________oo_((___ rotrroooonnapaaa ("apple")> piok("spple")
LLM Generated Program pask_program(): pick("apple")) pick("apple")
Real -wolld Constraint Violation
Robot Configuration Violation
Cannat pick up apple that does nat exist at the current location.
Cannot pick up two apples at once due to the robot having only one arn.
Figure 2: Examples of programs violating domain- specific constraints.
However, generating robot pro- grams presents unique challenges, robots interact with the real world and must adhere to robot- and environment-specffi constraints, which these existing approaches do not verify. For instance, as illustrated in Fig. 2, candidate program might instruct the robot to pick up an apple that not present at its current location (example 1) or attempt to pick up multiple objects simultaneously, which is physically impossible if the robot can only hold one time (example 2). These constraints are domain-specific to the robot's intended tasks, and while a developer may recognize such violations, automating their detection is non-trivial. One potential solution is to execute the program in robot simulator with well-defined environments. However, such simulations require pre-enumerating relevant entities and their states, which depend on the specific actions the program dictates. Since approaches like SELF-INSTRUCT generate diverse range programs, pre-enumerating all possible environments becomes impractical. Additionally, we observe that the generated instruction-prew proogramm pairs may be inconsistent; Fig. 10 illustrates this problem, where the instruction specifies checking for an apple, but the program fails to perform this check.
To address these challenges, this work introduces ROBO-INSTRUCT, a framework for gen- erating synthetic robot program training data to fine-tune ween- L.LSs for domain-
2

---

Published as a conference paper at COLM 2025
specific service robot tasks. As illustrated in Fig. 1, ROBO-INSTRUCT offers principled approach for robot developers to define task constraints and verify candidate programs against such constraints. Drawing inspiration from Angelic Execution (Broy & Wirsing, 1981), INSTRINT opportunistically infers entity properties and enforces correspond- ing constraints by synthesizing simulation environments as the program executes. Once violations are detected, ROBO-INSTRUCT then employs a rejection sampling mechanism by invoking SELF-INSTRUCT to generate new program based on the same instruction. To further address misalignment between the candidate instruction and program, ROBO- INSTRUCT incorporates an LLM-aided post-pree post-processing step that refines the instruction to better reflect the verified program's ss intent.
We show the effectiveness of ROBO-INSTRUCE by fine-tuning several LLMs generate domain-specific robot programs, and evaluating them using ROBOEVAL, (Hu et al., 2024), benchmark designed for service mobile robots. Our RosO-INSTRUUCT-inn--unnee models significantly outperform their corresponding base models, achieving an average 19.9% improvement in pass@1 scores, They also outperform their SELF-INSTRUCTT-fne-ttn EvoL-INsSTRUCT-fine-uundd counterparts by 9.7% and 7 7.2%, respectively, Moreover, the RoBo-INSTRUCTT-ie--tnee models surpass or match the performance of several larger code models, including GPT-4o-mini (OpenAI al., 2024a), Starcoder2-15B (Lozhkov et al., 2024), and -arpseeek-R--weent (DeepSeek-A et al., 2025).
and
2 ROBO-INSTRUCT
World (step
World Tstopp
Tolop
der tank pr
def task (r of in roant proni"appler "apple"))) "):
def task Cow_srrogaatt if is is rrost "asphe"): pick ("oppeeeee) go to( "o("kitchen" place( ("apple")
1f ss SiLnnontnhpppt )
pacel aappla))
pick: "oople")
touto""eitchee")
go "ittshn))""
place(" ("apple*)
else: pice( "apple")
Varify World
ptek("xople")
pick("xeple")
Success
AFECal
API Car pick
World roloc Poboc
Word Espsle
World
Verify World
H robot
rihot
Syetstctrue
Jbbjece
rrobtt
M rahnt
Goooll
(undet
Lbject
This program will be rejected because it fails simulation worlds
ab(xbart)
[pst[start)
(at((tarr)
World Jshpp
Worls (otop 21
World
"xppee"" pick!" upple") Tn("ketthen")
roont sf "ocooot"apppee"" "ample"): pick! aaple")
The model randomly apple is present
en tor "ittrhnn) pläce( placo("apole" (apple")
placel apple*)
Aftor the root picks up the apole, It becomes undefined whether
pick("aople"
(tcc(("
Fail
apple
World 2
16, Si.tereee your
Succosaj
pick
The model randomly infors that the apple is notpresent
Word Cropple
2
robot
Coobbo
Choole inject
The robot cannot sick
Oy (xttrt)
(yst(sters)
(uddsstFFals))
Cy (staart)
Eatt(stert)
(oxistfaalse)
Figure 3: Illustration of ROBO-INSTRUCT executing a task program while incrementally building the simulation environment. The environment starts with only the robot's initial position (gray, step 0). As the program runs, it branches into two possible execution paths. To evaluate each path, two simulation environments are sampled (world 1 and world 2). In this example, the program fails because attempts to pick up an apple that isn't present.
2.1 Overall Framework
ROBO-INSTRUCT generates task and robot program pairs training data to fine-tune open- weight LLMs for domain-specific service robot tasks. As shown in Fig. 1, ROBO-INSTRUCT first uses SELF-INSTRUCT to propose novel tasks. For each task, using in-context learning, it prompts a LLM to generate a candidate program to perform the task using the robot APIs in the given context (detailed prompts in Appendix A.5.2). Then ROBO-INSTRUCT verifies the candidate program by synthesizing a simulation environment on-the-fly as API functions are executed (explained in Sec. 2.2). When the simulator catches violations of
3

---

Published as a conference paper at COLM 2025
domain-specific constraints, it rejects the candidate program and re-prompts the LLM for new candidate program. If the program successfully terminates with no simulation failures, ROBO-INSTRUCT synthesizes additional simulation environments (up pre-defined limit) to check for the correctness the candidate program from different initial configurations and environmental states. RORO-INSTRUCT is thus able to catch candidate programs that are not robust to environmental variations. Finally, once the candidate program is verified, ROBO- INSTRUCT incorporates an LLM-assisted instruction-progrant revises natural language instructions using the verified candidate programs to enhance alignment between the two (as detailed in Sec. 2.3). Fig. shows an example of how ROBO- INSTRUCT executes and verifies candidate program while incrementally constructing the simulation environments (world and world 2) on-the-fly. In the following sections, we present these components in detail.
alignment procedure, which
Verifying Candidate Programs Against Domain-specific Constraints
To verify candidate programs, we introduce algorithm inspired by Angelic Execu- tion (Broy& & Wirsing, 1981), which infers program properties from incomplete API specs. As shown in Pseedocode
lets developers combine task constraints with robot AllIs, automatically synthesizing simulation environments and detecting constraint violations during execution. The algorithm is built around three core concepts essential for service robots to reason about:
1. Different entities, e-g. "apple", "kitchen".
2. The type of the entities, and hence their affordances, e-g., "apple" is object, you can pick it up; "kitchen" is a location, you can go to it, and it contains objects.
3. The state of the entities in the world, e.g,, the "apple" is in the "kitchen".
Pseudocode ROBO-INSTRUCT for Simulating Robot API calls.
def RoboInstruct_Sim (api__function_cal,, args):
world GecCurrentsimmlleiiiivviiooooon 3 entity_names Inferrinniiii/ammsssstntnttnnoon
args)
4
for entity_name in entity_names:
5
if EntitylsNewwworld, entity_name): # Synthesize new entity inferred from the API call
7
required_tye type InferRequiredffyeeaa. cattr//eeaapuunnntion.mmmm function_call,, args)
entity monttEntitylkthhmnttttttt. (world, entity_name, required_tyye type) The entity's state can be initialized ei randomly or deterministically, based the API function and its arguments
inferred_stare state
api_.unnttonncall, args)
12
else:
# Check if the entity's type and state is consistent with the API call required_type inferRequiredyyyeccifff.oo. ceapi_ffncciinncar args) inferred_ type GetEntityylyeeworllo entity_name) required_type inferred_type: return [None, 'TypeError'1
17
reate_rreqiremeens DeetstateeRquuirmeettstee entity_name) inferred_ statee_sstteeeeeeeeeeeee_eeeee InferNextSttat not lastratconsisteentniffrrerddddde state, state_requirementss) return [None, "stateInconsistenEEFrrr"
api_function_cal args)
21
22
# Update the state of the entity in the world
world Update@orrlStttteeoooll. Randomly sample value consistent with API call and the current world ret_val InferReturnVal (world, api_function_call, args)
entity_name, inferred_state,
26
return [ret_val, 'Success')
These concepts are closely tied to the robot APIs, where each API invocation during program execution updates the simulation environment. For example, the go_ so_to(loc) action takes

---

Published as a conference paper at COLM 2025
only entities of type "location" arguments, and executing it changes the state of the robot to be at the new location.
Unlike static simulation environments, ROBO-INSTRUCT synthesizes the simulation vironment on-the-fly. Consider an API function call like is iin_rom((bb)) which takes an argument of type "object" and checks whether that object is present in the robot's current location. When this API is called with a specific parameter (e.g,, is js_nn_ room("apple ROBO-INSTRUCT performs the following steps, as illustrated in I'seudocode 1. First, retrieves the current simulation environment and infers the entity name (eg., "apple") (lines 2-3). Next, it checks whether an entity with this name has already been initialized (line 6). If not, ROBO-INSTRUCT synthesizes the entity by inferring its type and state based on the APT call (line 7-11).
A feature of ROBO-INSTRUCT is that an entity's state can be initialized randomly or de- terministically, depending on the specific API function (line 10). For example, consider two API function calls: is iar_jin__oomappa room("apple"") and go_ "o_tto(kitchen") is is_in_ room("apple randomly initializes the state of the "apple", as the "apple" may or may not be present in the same room as the robot. In contrast, go_ so_to(kitchen") deterministically updates the robot's location to the "kitchen" (assuming the API is executed successfully). This flexibility allows ROBO-INSTRUCT to simulate diverse environments and test program robustness under varying conditions.
If an entity has already been initialized during program exe- cution, its type and state are checked against domain-specifie constraints (lines 14-21). First, the entity's type must remain consistent across different API calls (lines 14-17). For example, consider the program shown in Program Example 1. The API pick(obj) expects an argument of type object, while go_ go_to(loc)) expects a location 1, By executing API calls sequentially, ROBO-
def task_program(): pick("apple") go_to("apple")
Program Example
INSTRUCT first infers that "apple" is an object. Then when go_to("apple is called, ROBO- INSTRUCT detects a type inconsistency and returns with an error (line 17). Next, ROBO- INSTRUCT computes the requirements for the robot's next state based on the current world state and compares them with the inferred next state (lines 18-21). If the inferred state violates these requirements, ROBO-INSTRUCC raises an error (line 21). As illustrated in bullet point 4 of Fig. ROBO-INSTRUCT detects a constraint violation when the robot attempts to pick up an "apple" that does not exist in the environment. This mismatch between the expected and actual state leads ROBO-INSTRUCT reject the candidate program.
Another feature of ROBO-INSTRUCT is that the states of entities in the simulator resemble STRIPS-style planning (Fikes & Nilsson, 1971), where each state can be either "true" or "false", as illustrated in bullet points 1 and 3 of Fig. 3. However, unlike traditional STRIPS planning, ROBO-INSTRUCT also explicitly includes an "undefined" value for states. This value represents the default state of any entity not explicitly defined during program's execution. For instance, as shown in bullet point 2 of Fig. 3, after the robot picks up an apple, IOTR-INSTRUCT marks the apple's 's state as "undefined" since it does not track how many apples remain the environment and cannot determine whether an apple still exists at the robot's 's location. As a result, this state information is omitted in subsequent executions (a more detailed comparison with STRIPS planning is discussed in Appendix A.2).
Finally, INSTRUCT infers return values according to the API specification and the current simulation context. As shown in bullet points and 3 of Fig. 3, ROBO-INSTRUCT randomly decides whether the apple is present, which leads to different return values across simulation runs. program is considered valid if it terminates successfully in all simulated environments.
In this example, type compatibility check is strict (i.e., "apple" is only an object and no further inference is made about its location). Nevertheless, the algorithm is also capable of handling more advanced scenarios.
5

---

Published as a conference paper at COLM 2025
2.3 LLM-aided Instruction-Phogrra Alignment Procedure
A key challenge in generating synthetic training data is the mismatch between instruc- tions and programs e.g, a program may skip step implied by the instruction (see Ap- pendix Fig. 10). Rejection sampling only checks program validity, not whether it fully matches the instruction. Thus, valid programs can still fail to fulfill the instruction's 's intent.
To address this challenge, ROBO-INSTRUCT employs a post-processing procedure to align instructions with their corresponding programs. The key intuition behind this approach is that since the program has already been verified, it can remain fixed, and the lask shifts to finding an instruction that accurately aligns with the program. Hence, leveraging the advanced code understanding capabilities of modern LLMs (Rozière et al., 2024; Nam et al., 2024; Leinonen et al., 2023; Li et al., 2023; Lekshmi-Narayanee applies Chain-of-Thouge instructions, selecting the one that best reflects the program. Detailed prompt designs for this process are provided in AppendixA.5.4.
et al., 2024), ROBO-INSTRUCT reasoning (Wei et al., 2022) to generate and compare revised
Analysis and Experiments
3.1 Experiments Setup
Benchmark. In this work, evaluate on the ROBOEVAL benchmark (Hu et al., 2024), domain-specifie program generation benchmark for service mobile robots (See Appendix A.3 for details.). In this domain, service mobile robot can perceive objects, navigate to various locations, manipulate items, and communicate with humans. Accordingly, we design ROBO- INSTRUCT to align with the constraints of the APIs used in this benchmark. We use the pass@1 metric to assess the performance of LLMs in generating correct robot programs.
Data Generation. To generate a diverse dataset, we choose use the open-weight Llama-3 model (Grattafiori et al., 2024) with nucleus sampling to create instruction -rrotiioo--rogammmmmm pairs, setting the temperature T and the top p 0.95. The maximum resampling limit is capped at 3 to accommodate instructions that initially produce invalid programs, and each verification process duplicates the program 100 times to ensure robust probabilistic coverage of different execution branches. For the LLM used for post-processing, we empirically adjust the generation temperature to 0.3 to optimize performance (See Fig. 7). Furthermore, we assess the edit similarity between token sequences of each instruction pair in the dataset (Lee seeeence
et al., 2022), removing duplicates where the similarity score exceeds 0.6. The same similarity- based approach is used to decontaminate the dataset against the ROBOEVAL benchmark tasks (more details are presented in Appendix A.4.3).
Training Setup. We used PEFT (Hu et al., 2022) with unsloth (Unslothai, 2024) to fine-tune four popular open-weight LodeLLMs, including Codellama-Python (Rozière et al., 2024), Llama3 (Grattafiori et al., 2024), Qwen2.5-Codes (Hui et al., 2024), and Gemma2 et al., 2024). The learning rate is set to be 3e-5 with warmup ratio of 3% and a constant Ir scheduler. We employ the AdamW optimizer (Loshchilov & Hutter, 2019) with an effective batch size of 8, training each model for 5 epochs using a sequence length of 2048 tokens.
Baselines. We compare the performance of the ROBO-INSTRUCT fine-tuned models against the same models fine-tuned using two popular data generation methods: SELF- WWan et al., 2022) and INSS-UNSTUUCTTX et al., 2024). Additionally, we com- pare their performance against larger models, categorized into two groups: (1) propri- etary LLMs, including GPT (OpenAl et al., 2024b); and (2) open-weight LLMs, including Codellama-Pythont Starcoder2-15B (Lozhkov et al., 2024), and -eepseek-RI-Distil... Qwen-32B (DeepSeek-A) AI et al., 2025).
3.2 Is ROBO-INSTRUCT Effective at Generating Training Data to Fine- Small Language Model for Generating Domain-Specific Robot Programs?
Tab. 1 presents the average pass@1 results for different LLMs ROBOEVAL using two decoding settings: greedy decoding (T = 0) and nucleus sampling (T = 0.2). ROBO- INSTRUCT-fine-tune models outperform base models by average of 19.9% in pass@1

---

Published as a conference paper at COLM 2025
ROBOEVAL pass@1
Fine-tune
Model
Param
Licensing
T= 0
T = 0.2
GPT-4.5
88.75%
88.25%
Proprietary Proprictary Proprietary
GPT-4
83.75%
85.81% %
GPI-4o-mini
62.50%
61.63%
Codellama-Python
34B
46.25%
48.25%
Open
Starcoder2
15B
62.5%
60.94%
Open
DeepSeek-RI-Qwen
32B
67.50%
65.13%
Open
Codellama-Python
7B
40.00%
39.31%
Open
Self-Instruct
CodeLlama-Python
7B
55.00%
52.69%
Open
Evol-Instruct
CodeLlama-PPython
7B
57.50%
55.38%
Open
Robo-Instruct (ours)
CodeLlama-Python
68.75% 7B
66.00%
Open
Llama3
8B
42.5%
36.69%
Open
Self-Instruct
Llama3
8B
55.00%
53.75%
Open
Evol-Instruct
Llama3
8B
57.50%
54.87%
Open
Robo-Instruct (ours)
Llama3
8B
66.25%
62.44%
Open
Qwen2.5-Coder
7B
55.00%
55.25%
Open
Self-Instruct
Qwen2.5-Code
7B
62.50%
59.38%
Open
Evol-Instruct
Qwen2.5-Code 5-Coder
7B
65.00%
62.75%
Open
Robo-Instruct (ours)
Qwen2.5-Codd
7B
68.75%
67.00%
Open
Gemma2
9B
51.50%
52.00%
Open
Self-Instruct
Gemma2
9B
57.50%
57.88%
Open
Evol-Instruct
Gemma2
9B
60.00%
59.50%
Open
Robo-Instruct (ours)
Gemma2
9B
65.00%
62.63%
Open
Table 1: Pass@1 results of different LLMSs on RoBOEvAI. computed with greedy decoding T = 0 and nucleus sampling T = 0.2.
scores and surpass their SELF- BELL--ISTRRUCT-fn--tiie and EVOL--NNTTUUCT-fne--ur coun- terparts by 9.7% and 7.2%, respectively (analyses of the generated data Appendix A.4.3). Notably, despite having significantly fewer parameters, RORO-INSTRUCTT-fne-tie models match or exceed the performance of larger open-weight models and even the proprietary GPT-4o-mini.
3.3 Evaluating the Contributions of ROBO-INSTRUCT Components
T=0
T=0.2
Invalid
Method
pass@1
Improv.
pass@1
Improv.
Programs
Codellama-7B-Pythee
40.00%
+0%
39.31%
+0%
38.31%
SELF-INSTRUCT
55.00%
+15.00%
52.69%
+13.38%
20.94%
+Reject Unsolvable (RU)
60.00%
+20.00%
57.62%
+18.31%
23.38%
+
Verify Program + RU
63.75%
+23.75%
63.88%
+24.57%
14.13%
LLM-aided Align RU
58.75%
+18.75%
59.81%
+20.50%
23.44%
Both (ROBO-INSTRUCT)
68.75%
+28.75%
66.00%
+26.69%
17.07%
Table 2: Pass@1 results of different methods on ROBOEVAL computed with greedy decoding
= 0 and nucleus sampling T 0.2. The Invalid Programs column indicates the percentage of programs that result in execution errors when tested on ROBOEVAL tasks.
We conduct an ablation study to examine how verifying programs against domain-specific constraints (+ (erify Program) and applying the LLM-aided Instruction-Program Alignment procedure (+LLM-aided Align) affect the performance ROBO-INSTRUCT. Since SELF- INSTRUCT may generate instructions for which no corresponding valid program be generated given an instruction, we include Reject Unsolvable (RU) as an additional baseline. SELF--NSTRUCT+ keeps only the instructions that lead to at least one successful program
7

---

Published as a conference paper at COLM 2025
execution and removes those that do not produce any valid results. Tab. 2 shows the average pass@1 results from CodeLlama-7B-Pythoor fine-tuned on different datasets generated by each method. Results from SELF-INSTRUCT + RU indicate that simply discarding invalid instructions improves model performance. In addition, using either "Verify Program" or "LLM-aided Align" alone improves upon the baseline SEI.F-ISSTRUCT results, and incorporating both within ROBO-INSTRUCT achieves the best pass@1 performance. For more ablation experiments and analysis on the generated data, we refer the readers to Appendix A.1 for more results.of
LLM Generated Program
LLM Generated Program
2
1 def ;
1 def task_ jask_program()=
go to( gane roon")
list_ rooms get_all_roome rooms withwwhttheeeee robots [] for room in 1ist of_ of_rrons: if "robot" in is sin_roonn""Ta"i ("robot")> go_to(roon)
roon ""
say("Hello Jack")
response ask( "Jack",,
"want to play a game?", (YYes", "No [)
Real-world Constraint Violation
Violation
Line checks if Jack is in the roon. If he is absent, 1ine 5 raises an error, as it 1s illogical ask Jack question when is present.
Line "is is_in_room()) returns boolean, which leads to a Python runtine error due to type mismatch.
LLM Generated Program def task_ =ss__progran(;
3
LLM Generated Program def task _ask_progran(( in get_all_roons() if room 1- "living roon":
go to("iten storage roon") pick("iten storage roon")
4
go to(room)
pick("tay"))
yo_to("living roon") 7 if not is is_in_raon( "ooon""oy"): place("tay")
Entity Type Violation The robot cannot pick up location.
Robot Configuration Violation
If there are more than two roons, the rohot will atterpt pick up toys, resulting
violation of its configuration, as the rohot is equipped with only one arm.
Figure 4: StLF-INsTRUCTGGeeerteed Program Errors. Examples highlight errors that violate domain-specific constraints.?
3.4 Qualitative analysis of the generated program errors
We qualitatively analyze invalid programs identified by ROBO-INSTRUCT, as shown in Fig. 4. The first three examples are easily recognizable to humans flawed. However, the last example is more complex and involves an error when the robot can navigate to more than rooms. After the robot places toy in the living room, ROBO-INSTRUCT updates the environment to reflect that toy is now in the room (line 8). However, when the robot returns to the living room later (line 6), will not drop the item it's holding (line 8). As result, when the robot enters third room (line 4) and tries to pick up another toy (line 5), an error will occur because the robot is only capable of carrying one item at a time. This example demonstrates that ROBO-INSTRUE can detect invalid programs beyond those easily identifiable through human inspection.
3.5 Real-World Deployment Demo
We deployed the to generate and execute mobile robot programs in real-world environments, as illustrated in Fig. 5. Unlike GPT models, our locally deployed fine-tuned model offers significantly
fine-tuned model (on local 3080 Ti and an H100 server)
Programs have been adapted to succinctly demonstrate the types of errors and fit within the figure.
8

---

Published as a conference paper at COLM 2025
Models
GPT-4.5
GPT-4
GPT-4o-Mini Robo-Instruct (Local) Robo-Instruct (Server)
Inference Speed tokens/s
tokens/s
41 tokens/s
57 tokens/s
114 tokens/s
Table 3: Inference speed of different models.
faster program generation. Additional results on long-horizon tasks beyond ROBOEVAL are presented in Appendix A.6.
What would you like the robot to
Iwill go to Zarko's office
2
Comple list ingredients grilld
which them he has. Coma back and tsll me what ho coes not have.
Zarko's
progran():
User-Provided Instruction Generated program
start Lls curuureent locatien()
you have butter?
mlosing ingradiseta
No!
tol "Zaneote office))
for Angredtant de ("bread", "ctaeee", "butta""):
resporse ass(" ("rrkko"", 1o you hove Ingredient 72%, ("We', "wo"))
resgaase "Wo")
wiaxing._ignee Ergrediannsn appandi ingredkent:
Got It
"Ingewdtarte")
Zarko does have butter
Conwecing
Co siver
Generatte Code
Execunel
Figure Deployment of the ROBO-INSTRUCT fine-tuned model to generate programs based on user-provided instructions and execute them on the robot.
4 Related Work
LLMs for Robot Code Generation LLMs are performant in generating robot programs from natural language (Liang et al., 2022; Singh et al., 2023; Huang et al., 2023a). One common approach involves generating composable costmaps for motion planning, as seen in Voxposer (Huang et al., 2023b) for tabletop tasks and NavCon (Biggie et al., 2023) for navigation. LLMs are also effective at creating reward functions-Eurele (Ma et al., 2023; 2024) and Senguuggeto-Rewarrt skills through LLM-generated rewards, For high-level planning, LLM+p (Liu et al., 2023a) outputs PDDL plans, while Tidybot (Wu al., 2023) learns user preferences from examples to generate sequential task programs. RoboEval (Hu et al., 2024) targets service robots, generating and validating domain-specific programs for long-horizon tasks.
(Yu al., 2023) enable robots to learn complex
Generating Datasets For Fine-tuning LLMs To improve code generation, many studies create specialized datasets MMuennighoff et al., 2024; Köpf et al., 2023; Muennighoff et al., 2022).
et al., 2022) is popular approach that uses LLMs to generate synthetic data. This approach is later extended by Code Alpaca (Chaudhary, 2023) and Gorilla-LM (Patil et al., 2023) for code and MI APIs In addition, Evol-Instruct (Xu et al., 2024; Luo et al., 2024) proposes approach to iteratively update instructions to become more complex through different prompting strategies. OšS-Instruct (Wei et al., 2023) uses open-source code train Magicoder, matching GPT--3.-TTubo on HumanEval (Chen al., 2021). While prior work focuses on seed instruction generation, we explore post-processing methods, especially for robotics programs (Hu al., 2024), where we can effectively leverage constraints to filter out erroneous programs.
Relevance to Program Analysis Our approach is inspired by angelic execution (Broy & Wirsing, 1981), where we apply nondeterminism to resolve the types and state of input arguments for each predefined robot APT, in the context of LLM-based program generation. Outside the application of LLMs, related ideas have been explored in program analysis techniques such as symbolic execution, exemplified by KLEE (Cadar et al., 2008), which generates high-coverage tests for complex, environment-intnssin static analysis tools (Calcagno et al., 2015; Bessey et al., 2010; Ayewah et al., 2008) also demonstrate the effectiveness of analyzing codebases at scale to uncover bugs and enforce correctness properties.
programs. Large-scale

---

Published as a conference paper at COLM 2025
5 Conclusion, Limitation and Future Works
In this work, we introduce robot program training data to fine-tune open-weight LLMs for domain-specific service robot tasks. ROBO-INSTRUCT features novel algorithm to synthesize simulation environ- ments on-the-fly to check for any violations of domain-specifit constraints and an LLM-aided instruction alignment procedure that refines instructions to better match the generated pro- grams. Experimental results show that CORR--NSTRUCTTffne-tt
novel framework for generating synthetic
models significantly outperform baseline approaches using SELF-INSTRUCT and EVOL-INSTRUCT, while also matching or surpassing larger open-weight LLMs and proprietary models like GPT--o-mini in generating service robot programs. However, ROBO-INSTRUCT is not without limitations. The framework enforces necessary- but not sufficintt-conditions for program correctness: while violate at least one domain-specific constraint, passing program is not necessarily correct in all possible scenarios. For instance, consider a simplified program containing only the instruction pick_up(" up""buildi "). ROBO-INSTRUCT may synthesize scenario in which "building" is treated as pickable object, which is clearly unrealistic. As a result, while ROBO-INSIRUCT effectively detects many domain- specific violations, it may miss feasibility issues beyond the defined task constraints. Despite is effective in identifying
program that fails our checks is guaranteed
this limitation, our experiments demonstrate that wide range of domain-specifit specific violations. Future work could explore integrating ROBO- INSTRUCT within a reinforcement learning fine-tuning loop, allowing models to iteratively learn from violations, thereby improving their ability to generate robust and realistic robot programs for domain-specific applications.
Acknowledgments
This work was partly supported by NSF grants CCF-2313027, IIS-2416461, CAREER-2046955 and UT Austin through the Associate Professor Experimental (APX) program.
References
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. Program synthesis with large language models, 2021.
Nathaniel Ayewah, William Pugh, David Hovemeyer, J. David Morgenthaler, and John Penix. Using static analysis to find bugs. IEEE Software, 25(5): 2008. doi: 10.1109/ MS.2008.130.
Al Bessey, Ken Block, Ben Chelf, Andy Chou, Bryan Fulton, Seth Hallem, Charles Henri- Gros, Asya Kamsky, Scott McPeak, and Dawson Engler. A few billion lines of code later: using static analysis to find bugs in the real world. Commun. ACM, 53(2);66-75, February 2010. ISSN 0001-0782. doi: 1646353. 1646374.
URL https: http://doo.oo///0./1/1/// 1145/
Harel Biggie, Ajay Narasimha Mopidevi, Dusty Woods, and Christoffer Heckman. Tell me where to go: A composable framework for context-aware embodied robot navigation, 2023.
Manfred Broy and Martin Wirsing. On the algebraic specification of nondeterministic programming languages. In Proceedings of the 6th Colloquium on Trees in Algebra and Programming, CAAP '81, pp. 162-179, Berlin, Heidelberg, 1981. Springer-Verlag. ISBN 3540108229."
Cristian Cadar, Daniel Dunbar, and Dawson Engler. Klee: generation of high-coverage tests for complex systems programs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation, OSDI'OS, pp. 209-224 USA, 2008. USENIX Association.
unassisted and automatic
10

---

Published as a conference paper at COLM 2025
Cristiano Calcagno, Dino Distefano, Jeremy Dubreil, Dominik Gabi, Pieter Hooimeijer, Mar- tino Luca, Peter O'Hear,, Irene Papakonstantinou, Jim Purbrick, and Dulma Rodriguez. Moving fast with software verification. In Klaus Havelund, Gerard Holzmann, and Rajeev Joshi (eds.), NASA Formal Methods, pp. 3-11, Cham, 2015. Springer International Publishing- ISBN 978-3-319-17524--.
Sahil Chaudhary. Code alpaca: An instruction-folloving llama model for code generation. htmmm/ahh22211111111
https://gi thub. com/ com/sahi 128011 // 2023.
Mark Chen, Jerry Tworek, Heewoo Jun, et al. Evaluating large language models trained on code. 2021.
Zeming Chen, Alejandro Hernández-Cano, et al. Meditron-70b: Scaling medical pretraining for large language models, 2023.
DeepSeek-A,, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, 2025.
URL httph:tpp///aattii wttpaaawwww...rga////b////////// 12948.
Richard E. Fikes and Nils] J. Nilsson. Strips: a new approach to the application of theorem proving to problem solving. In Proceedings of the 2nd International Joint Conference on Artifi- cial Intelligence, IJCAI'71, pp. 608-620, San Francisco, CA, USA, 1971. Morgan Kaufmann Publishers Inc.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, et al. The llama herd of models, 2024. URL https: org/abs/ 1107.21783.
Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt. Measuring coding challenge competence with apps. NeurlPS, 2021.
Edward Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https: https: htttss///pperrview..r net/
Zichao Hu, Francesca Lucchetti, Claire Schlesinger, Yash Saxena, Anders Freeman, Sadanand Modak, Arjun Guha, and Joydeep Biswas. Deploying and evaluating 1lms program service mobile robots. JEEF Robotics and Automation Letters, 9(3):2853- 2024. doi:
0TT.0///.RAA2224.33666
Chenguang Huang, Oier Mees, Andy Zeng, and Wolfram Burgard. Visual language maps for robot navigation. In Proceedings of the IEEE International Conference Robotics and Automation (ICRA), London, UK, 2023a.
Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, and Li Fei-Fei. Vox- poser: Composable 3d value maps for robotic manipulation with language models. In 7th Annual Conference on Robot Learning, 2023b.
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, Kai Dang, Yang Fan, Yichang Zhang, An Yang, Rui Men, Fei Huang, Bo Zheng, Yibo Miao, Shanghaoran Quan, Yunlong Feng, Xingzhang Ren, Xuancheng Ren, Jingren Zhou, and Junyang Lin. Qwen2.5-coder technical report, 2024. URL https://arxxv.oor//aaaa. 21t.s//229..1..6.....
Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Due Minh Nguyen, Oliver Stanley, Richárd Nagyfi, Shahul ES, Sameer Suri, David Alexandrovich Glushkov, Arna Varma Dantuluri, Andrew Maguire, Christoph Schuhmann, Huu Nguyen, and Alexander Julian Mattick. Openas- sistant conversations democratizing large language model alignment. In Thirty-seunnth
11

---

Published as a conference paper at COLM 2025
Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL http:///weew...eeeeoo///nnnaeaaa wwt..//orruuiici//ooop.....
Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruiqi Zhong, Luke Zettlemoyer, Wen-Tau Yih, Daniel Fried, Sida Wang, and Tao Yu. Ds- Ds-1000: A natural and reliable benchmark for data science code generation. ArXio, abs/2211.11500 2022.
Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini. Deduplicating training data makes language models better. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Compulational Linguistics (Volume 1: Long Papers), Pp. 8424 8445, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/vi /v11/v1///// 2tt./22222..ccooo.... acl-long.577.
URL httpps://acaant
002/2022.
Juho Leinonen, Paul Denny, Stephen MacNeil, Sami Sarsa, Seth Bernstein, Joanne Kim, Andrew Tran, and Arto Hellas. Comparing code explanations created by students and large language models. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1, ITiCSE 2023, pp. 124-130, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9796400701382, 1155/387712 02.
doi: 10.1145 (0.1145/358710 /357700..
3588785. URL https:
Arun-Balajiee Lekshmi-Narayanan, Priti Oli, Jeevan Chapagain, Mohammad Hassany, Rabin Banjade, Peter Brusilovsky, and Vasile Rus. Explaining code examples in introductory programming courses: Llm vs humans, 2024. URL https://arxivv httpaaawww.....aaaaaaaae/ 2403. 05538.
Jierui Li, Szymon Tworkowski, Yingying Wu, and Raymond J. Mooney, Explaining competitive-levell programming solutions using llms. CoRR, abs /adss/230000000 220770033777 ,05337, 2023. doi: 10.48550/ / URL httt::::/doo.rr///// 48550/arXiv. 2307. 05337.
Yujia Li, David Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, et al. Competition-level code generation with alphacode. Science 378(6624): 1092-1097, 2022. doi: 10.1126/ science.abql15: URL https://ww/ wwww:weccnnceeeeeee org/ doi / 10. 10.11c/scceecce.. abll:ieeeeneaaa118....
Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code policies: Language model programs for embodied control. In ar Xiv preprint ar drXtv:2209.07753 2022.
Bo Liu, Yuqian Jiang, et al. LLM+P: Empowering Large Language Models with Optimal Planning Proficiency. ar Xii::2304.11477, 2023a.
Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by chatGPT really correct? rigorous evaluation of large language models for code gener- ation. In Thirty-seventth Conference on Neural Information Processing Systems, 2023b. URL net/forummid=1eeee 11 myyx110Cu7.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019. URL Bkg6RiCqY7.
Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poirier, Nouamane Tazi, et al. Starcoder 2 and the stack v2: The next generation, 2024.
Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. Wizardcoder: Empowering code large language models with evol-instruct. In The Twelffh International Conference on Learning Representations, 2024. URL https: //tperrve//eerevieww net hett/forumidd-unnniiieee
Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Eureka: Human-level reward design via coding large language models. ar Xiv preprint arviv Arxiv-2311 12931, 2023.
12

---

Published as a conference paper at COLM 2025
Yecheng Jason Ma, William Liang, Hungju Wang, Sam Wang, Yuke Zhu, Linxi Fan, Osbert Bastani, and Dinesh Jayaraman. Dreureka: Language model guided sim-to-real transfer. 2024.
Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong, Hailey Schoelkopf, et al. Crosslingual generalization through multitask finetuning, or arxiv preprint arXiv:2211.01786, 2022.
Niklas Muennighoff, Qian Liu, Armel Randy Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro Von Werra, and Shayne Longpre. Octopack: Instruction tuning code large language models. In The Twelfth International Conference Learning Represenfations, 2024. URL https: //tp:reeie//pereeiewwwweeeee net net/forum?
Daye Nam, Andrew Macvean, Vincent Hellendoorn, Bogdan Vasilescu, and Brad Myers. Using an Ilm help with code understanding. In Proceedings of the JEEE/ACM 46th International Conference on Software Engineering, ICSE '24, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400702174. doi: 00.1145/3597555 3639187. URL /ttp:tts/////o..rg///// 1140..19115990 3639187.
OpenAI, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, et al. Gpt-4o system card, 2024a. URL https://arxii.org/abb// 21276.
OpenAI, Josh Achiam, Steven Adler, al. Gpt-4 technical report, 2024b.
Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. Gorilla: Large language model connected with massive apis. ar preprint arXio:2305.15334 2023.
Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Défossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usunier, Thomas Scialom, and Gabriel Synnaeve. Code llama: Open foundation models for code, 2024.
Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. Progprompt: Generating situated robot task plans using large language models. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 11523-11530, 2023. doi: 10.1109/ M011199/CRRAS 000.444489..203.1161111
Karen Sparck Jones. A statistical interpretation of term specificity and its application in retrieval, Pp. 132-142. Taylor Graham Publishing, GBR, 1988. ISBN 0947568212.
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, et al. Gemma 2: Improving open language models a practical size, 2024. URL https://arxivv org/abs/ 2408. 2408.00118. 00118.
Unslothai. Unsloth: Finetune Ilama 3, mistral & gemma Ilms 2-5x faster with 80 URL https://githe: thub. htmm/unsooohaiiusslottoo Accessed: 2024-05-22.
Yizhong Wang, Yeganeh Cordi,Swarood Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language model with self generated instructions, 2022.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoe V Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances Neural Information Processing Systems, 2022.
Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. Magicoder: Source code is all you need, 2023.
13

---

Published as a conference paper at COLM 2025
Jimmy Wu, Rika Antonova, Adam Kan, Marion Lepert, Andy Zeng, Shuran Song, Jeannette Bohg, Szymon Rusinkiewicz, and Thomas Funkhouser. Tidybot: Personalized robot assistance with large language models. Autonomous Robots, 2023.
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. LizardLM: Empowering large pre-trained language models to follow complex instructions. In The Twelfth International Conference on Learning Representations, 2024. URL https://opemrreiiet wwwttorumt idtmmm/hnccc
Wenhao Yu, Nimrod Gileadi, Chuyuan Fu, Sean Kirmani, Kuang-Huei Lee, Montse Gonza- lez Arenas, Hao-Tien Lewis Chiang, Tom Erez, Leonard Hasenclever, Jan Humplik, Brian Ichter, Ted Xiao, Peng Xu, Andy Zeng, Tingnan Zhang, Nicolas Heess, Dorsa Sadigh, Jie Tan, Yuval Tassa, and Fei Xia. Language to rewards for robotic skill synthesis. Arxio preprint ar Xrxv:2306.0888 306406660666..... 2023.
14

---

Published as a conference paper at COLM 2025
A Appendix
A.1 Overview
In this appendix, we first outline the relationship between ROBO-INSTRUCT and the clas- sic STRIPS planning formulation in subsection A.2, providing new perspective on the proposed algorithm. Subsection A.3 shows mode detailed descriptions of the ROBOEVAL benchmark. In subsection A.4, we present additional ablation experiments to analyze the percentage of invalid programs generated by SELF-INSTRUCT and the effectiveness of the dveetion-sampling strategy combined with ROBO-INSTRUCT. We also explore how the generation temperature in the LLM-aided Instruction-Program Alignment Procedure im- pacts final performance and compare the dataset diversity produced by ROBO-INSTRUCT and SELF-INSTRUCT. Subsection A.5 lists the seed tasks used in ROBOEVAL and the CoT prompts. in subsection A.6, we report real-world experiments that empirically evaluate the performance of our fine-tuned model on two long-horizon tasks, which differ significantly from those in ROBOEVAL, and assess the model's latency in generating programs. Although this work focuses on service mobile robots, the proposed framework is adaptable to other domains. In subsection A.7, we offer toy examples showing how ROBO-INSTRUCT can be extended to verify programs by incorporating domain-specific constraints.
A.2 Relevance to STRIPS planning
The proposed ROBO-INSTRUCT shares significant similarities with the formulation of STRIPS planning. A STRIPS instance is typically represented as tuple (1, A, where I denotes the initial state of the simulation environment, G represents the desired goal state that the robot aims to achieve, A defines the set of actions available to transition between states, and P is the set of preconditions that must be satisfied before performing actions. Thus, ROBO-INSTRUCT can be reformulated align with the STRIPS formulation shown in Alg. 2. Each API invocation corresponds to an action, and its precondition consists of a set of literals, representing specific combinations of entities, types, and states.
To address this, we extend the classic STRIPS formulation by incorporating dynamically discovered literals. Unlike the conventional STRIPS approach, where each literal is bi- nary-True when defined and False when not-we introduce third value, "Undefined." This means a literal must be explicitly defined as either True or False; otherwise, it remains in the Undefined state. When an action requires a literal that is undefined, random value (True or False) assigned to it, and the literal is added to the state of the simulation environment (line 7). Once the precondition is fully defined, the action is executed, and domain-specific constraints are checked for any violations (line 10). This extension enables ROBO-INSTRUCT to handle arbitrary programs effectively.
Pseudocode 2 ROBO-INSTRUCT STRIPS(api fn, params, W)
1: Input: api_fn
The API function name The input received by the API invocation > The current state of the simulation environment b Get the parameter-specific precondition for api_fn fn b Loop through every literal in the precondition
2: Input: api Lpi_inputs
3: Input: W
4: p. GETPRECOND(apiLfh farrmm)
5: for E p do
6: if FFCHRccDDHNNEDWW.. is Undefined then
7: W < GROWWORLD((, W) Randomly instantiate the literal and grow W include
8: end if
9: end for
10:
retval, W + EXECUPDATH(apILLM W)
> Execute api_fn and update W
return retval
A.3 ROBOEVAL Benchmark
ROBOEVAL is domain-specific code generation benchmark, featuring a suite of 16 tasks designed to evaluate the ability of LLMs to understand custom APIs and generate programs for service robots. In this domain, service robot perceive objects, navigate to various
15

---

Published as a conference paper at COLM 2025
RoboEval Domain-Specific API Definitions
16 RoboEval Benchmark Tasks
cet_current__ocatioon
BlevtorTour
FhnBBokpack
GetDrink
Poll
a Get o 11st of a12 roces, def ret_aa)__roos() +> list[ser]
Task Instruction
object
current rooe.
Co the elevator. Wilt untll somrecne shows up and ask them they are here for the lour. yes, welzome frem to the university, sa them to killow you, and sake them to the main
ts_tt__ooeeeeesstt str) bool
Go specitfic naaed location.
conterence room. not, war for next person, When you ger to the conterence room, say you have artived at corference room and siso any enjoy your vise herel
def co_to(location: stn) >> None
a person a question, and offer set of spectiftc options the person respond. Returns the response selected by the person.
Canonical Solution
def task ()
ask(person
question
to clevacor")
options: list[st=])
and True:
If in, rocel rawee" perrson""
Say the nessage out loud.
responso ask? "Aer you here for the conferencet", ENNos", "No"1>
def syinessage str) -> None
resporua "Ves":
Pick an object you are not already holding one.
say? walcome to the university. Flease follow rw.")
only hote object
pick(abg: str)
tims.slemss13
der place(oby: str) None
place object conn are holding
11 say("We have arrived.
tol confarrnce rooe"))
time herm")
Figure 6: ROBOEVAL APIs and benchmark task example.
locations, manipulate items, and communicate with humans. Furthermore, the robot should be capable of basic commonsense reasoning and executing complex tasks that involve conditional and repetitive actions. To facilitate these capabilities, ROBOEVAL, defines a set of 8 API functions Python as skill primitives. Fig. 6 illlssrraes these function signatures and definitions, alongside an example task instruction and its canonical solution from the benchmark. In addition, unlike other popular code generation benchmark tasks (Chen et al., 2021; Austin et al., 2021; Li et al., 2022; Liu et al., 2023b; Lai et al., 2022; Hendrycks et al., 2021), the order of the robot's actions is crucial for successfully completing the specified tasks. For instance, in the task "bring me marker from the classroom that does not have whiteboard," the robot must check each classroom until finds one without whiteboard, whereas simply bringing back marker is insufficient. Hence, ROBOEVAL evaluates the generated program by executing it in a simulator to capture the action traces, which are subsequently validated for sequence correctness using temporal logic.
A.4 Ablation Experiments
A.4.1 the Effectiveness of the Rejection-Sampling Strategy
We analyze the percentage of instruction-program pairs discarded by ROBO-INSTRUCT at various maximum resampling limits, as shown in Fig. 7. Initially, with the maximum resampling limit set 0, disabling the rejection-sampling method, approximately 51% of the programs generated by SELF-INSTRUCT contain errors. As the limit increases, fewer programs are discarded. However, there is a diminishing return; even with the maximum resampling limit set to 10, about 15% of the instructions still result in invalid programs.
68
66
Programs (%)
Discarded
20
62
60
0
58
0
3
5
6
9
10
0.1
0.3
0.5
0.7
0.9
Maximum
Resampling Limit
InstAlign LLM Generation Temperature
Figure 7: Ablation Experiment Results
16

---

Published as a conference paper at COLM 2025
A.4.2 Instruction Aligument model temperature
We further investigate how varying LLM temperatures for generating the revised instruction in the LLM-aided Instruction-Progrant Alignment Procedure impact the performance of the fine-tuned model. Fig. shows the bar chart of the pass@1 score of the models fine- tuned over datasets generated using different LLM temperatures. The model performs the best when fine-tuned on the dataset generated using LLM temperature T = 0.3. As the temperature increases, observe a decrease in performance.
A.4.3 Analysis of Generated Dataset
0.14
Self-instruct; Median Score: 0.111
0.12
Robo instruct; Median Score: 0.120
0..
Percentage
0.10
HenCrrsgg voal
0.08
0.06
0.08
0.04
0.02
0.04
0.00
0.0
0.1
0.2
0.3
0.4
0.5
Cosine Similarity Score
0.00
Mumber Takera
Figure Cosine similarities between ROBOE- VAL and generated data.
Figure Token length distribution for each instruction-program pair.
Similar to Magicoder (Wei et al., 2023), we show the improvements from are not merely due to selection bias, i.e. including data more aligned with the distribution of ROBOEVAL tasks than SELF-INSTRUCT. We pair each sample from the generated dataset with task instructions and their canonical solutions, then compute cosine similarity using TF-IDF embeddings (Sparck Jones, 1988). Fig. shows comparable token similarities between both methods. Additionally, Fig. 9 presents the token length distribution, which also appears similar for both.
Method SELR-IST
Size
Ngram=4 Score
Synth. Loc. 956
Synth. Obj. 1060
5K
0.581
ROBO-INSTRUCT
5K
0.587
1025
928
Table 4: Dataset Statistics
Since ROBO-INSTRUCT does not rely on pre-defined simulation environments, we aim to assess the diversity of programs generated by SELF-INSTRUCT and whether RoBO- INSTRUCT can maintain this diversity. To do so, we measure the number of distinct entities, such as synthetic locations and objects. As shown Tab. 4, with dataset of only 5,000 samples, approximately 1,000 unique objects and locations are generated, highlighting that conventional robot simulations with pre-defined environments are insufficient. Additionally, Tab. 4 presents the n-gram diversity scores for each dataset, indicating that both distributions and dataset statistics are highly similar. This suggests that ROBO-INSTRUCT not only preserves but enhances the quality of generated data compared to SELF-INSTRUCT, rather than simply aligning the dataset with benchmark tasks.
A.5 Prompts
A.5.1 ROBOEVAL Seed Task Example
Seed Task Example 1:
17

---

Published as a conference paper at COLM 2025
1 Instruction: Go to Arjun's office, ask him if is ready to head out,
2 if and come back and tell me what he said
def task_program():
start_loc (etecton)ent_lcaation)))
5
go_to office")
response = ask("Arjun",
7
"Are ready to go?", ["Yes", "No"])
9
go_ (sstosstat_)oo) say("Arjun said: " response)
10
Seed Task Example 2:
1 i Instruction: Ask Alice if she needs 1, 2, or 3 boxes.
i Go to the storage room and ask if they have that many boxes.
If so, go place the boxes in Alice's office. Otherwise, tell Alice you could not get boxes.
5 def task_program():
go_to("Alice'ss office")
num_boxes ask("Alice",
8
"How many boxes you need?",
9
["1", "2", "3"])
go_to("storage room")
response = ask("",
"Do you have" num_boxes boxes?",
["Yes", "No"])
14
if response == "Yes":
for in (ingeeiint pick("box")
vo_to("Alice office")
("box"""
19
go_to("storage raom")
else:
go_ "o_to("Alice'' office")
("I could not get the boxes")
Seed Task Example 3:
1 # Instruction: Check if there is a red marker the main
# office, and if so, tell Eve that there is a marker there.
# If not, go to the supply room and bring red marker
main office.
5 def task_program()>
go_to("maii office")
red_marker__fe -oundd ii__in_romm"rerr marker") if red red_marker_foude
9
g"_ve office")
10
sayere is a red marker in the main office")
else:
sospply room")
pick("red marker")
14
go_to("maii office")
15
place ("ace("red marker")
Seed Task Example 4:
# Instruction: Check every classroom if there is whitobdd
Go to Aiden's office to tell him which room does not
18

---

Published as a conference paper at COLM 2025
have whiteboard. Come back and tell task is completed. def task_program():
5
start_loc get_cuurent__ocatioon dist_of_rooms = get_all_roomss
wttthhoutwwitebbbarrec = []
for room in list_of_root _
9
"classroom" not in room:
10
continue
go_to(room)
if not wariin.room"hhitebbarrd""
room_wi wwithhtht_hhht whi teboard. (room)
go_ "o"ide"AAdeen' office")
if wtwp://wthhhotwwwteeeeeee
> 0:
message
for room in whwp:wiithhtuuwhhteee
message room
19
message += "do not have whiteboard"
else:
message "all classrooms have whiteboard" say (message)
go_to(start_lot)
say task is completed")
Seed Task Example 5:
1 # Instruction: Go to the kitchen and wait for someone
# to show up. When someone shows up, ask them to open # the fridge, then pick up diet coke. Finally, put the diet coke in the living room. 5 def task_program():
go_to("kitchen")
while True:
if iar_in_roommppesoon"
9
response ask("",
10
"Please open the fridge", ["Yes", "No"])
if response = "Yes":
pick("dies cake")
14
break time. sleep(1)
go_to("living) room")
place(" ("diet coke")
Seed Task Example 6:
2 Instruction:
Take bed sheet from the laundry put each bedrooms.
3 def task_program(): start_loc get_curreent_occtioon list_of_rooms get_all_rooms{
for room in list_of_rocct
7
"bedroom" not room: continue
8
go_to("laundry room")
pick("bed sheet")
go_to go_tt(rromm)
12
place place("ber ("bed sheet")
go_to(start_loc)
19

---

Published as a conference paper at COLM 2025
A.5.2 Prompts to Generate Synthetic Dataset Using SELF-INSTRUCT
1 are a helpful assistant. Here is a robot that has the
2 following capabilities:
3 def [et_currren_locatioon -> str:
def get_all_rooms() -> list[str]:
5
(s_jeet : str) -> bool:
6 def go to_ttolocation str) None:
7 def ask(person str, question : str, options: list[str]) -> str:
def say(mass str) -> None:
def pick(obj: str) None:
place(obj: str) -> None:
11 Generate an interesting robot task that can be accomplished using the above capabilities.
{SEED EXAMPLE 1}
16
Generate an interesting robot task that can be accomplished using the above capabilities.
(SEED EXAMPLE 6)
20
Generate an interesting robot task that can be accomplished using the above capabilities.
A.5.3 LLM-aided Instruction-PPooranl Alignment Procedure
Validated Program
Choose the best instruction for the given program from the original Instruction and the revised Instruction
Write down step by step what the program instruction
def task p)sss_praaaa(): pick("apple") go "e_tt(""kia ("("kitchen") place(" ""apple)
</>
Original Instruction
Revised Instruction
LLM
LLM
If there apple, bring the kitchen.
Bring an apple
choose
kitchen.
Revised Instruction
Figure 10: Overview of the LLM-aided Instruction-Programm Alignment Procedure.
A.5.4 CoT Prompts for LLM-aided Instruction-Prooramm Alignment Procedure
### Role
2 are an expert at understanding robot programs.
3 You will be given a task instruction and robot program pair.
4 However, the instruction may not align with the program well.
You need to correct the task instruction to match the given robot program.
7 ### Context
9 The robot only has access to the following APIs and standard Python functions def wtw.ccrree__occatinnt -> str:
def get_all_rooms list[str]:
12 def is_in__ee is_in_roomcobeet : str) ~> bool:
def go_ to_ooooocaiion str) >> None:
(persnn str, question str, options: list[str]) >> str:
say (message str) None:
20

---

Published as a conference paper at COLM 2025
def pick(obj: str) None:
def place(obj: str) None:
18
*h# Inputs
Original Instruction
This is task instruction that may align with the robot program Robot Program
23
This is python function starting with dee task_program():
### Task
1. Write down all the provided APIs used in the program and explain the effect each API in this program
27
2. Examine these APIs and write down step by step what the program does
3. Combine all the results above and rewrite the instruction
You need to be specific and clear in your final corrected instruction.
A.6 Real-world Experiments
A.6.1 Program Generation Results Beyond ROBOEVAL
We conducted a small qualitative experiment to assess the performance of the base model, SELF-INSTRUCT fine-tuned models, ROBO-INSTRUCT fine-tuned models, and GPTT-4 on long-horizon tasks. To showcase performance, created two new long-horizon tasks that differ significantly from those the ROBOEVAL benchmark. Using temperature setting of O, generated programs with each model and found that our ROBO-INSTRUCT fine-tuned model successfully produced correct programs, while both the base model and SELF-INSTRUCT fine-tuned model failed. Additionally, GPT-4 made instruction. We show the generation results from ROBO-INSTRUCT fine-tuned models and GPT-4 below.
the second
Task instruction 1:
Let's play game: Double and give it to the person. Start with If you see someone, tell 3 them how much money you have. Then ask if they would like to take the money now double the amount and give it to the next person. If they choose to take it, the game is over, and you should come back to Otherwise, double your money continue. If, in the end, no one takes 7 the money, tell me how much you still have.
2 dollar. to rooms A, B, C, D, E, F,
ROBO-INSTRUCT fine-tuned model (Ours) [Success]
def task_programm):
start_loc get_current. location() (
list_of_rooms ["A", "B", "C", "D" "E", "F", "6"]
4 money for room in list_of_rooms:
5
go_to(room
if is sss_in_rromm"person
say have stoone))))
9
response ask("", """would you like to take the money now or double the amount and give it to the next person?""*, ["Take now", "Double and give"]) if response "Take now": go_ (o_to(start_loc)
10
18
Sahhe game is over, gave the money to room) return
else:
money
21

---

Published as a conference paper at COLM 2025
go_ so_to(start_loc)
NNy one took the money, still have $" str (money))
GPT-4 [Success]
def proo.rrao))))::
start_loc [ot_cuurret__ocation))) ["A", "B", "C", "D", "E", "F", "G"]
money
5
for room in rooms:
go_to(room) if "s__in_romm(perron"))
say("I have " (money) dollars")
response ask("", """woudd you like to take the money now or double the amount and give it to the next person?"",, ["Take it", "Double it"]) if response "Take it":
10
saym"Game over. The person room " room took the money.")
14
so_to(start_loc) return
else:
money
19
go_to so_to(start_lec
NN one took the money. still have str (money) + dollars.")
Task instruction 2:
have table, chair, and a monitor 2 there. If any of these items are missing, go to Jason's 's office and
Go to my office and check
3 see if he is there. If he is, ask him if can barrow the missing items.
If he agrees, pick each missing item and bring it to my office. If Jason is not in his office he says no, back and tell the reason.
ROBO-INSTRUCT fine-tuned model (Ours) [Success]
1 def task_progran():
2 start_loc cet_curreet__lcctiion)
gom office")
table_found is_in_roomer rooainnnoommm"al""""""
5
chair_found is_in_room(""" room("chiir")
6
monitor_found is_ is_in_room ("monitor")) if not table_found:
7
go_to("Jason' 's office") if is_in_room(""sson))
response ask("" Jason", "Can borrow the table from you?",
11
["Yes", "No"]) if response = "Yes":
go_sooss office")
pick("table")
15
go_yo("my office")
16
place("table"))
else:
go go_to(start_loc)
Sasoon said no")
20
else:
go_ go_to(start_loc)
sasonn
Jason is not in his office")
if not fhund_found:
22

---

Published as a conference paper at COLM 2025
co_too"Jason's office")
"s__innroom""aaon"):
26
response ask(")ason) "Can borrow the chair from you?",
["Yes", "No"])
if response == "Yes":
go_too"Jasoon' office")
pick("chair")
go_to("my office")
place("chair")
else:
so_to(start_loc)
35
say("Jason said no")
else:
Bo_to(start_loc)
say("Jason is not in his office")
if not monitor_found:
40
go_ "o_'oo""asso'' office")
if rs__in_room"Jason")
response aak(s"Jsoon)) "Can borrow the monitor from you?",
["Yes", "No"]) response "Yes":
45
go_to(" ss office")
pick("monitor"")
go_to("my office")
place tor""
49
else:
so_to(start_loc)
say("Jason said no")
else:
go_ (ostarttr__))
54
say is not in his office")
20_to(start_ooc)
say("t is completed")
GPT-4 [Failure]
def pask_programm):
start_loc get_current__lcattiong go_to office")
missing_items
5
for item in ["table", "chair", "monitor"]:
if not is_in_roomittem: missing_items append
8
if len(missinn_iitmms) go_too"Jasoon office")
9
10
if is_in_roomm" roommo"sss"""
for item in sissing_items:: response ask("Jason", "Can borrow your item
"?", ["Yes", "No"I) if response "Yes":
14
# BUG: did not return to Jason's office after going back to my office pick(item) go_ co_to("my office") place(item)
18
19
else:
go_to(start_loc)
say Jasson did not allow borrow his item)
return
23
else:
go_to(start_loc)
say(" is not in his office")
return
23

---

Published as a conference paper at COLM 2025
so_to(start_loc)
says" is completed")
24

---

Published as a conference paper at COLM 2025
A.6.2 Generation Latency
What would you like the robot to do??
Go the elevator. Wait someome shows up and ask them they are here for the tour. yes, welcome them the university, tell them to fellow you, and take them the main conference room, If conference room and also say enjoy your visit here!
conference say you
the
p:oggrm()):
start, get_surment, Incation()
(cceso"c""""")
16 askk"",
tour?", (hres", "ho"1)
"vex":
say("aalccee the uniowrsity! Flease follow m).")
(e_too"naann confenence rooe")
break
time sleep (3)
say have anrived at the conference reon. Enjoy your visit here!") (o starttrt__aa _lor)
Execute!
Generate Code
Connecting
Connected enerating generated 1162.00ms..
Generated by our model
(a) Example Program 1 generated by our fine-tuned model.
What would you like the robot do??
Go to the elevator. Wait until someone shows and ask them they are here for the tour. yes, welcome them the university, tell them follow and take them the main conference get conterence room, say you have
wartfor next person. conference room and also say enjoy your visit here!
def taxe, progran():
20. boT clevator")
shile True:
ix, rune "perran"):
rwsponse askk"", you here for tour?", ("Ves", "ta"")
response
say
roos.")
tain conforance poon")
conference visit hore!")
break:
timc: sloep(1)
Execute!
Generate Code
Connecting server... Connected to server. enerating code... Code generated in 6822.60ms.
Generated by GP1-4
(b) Example Program 1 generated by GPT-4.
Figure 11: Our fine-tuned model is approximately 6x faster in inference speed than GPT-4 (Part 1).
25

---

Published as a conference paper at COLM 2025
What would you like the robot do?
Ge to the elevator. Wait until someone shows and ask them they are here for the tour. yes, follow
the university, tell next person, When you get conference room, saly you have conference room and also sey enjoy your visit here!
take them main conference
the
def tasc, prograat():
bol "o("llleattr"
shile Thue:
1F 15, rooor" (perrson"""
nesponse ask("", you hare for tour?", ["Ves", "Io"") visit here!")
Pesporde "Ves":
sayl
roue"))
sty("Me have break
conference
time. sloup(3)
Executo!
Generatee Code
Connecting server.. Connected to server. enerating codo... Code generated in 6822.60ms.
Generated by GPT-4
(c) Another Example Program generated by GPT-4.
What would you like the robot do? the them university, tell follow you, take main conferance not wait for the noxt person. When you get the conference room, say you have arrived the conference room and also
ask
visit here!
Mhlls True:
1f 18 roce "persn""):
respome aski "Are you here for the tourl", ("Ves", "ta")
resppnnee "Ves""
universtty!! Please Follow to the wadn con ference room." go, conferince room")
styc"Melcoree
say""We have arrived
conference roce. Endoy your visit here!")
slmpp(1)
Connecting server.. Connected to server. enerating code... Code generated 6822.60ms.
Generated GPT-4
(d) Another Example Program generated by GPT-4.
Figure 12: Our fine-tuned model is approximately 6x faster in inference speed than GPT-4 (Part 2).
A.7 Toy Examples Beyond Service Mobile Robots
A.7.1 Robot with low-level controls
Consider tabletop manipulation scenario with potential API function, is_rotate(robot_griinn soroott_rippe______ name, radians), where the robot's gripper has physical constraint, allowing rotation only within the range [- π radians. For the following generated program snippet:
def (ask_program()
2 r"letf hand", math. pi/6)
3 rotate("left hand", math.pi/6)
rotate("left. hand", math.pi/6)
26

---

Published as a conference paper at COLM 2025
ROBO-INSTRUCT will first infer that "left hand" is entity of the robot gripper type. Then, regardless the initial configuration of the gripper, ROBO-INSTRUCT will throw an enror because the program causes the gripper to exceed its allowable range of motion.
A.7.2 Al-powered personal digital assistant
Consider broader application than robotics: code generation for Al-powered personal digital assistant. This Al assistant could handle scheduling events using an API function like schedule on calendar(event,, start time, duration). Given the instruction: "My schedule is free tomorrow morning. Please create two 1-hour timeslots for office hours for my robotics and deep learning class." The assistant could generate a program to create these timeslots:
def task_programe am)):
2
schedule_on__calendarr ("robotics class office hour",
"9:30 am", "1 hr")
schedule_on_ccllendar ("deep learning class office hour", "10:00 am", "1 hr")
In this example, ROBO-INSTRUCT needs to reason about the entities "robotics class office hour" and "deep learning class office hour", which are categorized as event types. The event type indicates that these entities have associated timeslots. The state of these entities is defined by the time they occur: robotics class office hour is set for 9:30-10:30 am, and deep learning class office hour is for 10:00-11:00 am. During evaluation, ROBO-INSTRUCT can identify time conflict between these two office hours and thus determine that the generated program is invalid.
27


Reviews:
[{"review_id": "CAhC1K0Tiy", "reviewer": "Reviewer_PKmn", "summary": "This work proposes a method to generate *grounded* synthetic training data for service robots.\n\nA naive approach to generate novel tasks, e.g., using self-instruct, results in tasks that aren't solvable or compatible with the capabilities of models, and constraints and initial state of the environment. To address this issue, authors develop an approach to generate task-specific environments on the fly.\n\nIn particular, they assume executable robot APIs. They few-shot prompt LLMs to generate pairs of instructions and solution codes. They then invoke a type and constraint satisfaction-based approach to initialize a compatible starting state for the environment. If it doesn't satisfy, they reject the sample, otherwise, they find more initial states that pass the solution to ensure it is robust. Finally, once the solution has been established, they regenerate the instruction to \"align\" the instruction and solution.\n\nUsing this approach, they generate training data using RoboEval APIs, and show that training on this data leads to substantial improvements for small open models (more so than the baseline of just plain self-instruct and evol-instruct).", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The improvements over baselines are consistent and substantial.\n- A novel and scalable approach to generate robot training environments on the fly.\n- The paper is clearly written and suitable for this conference.", "reasons_to_reject": "The experiments are shown on only one dataset, RoboEval. It's unclear if the results will hold up elsewhere.\n\nI should note, though, although I am quite familiar with Code LLMs and their evaluations, I am not much familiar with robot evaluation benchmarks. So feel free to correct me if you think there's a good reason to choose only one here.", "questions_to_authors": "- Have you tried using different/larger model for data generation? If so, do you see different/improved downstream results?\n- Are comparisons with baseline data synthetis methods, OpenInstruct and EvolInstruct, size normalized? I.e., all three methods use the same sized training data?\n- Have you tried training models larger than 8B? Using models under 8B is reasonable given your on-device deployment motivation. But I am still curious if these improvements also hold up for training larger models.", "extra_scores": {}}, {"review_id": "aFtNjxefm4", "reviewer": "Reviewer_LLwZ", "summary": "This paper describes a method to generate a dataset of programs in the service robotics domain from from natural language instructions. Using an existing large language model (LLM), the authors create a superset of initial instructions--program pairs. They automatically check the programs are valid using a simulator. The key idea of the simulation is that the authors infer properties and constraints on the objects and physical environment of the simulation. They can then reject generated programs that could not execute in the physical world. The authors then use the remaining programs to fine-tune instruction-based large language models. They also revise the text of the instructions--program pairs to align better the textual instructions with the working programs. The authors carried out an evaluation of fine-tuned models of open weight LLMs and a comparison with two other generators of instructions--program pairs. They report improvements over the other systems.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* The idea is interesting and seems new\n* The description is very pedagogical.\n* The appendices contain many valuable details\n* The authors used clear figures to explain the principles of their method.\n* The evaluation on the RoboEval dataset is thorough and shows an improved performance over Self-Instruct and Evol-Instruct", "reasons_to_reject": "* The task examples are a bit simplistic and it would be interesting to discuss how the system performs with more complex cases\n* The description of the limitations is quite sketchy, which makes it difficult to assess what the system can really do beyond RoboEval\n* There is no analysis of nonworking programs. It would help the reader understand the limitations of the system \n* The authors did not commit to releasing their code. Although the appendix provides details, it would be rather difficult to reproduce this system", "questions_to_authors": "Will you release your code?", "extra_scores": {}}, {"review_id": "rClFVvjX1w", "reviewer": "Reviewer_txhX", "summary": "The paper proposes a synthetic data generation method that generates a diverse set of (natural language instruction, robot program) pairs from a starting seed set. The method is based on self-instruct and uses what seems like a dynamic program analysis method to verify the validity of generated robot programs and discards invalid programs. Finally, the valid remaining programs are checked for alignment with the natural language instruction, and the instruction is rewritten to align with the program. This gives a high quality dataset for tuning smaller, specialized models that can be deployed on a robot.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper communicates the method clearly, is well-written, and is a useful addition to the toolbox of synthetic data generation methods.\n\n- Introduces program analysis methods to the language model community.", "reasons_to_reject": "- The application of program analysis to synthesized programs to synthetic data generation is sound, but I'm unsure of whether there is anything novel in the program analysis method itself. Still, this might of interest to the language model research community who might not be aware of the body of work in program analysis. It would be good to cite relevant work from venues such as PLDI, OSDI, SOSP, so that interested readers can follow-up.\n\n- As the results in the paper show, many of the frontier models are outperforming the models finetuned on the proposed synthetic data generation method. So, the value of the proposed method in forward looking terms is not so clear. Would it make sense to show results on distillation from larger models and compare that with your method? Of course, distillation from a larger, often closed model is not always an option, but it would be good to know how your method holds up.\n\n- Not forward looking: Language models have improved significantly over the past 2 years, and evidence is mounting that late stage task-specific finetuning does not help as much as before and might even harm generalization. So, would it make sense apply your program verification methods directly in the RL stage? i.e., prompt the model with the task and use your method to provide a reward based on the validity of the generated program.", "questions_to_authors": "- There is a vast literature on program analysis that seems relevant to your work. For example, see \"KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs\" or \"A few billion lines of code later: using static analysis to find bugs in the real world\". Can you kindly cite some relevant program analysis papers since it seems connected to your approach?", "extra_scores": {}}]

Author rebuttal:

Meta-review: Empiricism, Data, and Evaluation: Mixed/Acceptable.

Reviewers highlighted thorough experimental design and substantial improvements over baselines, albeit on only one dataset (RoboEval), raising questions about generalizability. The evaluation is rigorous within its scope but narrow in breadth.

Technological Impact: Great.

All reviewers recognized the novel, scalable framework for robot program verification as a valuable contribution. The authors committed to releasing code, datasets, and models. The approach introduces program analysis techniques to the LLM community in a practical way.

Ambition, Vision, Forward-outlook: Mixed/Acceptable.

One reviewer questioned the forward-looking value given that frontier models outperform the approach. In my opinion, even if the specific results fade in relevance, the suggested approach has more fundamental research value. The proposed task+code data generation approach may offer a solution to the instruction dilution problem that seriously limits applicability of frontier models to robot code generation. There is room to extend it to physical-based constraints, but that was clearly out of scope for the paper.

Understanding Depth, Principled Approach: Great.

Reviewers appreciated the principled application of program analysis to robot program verification. While one reviewer questioned the novelty of the program analysis itself, all agreed the adaptation to embodied AI represents a meaningful technical contribution.

Clarity, Honesty, and Trust: Great.

Universal praise from reviewers for clear presentation, writing, helpful figures, and detailed appendices. Authors were transparent about limitations and honest about resource constraints during the rebuttal.

Overall: The submission excels on 3 out of 5 criteria, and meets the bar on the other 2.
Decision: Accept