Title: InstructFlow: Adaptive Symbolic Constraint-Guided Code Generation for Long-Horizon Planning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Code generation; Task planning; Large language models
Abstract: Long-horizon planning in robotic manipulation tasks requires translating underspecified, symbolic goals into executable control programs satisfying spatial, temporal, and physical constraints. However, language model-based planners often struggle with long-horizon task decomposition, robust constraint satisfaction, and adaptive failure recovery. We introduce InstructFlow, a multi-agent framework that establishes a symbolic, feedback-driven flow of information for code generation in robotic manipulation tasks. InstructFlow employs a InstructFlow Planner to construct and traverse a hierarchical instruction graph that decomposes goals into semantically meaningful subtasks, while a Code Generator generates executable code snippets conditioned on this graph. Crucially, when execution failures occur, a Constraint Generator analyzes feedback and induces symbolic constraints, which are propagated back into the instruction graph to guide targeted code refinement without regenerating from scratch. This dynamic, graph-guided flow enables structured, interpretable, and failure-resilient planning, significantly improving task success rates and robustness across diverse manipulation benchmarks, especially in constraint-sensitive and long-horizon scenarios.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nInstructFlow is a symbolic, feedback-driven framework for robotic code generation that decomposes tasks using a hierarchical instruction graph. When execution failures occur, a constraint generator analyzes the feedback to infer symbolic constraints that identify the root cause of the problem. These constraints then guide targeted, efficient code repairs without regenerating the entire plan from scratch. This method significantly enhances robustness and task success rates in complex, long-horizon manipulation scenarios.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths:\n\n- Extends the PRoC3S [3] robot planning framework with symbolic reasoning features and constraint repair functionality.\n- More efficient and robust than PROC3S's reactive  method of starting over, which lacks mechanisms to reason about the task-level causes of failures. The system uses a graph as an internal plan representation that can be adjusted mid-stream.\n- This focus on generalization is a step towards more intelligent and less brittle robotic systems. Instead of solving just the immediate problem, the system learns a more fundamental rule which can help LLM planners to correctly refine the plan.\n\n## Weaknesses:\n\n- The symbolic inference phase of the constraint generator relies on unverified outputs. If these are incorrectly identified, the plan repair would most likely be incorrect.\n- Lack of real-world experiments evaluating the sim-to-real gap.\n\n### Typos:\n\n- L105 a LLMis\n- L627: offse_x\n\nQUESTIONS:\n1. Regarding the limitations of InstructFlow, what analysis has been conducted on its failure cases or the scenarios in which its performance is suboptimal? Furthermore, which component within the framework (say, the Symbolic Constraint Generator) is most frequently the point of failure?\n2. Have metrics such as wall-clock computation time and the average number of feedback queries been benchmarked for InstructFlow? If so, what are the quantitative improvements in planning efficiency and query complexity relative to the PROC3S baseline?\n\nLIMITATIONS:\n- The framework's efficacy is reduced in scenarios requiring precise manipulation of objects with irregular geometries as evidenced by the suboptimal success rates in the Arrange-YCB domain, which were 60% for packing and 70% for stacking, respectively.\n- Dependency on closed LLM models accessed via a cloud API. There is a lack of analysis of the framework's viability with open-source or locally hosted models, which is critical for applications in autonomous systems where network-independent, real-time processing is often a requirement.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the brittleness of large-language-model planners on long-horizon robotic manipulation tasks, where naive code generation often produces physically infeasible plans and, upon execution failures, resorts to blind retries without diagnosing root causes. To overcome this, the authors propose InstructFlow, a multi-agent framework that constructs a hierarchical instruction graph to decompose high-level goals into semantically typed subgoals, embeds a Constraint Generator that analyzes failure traces (e.g., collisions) to induce precise symbolic predicates and inject them back into the graph, and orchestrates a targeted, graph-guided code-generation loop that updates only the affected subgraph rather than regenerating the entire plan. Empirical results on drawing, block-stacking, and YCB object-manipulation benchmarks demonstrate marked improvements in success rates and robustness compared to existing LLM-based planners.\n\nSTRENGTHS AND WEAKNESSES:\nPros\n- InstructFlow achieves substantial gains in task success(20-40 percentage points over strong baselines)across drawing, block-stacking, and YCB benchmarks, demonstrating its robustness in long-horizon planning under physical constraints.\n- The hierarchical instruction graph cleanly decomposes high-level goals into typed subgoals, providing clear semantic and spatial structure for both initial plan synthesis and targeted repair.\n- Its automatic symbolic constraint induction converts raw failure traces (e.g. collisions) into reusable predicates that guide focused code refinement, no per-task hand-crafting of constraints is needed.\n\nCons\n- The multi-agent, graph-guided pipeline incurs extra LLM calls and bookkeeping (planning, code generation, constraint induction), leading to higher computational overhead and latency compared to flat planning methods.\n- Setting up the domain requires defining action templates, object/region vocabularies, and API mappings-a nontrivial engineering effort that may slow adoption in new environments.\n- In scenarios with simple, short-horizon tasks, the marginal benefit of structured repair can be limited, suggesting diminishing returns when failures are rare.\n\nQUESTIONS:\nWriting issues\n- Line 105 in the PDF reads \"a LLMis instructed...\", missing a space (\"LLM is\").\n- In Figure 1's caption, \"Phsical Diagnosis\" is a typo, should be \"Physical Diagnosis.\"\n- In Figure 1 and Figure 2, \"griper\" is misspelled; it should read \"gripper\".\n\nQuestions\n- How sensitive is InstructFlow's performance to the clarity and structure of the user-written instruction scaffold (e.g., decomposition prompts, action templates)? Have you evaluated how variations or errors in those instructions impact success rates and repair effectiveness?\n- Constraint Generator assumes clean, structured failure traces (e.g. precise collision logs), but real‐world feedback is often noisy or incomplete, how does InstructFlow handle ambiguous or incorrect feedback data?\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents InstructFlow, an LLM-based system that combines symbolic planning and constraint-guided code repair for long-horizon robotic manipulation. The central idea is to represent tasks via a hierarchical instruction graph, which is refined using symbolic feedback when execution failures occur. The system shows improved success rates on simulation benchmarks compared to PRoC3S, CaP, and LLM^3.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThe separation between planner, code generator, and constraint agent is conceptually clean and enables modular plan repair.\nBy using declarative constraints and spatial relations, the method enables interpretable and targeted adjustments.\nThe system outperforms baselines across several task domains and includes ablation studies to isolate component contributions.\n\nWeaknesses:\nThe rationale behind structuring tasks as a symbolic instruction graph is not well-justified. It’s unclear why this representation is fundamentally superior to flat sequence planning or dynamic reactive strategies. The benefit seems incremental.\nThe paper primarily demonstrates simple, linear subgoal sequences. There’s no evidence that the instruction graph needs to go beyond a list of ordered subtasks. Are there any realistic scenarios where the graph structure becomes nonlinear (e.g., with branching, loops, or dependencies)?\nThe method relies on accurate symbolic representations of the scene and object poses. It is unclear how it would generalize to perceptual uncertainty or sensor noise, common in real-world settings. \nAll experiments are in simulation. No attempt is made to show robustness in hardware or even perception-grounded simulation with visual inputs.\nWhile symbolic constraint induction is novel, the system appears to rely on a hand-designed set of predicates, limiting generality and scalability.\n\nQUESTIONS:\nWhat is the concrete benefit of using an instruction graph over simply sequencing high-level actions with reactive refinement?\nCan you provide compelling examples where the graph becomes non-linear (e.g., has branches, merges, loops), and this structure is essential for solving the task?\nWould a flat plan with dynamic constraint-aware sampling perform similarly, especially if paired with a constraint checker?\nIf you are using the same test suites in PRoC3S, why are the reported numbers different from the original paper? If the variance of the performance is high, maybe you need more trials?\nWhy is the generated constraint ``reusable’’? (line 331) The constraint is specific to a particular task setting/block positioning, why would it be reusable?\nHow would the proposed system generalize this to real-world tasks that cannot provide systematic feedback in textual format?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
The paper introduces InstructFlow, a modular, multi-agent framework for robotic code generation. It uses a hierarchical instruction graph and symbolic constraint reasoning for task planning. The model takes advantage of dynamically feedback from execution failures, which it integrates into symbolic constraints.

STRENGTHS AND WEAKNESSES:
Strengths:

- Combines probabilitisc and symbolic comnstraints for task planning, which I believe is novel.

- Sizable improvements across multiple domains compared to strong baselines like PRoC3S and LLM3: 20–40%

- Careful ablation analysis

- Interpretability of the produced plans

- Allows recovery from bad initial plans

- Detailed description of the experimental setup, including task domains, constraints, baselines, and compute resources, which should enable reproducisbility.


Weaknesses:

- All results are based on simulations; however, nowadays in robotics, it is standard to have also real-world experiments with actual robots.

- Limited kinds of tasks: table top manipulation

- No Limitations section (even though some limitations are discussed in the text).

Minor:

- The authors refer to intuition in a couple of places; this is not a scientific method.

QUESTIONS:
Questions: 

- Would InstructFlow generalize across robotic domains beyond tabletop manipulation, e.g., mobile navigation, assembly?

- How much doe sthe method depend on the underlying language model (e.g., GPT-4 vs. smaller models)?

- Can the symbolic constraint vocabulary be learned or adapted online instead of being manually defined?

LIMITATIONS:
No dedicated section on this.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-05