How to Auto-optimize Prompts for Domain Tasks? Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge Adaptation
Authors: Yang Zhao
Pu Wang
Hao Frank Yang
Venue: NeurIPS 2025 poster ()
Keywords: LLMs
Prompt Optimization
Expert Knowledge Integration
Task-adaptive Prompting
Primary area: deep_learning

Abstract: Designing optimal prompts and reasoning processes for large language models (LLMs) on domain-specific tasks is both necessary and challenging in real-world applications. Determining how to integrate domain knowledge, enhance reasoning efficiency, and even provide domain experts with refined knowledge integration hints are particularly crucial yet unresolved tasks. In this research, we propose Evolutionary Graph Optimization for Prompting (EGO-Prompt), an automated framework to designing better prompts, efficient reasoning processes and providing enhanced causal-informed process. EGO-Prompt begins with a general prompt and fault-tolerant initial Semantic Causal Graph (SCG) descriptions, constructed by human experts, which is then automatically refined and optimized to guide LLM reasoning. Recognizing that expert-defined SCGs may be partial or imperfect and that their optimal integration varies across LLMs, EGO-Prompt integrates a novel causal-guided textual gradient process in two steps: first, generating nearly deterministic reasoning guidance from the SCG for each instance, and second, adapting the LLM to effectively utilize the guidance alongside the original input. The iterative optimization algorithm further refines both the SCG and the reasoning mechanism using textual gradients with ground-truth. We tested the framework on real-world public health, transportation and human behavior tasks. EGO-Prompt achieves 7.32\%–12.61\% higher F1 than cutting-edge methods, and allows small models to reach the performence of larger models at under 20\% of the original cost. It also outputs a refined, domain-specific SCG that improves interpretability.

How to Auto-optimize Prompts for Domain Tasks? Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge Adaptation
Yang Zhao Pu Wang Hao Frank Yang
Johns Hopkins University
[yzhao229, pwang80, haofrankyang edu
Project page: https::/mmeemeyyaaggg waaaanngg.gggttte 1o/EC0Prompt/
Abstract
Designing optimal prompts and reasoning processes for large language models (LLMs) on comaiin-ppccffc tasks is both necessary and challenging in real- world applications. Determining how to integrate domain knowledge, enhance reasoning efficiency, and even provide domain experts with refined knowledge integration hints are particularly crucial yet unresolved tasks. In this research, we propose Evolutionary Graph Optimization for Prompting TEGO-Prompte an automated framework to designing better prompts, efficient reasoning processes and providing enhanced causal-informed process. 160-Prompt begins with a general prompt and fault-tolerant initial Semantic Causal Graph (SCG) descriptions, constructed by human experts, which is then automatically refined and optimized to guide LLM reasoning. Recognizing that sxpert-defined SCGs may be partial imperfect and that their optimal integration varies across LLMs, EGO-Promme integrates a novel causal-guided textual gradient process in two steps: first, generating nearly deterministic reasoning guidance from the SCG for each instance, and second, adapting the LLM to effectively utilize the guidance alongside the original input. The iterative optimization algorithm further refines both the SCG and the reasoning mechanism using textual gradients with ground-truth. We tested the framework on real-world public health, transportation and human behavior tasks. EGO-Promp achieves 7.32%-12.6 % higher Fl than cutting-edge methods, and allows small models to reach the performence of larger models at under 20% of the original cost. It also outputs a refined, domain-speciif SCG that improves interpretability.
Introduction
Foundation models, particularly Large Language Models (LLMs), are increasingly being adapted for domain-specific tasks, providing reasoning and decision support across real-world applications, such as public health [+33. transportation 477. medical treatment g 9. and robotics [T0H22]. For these models to be used effectively in particular domains, additional task adaptations are usually necessary. Prompt engineering has emerged as the primary, flexible, and cost-effective method for this adaptation [ [6] In this process, domain experts generally incorporate specialized knowledge and priors into prompt design by structuring the prompts and excluding irrelevant information to enhance the reasoning process. However, experts can also inadvertently introduce assumptions, mechanisms, even biases in prompt engineering [17] [8]. Therefore, critical questions arise how can we optimize prompts and reasoning procedures, and discover better combinations that integrate structured domain knowledge for domain-specife tasks? Furthermore, how can we optimization?
automate this
"Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Seeking the potentially better prompt and reasoning procedure for comain--peccfic tasks without fine- tuning involves three challenges: 1) Domain-knowledge Adaptation involves organizing extensive domain knowledge in textual form to maximize task performance while minimizing bias [19 20] For example, in traffic crash modeling 4 21), L.LMs must consider a broad range of textual inputs, including driver attributes, vehicle characteristics, infrastructure conditions, environmental factors, and driving behaviors, etc. Although these elements are represented in text, factors such as word choice, linguistic description, level of detail, and paragraph organization can influence how effectively the model learns and reasons about the domain. 2) Optimal Domain- -adapiive Reasoning recognizes that even with high-quality textual descriptions, physical priors still play crucial role. Domain- specific conditional distributions and causal graphs strongly influence the reasoning process and performance 22 Leveraging these priors can further guide the reasoning procedure, resulting in a more efficient overall process (23]. Without explicit guidance, L.LMs may overlook key domain relationships [19, leading to hallucinations or outputs lacking verifiable evidence. To bridge the gap between a model's 's general reasoning abilities and the domain-specific priors needed for robust inference, it is crucial to incorporate causal structures and other form of domain representations. 3) Task Evolutionary then considers whether LLMs can continue refining their reasoning once domain knowledge and causal relations are effectively integrated. For instance, can active knowledge or ground truth data help a domain-adaptive prompt match or surpass state-of-the-art legacy models? If so, the resulting improved domain knowledge and causal relations can further support evolving domain knowledge and uncover hidden factors, thereby reinvigorating expert-driven research.
In response, promising approach is to leverage graph-based structural domain knowledge to guide both prompt design and the reasoning process [24]. Graph data offers explicit representations of entities and their conditional relationships, capturing key domain features correlations, whether factual (e.g., alcohol involvement increases crash severity regardless of consumption level) or causal (e.g.. how alcohol consumption, road-surface conditions, and other factors jointly affect crash severity). By integrating these explicit relational structures, LLMs can better connect textual descriptions to domain- specific causal dependencies, reducing the likelihood of overlooked relationships or unsupported conclusions. Recent work has explored knowledge and causal graphs to enhance LLM reasoning [24]- 26). often via Aetrieval-Augmente or facts to guide the model during inference 124 25 While this line of research is promising, there remain critical challenges for adaptive reasoning:
Generation (RAG) [2 29], which retrieves relevant subgraphs
1. Limited and Incomplete Domain Knowledge and Graphs. In real-world settings, domain experts often have access only to partial knowledge or graphs for prompt design and reasoning 330. However, many methods assume the availability of fully curated knowledge or causal graphs [26]3]] 31), which is an unrealistic and task-inflexible requirement (e.g., in RAG scenarios where no relevant knowledge graphs exist).
2. Interpretability and Knowledge Refinement. Information in external graph databases may be inaccurate, yet existing methods often treat it as ground-truth reference, potentially leading to performance degradation 18 32 33).
3. Fixed External Priors. Many current methods depend on fixed, external graph databases that can lack coverage or fail to actively evolve with the domain, limiting their adaptability emerging contexts or newly available information 24
4. Automated Evolution. In existing methods, the interaction is typically one-way: external knowledge is used to strengthen LLM reasoning, but the model does not feed back correc- tions or enhancements to the experts. As a result, experts must invest additional effort to extract new insights from the model's outputs 22 26
Given these constraints, effectively adapting LLMs to novel tasks often requires an active process of graph integration and evolution for improved prompt design and reasoning. In this research, we pro- pose EGO-Prompt (Exolutionary Graph Optimization for Prompt) for evolutionarily incorporating domain structural knowledge into more effective reasoning in LLMs. As shown in Figure I we begin by representing expert knowledge as an active, expert-constructed Semantic Causal Graph (SCG) g. EGO-Prompt then decomposes the graph-guided reasoning process into two stages: 1) Generating cittace--ppecfic reasoning guidance z" (ar, 9) derived from the SCG G and the input data v, subject to output constraints, and 2) Performing reasoning conditioned on this guidance. This process can be abstracted as the following equation:
2

---

Domain Specific Tasks
Textualized Structural
Domain Knowledge
Graph-based Structural Knowledge
[-]]
Driving a work zone after drtrking dangerous (-.]
Public Health Urban Planingg Traffic Safety
Rosse
Knowledge Correction
EXPERT
(1) Structural Domain Knowledge Integration
Vesther
Domain Knowledge Cap
Severty
Graph Promot
Graph Optimzation
Nnowledgge Sources
[Craphennanced Promps]
(horgght) Prokeken
weather snowy. driver's
Ground Truth
Smoodedge
BAC 4 mpldt.. The road purffoce
Loss
"The [prediction] is (gggresshhe
dry, The driver di drives aggresiively.
Tectull Cradieens
Methods
Cost
Performance
Sources
[Syseem Prompt]
"[Prompt] is ambiiuuuu (...)'
Fine tuniinnig
$
tiiin
from graph
aPrumpt
[-.]
Bloss
"[Grophj incorrett
Optimization
Reworring
Y
Better
Better Graph
[Prediction]
decortption
EGO-Prompt
(+) Graph Guided Reasoning
XThink cag by step
michodo conatructa resheeeee bring soral exparies.
Optimization
(3 Prompt & Graph Optimisation
(a)
(b)
Figure I: Overview of the proposed EGO-Prompy (a) LLMs often struggle with domain- tasks due to the optimal prompt design and domain knowledge gap. Existing methods rely on the external database established graph. In comparison, EGO-Prompt evolutionarily incorporates expert knowledge with minimal cost. (b) We represent external knowledge as graph-based structure. graph-enhanced prompt then generated guide the LLM''s reasoning. Both the graph and the prompt are iteratively optimized using textual gradients from ground-truth data.
Instance-speciffc Reasoning Guidance
plu x. () = ply 2, 2 G))
(I)
where y is the target reasoning result. The derivation is provided in Appendix 8.1 With this reasoning workflow, both the SCG and the reasoning process are then jointly refined through an evolutionary optimization algorithm that learns factual patterns from ground-truth data. This dynamic adaptation not only enhances the model's reasoning accuracy but also improves the quality of the SCG and its alignment with the prompt. Experiments across three domain-specific tasks demonstrate that EGO-Prompt consistently outperforms previous methods, achieving average F1 improvement of 7.32%-12.61 % over the strongest baseline. Moreover, EGO-Prompt enhances the performance of lightweight models such as GPT-4o mini, surpassing reasoning models like o4-mini and ol, despite their inference costs being 6 to 140 times higher.
Related Works
Generalized Multi-step Reasoning. Currently, enhancing the multi-step reasoning capabilities of
LLMs is primarily achieved through two turning-free approaches (3 In-Context Learning (ICL)
40 The open-ended nature of the reasoning space in ICL makes identifying optimal demonstrations and paths challenging
35/87 and Chain-of-Though (CoT) [4 13 38), 2) Prompt Optimization
38 41 Prompt optimization aims to find the best prompt to guide LLMs towards better reasoning, with Automatic Prompt Optimization (APO) automating this process through candidate generation, evaluation, and filtering [ 42 A recent APO framework, TextGrad [6), draws inspiration from deep learning optimization methods (e.g.. PyTorch [4)). However, it is highly prone to overfitting the training set after several iterations, where the adjusted prompt focuses on case-by-cas details rather than general feature distribution.
Recent works have also explored using Reinforcement Learning (RL) to edit and optimize prompts for L.M reasoning 52 TEMPERA 46 employs attention-based policy model to edit prompts for L. and uses logits difference between the L.LM's predictions and the ground truth [45] as the reward function to fine-tune the policy. This approach effectively improves performance by producing

---

higher-qualig, prompts. However, such methods rely on access to the model's logits during training. Recent studies have extended this RL framework to black-box LLMs by adopting logit-free reward designs (50) 49 While these RL methods can be more stable after careful fine-tuning compared to APO methods (thanks to their numerical optimization framework) they still require fine-tuning a policy model. Importantly, the policy model must be reasonably strong, ideally not much weaker than the target LLM (typically 7B parameters or larger), which imposes substantial demands on fine-tuning expertise and computational resources. In our tasks, the motivation is to leverage available LLMs for better adaptation to domain-speciffe applications (e.g. public health, transportation), where end users are often domain experts with limited access to large-scale computation.
Table 1: Taxonomy of related literature. The v represents the technique is used or implemented by the method. The A means the method needs manual designed prompts as seeds start the optimization process.
Prompt Engineering
Domain Knowledge Database
Graph-Enbanced Enhanced Reasoning
Optimization Automated Optimizationn
Know kelge Reference
Knowklgee Reasoning Guidance Graph Conrection
RAG (22
JCL 23
Zero-Shot-Cal
APE 13
ProTeGi 43
1 erall
24
RoG 25
6 Reaseeet
20
TextGa
1b
Luo ct al.
EGO-Prompt
Domain Adaptive Reasoning. LLMs often need external knowledge for effective reasoning in applicable or evolving domains (c.g., robotics [0]. public health modeling [T] urban planning
15 55,,
traffic safety 44] [56) autonomous driving [57 58 Retrieval-Augmented Generation (RAG)
27 is one of the solutions by retrieving relevant information from corpus based on the input query. However, suitable text corpora not always available, and simply incorporating retrieved text does not guarantee improved reasoning processes [59]. To address this, structured knowledge rep- resentations like Knowledge Graphs and Causal Graphs offer promising alternatives. Knowledge Graph can explicitly represent entities and relations, enabling more structured reasoning guidance compared raw text retrieval. Existing methods focus on retrieve related paths from knowledge graph database to guide LLMs' reasoning (1) Causal Graph, in particular, offer detailed causal information distinct from general KGs [53 60). Most existing mentioned graph integration methods are static, utilizing pre-defined graph database, which limits the generalization capabilities of LLMs and may introduce biases originating from domain-specific graphs (see Table D. Therefore, the primary objective of this research is to develop an evolutionary information integration approach that organically merges structural graph priors with the flexibility of textual information, thereby enabling an optimal reasoning process for real-world domain applications.
3 Prompt Optimization through Textual Gradients
Textual gradients is one type of automatic prompt optimization method that leverages the natural language feedback generated by LLMs to iteratively refine and enhance various components of AI systems 42 16]. The core idea is to emulate the barward-backwand learning frameworks such as PyTorch [43), enabling the system to update prompts through a loop of evaluation and revision. The entire process can be divided into a textual forward and a textual backward phase [1]:
learning paradigm in deep
Textual Forward. For classification task, given a system prompt Pays input data Ti, and its corresponding label yı, the forward model MM generates a prediction ÿ = My (#4; Psys).
Textual Backward. Distinct from traditional numerical learning frameworks, the textual gradients 16] leverages a text-based loss function L to evaluate the alignment between the prediction J. and the ground truth yı- For example:
method 42
"Prediction matches the ground truth.", if if y: otherwise
C(yi, v1)
(2)
"Prediction does not match.",

---

In conventional deep learning frameworks such PyTorch and used update parameters via gradient descent, contrast, textual gradients emulate this process by using LLMs to generate natural language feedback that guides prompt revision such as The prompt can be improved by [strategies]. The textual gradient of Psys with respect to the loss is defined as:
43
gradients are computed numerically
dc:
dc:
Dū. Pays
V Ci Pop
(Chain Rule)
Opsys
(3)
Oc,
=MB Pays Dis
(Implementation),
ajj
where MB denotes the textual backward engine, typically stronger than the forward model M The system prompt for the MM 3 is omitted here. The quantity ag: feedback generated by the backward engine, indicating the improvement direction for the predictions. This gradient follows from the chain rule and can be accumulated across iterations by concatenating past gradients [66] The updated system prompt Psy then obtained by applying the textual gradient:
ac, MM (ūs, L.) represents the
P MMa(Pss Vay
Domain-Specific Reasoning with Expert Knowledge Guidance
4.1 Human-guided Graph Initialization
Textualization of Raw Data. Domain-specific tasks often involve knowledge and information in heterogeneous formats, such numerical, textual and tabular data. To enable LLMs process the structured data, common first step is to construct manually designed templates that convert relevant information into textual prompts [ 4]. Prompt shows an example of partial prompt used in crash prediction task. The full prompt can be found in the Appendix 8.7
Prompt 4.1: Organized Prompt Example of Crash Prediction Task
Predict the crash severity based the crash event details [...]
[Time] The crash occurred on April 29, 2022 at hour
[Dynamic Conditions] The light condition is Daylight. The weather condition is Clear. [...]
Graph Establishment. To better leveraging the reasoning process of LLMs with causal information, we propose the Semantic Causal Graph (SCG) as Directed Acyclic Graph (DAG), where nodes represent entities events extracted from the organized prompt, and edges denote causally-related semantic relations inferred from expert knowledge. Since this graph not used for strict causal inference, do not require it satisfy the causal Markov assumption the faithfulness assumption [62]. Formally, SCG for comain-specifii task can be represented as g = {(n,, Tij. n2) n4 ny E N, Pay R), where A is the set of semantic nodes and R is the set of semantic causal relations. Each node n, E N corresponds to information block extracted from the organized prompt. Each edge Pey R. denotes directed causal relation from n, Nj- capturing the semantic causal link between them as expressed in natural language. In EGO-Prompt, the initial construction of the SCG relies expert knowledge external data analysis.
The biggest difference between the proposed work and most existing works is that the original input of the SCG does not need to be perfectly accurate complete, given the evolutionary process of graph growth. Automatic correction will involved and discussed in the Section 6.2 Below an example SCG prompt for the crash prediction task and full SCG is shown in the Appendix 8.8
Prompt 4.2: SCG Example for Crash Prediction Task
CAUSAL STATEMENT 1: [Person Status] affects [Severity].
The driver's Blood Alcohol Content (BAC) significantly increases the probability of fatal crashes. [...]
5

---

Oriaaized Prompt
Graph
[x]]
O
Initalie
Add
Delete
Edit
Beneft
Graph Description Model [Mc²]
ESPERI
Causal System [Peax]
Semandii
Casual Graph Update
Prompt
Causal Graph
The reletion from to 8 incorrect [--]]
descrption [-.]]
(5)
Preciction Optimizer
Beasoning Guidance
[Node affected (Modie with
The guiiinne can improved
Uve the fevollack to Improve the varsbble [.r]
[+]]
Ec ())
Legend
Prediction Moćel [M
a4
System PromptUpsatt
EGO Promps Sasitero
Thiskstep by step
Prompt [Pays]
The prompt inserveel [..]]
reasoning guidance [..]
by step
[-]
Inputs B Varielites
(6)
(5)
Eq. (5)
Gf [x]]
Loos [4:]
Better Prompt & Graph
Optmizzttin
Prediction
[9]]
Prediction
Incomrect
Data Flow
Textual Forward
Textual Backward
Graph Prompt Update
Figure 2: The optimization process EGO-Prommt. The graph model generates reasoning guidance from the SCG and input prompt, which the prediction model uses to produce an output. Textual gradients are used to update the system prompt and refine the SCG through targeted operations.
4.2 Reasoning with Instance-specifie Guidance
Given the expert-construccte SCG g, our goal is to maximize the likelihood the ground-truth label y conditioned the input x and G, i.e. p(y g). Because & is global description that (i) always contains information irrelevant to the current case and (ii) can be partially missing for a particular instance (e.g. the BAC field may be None in Prompt feeding it directly the predictor is sub-optimal. Instead, as shown in Eq. (I). we first distill instance-specifie reasoning guidance, denoted z" (1x, 9). from the graph G based on the input æ. This guidance then used for predicting
the final results. As shown in Fig. 2 graph description model MP produces the reasoning guidance 2 for each input 24;
MMp (I:; Pemu: 9);
where Pean is causal- prompt steering MM to ground its deterministic generation process with 24 and G. The predictor My. then reasons jointly over x, and 2;:
0 = MM (I,, 2;; Pays).
(6)
with Psys instructing Mr to integrate the original description and the generated reasoning guidance. This two-stage factorization filters out extraneous or missing details G while preserving its causal structure, yielding cleaner and more informative context for the final prediction (see Section 6.1
4.3 Optimizing SCG and Reasoning Process Through Textual Gradients
Optimization through Textual Gradients. Ideally, well-constructed SCG and effective system prompt can substantially enhance the model's reasoning process and predictive performance. However, the performance may vary due to issues: (1) the SCG is incomplete or flawed, and (2) the system prompt fails to guide the model effectively. To address this, we innovated textual gradients method with graph priors [16 42 update flawed SCGs and optimize the system prompt, thereby enabling more effective SCG- based reasoning. We define the loss function L, = L(ū,, ys) similar to Eq. (2) to evaluate the difference between the final prediction ÿ, and the ground truth yı. To improve the system prompt, we compute its textual gradients C; using the chain rule in Eq. (3), which estimates how changes the system prompt P.ys influence the loss through the model's output. The textual gradients for the SCG G and the causal system prompt Pe.. can be formulated as:
02 V Li VzLi 02 OPc.
VgL. VzLi
0g'

---

where Vz Bg: De;
02,, 09 is the textual gradients for the reasoning guidance 2;.
With textual gradients, the system prompt, SCG, and causal system prompt can be updated based on Eq. f As shown in Figure 2 SCG refinement is constrained to three operations: (1) Add node from the candidate set.N A with its causal descrip- tion and links; (2) Delete node and its associated descriptions; and (3) Edit ex- isting descriptions which is incorrect or unnecessary.
Algorithm EGO-Prompt Iterative Optimization
Require: Pays cmu 9o dataset D = ((.,,))) 3)] steps T
1: Pays, Peau, g t- 1ss P g
2: Test on validation set and get F1 f + Fi Pam, 9;
3: for to-t= = 1 to I' do
4: Sample (1:,
V4) D; derive reasoning guidance st
(Eq. (5)
5:
De Eq.
L. + Eq. (V V con: SCca) + Eqs. and
6:
Stage 1: Update system prompt
7:
Psy t- Apply (Pass. Vsys)
Iterative Optimization. The goal of the proposed optimization is to identify the optimal system prompt Pays. SCG G. and causal system prompt Peam. These compo- nents are co-optimized by two optimizers our framework (see Figure each gov- erned by distinct update rules. Performing
8:
I" Fi (Payer S Pam: g)
9:
if f > f then Paysy +- Pagas f
10:
end
Stage 2: Update SCG & causal prompt
11:
P +- Apply(( Pem. Vem); ): G¹ < Apply(G, Vsca)
12:
s' F1 (Pays, P 9")
13:
if > then (Peus f) + (Pom. g')
all updates with single L.LM single
14: end
iteration may lead to suboptimal perfor- mance due to conflicting update signals.
15: end for
16:
return (Pays G, Pam)
Therefore, we adopt iterative optimiza-
tion strategy that updates the SCG and prompt components separately. One component is updated only when it yields individual performance improvement while the other component is fixed. We perform a fixed number optimization steps per task. The full optimization procedures are shown in Algorithm
5 Experiments on Three Domain Tasks
Experimental Objectives for Testing the Proposed EGO Prompt. We evaluate the proposed EGO Prompt based on three main criteria: 1) Performance Improvement: compare the performance of EGO Prompt with existing roopttoptimizztion frameworks to determine whether it yields better results. 2) Generalization: assess how well EGO Prompt generalizes across a wide range real- world tasks on diverse LLM model zoos 663 64] 3) Efficiency and Cost-Efffetiveness:: examine whether a paaalerrparameter, lower- wwwerrreasonin-ccpaacc
model (e.g., GPT-4o mini) can achieve comparable performance to larger model (c.g., 04-mini or ol) after optimization using EGO Prompt. Mcasure the resulting cost savings relative to the performance level of the larger model.
5.1 Datasets for Public Health, Transportation, and Human Behavior Modeling
To assess the generalization and cross-domain adaptivity of EGO Prompt, we evaluate its performance using three real-world applications with publicly available datasets: Pandemic [1. TrafficSafe [4] and Swissmetro [65]. These datasets span the domains of public health, traffic safety, and human behavior, respectively. LLMs have been extensively utilized in these domains due to their capability to interpret complex textual inputs (e.g., crash reports) and generalize knowledge across diverse scenarios (e.g., from COVID-19 19 future pandemics). Specifically, as shown in table in Table 2 1) Public Health Pandemic Hospitalization Dataset [ [6] consists of textual descriptions for forecasting pandemic hospitalization trends. It is constructed from CDC COVID-19 reports (67). reformulated into human-designed prompts. Each instance includes demographic information, COVID-19 case counts, vaccination rates, and hospitalization patterns for individual states, with the target variable being the hospitalization trend for the subsequent week. 2) Crash Modeling for Transportation TraffieSafe Dataset 68 7 comprises structured textual records derived from real-world crash reports across the United States [72]. Each entry provides details about crash time, location, weather conditions, road surface conditions, vehicle maneuvers, and driver behaviors, paired with the corresponding crash severity outcomes. 3) Human Behavior Modeling Travel Mode Choice Dataset [65] [33 originates from a stated- preeee--eeeeencce survey conducted in Switzerland. It presents respondents with hypothetical travel scenarios, prompting them to choose
7

---

between Swissmetro, traditional trains, and cars based on various factors including travel time, cost, and comfort.
Table 2: Overview of the Datasets Employed in This Study.
Dataset
Domain
Prediction Targets
Domain Task labels
Dataset Size
substantial decreasing. moderate decreasing. stable, moderate increasing, substantial increasing no apparent injury, minor injury, serious injury.
Pandemic
1
Public Health
Pandemic Trends
5,200
TrafficSufe
4
Traffic Safety
Truffic Crash Severity
16,188
fatal swissmetro, car, train
Swissmetro
65
Human Behavior
Travel Mode Choice
10,728
5.2 Settings
Experimental Settings. Following prior work on automatic prompt optimization randomly sample balanced subset from each dataset: 100 instances for validation, 100 for testing, and the rest for training. Due to the inherent stochasticity of LLM API calls (see Section 8.5), we repeat each experiment three times and report the best result to represent the achievable upper-bound performance under identical settings (same random seed). We report weighted Fl and accuracy, using batch size 3 to perform 6 to optimization steps per task.
16 42 we
63 as the backward engine MM across all experiments, as it is considered more powerful than the forward engine (74). For the forward engine MM, evaluate several mainstream commercial models [63 64), including gpt-4o www-mmiiii2-000------- 2024-07-18, apt-4.1-mini 20022022222---2
Models. We use gpt-4o-22 400--44--22-----0 20-4-00------
-awassspreeeew0------ The graph description model My is always set to be the same as MMr.
and gemini-2
Baseline Methods. We compare our approach against two APO methods: 1) ProTeGi 42 Pro-
TeGi is the first framework to use natural language "gradients" to generate feedback for prompt optimization, combining beam search with bandit-based selection to identify the optimal prompts. 2) TextGrad [16]. This method is introduced Section 3 We further incorporate three prompt engineering methods: 3) Prro-Shot-CoTT approach [14. which enables step-by-step reasoning with- out in-context examples; 4) PHP [37]] which progressively integrates historical model predictions as hints appended to the prompt to guide reasoning: and 5) Auto-CoT (75). which is an automatic prompting method that generates diverse CoT demonstrations using LLMs. RAG-based methods excluded due to the lack of accessible domain-specific graph databases for datasets. We also include the 6) Expert Organized Prompt, which uses the original input (specifically, the initial x; and Pays) to make direct predictions.
5.3 Results Comparison and Summary
Comparison with Cutting-edge Baselines. Table 3 shows the performance of EGO-Prompt compared to baselines on the Pandemic, TrafficSafe, and Travel Mode Choice datasets. EGO-Prompt consistently outperforms all methods, achieving average F1 gains 7.32% with GPT-4o mini, 12.61 % with Gemini 2.5 Flash, and 9.07% with GPT-5-mini over the best baseline. In contrast, some methods such as TextGrad show limited or even negative improvement in certain cases (e.g.. Pandemic with GPT-4o mini, Fl score only improve from 0.347 to 0.359). This is primarily attributed to the tendency of these methods to overfit the validation set (e.g., potentially generating overly long prompts with TextGrad), leading to performance decline on the test set.
Generalization. EGO-Prompt also brings consistent improvements across different backbone models. As shown in Figure [ it enhances performance on all three tasks when using GPT-4o mini, GPT-4. mini, GPT-5o mini, Gemini 2.0 Flash, and Gemini 2.5 Flash as the forward engine. Specifically, EGO-Prompt achieves relative Fl score gains ranging from 13.3% to 37.0% compared to the organized prompt baseline, showing robust generalization capability across both models and domains. See Appendix 8.2 for more results for commercial and open-source models.
Efficiency and Cost-Effectiveness. EGO- EGO-Prompt enables smaller LLMs to rival or exceed the accuracy of larger, costlier models. As illustrated in Figure EGO-Prompt effectively boosts the average performance of compact models, matching the effectiveness of the reasoning model 04-mini. Moreover, as shown in Figure the mean inference cost per 100 samples using EGO-Prompt is only $0.057, substantially lower than that of o4-mini ($0.33) and other reasoning models.

---

Table 3: Performance comparison with baselines.
Forward Engine
Method
Venue'JJournal
Pandemic
TraffieSafe
Mode Choice
Mean F1
Acc
Acc
Acc
Value
Imp.
Organized Prompt
0.360
0.347
0.300
0.232
0.459
0.406
0.328
Zero-Shot-Con (14
NeurIPS
0.370
0.361
0.280
0.221
0.494
0.435
0.339
3.2%
ProTeeG (42
EM
0.370
0.370
0.304
0.529
0.382
16.3%
GPT-4o mini
Auto-CoT 75
ICLR'23
0.380
0.352
0.320
0.220
0.447
0.462
0.345
5.0%
PHP 37
ICLR
0.330
0.327
0.320
0.268
0.376
0.370
0.322
-2.0%
TextGrad
Nature
0,380
0.300
0.506
EGO-Prompt
This work
0.410
0.399
0.380
0.333
0.506
0.498
0.410
24.9%
Organized Prompt
0.470
0.470
0.340
0.319
0.459
0.392
0.394
Zero-Shot-CoT [44
2e
0.490
0.490
0.400
0.390
0.482
0.398
0.426
8.1%
ProTeGi (12
EM
0.470
0.482
0.420
0.400
0.494
0.425
0.435
10.6%
Gemini 2.5 Flash
Auto-CoT [5
ICLR 23
0,380
0.352
0,340
0.230
0.447
0.462
0.348
6.0%
РHP 37
ICLR
0.520
0.515
0.360
0.334
0.494
0.436
0.428
8.6%
TextGrad [6
Nature 25
0.470
0,483
0,380
0,374
0.494
0.428
0.428
8.6%
EGO-Prompt
This work
0.540
0.546
0.430
0.428
0.518
0.499
0.491
24.6%
Organized Prompt
0,420
0.387
0.330
0.265
0.435
0.435
0.362
Bero-Shot CoTl 114
22
0.430
0.415
0.330
0.281
0.424
0.428
0.375
3.4%
ProTeGi C+
EMNLP'23
0.420
0.413
0.320
0.267
0.471
0.454
0.378
4.4%
GPT-5-mini
Auto-CaT 75
ICLR'23
0.410
0.408
0.310
0,280
0.506
0.470
0.386
6.6%
PHP 37
ICLR 24
0.430
0.397
0.320
0,240
0.435
0.431
0.356
-1.3%
TextGrad
Nature
0.430
0.250
0.230
0.494
0.485
0.377
EGO-Prompt
This work
0.460
0.448
0.340
0.305
0.529
0.511
0.421
16.3%
0.5
Organized Prompt
0.5
Ours (Gemini 2.5 Flash)
Mcan Fl Score
Prrompt
Mean Fl Score
04-mini, 0.404
0.45
0.4
Ours (GPT-4o mini)
o3
0.4
Gemini 2.5 Pro
0.3
o4-mini
ol
03-mini
0.2
0.35
0.05
0.3
3
10
GPT-4o
GPT-4.1
GPT-5
Gemini 2.0 Flash 2.5 Flash
Gemini
mini
mini
mini
Mean Price (USD per 100 samples)
Figure 3: Mean F1 Score Across Datasets.
Figure 4: Mean F1 Score and Mean Price with Reasoning Models.
Analysis and Insights
6.1 Ablation Study
Instance-specific Reasoning Guidance. The instance-specific reasoning guidance mechanism in Eq. (1) plays critical role in enhancing opti- mization effectiveness. As shown in Table 4 re- moving the decomposition process in Eq. (I) and using single model leads to noticeably worse performance. Specifically, the Fl scores across three tasks drop from 0.399, 0.333, and 0.498 to 0.397, 0.247, and 0.445, respectively.
Table 4: Ablation Study of Instance-specifi Rea- soning Guidance. GPT-4o mini is used for all the ablation study and analysis experiments.
Pandemic
TraflieSafe
Swissmetro
Model
Acc
F1
Acc
F1
Acc
F1
Single Model
0.390
0.397
0.340
0.247
0.482
0.445
EGO-Prompt
0.410
0.399
0.380
0.333
0.506
0.498
Model Components. We conduct an ablation study to evaluate the contribution of the graph description model (with variables G and Pen.) and the prediction model (with variable Psyys) to the overall performance. As shown Table our full framework, where both components are used and updated during optimization, achieves the best performance across all three tasks. Fixing either the graph description model or the prediction model leads to noticeable performance drops. Furthermore. removing the graph description model entirely (i.c., only using the prediction model, similar to the TextGrad method [16] results in consistently lower Fl scores.
"See Appendix 8.5 for the distribution.

---

Table Ablation Study of Model Components. checkmark (V) indicates the component used
and updated during optimization, while a triangle (A) indicates it is used but kept fixed.
Pandemic
TrafficSafe
Swissmetro
G Pa
Paps
Acc
F1
Acc
F1
Acc
F1
0.380
0.359
0.300
0.243
0.506
0.432
A
4
0.370
0.337
0.270
0.246
0,471
0.418
4
A
0.370
0.354
0.330
0.257
0,471
0.418
4
0,410
0,399
0,380
0.333
0.506
0.498
Iterative Optimization. We conduct an ablation study to assess the effectiveness of our optimization strategy by comparing three variants: (1) removing the optimization process entirely (W/O OPT.), (2) disabling the iterative optimization process (W/O ITERATIVE OPT.), and (3) using the full strategy in EGO-Prommt. As shown Table 6, the absence of optimization results in reduced performance, with F1 scores dropping to 0.359, 0.293, and 0,434 on three tasks, respectively. When iterative refinement removed, the Fl scores are moderately improved (0.369, 0.272, and 0.447) but still lag behind the full model.
Table 6: Ablation Study of Iterative Optimization.
Pandemic
TrafficSafe
Swissmetro
Base Model
Acc
F1
Acc
Acc
OPT.
0.360
0.359
0.293
0.434
W/O ITERATIVE OPT.
0.410
0.369
0.310
0.272
0.447
0.447
EGO-Prompt
0.410
0.399
0.380
0.333
0.506
0.498
6.2 Automatic SCG Correction
EGO-Prompt automatically refines a human- designed SCG during iterative optimization by restricting the graph description updates to three operations: addition, deletion, and modification of causal description. These targeted updates correct biases or flaws in the expert-construuted SCG, guided by ground-truth data. Figure [5] illustrates this correction process on the Pandemic dataset, where EGO-Prompt identifies and incorporates missing connections (e.g., from Heal thcare System Condition to per 100k) and removes weak incorrect ones (e.g., from Demographic Information to Restriction Policy Response). The system prompt is also co-optimized throughout this process. Additional corrected SCGs and adjusted system prompts for other datasets provided
HCS
Iteakheare System
persook
DI
VC
PI
H
Demagrann lnfor nation
nacion Covorage
Fopuibtion Immurtty
Repprttd Coces persock
(Torget)
Stallng Cond
Hespitaltaatinn
Figure 5: Automatic SCG Correction for the Pan- demic Dataset. Green line denotes newly added relations; red dash denotes deleted relation.
in Appendix 8.9
7 Conclusion
We propose a novel method called EGO-Prompt, which integrates expert knowledge to enhance the adaptivity and interpretability of LLMs for domain-specifie tasks. By decomposing graph- based reasoning into two stages: generation of reasoning guidance and model reasoning conditioned on that guidance, EGO-Prompt not only outperforms existing prompt optimization baselines, but also enables automatic correction of the SCG and refinement of the reasoning process. In future work, EGO- Prompt holds promise not only for broader range of domain-specific applications but also in other emerging directions such as Dynamic RAG (real-time updates to the graph database), domain-specific knowledge graph construction, and causal discovery. One noticeable limitation of EEO--rrompp that it requires additional computational resources for causal-guided textual gradient, especially when scaling larger sample sets. Another limitation is the results can be unstable due API variability and the inherent sensitivity of the Textual Gradients method (see Appendix 8.5 and 8.6 for details).
10

---

Acknowledgments
Yang Zhao acknowledge the fellowship from JHU Amazon Initiative for Interactive AI (AI2AI).
References
[1] Hongru Du, Jianan Zhao, Yang Zhao, Shaochong Xu, Xihong Lin, Yiran Chen, Lauren M Gardner, and Hao Frank Yang. Advancing real-time pandemic forecasting using large language models: A covid- 19 case study. arXiv preprint arXiv:2404.00
2024.
[2] Jianing Qiu, Kyle Lam, Guohao Li, Amish Acharya, Tien Yin Wong, Ara Darzi, Wu Yuan, and Eric Topol. Llm-based agentic systems in medicine and healthcare. Nature Machine Intelligence, 6(12):1418-142) 2024.
[3] Jan Clusmann, Fiona Kolbinger, Hannah Sophie Muti, Zunamys Carrero, Jan-Niklas Eckardt, Narmin Ghaffari Lalch, Chiara Maria Lavinia Löffler, Sophie- Sophie-Caroline Caroline Schwarzkopf, Michaela Unger, Gregory P Veldhuizen, et al. The future landscape of large language models medicine. Communications medicine, 3(1):141, 2023.
[4] Yang Zhao, Pu Wang, Yibo Zhao, Hongru Du, and Hao Frank Yang. Safetraffic copilot: adapting large language models for trustworthy traffic safety assessments and decision interventions. Nature Communications, 16(1):8846, 2025.
[5] Zijian Zhang, Yujie Sun, Zepu Wang, Yuqi Nie, Xiaobo Ma, Peng Sun, and Ruolin Li. Large language models for mobility in transportation systems: A survey on forecasting tasks, arXiv preprint arXiv:2405.02357 2024.
[6] Xusen Guo, Qiming Zhang, Junyue Jiang, Mingxing Peng, Meixin Zhu, and Hao Frank Yang. Towards explainable traffic flow prediction with large language models. Communications in Transportation Research, 4:100150, 2024.
[7] Li Chen, Penghao Wu, Kashyap Chitta, Bernhard Jaeger, Andreas Geiger, and Hongyang Li. End-to-end autonomous driving: Challenges and frontiers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[8] Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez. Ting Fang Tan, and Daniel Shu Wei Ting. Large language models in medicine. Nature medicine, 1930):130011400 2023.
[9] Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et Large language models encode clinical knowledge. Nature, 622-7972) 2023.
[10] Jiaqi Wang, Enze Shi, Huawen Hu, Chong Ma, Yiheng Liu, Xuhui Wang, Yincheng Yao, Xuan Liu, Bao Ge, and Shu Zhang, Large language models for robotics: Opportunities, challenges, and perspectives. Journal of Automation and Intelligence, 2024.
[11] Michael Ahn, Debidatta Dwibedi, Chelsea Finn, Montse Gonzalez Arenas, Keerthana Gopalakr- ishnan, Karol Hausman, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, et al. Autort: Embodied foundation models for large scale orchestration of robotic agents, arXiv preprint 2rXiv:2401.12966 2024.
[12] Bingqian Lin, Yunshuang Nie, Ziming Wei, Jiaqi Chen, Shikui Ma, Jianhua Han, Hang Xu, Xiaojun Chang, and Xiaodan Liang. Navcot: Boosting llm-based vision-and and--and-dannd language naviga- tion via learning disentangled reasoning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(7):54555557, 2025. doi: 10. 109/TPAMM.2025.35544
[13] Yongchao Zhou, Andrei Toan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers. In The Eleventh International Conference Learning Representations, 2022.

---

[14] Takeshi Kojima. Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners, Advances in neural information processing systems, 15:22199-222111 2022.
[15] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. 2022.
Advances in neural information processing systems,
[16] Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Pan Lu, Zhi Huang, Carlos Guestrin, and James Zou. Optimizing generative ai by backpropagating language model feedback. Nature, 639(8055):609-611 2025.
[17] Aniket Vashishtha, Abbavaram Gowtham Reddy, Abhinav Kumar, Saketh Bachu, Vineeth N Balasubramanian, and Amit Sharma. Causal inference using 1Im-guided discovery. arXiv preprint arX1v:2310.15117, 2023.
[18] Stephanie Long, Tibor Schuster, and Alexandre Piché. Can large language models build causal graphs? arXiv preprint (iti:2303.00279,, 2023.
[19] Rohan Alur, Manish Raghavan, and Devavrat Shah. Human expertise in algorithmic prediction. Advances in Neural Information Processing Systems, 37:138088-1388 2024.
[20] Xinyuan Wang, Chenxi Li, Zhen Wang, Fan Bai, Haotian Luo, Jiayou Zhang, Nebojsa Jojic, Eric Xing, and Zhiting Hu. Promptagent: Strategic planning with language models enables expert-level prompt optimization. arXiv preprint arXiv:23100 16427, 2023.
[21] Hao Zhen and Jidong J Yang. Crashsage: A large language model-centered framework for contextual and interpretable traffic crash analysis. arXiv preprint arXiv:2505.07853 2025.
[22] Bailin Wang. Zi Wang, Xuezhi Wang, Yuan Cao, Rif A Saurous, and Yoon Kim. Grammar prompting for domain-specific language generation with large language models. Advances in Neural Information Processing Systems, 56:65030-65055. 2023.
[23] Alon Talmor, Oyvind Tafjord, Peter Clark, Yoav Goldberg, and Jonathan Berant. Leap-of- thought: Teaching pre-trained models to systematically reason over implicit knowledge. Ad- vances in Neural Information Processing Systems, 33:20227-20237 2020.
[24] Shiyang Li, Yifan Gao, Haoming Jiang, Qingyu Yin, Zheng Li, Xifeng Yan, Chao Zhang, and Bing Yin. Graph reasoning for question answering with triplet retrieval. arXiv preprint arXiv:2305. 18742, 2023.
[25] Linhao Luo, Yuan-Fang Li, Gholamreza Haffari, and Shirui Pan. Reasoning graphs: Faithful and interpretable large language model reasoning. arXiv preprint arXiv:2310.01061, 2023.
[26] Haoang Chi, He Li, Wenjing Yang. Feng Liu, Long Lan, Xiaoguang Ren. Tongliang Liu, and Bo Han. Unveiling causal reasoning in large language models: Reality or mirage? Advances in Neural Information Processing Systems, 77:96640-966777 2024.
[27] Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for inonsivge-intenivve intensive nlp tasks. Advances neural information processing systems, 33:9459- 9474, 2020.
[28] Boci Peng, Yun Zhu, Yongchao Liu, Xiaohe Bo, Haizhou Shi, Chuntao Hong, Yan Zhang, and Siliang Tang. Graph serieval-auugmente generation: A survey. arXiv preprint 240.02200,. 2024.
[29] Qinggang Zhang, Shengyuan Chen, Yuanchen Bei, Zheng Yuan, Huachi Zhou, Zijin Hong, Junnan Dong, Hao Chen, Yi Chang, and Xiao Huang. A survey of graph retrieval-augmeenee generation for customized large language models. arXiv preprint arXiv:2501.1395 2025.
[30] Zhou Yu, Xuecheng Ouyang, Zhenwei Shao, Meng Wang, and Jun Yu. Prophet: Prompting large language models with complementary answer heuristics for bnowledge-based visual question answering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 4788)6677766888
2025. doi: 10. 11199TTPAM222225552222...
12

---

[31] Jianing Wang, Qiushi Sun, Xiang Li, and Ming Gao. Boosting language models reasoning with chain-of-knowen knwwleeg prompting. arXiv preprint arXiv:2306.00442,, 2023.
[32] Bingcong Xue and Lei Zou. Knowledge graph quality management: A comprehensive survey, IEEE Transactions on Knowledge and Data Engineering, 35(5):4969-49)) 4988, 2022.
[33] Kai Guo, Harry Shomer, Shenglai Zeng. Haoyu Han, Yu Wang, and Jiliang Tang. Empowering graphrag with knowledge filtering and integration. arXiv preprint arXiv:2503. 3804, 2025.
[34] Wayne Xin Zhao. Kun Zhou, Junyi Li. Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang. Junjie Zhang, Zican Dong, al. A survey of large language models. arXiv preprint arXiv:2303. 18223, 1(2), 2023.
[35] Tom Brown, Benjamin Mann, Nick Ryder, Melamie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 13:1877-1901, 2020.
[36] Andrew K Lampinen, Ishita Dasgupta, Stephanie CY Chan, Kory Matthewson, Michael Henry Tessler, Antonia Creswell, James McClelland, Jane X Wang, and Felix Hill. Can language models learn from explanations in context? arXiv preprint 2040v2200.022329, 2022.
[37] Chuanyang Zheng. Zhengying Liu, Enze Xie, Zhenguo Li, and Yu Li. Progressive- Prompting Improves Reasoning in Large Language Models, October 2024. URL /ttp:// wttpl//org/bbbb////e// 09797 arXiv:2304.0979 [es].
[38] Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoe Le, and Ed Chi. Least-to- Most Prompting Enables Complex Reasoning in Large Language Models, April 2023. http:///:///axi org/aba ////bb//// 0atss/2205..00 10625 arXiv:2205. 10t.:2205..0.55555 [es].
[39] Han He, Qianchu Liu, Lei Xu, Chaitanya Shivade, Yi Zhang, Sundararajan Srinivasan, and Ka- trin Kirchhoff. Crispo: Multi-aspect suggguuu---gggstionn- guided automatic prompt optimization for text generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 24014-24022, 2025.
[40] Kiran Ramnath, Kang Zhou, Sheng Guan, Soumya Smruti Mishra, Xuan Qi, Zhengyuan Shen, Shuai Wang, Sangmin Woo, Sullam Jeoung, Yawei Wang. Haozhu Wang. Han Ding, Yuzhe Lu, Zhichao Xu, Yun Zhou, Balasubramaniam Srinivasan, Qiaojing Yan, Yueyan Chen, Haibo Ding, Panpan Xu, and Lin Lee Cheong. Systematic Survey of Automatic Prompt Optimization Techniques, April 2025. URL [cs].
org/ abaa 2502. 16923 arXiv:2502.16923
[41] Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic Chain of Thought Prompting in Large Language Models, October 2022. URL htt.::ttp//arxii.rr//bbbbb 03493 arXiv:2210.03493 [cs].
[42] Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, and Michael Zeng. Automatic prompt optimization with" gradient descent" and beam search. arXiv preprint arXhv:2305.03495. 2023.
[43] A Paszke. Pytorch: An imperative style, nigh-peeformance deep learning library. arXiv preprint 1rXiv:1912.01703, 2019.
[44] Han Guo, Bowen Tan, Zhengzhong Liu, Eric P Xing, and Zhiting Hu. Efficient (soft) q-learning for text generation with limited good data. arXiv preprint arXiv:2106.07704 2021.
[45] Mingkai Deng, Jianyu Wang, Cheng- Piig Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric Xing, and Zhiting Hu. Riprompt: Optimizing discrete text prompts with reinforce- ment learning. arXiv preprint 2rXiv:2205.12544 2022.
[46] Tianjun Zhang, Xuezhi Wang, Denny Zhou, Dale Schuurmans, and Joseph E. Gonzalez. Tempera: Test-time prompting via reinforcement learning. arXiv preprint arXiv:2211.11991 2022.
13

---

[47] Chengzhengxu Li, Xiaoming Liu, Yichen Wang. Duyi Li, Yu Lan, and Chao Shen. Dialogue for prompting: A volicy-gradient-beee discrete prompt generation for few-shot learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18481-18489, 2024.
[48] Shivam Shandilya, Menglin Xia, Supriyo Ghosh, Huiqiang Jiang, Jue Zhang, Qianhui Wu, and Victor Rühle. Taco-rl: Task aware prompt compression optimization with reinforcement learning. arXiv preprint arXiv:2409.13035, 2024.
[49] Weize Kong, Spurthi Amba Hombaiah, Mingyang Zhang, Qiaozhu Mei, and Michael Bendersky, Prewrite: Prompt rewriting with reinforcement learning. arXiv preprint arXh::2401.08155 2024.
[50] Minchan Kwon, Gaeun Kim, Jongsuk Kim, Haeil Lee, and Junmo Kim. Stableprompt: Au- tomatic prompt tuning using reinforcement learning for large language models, 2024. URL htlps://arxiv. org/ubs/2410.07622
[51] PaweL. Batorski, Adrian Kosmala, and Paul Swoboda. Prl: Prompts from reinforcement learning. arXiv preprint arXiv:2505. /4412, 2025.
[52] Anita Kriz, Elizabeth Laura Janes, Xing Shen, and Tal Arbel. Prompt4trust: A reinforce- ment learning prompt augmentation framework for clinically-aligned confidence calibration in multimodal large language models. arXiv preprint MrXiv:2507.09279, 2025.
[53] Hang Luo, Jian Zhang, and Chujun Li, Causal graphs meet thoughts: Enhancing complex reasoning in graph-augmented llms. arXiv preprint arXiv:2501.14882, 2025.
[54] He Zhu, Wenjia Zhang, Nuoxian Huang, Boyang Li, Luyao Niu, Zipei Fan, Tianle Lun, Yicheng Tao. Junyou Su. Zhaoya Gong, et al. Plangpt: Enhancing urban planning with tailored language model and efficient retrieval. arXiv preprint arXiv:2402. 192733,,..., 2024.
[55] Yu Zheng, Longyi Liu, Yuming Lin, Jie Feng, Guozhen Zhang, Depeng Jin, and Yong Li. Urbanplanbench: A comprehensive urban planning benchmark for evaluating large language models, arXiv preprint urXiv:2504.21022 2025.
[56] Hao Zhen, Yucheng Shi, Yongcan Huang, Jidong Yang, and Ninghao Liu, Leveraging large language models with chain-of-thought and prompt engineering for traffic crash severity analysis and inference. Computers, 13(9):232, 2024.
[57] Ming Nic, Renyuan Peng. Chunwei Wang, Xinyue Cai, Jianhua Han, Hang Xu, and Li Zhang. Reason2drive: Towards interpretable and chain- reasoning for autonomous driving. In European Conference on Computer Vision, pages 292-308. Springer, 2024.
[58] Shili Sheng, Erfan Pakdamanian, Kyungtac Han, Ziran Wang, John Lenneman, David Parker, and Lu Feng. Planning for automated vehicles with human trust. ACM Transactions Cyber-Physical Systems, 6(4):1-21, 2022.
[59] Scott Barnett, Stefanus Kurniawan, Srikanth Thudumu, Zach Brannelly, and Mohamed Ab- delrazek. Seven failure points when engineering retrieval augmented generation system. In Proceedings of the JEEF/ACM 3rd International Conference on Al SeitineeriinSSfftwrr Engineering for AI, pages 194-199, 2024.
[60] Junda Wu, Tong Yu, Xiang Chen, Haoliang Wang, Ryan Rossi, Sungchul Kim, Anup Rao, and Julian McAuley. Decot: Debiasing -ftthuggtt for www.lede-intensive tasks in large language models via causal intervention. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers), pages 14073-14087 2024.
[61] Dengyun Peng, Yuhang Zhou, Qiguang Chen, Jinhao Liu, Jingjing Chen, and Libo Qin. Dipo: Towards a robust, efficient, and generalizable prompt optimization framework from a deep- learning perspective. arXiv preprini arXiv:25033 13413, 2025.
[62] Judea Pearl. Causal inference in statistics: An overview. 2009.
[63] OpenAl. Gpt-4o technical report. httpp:///ppeaiiicco///eeessrs//p 2024.
14

---

[64] Google DeepMind. Gemini 1.5 technical report.
https://deopiind.gomm
technologies, /gmini// 2024.
[65] Kay W. Axhausen. Swissmetro. httppp///ww.rreearh--oollleeee ethz. ch/handle/
20.500, 11850/69494 2013. ETH Zurich Research Collection. Survey data collected in March 1998 on trains between St. Gallen and Geneva, Switzerland.
[66] Kin On Kwok, Tom Huynh, Wan In Wei, Samuel YS Wong, Steven Riley, and Arthur Tang. Utilizing large language models in infectious disease transmission modelling for public health preparedness. Computational and Structural Biotechnology Journal, 23:3254-3257, 2024.
[67] Ensheng Dong, Hongru Du, and Lauren Gardner. An interactive web-based dashboard to track covid-19 in real time. The Lancet infectious diseases, 20(5):533-534, 2020.
[68] Yanyong Guo, Jibiao Zhou, Yao Wu, and Jingxu Chen. Evaluation of factors affecting e-bike involved crash and e-bike license plate use in china using a bivariate probit model. Journal of advanced transportation, 2017(1):214255,, 2017.
[69] Ramina Javid, Eazaz Sadeghvaziri, Seyedmirsajad Mokhtarimousavi, and Hananeh Omidi. Predicting dicycle-involver crashes in the scag region: A machine learning analysis using hsis data from california state. In International Conference on Transportation and Development 2024, pages 305-314, 2024.
[70] Dominique Lord and Fred Mannering. The statistical analysis of crash-frequency data: A review and assessment of methodological alternatives. Transportation research part A: policy and practice, 44(5):291-305, 2010.
[71] Kundan Parajulee, Joshua Q Li, and Cody Hamblin. Multi-scale temporal analysis of connected vehicle data for safety analysis. In International Conference Transportation and Development 2025, pages 533-543, 2025.
[72] U.S. Department of Transportation, Federal Highway Administration. Highway Safety Informa- tion System (HSIS). https://highwag: dot gov, 2025, Accessed: January 13, 2025.
[73] Weike Lu, Jun Liu, Xing Fu, Jidong Yang, and Steven Jones. Integrating machine learning into path analysis for quantifying behavioral pathways in bicycle-motor vehicle crashes. Accident Analysis & Prevention, 166:20 2022.
[74] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric. P Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging au-uuee with mt-bench and chatbot arena, 2023.
[75] Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models. arXiv preprint arXiv:22 10.03493, 2022.
[76] Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309. 2023.
[77] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412. 19437, 2024.
[78] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21788 2024.
[79] Horace He and Thinking Machines Lab. Defeating nondeterminism in 11m infer- ence. Thinking Machines Lab: Connectionism, 2025. doi: 10.644334/ml.20250000
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The motivation is detailed in Section | The method is described in Section 4 The results are listed in Section S
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitation section included Section 8.6
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
16

---

Justification: See Section 8.1
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The settings are listed in Section and the full prompts needed are listed in the Appendix.
Guidelines:
The answer NA means that the paper does include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
17

---

Answer: [Yes]
Justification:
The code will shared in GitHub upon paper's acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ /pippe.cc/pullic/guuii wwwpc//ggue////eeeeeoooc CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The settings are listed in Section 5.2
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Please see Appendix 8.5 for the evaluation of variability of our method.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
18

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Section 8.4 for detailed discussion of both inference and training costs.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httpge///ourrrp...cc/ac/////eeea
Answer: [Yes]
Justification: Our research does not involve human subjects, sensitive data, or applications with foresceable socictal impact. All datasets used are publicly available and ethically sourced.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
See Section 8.6
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: NA.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All datasets and models used are publicly available and properly cited with their licenses respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
20

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
All new assets are well documented and provided in anonymized form for review.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification: NA.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: NA.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
21

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
Our method incorporates LLMs (e.g.. GPT-4o) as core components for rea- soning and optimization in specific tasks. Their usage is clearly described in the methodology and experimental setup.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, ntn-standard components.
Please refer to our LLM policy atteee//heerrippcccc///oeerrrrrs for what should or should not be described.
22

---

Technical Appendices and Supplementary Material
8.1 Generation of Reasoning Guidance
Derivation. Let denote the structured prompt for the domain-specifet task, 9 the semantic causal graph (SCG), and z the reasoning guidance distilled from (x, g). Starting from the product rule,
ply xx 9)
Σ p(y, 2.
(8)
Σ ply rl a, (. p(= 2 x)
(9)
(A1) Conditional independence. Because the SCG G is identical for every data point, once (æ, 2) given, the remaining information in G is irrelevant to y:
ply x. 2) = pvy 2, 2).
Applying (A1) to 9 yields
ply [. 1
Erly ply x, p(z 1)
(10)
(A2) Deterministic guidance. To guarantee that the graph description model My yields single, repeatable guidance for each (z, 9), we impose the following output constraint in the optimizer:
Prompt 8.1: Output Constraint for the Graph- Draph--escription Model
Format
<Causal Description>
Provide a numbered list of causal statements grounded in the supplied causal relations and crash details. Each statement must explicitly articulate the causal mechanism whenever is available.
</Causal Description>
Because every instance is processed with the same causal system prompt Peau and this constraint, the model behaves almost deterministically, inducing the mapping
2 (x,9) (M 9; P.m.).
so that the posterior over collapses to Dirac delta, p(= x. x. = = ()) Substituting this
into IOp:
pΦy x, p(y (z, 9))
0 - (x,
pvy 2 (z, 9))
(11)
Equation 11] shows that, under assumptions (A1) the global graph & can be replaced by the sample-specific deterministic guidance 2 (x. 9) without losing any predictive information.
23

---

8.2 Performance Comparison of Open Source Models and Commercial Models
In addition the commercial models GPT-40-mini, GPT-4.1-mini, GPT---mini, Gemini 2.5 Flash, and Gemini 2.0 Flash, we extended our experiments to evaluate EGO-Prompt on open-source models, including Qwen3 (1.7B, 8B. 14B, 32B) (76,, DeepSeek- V3 [77). and Llama-3.3 (8B, 70B), and I. Llama--SScou-117 17B 78 Table 7 shows the experiment results. Several findings can be summarized:
EGO-Prompt is efffcttive for open source models. EGO-Prompt boosts the performance of open-source models by 49.9%, which substantially higher than its improvement on commercial models (22.7%).
Commercial models achieve better and more stable performance. Across both the initial organized prompt and the optimized settings, commercial models consistently outper- form open-source models, achieving higher Fl scores and greater robustness across tasks compared with open-source models (0.427 0.351).
Model size substantially influences the performance ceiling. Models with more parame- ters achieve better optimized performance, consistent with the scaling law. For example, the Qwen3 series improves from an Fl score of 0.265 for the 1.7B model 0.390 for the 32B model. DeepSeek- V3, with 671B parameters, achieves the highest performance among the open-souree models. These results suggest that our method can consistently enhance performance across models of varying sizes.
enhances reasoning smaller models. Models such as Qwen3-1.7h and Qwen3-8B fail to generate structured outputs under the initial organized prompt. However, with ProOPr they are able to produce more structured predictions and follow certain reasoning paths to make predictions, although their overall performance remains limited.
Table 7: Performance comparison of open source models and commercial models across three datasets.
Pandemic
TraffleSafe
Mode Choice
Type
Base Model
Mean F1 (1%)
Ace
Acc
F1
Acc
F1
Qwen-1.7B
0.120
0.158
0.110
0.120
0.047
0.084
0.121
Owen 1. 1.7B with EGO
0.190
0.178
0,200
0.230
0.412
0.386
0.265
(1119.5%)
Qwen-88 8B
0.060
0.220
0.100
0.059
0.096
Owen 8B with EGO
0.140
0.199
0.290
0.202
0.459
0.427
0.276 (1200.1 %)
Qwen-1
0.299
0.330
0.227
0.424
0.309
0.279
Qwen-14B with EGO
0.390
0.339
0.340
0.271
0.471
0.457
0,356 (127.7 %)
Qwen-328
0.340
0.284
0.310
0.229
0.412
0,321
0,278
Qwen 32B with EGO
0.430
0.405
0.350
0.285
0.494
0.481
0.390 (740.3 3%)
Open Source
DeepSeek-V3
DeepSeek-V3 with EGO
0.460
0.210
0.464
0.231
0.400
0.340
0.371
0.211
0.506
0.459
0.501
0.457
0.445 (148.7
0.300
LLaMA-3.3-88
0,350
0.274
0.310
0,210
0.310
0,210
0,231
LLaMA-3.3 EGO
0.377
0.335
(136.8%) %%
1.
0.370
0.317
0.250
0.176
0.400
0.275
0.256
LLaMA-3.3-70B with EGO
0.420
0.378
0,340
0.273
0.459
0.419
0.357
(139.5
Llama-4-Scout-17B
0.360
0.307
0.260
0.237
0.294
0.287
0,277
Llama-4-Souu-112 with EGO
0.420
0.376
0.320
0.268
0.482
0.459
0.368
(133.0
Average
0.279
0.248
0.271
0.193
0.313
0.262
0,235
Average with EGO
0.361
0.341
0.324
0.275
0.461
0.438
0.351 (749.9 %
GPT-4o-mini
0.360
0.347
0.300
0.232
0.459
0.406
0,328
GPT-4o-miai with EGO
0.410
0.399
0.380
0.333
0.506
0.498
0,410
(724.9%
GPT-4.1-mie
0.400
0.374
0.350
0.246
0.506
0.468
0.363
GPT-4.I-mini with EGO
0.430
0.420
0.370
0.292
0.553
0.522
0.411
11.2 %)
GPT min
0.420
0.387
0.330
0.265
0.435
0.435
0.362
Commercial
GPT-S-mini with EGO
0.460
0.448
0.340
0.305
0.529
0.511
0.421 (1 [116.39 %)
Seminn-2.5-Fllas
0.470
0.470
0.340
0.319
0.459
0.392
0.394
Gemini-2.5 1.5-ii--2-FFlaa with EGO
0.540
0.546
0.430
0.428
0.518
0.499
0.491 (?24.6 %
Gemini
0.330
0.264
0.259
0.351
Gemini-2.0 Flash with EGO
0.430
0.412
0.400
0.327
0.506
0.470
0.4033 (137.00% %))
Average
0.390
0.369
0.330
0.265
0.424
0.410
0.348
Average with EGO
0.454
0.445
0.384
0.337
0.522
0.500
0,427 (122.7% %)
24

---

8.3 Human-Expert Involvement Evaluation
To assess how human involvement and the completeness of the SCG affect the reasoning process, we evaluate our method under several SCG completeness settings (Table 8): 1) Reversed SCG. To test sensitivity to incorrect causal structures, we reversed all edges the expert-designed SCGs used in the main experiments (see Figure 6). This allows us compare performance differences between mostly correct and mostly incorrect SCGs. 2) Empty SCG. We removed the initial SCG and let the model construct it during optimization. This tests whether providing good initial SCG improves performance and whether SCG can be constructed automatically. 3) 33% and 66% % SCG. These settings randomly retain only 33% 66% of the original cauasl edges, reflecting different degrees of SCG completeness. The key observations are as follows:
Incorrect SCG degrades reasoning performance. On the Pandemic dataset, the reversed SCG substantially degrades the performance (F1 from 0.359 0.303), suggesting that incorrect prior knowledge can misguide the optimization process.
Randomly removing causal edges can impair the model's reasoning capability. The initial SCG represents complete reasoning path. Although may not entirely correct, maintains structural integrity and connected as DAG. Random removal of edges disrupts this structure, leading to degraded performance. For example, the 33% SCG setting yields a lower mean Fl score than the Empty SCG, indicating that a partially broken causal graph may be worse than having no guidance at all.
Greater completeness in the SCG supports better reasoning. The 66% SCG setting achieves performance close to that of the full SCG, suggesting that more complete causal guidance helps the model reason more effectively.
If a user is unsure about the completeness or correctness of the SCG, it is advisable to start with an empty SCG and iteratively refine it. This allows gradual construction of reliable causal graph without risking the adverse effects of incorrect or incomplete edges.
Healticcre System Condition
Hospitaliaation per rook
Demographic Information
Vaccination Coverage
Population Inmurrity
RC Reparied Cases per
(Targat) of
Hoospital Staffing Condition
Restriction Policy Response
Mospitalization
Figure 6: Reversed Pandemic SCG.
Table 8: Performance under different SCG settings. We use GPT-4o-mini as the forward engine.
Pandemic
TrafficSafe
Mode Choice
SCG Setting
Mean F1
Aec
Acc
Acc
No SCG
0.380
0.359
0.300
0.243
0.506
0.432
0.345
Reversed
0.310
0.303
0.290
0.260
0.482
0.457
0.350
Empty
0.370
0.372
0.330
0.317
0.506
0.470
0.394
33%
0.390
0.389
0.300
0.270
0.459
0.462
0.378
66%
0.400
0.387
0.310
0.314
0.506
0.493
0.402
Full
0.410
0.399
0.380
0.333
0.506
0.498
0.421
25

---

8.4 Analyzing the Trade-off Between Performance and Cost
As shown in Table 9 we evaluate the inference performance and cost of our model compared to mainstream reasoning models, including OpenAl models o3 23-2025-04-101
and Costs for OpenAl models were recorded based on actual API platform expenses, while costs for Google models were estimated from imput and output token counts. Results show that our EGO-Prompt (GPT-40 mini) matches the performance o4-mini at roughly one-sixth the cost, and outperforms ol at over one-hundredth the cost.
od-minn-2225500-16,
o1-2024-12-17
Google's gemini-2. 5- S--poo-previee-5------
Table 9: Comparison of model performance with their price. Price measured in USD per 100 inferences.
Pandemic
TrafficSafe
Swissmetro
Model
Acc
F1
Price
Acc
Fl
Price
Acc
F1
Price
o3-mini
0,390
0.383
S0.50
0.330
0.282
$0.57
0.494
0,442
$0.45
o4-mini o1
0,430
0.418
S0.33
0.380
0.334
$0.26
0.494
0.456
$0.29
0.410
0.409
$10.19
0.310
0.219
$7.18
0.541
0.496
$7.04
o3
0.450
0,425
S3.06
0.410
0.385
$2.12
0.506
0.444
$3.32
Gemini 2.5 pro
0,400
0.399
S0.79
0.410
0.411
$0.45
0.471
0,402
$0.19
EGO-Prompt (GPT-4o mini)
0,410
0.399
S0.04
0.380
0.333
$0.08
0.506
0.498
$0.05
EGO-Prompt (Gemini 2.5 Flash)
0.540
0.546
50.06
0.430
0.428
$0.07
0.518
0.499
$0.13
As our model optimizes the reasoning process using ground truth data, the training incurs additional cost. The cost of our method throughout the optimization process is evaluated using GPT-4o-mini as the forward engine and GPT-4o as the backward engine. Each optimization step includes two forward passes through both the graph description model and the prediction model, two backward passes to iteratively update the SCG, system prompt, and causal system prompt, as well two evaluations on the validation set and zero or one evaluation on the test set (see Algorithm [] We recorded the actual costs using the OpenAl API platform during the optimization and reported the average cost over three steps for each task. As shown in Table T0 each optimization step for the three tasks costs approximately $0.3-$0.4. Given that each task typically requires 6 to 12 optimization steps, the total cost for the full optimization process ranges from approximately $2 to $5 per task. However, in real-world domain-speciff tasks, such as traffic safety modeling with hundreds of thousands of crashes reported annually in single state, the training cost becomes relatively negligible compared the inference cost.
Table 10: Optimization cost per step for each dataset, measured in USD.
Dataset
Opt. Cost per Step (USD)
Pandemic
0.310
TrafficSafe
0.403
Swissmetro
0.313
8.5 Variability of the Performance
Due to the inherent non-deterministic nature of LLM outputs (unstable even when tempera- ture=0) [79), as well as the instability of the Textual Gradient method, our results exhibit fluctuations within a certain range. Figure 7 presents the box plots of the performance of the organized prompt and EGO-Prompp across 10 independent runs on three tasks. We observe that our method may exhibit up to 20% variability performance across the three tasks. In some cases, as only a very limited number of training examples are used in our method, the model may overfit to the validation set in certain training step, leading to degraded performance on the test set. Addressing this issue will be an important direction for future work.
26

---

Organized Prompt
PromPr
Weighted F1
0.45
0.40
0.35
0.30
0.25
0.20
Swiss
TrafficSafe
Pandemic
Figure 7: Performance of organized prompt and EGO-Prompt across 10 independent runs on three tasks. We use GPT-4o-mini as the forward engine.
8.6 Limitations
The primary limitation of our work lies in the inherent variability introduced by the API-based inference process. To explore the upper-bound performance under identical settings (same random seed), we perform multiple runs for each experiment and report best results. However, similar to other prompt optimization methods, this stochasticity cannot be fully eliminated and may still influence performance evaluation. Additionally, due to the relatively small number validation and test examples (consistent with prior work prompt optimization) there is risk of overfitting to the validation set in certain runs. Future work may address these challenges by incorporating more robust evaluation protocols and exploring prompt tuning strategies that are less sensitive to sampling noise.
In addition, our method can enhance the performance of L.LMs on domain-specifie tasks, thereby en- abling more effective decision support (e.g., identifying turning points in pandemic trends). However, the reasoning processes of LLMs may produce incorrect or unreliable results for certain cases. Future research could focus improving the trustworthiness of LLMs in such Nomain-specinf applications.
27

---

8.7 Example of Organized Prompts
Prompt 8.2: Example of Organized Prompts for Pandemic Dataset
[System Prompt] Predict the trend of hospitalizations for the next week based the pandemic details provided between <Pandemic Description> and <\Pandemic Description>,
Provide a single prediction enclosed in < using one of the following labels: <substantial decreasing?, <moderate decreasing?, <stable?, <moderate increasing>, <substantial increasing>.
Definitions:
"Substantial" refers to changes greater than 3.
"Moderate" corresponds changes between and "Stable" is defined as changes between -1 and 1.
The final line of your response must follow this format: <VALUE>, where VALUE is your prediction.
<Pandemic Description>
(Demographic Information] Vermont, with one of the smallest populations and one of the smallest Black demographic groups, voted Democratic in the recent Presidential election.
[Healthcare System Condition] During the pandemic, Vermont's healthcare systems per- formed among the best, with above- wateeenatinallvvrraae Access and Affordability, excellent Prevention and Treatment, better-than-average population health conditions, and reduced Income Disparity.
[ICU and Hospital Stafling Condition] Vermont had ICU stress levels near the national average, but hospital staffing shortages worse than the national average.
[Vaccination Coverage] As of now, 81 % of the population has received least vaccine dose (Rapid Increase trend), 71 % are fully vaccinated (Moderate Increase trend), and 23% received boosters (Rapid Increase trend).
[Population Immunity] Around 28% the population reported infections in the past three months, and population immunity is showing Rapid Increase.
[Restriction Policy Response] School closures were recommended, but there were no restrictions for workplaces gatherings among elderly patients. Isolation was recommended, and visitor restrictions were in place.
(Hospitalization per 100k] The average number of COVID-19 hospitalizations per 100K over the past five weeks was 9.8. Hospitalizations remained relatively stable, mostly between 9.0 and 11.2. A slight increase was observed in the most recent week, with rate of change of 1.2. Volatility in hospitalization numbers was minimal, indicating consistent trends.
[Reported Cases per 100k] In the most recent five weeks, reported COVID-19 cases 100K showed a fluctuating trend. The average was 292.6. Cases declined from 263.8 216.0 over the first three weeks, then sharply increased to 340.1 in the fourth week and 398.7 in the fifth. These changes indicate significant uptick recent weeks, with inconsistent weekly trends.
<\Pandemic Description
[Trend of Hospitalization (Ground Truth)] <moderate increasing>
28

---

Prompt 8.3: Example of Organized Prompts for TrafficSafe Dataset
[System Prompt] Predict the crash severity reasoning the causal descriptions provided between the crash event details provided between <Crash Description. and <Crrahh Description>.
Provide a single prediction enclosed in < > using one of the following labels: <no apparent injury?, <minor injury?, <serious injury>, <fatal>. The last line of your response should only be of the following format: <VALUE> where VALUE your prediction.
<Crash Description>
[Time] The crash occurred on April 29, 2022 at hour 16.
[Position] The crash occurred in Champaign, within an Unincorporated area. It did occur in work zone.
[Dynamic Conditions] The light condition is Daylight and the weather condition is Clear.
[Infrastructure] The crash is not at an intersection. The traffic control device is Other Regulatory Sig.
[Road Surface] The road surface condition Dry. The road defect condition is nan.
[Road Level] The trafficway is Not Divided Two-way. The functional class of the roadway is Minor Arterial. The roadway class is Rural Lane Roads.
[Driver Behavior] The primary behavior is Driving On Wrong Side/Wrong Way, and the secondary behavior Improper Lane Usage. The crash
hit-and-run incident.
[Vehicle 1 Vehicle Information] The vehicle had a defect of None and was manufactured in 2004.
(Vehicle 2 Vehicle Information] The vehicle had a defect of None and was manufactured in 2002.
(Vehicle Vehicle Status] The unit locates at On Pavement (Roadway). The vehicle's maneuver prior the crash was Passing/Overtaking and it was traveling in the North direction. (Vehicle Vehicle Status] The unit locates at On Pavement (Roadway). The vehicle's maneuver prior the crash was Straight Ahead and it was traveling in the South direction.
[Person Person Information] This person was in Vehicle Unit 1. The person involved is Driver, aged 39. Gender is Male. [Person Person Information] This person was in Vehicle Unit 2. The person involved is Driver, aged 70. Gender is Male.
[Person Person Status] The driver's blood alcohol content .000. Distraction status: No. Safety equipment used: Shoulder and Lap Belt Used. [Person Person Status] The driver's blood alcohol content is Not Tested. Distraction status: No. Safety equipment used: Shoulder and Lap Belt Used.
<\Crash Description>
[Crash Severity (Ground Truth)] <fatal>
29

---

Prompt 8.4: Example of Organized Prompts for Swissmetro Dataset
[System Prompt] Predict the travel mode choice reasoning the causal descriptions pro- vided between <Causal Description> and <\Causal Description', and the traveler details provided between <Traveler Description' and <\Traveler Description>, Provide a single prediction enclosed in < using one of the following labels: <swissmetro>, <car>, <train
The final line of your response must follow this format: <VALUE>, where VALUE is your prediction.
<Traveler Description> [rripppurpos]] The purpose of the trip is business. trip_paid_byy Traveler trip is paid by oneself. [luggage] Traveler has no luggage.
[first class_llss The traveler earms 100,000 CHF and does travel in first class, [rail pass] Traveler does not have rail-system annual season ticket.
] This trip starts at VD and ends at ZH. (options_coun) Traveler has two possible travel options.
[swissmetro time_cost] Swissmetro's travel time is 63 minutes and it costs 57 CHF. [swissmetro ] The headway of Swissmetro is 10 minutes. [train time_tcmeeeeeeeeee _eeeec_os]______ Train's 's travel time is 192 minutes and it costs 52 CHR. [train trran_hheawway The headway of train is 30 minutes.
[income] Traveler's annual income is between 50,000 and 100,000 CHF. [age_range] The traveler is between 39 and years old. [gender] The traveler is female. <\Traveler Description>
[Travel Mode Choice (Ground Truth)] <car>
30

---

8.8 Initial SCG
Prompt 8.5: Initial SCG for Pandemic Dataset
CAUSAL STATEMENT I: [Demographic Information] affects [Vaccination Coverage] and [Restriction Policy Response].
Older or vulnerable populations often have higher vaccination uptake and more likely to be targeted by stricter restrictions.
CAUSAL STATEMENT [Healtheare System Condition] affects [Vaccination Coveragel and [Population Immunity]. Regions with better healthcare access can distribute vaccines more effectively and maintain higher baseline immunity.
CAUSAL STATEMENT 3: [ICU and Hospital Stafling Condition] affects [Restriction Policy Response].
When ICU beds are full or stafling is limited, governments tend to implement stricter control policies.
CAUSAL STATEMENT 4: |Vaccination Coverage] affects [Population Immunity].
Higher vaccination coverage directly increases the proportion immune individuals in the population.
CAUSAL STATEMENT 5: [Population Immunity] affects [Reported Cases per 100k] and [Hospitalization per 100k].
Stronger immunity reduces both the number of new infections and the chance of severe cases needing hospitalization.
CAUSAL STATEMENT 6: [Reported Cases per 100k] affects [Hospitalization per 100k] and [Restriction Policy Response].
A rise in reported cases usually precedes more hospital admissions and can trigger policy tightening.
[Hospitalization per 100k] affects [Restriction Policy Response]. High hospitalization levels often lead to immediate government intervention to limit further spread.
CAUSAL STATEMENT 7:
CAUSAL STATEMENT 8: [Hospitalization per 100k] and [Restriction Policy Responsel affect [Change of Hospitalization Next Week]. The trends of hospitalization in past weeks have strong relation with change of hospitalization next week.
31

---

Prompt 8.6: Initial SCG for TrafficSafe Dataset
CAUSAL STATEMENT 1: [Person Status] affects [Severity].
The driver's Blood Alcohol Content (BAC) significantly increases the probability of fatal crashes.
CAUSAL STATEMENT 2: [Position] affects [Severity].
Work zones can increase the probability of serious and fatal crashes. Driving in work zones after drinking is especially likely to cause severe fatal crashes.
[Driver Behavior] affects [Severity). Aggressive driving and impairment-related behavior pose higher risk than other driver behaviors.
CAUSAL STATEMENT 3:
Prompt 8.7: Initial SCG for Swissmetro Dataset
CAUSAL STATEMENT 1: [Gender] and [Age] affect [Trip Purpose] and [Luggage].
Younger travelers are more likely to travel for education or leisure and carry luggage; older travelers more often travel for business with less luggage.
CAUSAL STATEMENT 2: [Income] affects [First Class], [Rail Pass], and ['Tri__Paid __]] High-income travelers are more likely to choose first class, own rail pass, and pay for the trip themselves.
CAUSAL STATEMENT 3: [Trip Purpose] affeets [Trip Trid By] and [Luggage]. Business trips are often employer-paid and involve less luggage; leisure trips are usually self-paid and involve more.
CAUSAL STATEMENT 4: [Origin and Destination] determine [Travel Options], [Travel Time], and [Headway]. Major city pairs offer more modes, shorter travel time, and higher frequency.
CAUSAL STATEMENT 5: [Trip Purpose] affects [Travel Mode Choice].
Business travelers tend to prefer faster, more reliable modes; leisure travelers may prioritize cost or flexibility.
CAUSAL STATEMENT 6: [First Class] affects |Travel Mode Choice]. Travelers choosing first class are more likely to select Train or Swissmetro over Car for comfort.
CAUSAL STATEMENT [Rail Pass] affects [Travel Mode Choice]. Travelers with rail pass are more likely to use Train or Swissmetro due to lower perceived cost.
CAUSAL STATEMENT [Luggage] affects [Travel Mode Choice). Travelers with heavy or bulky luggage may prefer Train or Car.
CAUSAL STATEMENT [Trip_ Paid By] affects [Travel Mode Choice].
If the trip employer-paid, travelers tend to choose faster or more comfortable modes like Swissmetro; self-paid, they prefer cheaper options like standard Train or Car.
CAUSAL STATEMENT 10: Travel Time] and [Headway] affect [Trrvell Mode Choice]. Business travelers are more sensitive to time and prefer faster and frequent modes; leisure travelers may tolerate longer travel time or wait the mode is cheaper or more flexible.
32

---

8.9
Refined SCG
Prompt 8.8: Refined SCG for Pandemic Dataset (Compared to Initial)
CAUSAL STATEMENT I: [Demographic Information] affects [Vaccination Coverage and [Restriction Poliey Response).
Older or vulnerable populations often have higher vaccination uptake. and-are mere more-likel--eee be be-targeted by by-sttieter atrietee-restrietten
CAUSAL STATEMENT 2: [Healthcare System Condition] affects [Vaccination Coverage] and affects [Population Immunity].
Regions with better healtheare access can distribute vaccines more effectively and maintain higher baseline immunity.
CAUSAL STATEMENT 3: [ICU and Hospital Staffing Condition] affects [Restriction Policy Response].
When ICU beds full or staffing is limited, governments tend implement stricter control policies.
CAUSAL STATEMENT 4: Vaccination Coveragel affects [Population Immunity).
Higher vaccination coverage directly increases the proportion immune individuals in the population.
CAUSAL STATEMENT 5: [Population Immunity] affects [Reported Cases per 100k] and [Hospitalization per 100k].
Stronger-mmunitty reduces both the number of new inwe-infcttionseeo and the chance of severe-cases cases needing hospitalization. Stronger immunity reduces the number of new infections.
CAUSAL STATEMENT 6: [Reported Cases per 100k] affects [Hospitalization per 100k). and (Restrietionn Peliee- Responsel
rise reported cases usually precedes more hospital admissions. and ean trigger policy tightening.
CAUSAL STATEMENT 7: [Hospitalization per 100k] affects [Restriction Policy Response]. High hospitalization levels often lead to immediate government intervention to limit further spread.
CAUSAL STATEMENT 8: [Hospitalization per 100k] and [Restriction Policy Response] affect [Change of Hospitalization Next Week]. The trends of hospitalization in past weeks have strong relation with change of hospitalization
next week.
CAUSAL STATEMENT 9: [Healthcare System Condition] affects [Hospitalization per 100k].
Poor healthcare system performance can lead to higher hospitalization rates due to inadequate prevention and treatment measures.
33

---

Prompt 8.9: Refined SCG for TrafficSafe Dataset (Compared to Initial)
CAUSAL STATEMENT 1: [Person Status] affects [Severity].
The driver's Blood Alcohol Content (BAC) will significantly increase the probability of <FATAL> crashes.
CAUSAL STATEMENT 2: [Position] affects [Severity].
Work zone increase the probability of <SERIOUS INJURY> and <FATAL> crashes. Driv- ing in a work zone after drinking is very likely to cause <SERIOUS INJURY> or <FATAL> crashes.
CAUSAL STATEMENT [Driver Behavior] affects [Severity].
Aggressive driving and impairment-related behavior more risky than other driver behav- iors.
CAUSAL STATEMENT 4: [Road Conditions] affects [Severity].
Icy road conditions can significantly increase the likelihood of <SERIOUS INJURY> or <FATAL> outcomes due to loss of vehicle control.
CAUSAL STATEMENT 5: [Safety Equipment] affects [Severity]. The use of safety equipment, such as shoulder and lap belts, is likely to reduce the severity of injuries in the event of crash.
CAUSAL STATEMENT 6: [Road Level] affects [Severity].
Two-way undivided roads can increase the likelihood of <SERIOUS INJURY> <FATAL> outcomes due to the potential for head-on collisions.
CAUSAL STATEMENT 7: [Dynamic Conditions] affects [Severity].
Daylight and clear weather conditions are generally associated with lower crash severity, reducing the likelihood of <SERIOUS INJURY> or <FATAL> outcomes.
CAUSAL STATEMENT [Infrastructure] affects [Severity].
The presence of traffic control devices, such as stop signs, can influence crash severity by regulating vehicle flow and reducing collision risks.
CAUSAL STATEMENT 9: [Vehicle Information] affects [Severity].
Older vehicles or those with unknown defects may contribute to higher crash severity due to potential safety feature limitations.
34

---

Prompt 8.10: Refined SCG for Swissmetro Dataset (Compared Initial)
CAUSAL STATEMENT 1: [Gender] and AAge_Rangee affect [Trip Purpose] and [Luggage]] Younger travelers are more likely to travel for education or leisure and carry luggage; older travelers more often travel for business with less luggage.
CAUSAL STATEMENT [Income] affects [First Class] frait and [Self-Paid]] high-income travelers more likely to choose first class, rail pass, and pay for the trip themselves.
CAUSAL STATEMENT [Trip Purpose] affects [Self-Paid] and [Luggage]. Business trips are often employer-paid and involve less luggage; leisure trips are usually self-paid and involve more.
CAUSAL STATEMENT 4: [Origin and destination] determine [travel options], [travel time], and [headway]
(major-eity pairreeettttttt---teeeeees offer more-mosss, modes, soandes,shrrtte shorter travel time, and aad-highhr--ee frequeee))
CAUSAL STATEMENT 5: "Frip-purpens affects [travel medeelll-eeeeeeee chenee-
(business tend to fre-rrefer faster-fatter, more reliable mode;; leisure travelers may priioiiii eest-er flexibility)
CAUSAL STATEMENT (First (First-elass) alaest affeets (travel mode cheiie (travelers shreeeles-ehoesingg first elass are likely to seleet Train Swissmetro--vver Car for comfort)
CAUSAL STATEMENT 4: [Rail Pass] affects [Travel Mode Choice]. Travelers with rail pass are more likely to use Train or Swissmetro due to lower perceived cost.
CAUSAL STATEMENT [Luggage] affects [travel mode choice] (travelers with heavy or bulky luggage may prefer Train or Car)
CAUSAL STATEMENT 5: Iirip_Paid___ BB] affects l'Travel Mode Choice]. If the trip is employer-paid, travelers tend to choose faster or more comfortable modes like Swissmetro; if self-pait paid, they prefer cheaper options like standard Train or Car.
CAUSAL STATEMENT [Travel Time] and [Headway] affect [Travel Mode Choice]. Business travelers more sensitive to time and prefer faster and frequent modes; leisure travelers may tolerate longer travel time or wait the mode is cheaper or more flexible.
35

---

8.10
Constraints for SCG Optimizer
Prompt 8.11: Constraints for SCG Optimizer for Pandemic Dataset
Prompt Format
Your revised prompt must follow the structure below:
<SYSTEM PROMPT>
(system prompt)
<\SYSTMM PROMPT>
<Causal Relations>
(causal relations)
<\Causal Relations>
<Output>
(output format, fixed, don't revise this)
<'Output>
Causal Relations Guidelines
Only include causal relations between nodes for which corresponding information is available in the input description:
DDeoggrphhic Information]
[Population Immunity]
[hearthcare System Condition]
[Restriction Policy Response]
[ICU and Hospital Staffing Condition]
[Hospitalization per 100k]
[Vaccination Coverage]
[Reported Cases per 100k]
Operations
You can only the following operations: [1] Add new causal relations they are clearly supported by the input. Do not make assumptions without evidence. Use the format:
[Node A] affects [Node B]
(Explanation of how [Node A] affects [Node B])
Both [Node A] and [Ndee B] must come from the list above. You may include node-to- node relations not involving the final prediction target to support broader reasoning and imputation.
[2] Modily existing causal relations. You may:
Replace [Node affects [Node B] with more accurate link such [Node affects [Node C]
Update the explanation for clarity or correctness
[3] Delete any causal relation that is unsupported may negatively impact model inference. Remove both the relation and its explanation.
36


Reviews:
[{"review_id": "zGZSrshUAE", "reviewer": "Reviewer_KdBz", "summary": "The paper introduces EGO-Prompt, a framework designed to optimize prompt design and reasoning for LLMs in domain-specific tasks by incorporating expert-defined causal graphs. It begins with imperfect Semantic Causal Graphs (SCGs) and iteratively refines both the SCG and system prompts using textual gradients. This method produces instance-specific reasoning guidance, enhancing model accuracy without requiring fine-tuning. Applied to public health, transportation, and behavioral datasets, EGO-Prompt outperforms prior methods with F1 score improvements of up to 12%, while also reducing inference costs.", "questions": "In addition to addressing the concerns raised under the weaknesses section, we encourage the authors to clarify the following questions:\n\n1）\tCould the authors elaborate on the distinction between “expert knowledge” and “database knowledge” as described in the introduction and illustrated in Figure 1(a)? Specifically, why are RAG methods considered unsuitable for utilizing expert knowledge in this context?\n\n2）\tHow do the authors ensure that prompting with natural language effectively captures the complex, higher-order relationships inherent in graph structures? A discussion on the limitations and representational fidelity of such text-based graph encodings would be beneficial.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1)\tThe paper presents a well-structured and comprehensible pipeline for integrating domain knowledge into LLM reasoning, making the methodology easy to follow and replicate.\n\n2)\tThe use of SCGs offers a structured and flexible way to encode and evolve domain knowledge, enhancing the model’s ability to reason effectively in complex, real-world settings.\n\n3)\tThrough detailed ablation studies, the paper validates the individual impact of its core components—such as instance-specific reasoning guidance and iterative optimization—on overall performance.\n\nWeaknesses:\n\n1)\tThe paper does not compare EGO-Prompt against reinforcement learning-based prompt optimization methods (e.g., [1]), which are important alternatives in the field and could provide valuable benchmarks.\n\n2)\tAlthough six challenges are listed, their interrelationships and how the proposed method specifically addresses each are not clearly articulated. For instance, it is unclear how using a graph-based structure helps resolve the “task evolutionary” challenge or how the components of EGO-Prompt systematically map to the stated problems.\n\n3)\tThe claim that EGO-Prompt reduces human involvement lacks empirical support. Since the approach still relies on expert-created prompts and initial causal graphs, a quantitative analysis—such as testing performance under varying degrees of SCG completeness—would clarify the true extent of human dependency.\n\n4)\tSome experimental outcomes are counterintuitive. For example, in Table 5, using the graph seems to yield worse performance on the Swissmetro dataset compared to not using it (as seen in Table 4). The paper does not provide sufficient explanation for such discrepancies, which raises concerns about the method’s robustness across tasks.\n\nReference\n\n[1] Tempera: Test-time prompting via reinforcement learning.", "was_revised": true, "final_justification": "My major concerns have been addressed. I have raised the score.", "rebuttal": ">  **W1: The diference with RL-based methods.**\n\nThanks! We understand the question and agree that comparing with existing RL methods would be interesting. However, we believe such a comparison would not be fair or meaningful, as our goal differs fundamentally from RL-based prompt optimization. We aim to develop a generalized prompt optimization framework applicable for all LLMs in domain-specific tasks, rather than optimizing prompts for individual models with RL on each domain task.\n\n- **Distinction of prompt optimization paradigms.** Prompt optimization methods can be broadly categorized into two types: (1) continuous optimization, which directly updates model parameters; and (2) discrete optimization, which searches for improved prompts in natural language form. Most RL-based methods fall into the continuous category and are designed for open-source models.\n\n- **RL optimization may not be integrable or generalizable in closed-source LLMs. Our motivation is to leverage all LLMs for better domain-specific task adaptations (e.g., public health, transportation).** The end users are often domain experts with limited access to computational resources. Recent RL-based methods for prompt optimization typically require local fine-tuning or policy learning with open-source models [1,2], which conflicts with our motivation of lightweight and broadly accessible solutions for domain experts. In practice, if servers, model weights, and labeled data are available (always true in domain specific tasks), fine-tuning can be a stronger alternative.\n\n- **We add more baselines for both close- and open-source LLMs.**. Please see response to **Q3 of Reviewer UAca**.\n\nWe have revised the discussion to clarify our motivation and distinguish our approach from RL-based work [1,2] from scope and targets.\n\n>  **W2: How the proposed method addresses challenges.**\n\nThanks! We emphasized three challenges for seeking the potentially best prompt and reasoning procedure for domain-specific tasks without fine-tuning.\n- Domain-knowledge Adaptation. We encode expert knowledge using a SCG that captures abstract causal relationships.\n- Optimal Domain-adaptive Reasoning. The SCG guides the reasoning process, improving efficiency and performance.\n- Task Evolutionary. We dynamically refine the SCG using ground-truth data, which in turn updates expert knowledge through a reverse feedback loop.\n\nWe also overcome key limitations of exsiting RAG-based methods:\n- Partially Observed Domain Graphs. We avoid reliance on external databases by using lightweight, domain-specific SCGs.\n- Graph Priors Evolution. Please see W4 for details.\n- Optimal Graphical-guided Reasoning Process. Please see Q2 for details.\n\nWe have revised the introduction section of the manuscript to provide a simpler and more direct description.\n\n>  **W3: Quantitative analysis of human dependency.**\n\nThanks! To better understand the impact of human-provided SCGs, we conducted additional experiments under two settings:\n\n- **Reversed SCG.** To test sensitivity to incorrect causal structures, we reversed all edges in the expert-designed SCGs used in the main experiments. This allows us to compare performance differences between mostly correct and mostly incorrect SCGs.\n\n- **Null SCG.** We removed the initial SCG and let the model construct it during optimization. This tests whether providing a good initial SCG improves performance and whether SCG can be constructed automatically.\n\n|Setting        |EGO Opt.|Pandemic F1|TrafficSafe F1|Swissmetro F1|\n|---------------|--------------|------------|----------------|---------------|\n|Organized Prompt||0.347|0.232|0.406|\n|Reversed SCG   ||0.319|0.160|0.277|\n|Reversed SCG   |✓|0.303|0.260|0.457|\n|Null SCG       ||0.330|0.224|0.295|\n|Null SCG       |✓|0.372|0.317|0.470|\n|EGO-Prompt     |✓|**0.399**|**0.333**|**0.498**|\n\nFrom the experiment results, we found:\n\n- **Reversed SCG degrades reasoning performance.** Without optimization, reversing all causal edges introduces misleading priors and reduces performance.\n- **Reversed SCG damages prompt optimization.** On the Pandemic dataset, performance worsens even after optimization (F1 drops from 0.319 to 0.303), suggesting that incorrect prior knowledge can misguide the optimization process.\n- **EGO-Prompt can recover from a null SCG while carefully designed SCG is better.** A null SCG causes a slight initial drop (e.g., Pandemic F1: 0.347 → 0.330). This is expected, as the model receives no causal guidance at the start. After optimization, the model can construct a meaningful SCG, resulting in performance recovery. \n- **Correct SCG can yield better performance.** The expert-designed SCG consistently leads to the highest performance, both before and after optimization. In contrast, reversed SCGs degrade performance and obstruct optimization, while null SCGs hinder initial reasoning but allow recovery through learning.\n\n>  **W4: Lack explaination to Table 4&5.**\n\nThanks! These experimental results directly link to the motivation of our work.\n\n- **Table 4.** Table 4 shows two results: (1) using a single model to integrate the system prompt and SCG; and (2) our proposed method, which uses instance-specific reasoning guidance and a separate model to translate the SCG into textual guidance. The results show that our model outperformed than the single model design. As both experimrnts incorporate SCG, this results are not related to the performance degration issue brought by graph. \n- **Why graph degrade the performance in Table 5.** In Table 5, we observe that fixing either the SCG or the system prompt leads to degraded performance, while jointly optimizing both yields the best results. This is due to potential inaccuracies in expert-designed SCGs and misalignment when the prompt isn't adapted to the SCG.\n- **How SCG handle with the flawed graph.** We focus on aligning expert knowledge with the LLM’s reasoning process. By introducing the SCG, this focus becomes **aligning the graph-based information with the textual information** and **jointly optimizing both the graph and the prompt** to better guide the reasoning. So, we use the instance-spcific guidance to describe the graph as a better format to be integrated in textual prompt, and develop the iterative optimization method to co-optimize the SCG and system prompt. This approach allows the SCG to be corrected during training, while the system prompt is simultaneously adapted to effectively leverage the SCG for better performance.\n\n>  **Q1: Distinction between “expert knowledge” and “database knowledge”.**\n\n- **Graph database encodes entity-level factual knowledge**, e.g., (\"Crash #1234\", \"occurredOn\", \"Highway-99\"). This brings knowledge, but can not guide better reasoning. In contrast, **expert knowledge typically appears as conditional rules, heuristics, and mental models that operate under uncertainty,** focusing on the reasoning process rather than static facts. For example, “If the driver is fatigued and the road is slippery, the crash risk increases markedly”. Such judgments require abstract causal relations and contextual understanding that are difficult to encode as static database entries and cannot be reliably retrieved by RAG-based methods.\n- **SCG is an effective tool for representing expert knowledge.** SCG serves as a powerful framework to capture expert knowledge by combining a structured graph that encodes causal relationships with textual descriptions that convey factual information. Moreover, SCG can be losslessly integrated into the reasoning process of LLMs (see Equation (1) and Appendix 8.1) and dynamically updated through prompts using the Textual Gradient method.\n- **Why RAG is unsuitable here.** RAG methods rely on retrieving discrete, surface-level information chunks from a knowledge base. However, in our setting:\n\n    - There is **limited existing large-scale domain-specific knowledge graph database**. This is the common settings in real-world applications.\n    - Even if such a database existed, **retrieving knowledge fragments is insufficient to guide structured reasoning** across multiple interacting factors, as required in our tasks. SCG goes beyond retrieval and external knowledge supplementary and instead structures the model’s reasoning path.\n\n>  **Q2: Can EGO-Prompt capture the complex, higher-order relationships in graph?**\n\nThanks! We fully agree that this is a challenging but insightful idea. SCG provides a compact representation of expert reasoning by encoding abstract causal relationships between key concepts (see Q1). While graphs capture structural dependencies, they often lack the contextual richness needed for nuanced decision-making. For example, a graph might indicate “alcohol increases crash risk,” but not capture conditions like “young drivers at night in rural areas face higher risk.”\n\n**Instance-specific reasoning guidance bridges the gap between structured graph and natural language.** Graph-structured information, such as SCGs, is not directly compatible with LLM inputs in its raw form. To better integrate graph information into natural language prompts, we introduce a causal description model (see Figure 2) that aligns SCGs with natural language. By satisfying the Conditional Independence and Deterministic Guidance conditions (see Section 8.1), the model produces lossless textual descriptions that better preserve the high-order dependencies encoded in the graph while making them accessible to language models. Our experiments (Table 4) show that this design substantially improves performance over directly feeding graphs into the LLMs — **so the answer is yes.** We are also working on improving causal graph representation through symbolic graph representations.\n\n## References\n[1] Zhang, T., , et al. Tempera: Test-time prompting via reinforcement learning. ICLR. 2023\n\n[2] Kwon, M.,, et al. StablePrompt: automatic prompt tuning using reinforcement learning for large language models. EMNLP. 2024"}, {"review_id": "5UyaULxY6j", "reviewer": "Reviewer_DhWo", "summary": "The work proposes a method for optimising prompts and knowledge graphs to iteratively improve performance on tasks in 3 real-world domains.", "questions": "To what extent does the mathematical appearance of Equations 7 & 8 actually add to the rigor of the method?\n\nL517 (Appendix 8.1): Is the Dirac Delta function being used on discrete values?  Doesn't that make it an Indicator function?", "limitations": "yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "The proposed strategy of both prompt and knowledge graph optimisation is quite interesting, and (guessing from the multiple references to real-world application areas) can be of practical importance.  However, there are aspects of the write-up that detract from the core idea being presented.\n\n\n'Optimal' is used rather liberally throughout the paper.  While it is true that the Optimiser being proposed aims to produce 'More effective' or 'Improved' versions, suggesting that the results are likely _optimal_ is a stretch (particularly since the end results are <<100% accuracy).  Having the method being called 'Optimiser' is fine, but in several places in the paper the output is called 'optimal' without justification.  For example:\n\n* L118 'enabling an optimal reasoning process' (while 'optimal' is a good objective, your research doesn't 'thereby enable' it).  \n* L201 \"may lead to suboptimal performance\" - really means that it didn't work well when you tried it, correct?  Breaking the process into several steps (Algorithm 1) isn't actually achieving 'optimal performance' - it's just a better Algorithm than the previous one...\n* L253 'to identify the optimal prompts' -> 'to identify improved prompts'\n* Figure 1b : 'Best XYZ' could be 'Improved XYZ' (or Best *so far*)\n\nThe language around optimal outputs permeates the paper (i.e. there are many more example than the above).\n\n\nAround L136 (and equation 3+4) the idea of 'Textual Gradients' is given a mathematical formality which is way beyond the actual notion of derivatives.  At most these should be delta updates (rather than actual formal derivatives).  For instance, L137 : \"Chain rule\" of these gradients, is really stretching the point.  I realise that this idea can be found in the TextGrad work (now Nature paper), but overloading actual mathematical notation with LLM guesses at text deltas doesn't make it an actually good formalism.\n\n\nI am not familiar with the datasets being tested against, but it does appear that the results shown may be only marginal improvements.", "was_revised": true, "final_justification": "Rating remains the same, despite Authors considerable efforts at improving the wording of the paper : These changes make it a much better paper (IMHO).  However, even considering these changes, and rebuttal experiments, the overall system described does not rise above my own mental acceptance hurdle.", "rebuttal": ">  **W1: 'Optimal' is used liberally.**\n\nThank you for the helpful suggestion. We have revised all relevant expressions in the manuscript accordingly. The updates are summarized below:\n\n|**Position**|**Original**|**Revised**|\n|------------|------------|-----------|\n|Page 1, Line 6|Optimal prompts|Improved prompts|\n|Page 2, Line 39|Optimal Domain-adaptive Reasoning|Improved Domain-adaptive Reasoning|\n|Page 2, Line 70-71|Optimal reasoning|Effective reasoning|\n|Page 2, Line 74|Optimal prompts design|Better prompts design|\n|Page 2, Line 76|LLM optimal one-shot reasoning|More effective one-shot reasoning in LLMs|\n|Figure 1|Best/Optimal Prompt/ Graph|Improved Prompt/Graph|\n|Page 4, Line 118|Optimal reasoning process|Better reasoning process|\n|Figure 2|Best Prompt & Graph|Improved Prompt & Graph|\n|Page 6, Line 195|Optimal system prompt|Improved system prompt|\n|Page 6, Line 201|XXX may lead to suboptimal performance|XXX may lead to inferior performance|\n|Page 7, Line 253|Optimal prompts|Better prompts|\n\n>  **W2: The idea of 'Textual Gradients' is given a mathematical formality which is way beyond the actual notion of derivatives.**\n\nThank you for the constructive suggestion. \n\n- **We fully agree with your comment and have revised the notational system.** In our original manuscript, we adopted the formulation used in the TextGrad paper (see Figure 2 of [1]), where derivatives and chain rules are expressed using the partial derivation [1]. However, we fully understand your concern regarding the potential over-formalization and agree that such notation may overstate the actual mathematical rigor behind textual feedback. In fact, this concern resonated with us as well when we first encountered those equations. In light of this, we have revised our equations to more appropriately reflect delta-style updates rather than formal gradients, and we have added clarifying explanations to reduce the risk of misinterpretation. \n\n| **Eq. No.** | **Original** | **Revised** |\n|------------|--------------|----------------------------|\n| **3** | $\\nabla\\_{\\mathcal{P}\\_{\\text{sys}}} \\mathcal{L}(\\hat{y}\\_i\\, y\\_i) = \\frac{\\partial \\mathcal{L}(\\hat{y}\\_i\\, y\\_i)}{\\partial \\mathcal{P}\\_{\\text{sys}}} = \\mathcal{M}\\_B(x\\_i\\, y\\_i\\, \\mathcal{P}\\_{\\text{sys}}\\, \\mathcal{L}(\\hat{y}\\_i\\, y\\_i))$ | $ \\Delta \\mathcal{L}\\_i = \\mathcal{M}\\_B(\\hat{y\\_i}\\, \\mathcal{L}\\_i)\\, \\Delta P\\_{\\text{sys}}=\\mathcal{M}\\_B( P\\_{\\text{sys}}\\, \\hat{y\\_i}\\, \\Delta \\mathcal{L}\\_i ) $ |\n| **7** | $\\nabla\\_{\\mathcal{P}\\_{\\text{sys}}} \\mathcal{L}\\_i = \\frac{\\partial \\mathcal{L}\\_i}{\\partial \\hat{y}\\_i} \\cdot \\frac{\\partial \\hat{y}\\_i}{\\partial \\mathcal{P}\\_{\\text{sys}}}$ | Removed, refer to Eq. (3) |\n| **8** | $\\nabla\\_{\\mathcal{G}} \\mathcal{L}\\_i = \\nabla_{z^{\\*}\\_i} \\mathcal{L}\\_i \\cdot \\frac{\\partial z^{\\*}\\_i}{\\partial \\mathcal{G}}\\, \\quad \\nabla\\_{\\mathcal{P}\\_{\\text{cau}}} \\mathcal{L}\\_i = \\nabla_{z^{\\!*}\\_i} \\mathcal{L}\\_i \\cdot \\frac{\\partial z^{\\*}\\_i}{\\partial \\mathcal{P}\\_{\\text{cau}}}$ | $\\Delta P\\_{\\text{cau}}=\\mathcal{M}\\_B( P\\_{\\text{cau}}, z^{\\*}_i\\, \\Delta z^{\\*}\\_i)\\, \\Delta \\mathcal{G}=\\mathcal{M}\\_B(\\mathcal{G}\\, z^{\\*}\\_i\\, \\Delta z^{\\*}\\_i)$\\, where $\\Delta z^{\\*}\\_i=\\mathcal{M}\\_B(\\hat{y}_i,z^{\\*}\\_i\\,\\Delta \\mathcal{L}\\_i )$ |\n\n- **Textual Gradient is only used as the iteration optimizer, like Adam. We also use it as a baseline method.** Our core idea is the allignment of expert knowledge with LLM reasoning process. By introducing the SCG, this focus becomes aligning the graph-based information with the textual information and jointly optimizing both the graph and the prompt to better guide the model’s reasoning. To achieve this, we use the instance-spcific guidance to describe the graph as a better format to be integrated in textual prompt, and develop the iterative optimization method to co-optimize the SCG and system prompt. This approach allows the SCG to be corrected and completed during training, while the system prompt is simultaneously adapted to effectively leverage the SCG for improved performance. **In this entire process, Textual Gradient is simply used as a black-box optimizer**. We included Section 3 to briefly introduce the core idea of Textual Gradient, particularly because many readers may be unfamiliar with it and we have properly cited the original TextGrad work.\n\n\n> **W3: The results shown may be only marginal improvements.**\n\nThank you for your comment. We understand your concern regarding the performance improvements. However, given that the performance of LLMs is generally constrained by **scaling laws**, we would like to clarify that our contributions extend beyond prediction accuracy and also include improvements in **cost-efficiency, cross-domain generalization, interpretability, and the co-pilot design**.\n\n- **High accuracy.**As shown in Table 3 and the **long table in response to reviewer qzNs**, our method improves the F1 score by approximately 20%-50% over the organized prompt baseline across multiple LLMs, and achieves 8%–12% gains over SOTA prompt optimization methods on diverse tasks.\n\n- **Cost-efficiency.** As shown in Figure 4 and Table 7, we make **GPT-4o-mini comparable with reasoning models like o3 and o4-mini** which requires **6 to 140 times higher cost**.\n\n- **Cross-domain generalization.** Existing RAG-based methods require access to external databases, which are often unavailable in domain-specific tasks. This limits their generalization ability. Our method is designed for scenarios without existing graph databases and uses SCG to support LLM reasoning. SCG requires only minimal expert effort and can be dynamically corrected during optimization, making the approach lightweight and easily adaptable to other tasks. Please see response to **Q2 of Reviewer qzNs** for more detailed experimrnts.\n\n- **Interpretability.** As shown in Figure 5, our unified textual framework enables updating the SCG based on feedback from prediction errors. This offers a direct observation of flaws in expert-provided knowledge and improves transparency in the reasoning process.\n\n- **Co-pilot design.** Our method integrates expert knowledge directly into the LLM reasoning process, enabling the expert to guide and refine the model’s predictions. In turn, the LLM can learn from ground truth feedback and iteratively update the expert-provided knowledge. This co-pilot mechanism fosters a more interactive and adaptive decision-making process in domain-specific tasks.\n\n>  **Q1: How Equations 7 \\& 8 actually add to the rigor of the method.**\n\nThank you for the comment. We added Equations 7 and 8 with the following considerations:\n\n- **Explicitly formulate the update process of the variables.** Since our method involves two LLMs and updates three key components (the system prompt, the causal system prompt, and the SCG), we aimed to clearly specify how each variable is updated and which information it depends on. For instance, Equation 8 illustrates that the SCG is updated through the textual gradient of the reasoning guidance, which is approximated through a chain of feedback signals propagated from the prediction error back to the guidance itself. We acknowledge that the original equation may have appeared somewhat difficult to follow, but we believe it is important for clarifying the update process. Based on your suggestion in W2, we have revised the gradient notation using delta-style updates, which improves readability.\n\n- **Readers may not familiar with the chain rule in Textual Gradient.** As noted in W2, some readers may not be familiar with the concept of Textual Gradient and its underlying propagation process. We believe that Equations 7 and 8 can help facilitate understanding by providing a structured view of how feedback signals are propagated through different components of our framework.\n\n>  **Q2: Dirac Delta function being used on discrete values make it an Indicator function.**\n\nThank you for your careful observation. You are correct that the Dirac delta function is technically defined over continuous domains, whereas in our formulation, the reasoning guidance is discrete (i.e., a sequence of textual statements, though from a vast space).\n\nTo reflect this more accurately in the discrete setting, we have revised the expression to use an indicator function:\n\n$$p(y \\mid x\\, \\mathcal{G}) = \\sum\\_{z} p(y \\mid x\\, z)\\mathbb{I}\\left[z = z^{\\*}(x\\, \\mathcal{G})\\right] = p\\left(y \\mid x\\, z^{\\*}(x\\, \\mathcal{G})\\right)$$\n\n\n\n>  **Paper Formatting Concerns: Tables in Section 6, grammer corrections.**\n\nThank you for your suggestion. We have now revised the layout and corrected the grammatical errors accordingly.\n\n## References\n[1] Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Pan Lu, Zhi Huang, Carlos Guestrin, and James Zou. Optimizing generative ai by backpropagating language model feedback. Nature, 639(8055):609–616, 2025."}, {"review_id": "5kSac3EKf8", "reviewer": "Reviewer_UAca", "summary": "The paper proposes the Evolutionary Graph Optimization for Prompting (EGO-Prompt), an automated framework designed to craft optimal prompts, refine reasoning processes, and enhance causal inference. The framework starts with a general prompt constructed by human experts and an initial fault-tolerant Semantic Causal Graph (SCG) description, then guides LLM reasoning through iterative automated optimization.  \nThe framework was tested on real-world tasks in public health, transportation, and human behavior. Results show that EGO-Prompt achieves an 8%-12% improvement in F1-score compared to state-of-the-art methods while generating refined domain-specific SCGs.", "questions": "1. How much extra inference cost does it require compared to ​Zero-shot-CoT​ or ​Organized Prompt​?\n\n2. The baseline methods compared in the paper do not include other graph-based approaches. Are there such baselines available?\n\n3. How does it perform against automated ICL methods like ​AutoCoT​?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThe paper proposes a novel framework that employs a forward-backward simulation process to iteratively optimize the SCG and instance-specific prompts. \nThis approach generates more refined domain-specific prompts, enabling LLMs to enhance their reasoning steps and improve performance in specialized domains.\n\nWeaknesses: \n1. The method proposed in the paper appears to be an optimization based on TextGrad combined with some SCG techniques—the approach is relatively simple; \n\n2. The additional computational overhead incurred during the optimization process remains unclear.\n\n3. The results show that ProTeGi (GPT-4o mini) achieves higher accuracy than the proposed method in certain settings. Additionally, the performance gap between Zero-shot-CoT and EGO is relatively small.", "was_revised": true, "final_justification": "The authors' experiments demonstrate its low computational cost and strong performance across multiple models, so I will raise my score.", "rebuttal": "> **W1: The approach is relatively simple.**\n\nThank you for the comment. Our method innovatively integrate domain expert knowledge into LLMs for better reasoning in domain-specific tasks. Our contribution includes\n\n- **Expert knowledge guided LLM raesoning in domain-specific tasks.** Existing methods for domain-specific reasoning often rely on external graph databases to retrieve structured knowledge. However, this database is often unavailable in real-world scenarios. We use SCG to encode expert knowledge which not only provides explicit causal relations to guide LLM reasoning but also evolves during the optimization process, making the method adaptive and dynamic.\n\n- **Lossless instance-specific reasoning guidance which aligns the structural graph information with textual information.** Our experiments reveal that directly feeding SCGs into a single LLM yields degraded performance (see Table 4). This suggests that graph-structured information, such as SCGs, is not directly compatible with LLM inputs in its raw form. To better integrate graph information into natural language prompts, we introduce a causal description model (see Figure 2) that aligns SCGs with natural language. By satisfying the Conditional Independence and Deterministic Guidance conditions (see Section 8.1), the model produces lossless textual descriptions that effectively preserve the high-order dependencies encoded in the graph while making them accessible to language models.\n\n- **Overall Improvement in performance, cost-efficiency, cross-domain generalization, interpretability, and the co-pilot design.** Please see response to W3.\n\n- **Textual Gradient is only used as the iteration optimizer, like Adam. We also include it as a baseline method.** Please refer to the response to W2 of Reviewer DhWo for details.\n\n**Being simple and effective is a merit, especially in real-world tasks.** Our method is a multimodal prompt optimization framework that incorporates graph-based priors for real-world domain tasks (e.g., transportation, public health, and human behavior modeling) that require low cost, high interpretability, and strong reasoning quality. Through careful system design and thorough empirical validation, we achieve clear improvements in these aspects (**we also added 6 more experiments on open-source LLMs, see the long table in reviewer qzNs**). Our lightweight, plug-and-play method offers a practical solution with merit that can be readily adopted in real-world decision-making systems. \n\n>  **W2: The additional computational overhead incurred during the optimization process is unclear.**\n\nThank you for raising this important point. We have carefully evaluated the cost of our method throughout the optimization process, using GPT-4o-mini as the forward engine and GPT-4o as the backward engine. Each optimization step includes two forward passes through both the graph description model and the prediction model, two backward passes to iteratively update the SCG, system prompt, and causal system prompt, as well as two evaluations on the validation set and zero or one evaluation on the test set (see Algorithm 1). We recorded the actual costs using the OpenAI API platform during the optimization and reported the average cost over three steps for each task.\n\n|Dataset|Opt. Cost per Step (USD)|\n|----|----|\n|Pandemic|0.310|\n|TrafficSafe|0.403|\n|Swissmetro|0.313|\n\nAs each task typically requires 6 to 12 optimization steps, the total cost for the full optimization process ranges from approximately \\\\$2 to \\\\$5 per task. We have added this information to the appendix for reference.\n\n>  **W3: ProTeGi (GPT-4o mini) achieves higher accuracy than the proposed method in certain settings. The Performance gap between Zero-shot-CoT and EGO is relatively small.**\n\nThank! We understand your concern regarding the performance improvements. However, given that the performance of LLMs is generally constrained by **scaling laws**, we would like to clarify that our contributions extend beyond prediction accuracy and also include improvements in **cost-efficiency, cross-domain generalization, interpretability, and the co-pilot design**.\n\n- **Cost-efficiency.** As shown in Figure 4 and Table 7, we make **GPT-4o-mini comparable with reasoning models like o3 and o4-mini** which requires **6 to 140 times higher cost**.\n\n- **Average 8%–12% accuracy gain.** As shown in Table 3 and the **long table in response to reviewer qzNs**, our method improves the F1 score by approximately 20%-50% over the organized prompt baseline across multiple LLMs, and achieves 8%–12% gains over SOTA prompt optimization methods on diverse tasks. Although ProTeGi (GPT-4o mini) achieves slightly higher accuracy than our method on the Swissmetro dataset, our F1 score remains superior (0.498 vs. 0.481). **Considering that ProTeGi requires extensive search and more API calls to find the optimal prompt, our method still outperforms it with an 8.5% higher mean F1 score.**\n\n- **Cross-domain generalization.** Existing RAG-based methods require access to external databases, which are often unavailable in domain-specific tasks. This limits their generalization ability. Our method is designed for scenarios without existing graph databases and uses SCG to support LLM reasoning. SCG requires only minimal expert effort and can be dynamically corrected during optimization, making the approach lightweight and easily adaptable to other tasks. Please see response to **Q2 of Reviewer qzNs** for more detailed experimrnts.\n\n- **Interpretability.** As shown in Figure 5, our unified textual framework enables updating the SCG based on feedback from prediction errors. This offers a direct observation of flaws in expert-provided knowledge and improves transparency in the reasoning process.\n\n- **Co-pilot design.** Our method integrates expert knowledge directly into the LLM reasoning process, enabling the expert to guide and refine the model’s predictions. In turn, the LLM can learn from ground truth feedback and iteratively update the expert-provided knowledge.\n\n>  **Q1: Extra inference cost compared to Zero-shot-CoT or Organized Prompt.**\n\nThank you for the helpful comment. We evaluated the inference cost using GPT-4o-mini. Each method was tested on the test set and run 5–10 times. The actual expenses were recorded on the OpenAI API platform, and we report the average cost per 100 inferences in USD. The table below compares the accuracy, F1 score, and cost (per 100 examples) across Organized Prompt, Zero-Shot-CoT, and EGO-Prompt. EGO-Prompt incurs approximately 2–3 times higher cost than the Organized Prompt baseline and about 1.5 times higher than Zero-Shot-CoT. However, this additional cost yields notable performance improvements, achieving results comparable to reasoning models such as o4-mini or even o3 which requires 6 to 140 times higher cost.\n\n|Setting|Pandemic Acc|Pandemic F1|Pandemic Cost|TrafficSafe Acc|TrafficSafe F1|TrafficSafe Cost|Swissmetro Acc|Swissmetro F1|Swissmetro Cost|\n|-------|------------|------------|--------------|-------|----|-------|------|--------|---------|\n|Organized Prompt|0.360|0.347|0.016|0.300|0.232|0.020|0.459|0.406|0.019|\n|Zero-Shot-COT|0.370|0.361|0.028|0.280|0.221|0.048|0.494|0.435|0.034|\n|**EGO-Prompt**|**0.410**|**0.399**|**0.042**|**0.380**|**0.333**|**0.078**|**0.506**|**0.498**|**0.048**|\n|o4-mini|0.430|0.418|0.330|0.380|0.334|0.260|0.494|0.456|0.290|\n\n>  **Q2: Graph-based basselines.**\n\nThank you for the comment. One of our key motivations is to address the gap in domain-specific tasks where no existing structured graph database is available. As Please refer to response to **Q1 of Reviewer #KdBz.**\n\n>  **Q3: Automated ICL methods like AutoCoT**\n\nThank you for the suggestions. We added two additional Automated ICL baseline methods as follows: \n\n- **Auto-CoT [1]** is an automatic prompting method that generates diverse CoT demonstrations using LLMs. Following its framework, we uniformly sampled 200 training examples for each task and clustered them into 5 groups based on semantic similarity. Within each group, we used the zero-shot \"Let's think step by step\" prompt to generate CoT reasoning chains for all samples. We then selected the CoT example that produced the correct answer to serve as the demonstration for that cluster.\n\n- **DSPy [2]** is a SOTA LLM programming and prompt optimization framework. We use BootstrappedFewShotRandomSearch optimizer with 10 candidate programs and 5 few-shot examples. The optimizer identifies correct reasoning traces as candidate demonstrations and performs random search over subsets (up to 5 shots) to find the prompt configuration that maximizes task accuracy.\n\nWe tested both baselines using GPT-4o-mini as the forward model, and the results are shown in the table below. We observe that the automatic CoT methods perform poorly on our datasets. This aligns with our motivation: LLMs often lack sufficient domain knowledge for these tasks and struggle to learn effective reasoning traces from limited demonstrations, especially given that our input texts are substantially longer than those typically used in CoT benchmark tasks.\n\n|Method|Venue/Journal|Pandemic Acc|Pandemic F1|TrafficSafe Acc|TrafficSafe F1|Swissmetro Acc|Swissmetro F1|Mean F1 (Improvements)|\n|------|--|--|------|----|---|----|----|------|\n|Organized Prompt|—|0.360|0.347|0.300|0.232|0.459|0.406|0.328 (—)|\n|**Auto-CoT [1]**|**ICLR'23**|**0.380**|**0.352**|**0.340**|**0.230**|**0.447**|**0.462**|**0.348 (6.0%)**|\n|**DSPy [2]**|**ICLR'24**|**0.380**|**0.374**|**0.370**|**0.285**|**0.459**|**0.397**|**0.352 (7.2%)**|\n|**EGO-Prompt**|**This work**|**0.410**|**0.399**|**0.380**|**0.333**|**0.506**|**0.498**|**0.410 (24.9%)**|\n\n\n## References\n[1] Zhang, Z., et al. Automatic chain of thought prompting in large language models. ICLR. 2023.\n\n[2] Khattab, O., et al. Dspy: Compiling declarative language model calls into self-improving pipelines. ICLR, 2024."}, {"review_id": "nxSHILK599", "reviewer": "Reviewer_qzNs", "summary": "- This paper introduces EGO-Prompt, a novel framework for optimizing domain-specific reasoning in LLMs by integrating expert-defined semantic causal graphs (SCGs) and evolving them via textual gradients. The method decomposes reasoning into guidance generation and prediction stages and efficiently reduces inference costs.", "questions": "- I found that the LLMs used in the experiments are all closed APIs like GPT-40-mini and Gemini-2.5-Flash. These LLMs are mature commercial products with better instruction-following ability. Can EGO-Prompt be applied on open-source LLM backbones like llama or Qwen? For domain-specific application, it seems that privatized deployment of small-scale LLM is a necessity and open-source LLMs might be a better choice.\n- Does EGO-Prompt have cross-domain generalization ability? For example, train EGO-Prompt on one domain and generalize to another similar but not equal domain.\n- What is the cost for manual design of SGGs? Can SGG also be constructed automatically by the LLMs?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n- EGO-Prompt's two-stage reasoning process—generating instance-specific guidance from SCGs and optimizing prompts via textual gradients effectively bridges domain knowledge with LLM capabilities. This is a novel prompt engineering design.\n- The framework enables lightweight models (e.g., GPT-4o mini) to achieve or surpass state-of-the-art performance.\n\n## Weaknesses\n- The framework assumes human experts provide initial SCG, introducing potential biases that optimization may not fully correct. This design gap is not rigorously tested against imperfect expert inputs, weakening the method's autonomy and real-world applicability where expert knowledge is sparse or noisy.\n- The datasets used in experiments are relatively small-scale, and the number of specific domain is limited.\n- See questions for more question about this paper.", "was_revised": true, "final_justification": "I'd like to keep positive score as authors make enough response to my questions.", "rebuttal": ">  **W1: Lack of test with imperfect expert inputs.**\n\nThank you for the constructive comment. We fully agree that imperfect expert inputs are common in real-world applications, and it is important to evaluate the robustness of our method under such conditions. To evaluate this, we have conducted additional experiments where we evluate our model's performance under imperfect expert inputs. Please see response to **W3 of Reviewer KdBz**\n\n>  **W2: The datasets are small-scale, and the number of specific domain is limited.**\n\nThank you for the comment. \n\n- **We follow the standard settings used in discrete prompt optimization methods.** The commonly used benchmarks are GSM8K [1], a dataset of 7,473 training and 1,319 test samples of grade-school math word problems, and Big-Bench Hard (BBH) [2], which consists of 23 reasoning tasks with up to 250 examples per task, covering symbolic, logical, and commonsense reasoning. For BBH, during prompt optimization, each task is typically split into training, validation, and testing sets, with around 80 samples in each split. This is the common settings for those prompt optimization methods like Textual Gradient [3]. In our evaluation, we use datasets from three different domain, each containing approximately 300 samples for optimization, which is comparable in scale to existing benchmarks and methods.\n\n- **Our prompt length is substantially longer than commonly used benchmarks.** The average prompt length in GSM8K and BBH is around 100 tokens per question. In contrast, our datasets feature an average prompt length of approximately 500 tokens, incorporating much more complex contextual information and interactions among multiple factors.\n\n- **We cover multiple domains with real-world complexity.** While existing benchmarks often focus on a single type of reasoning task (e.g., given A and B, what is C), our datasets span three distinct domains: public health (Pandemic), traffic safety (TrafficSafe), and human behavior (Swissmetro). These domains involve diverse data structures, contextual dependencies, and decision factors. We also plan to extend our method to additional real-world application areas in future work.\n\n\n\n> **Q1: Can EGO-Prompt be applied on open-source LLM?**\n\nThank you for the helpful comment. In addition to the commercial models GPT-4.1-mini, GPT-4o-mini, Gemini 2.5 Flash, and Gemini 2.0 Flash, we extended our experiments to evaluate EGO-Prompt on open-source models, including Qwen3 (1.7B, 8B, 14B, 32B), DeepSeek-V3, and Llama-4-Scout-17B. We summarize several findings below:\n\n|Type|BaseModel|PandemicAcc|PandemicF1|TrafficSafeAcc|TrafficSafeF1|SwissmetroAcc|SwissmetroF1|MeanF1(improvements)|\n|---|---|---|---|---|---|---|---|---|\n|**Open models**|Qwen-1.7B(Base)|0.120|0.158|0.110|0.120|0.047|0.084|0.121|\n||+EGO-Prompt|0.190|0.178|0.200|0.230|0.412|0.386|**0.265(↑119.5%)**|\n||Qwen-8Bc(Base)|0.060|0.080|0.220|0.100|0.059|0.096|0.092|\n||+EGO-Prompt|0.140|0.199|0.290|0.202|0.459|0.427|**0.276(↑200.1%)**|\n||Qwen-14B (Base)|0.360|0.299|0.330|0.227|0.424|0.309|0.279|\n||+EGO-Prompt|0.390|0.339|0.340|0.271|0.471|0.457|**0.356(↑27.7%)**|\n||Qwen-32B (Base)|0.340|0.284|0.310|0.229|0.412|0.321|0.278|\n||+EGO-Prompt|0.430|0.405|0.350|0.285|0.494|0.481|**0.390(↑40.3%)**|\n||DeepSeekV3 (Base)|0.210|0.231|0.340|0.211|0.459|0.457|0.300|\n||+EGO-Prompt|0.460|0.464|0.400|0.371|0.506|0.501|**0.445(↑48.7%)**|\n||Llama-4-Scout-17B-16E-Instruct-FP8(Base)|0.360|0.307|0.260|0.237|0.294|0.287|0.277|\n||+EGO-Prompt|0.420|0.376|0.320|0.268|0.482|0.459|**0.368(↑33.0%)**|\n|**Close models**|GPT-4o-mini (Base)|0.360|0.347|0.300|0.232|0.459|0.406|0.328|\n||+EGO-Prompt|0.410|0.399|0.380|0.333|0.506|0.498|**0.410(↑24.9%)**|\n||Gemini-2.5-Flash (Base)|0.470|0.470|0.340|0.319|0.459|0.392|0.394|\n||+EGO-Prompt|0.540|0.546|0.430|0.428|0.518|0.499|**0.491(↑24.6%)**|\n||GPT-4.1-mini (Base)|0.400|0.374|0.350|0.246|0.506|0.468|0.363|\n||+EGO-Prompt|0.430|0.420|0.370|0.292|0.553|0.522|**0.411(↑13.2%)**|\n||Gemini-2.0-Flash (Base)|0.300|0.268|0.330|0.264|0.259|0.351|0.294|\n||+EGO-Prompt|0.430|0.412|0.400|0.327|0.506|0.470|**0.403(↑37.0%)**|\n\n- **EGO-Prompt is effective for open source models.** Excluding Qwen3-1.7B and Qwen3-8B, EGO-Prompt improves the performance of open-source models by 27.7%–48.7%, comparable to its improvements on commercial models.\n\n- **Commercial models achieve better and more stable performance.**  Across both the initial organized prompt and the optimized settings, commercial models consistently outperform open-source models, achieving higher F1 scores and greater robustness across tasks.\n\n- **Model size substantially influences the performance ceiling.** We can find the model with more parameters achieve better optmizaed performance, consistent with the scaling law. For example, the Qwen3 series improves from an F1 score of 0.265 for the 1.7B model to 0.390 for the 32B model. DeepSeek-V3, with 671B parameters, achieves the highest performance among the open-source models. These results suggest that our method can consistently enhance performance across models of varying sizes.\n\n- **EGO-Prompt enhances reasoning in smaller models.** Models such as Qwen3-1.7B and Qwen3-8B fail to generate structured outputs under the initial organized prompt. However, with EGO-Prompt, they are able to produce more structured predictions and follow certain reasoning paths to make predictions, although their overall performance remains limited.\n\n**In conclusion, commercial models are a better choice for our method.** Motivated by the need to better integrate expert knowledge into the LLM reasoning process, we design our framework to minimize deployment complexity and maximize usability for non-technical users. Since most end-users may lack the infrastructure to run models locally, our key focus is on improving the reasoning performance of black-box LLMs for domain-specific tasks. As commercial models typically exhibit better instruction-following ability, we recommend using lightweight commercial LLMs as the forward engine, although our method is compatible with both open-source and closed-source models.\n\nWe have revised our manuscript to incorporate the above results and discussion into the appendix.\n\n>  **Q2: Does EGO-Prompt have cross-domain generalization ability?**\n\nThanks! And the answer is yes in real-world applications. We tested the EGO Prompt method across three different real-world tasks (transportation, human choice and public health), achieving clear performance and cost advantages. More specifically, our method demonstrates generalization ability in the following two aspects:\n\n- **Domain-level Generalization.** We evaluate generalization by directly applying the SCG and system prompt optimized on one dataset to another, testing the model’s ability to transfer across domains. In the TrafficSafe dataset, we predict crash severity using crash report data from the HSIS [4]. Our original model was optimized on data from Illinois, which we now adapt to Washington State. These two datasets differ in several respects. For instance, Washington includes features such as daily road traffic volume, which are not available in the Illinois dataset. Additionally, the distribution of crash severity varies across states ,as Washington has fewer fatal and serious injury cases compared to Illinois. \n\n  The table below summarizes the performance on the Washington dataset using the SCG and system prompt optimized on the Illinois dataset. We uniformly sampled 100 instances from the Washington dataset across all crash severity classes. GPT-4o-mini and Gemini 2.5 Flash were used as the forward engine. We tested three configurations: replacing the initial SCG, the system prompt, or both with those optimized on the Illinois dataset, and recorded the corresponding performance on the test set. The results indicate that transferring either the SCG or the system prompt individually leads to performance gains, while combining both yields the most substantial improvement.\n\n| Setting                                  | GPT-4o-mini Acc | GPT-4o-mini F1 | Gemini 2.5 Flash Acc | Gemini 2.5 Flash F1 |\n| -- | ---- | --- | -- | -- |\n| Organized Prompt              | 0.300           | 0.222          | 0.350                | 0.312               |\n| IL SCG Only                                 | 0.340           | 0.251          | 0.390                | 0.353               |\n| IL System Prompt Only                       | 0.330           | 0.272          | 0.410                | 0.365               |\n| IL SCG + System Prompt | **0.340**       | **0.285**      | **0.440**            | **0.401**           |\n\n- **Task-level Generalization.** Our method is easily adaptable to new tasks, requiring only the redesign of the SCG. As shown in Table 3 of our manuscript, EGO-Prompt consistently improves reasoning performance across diverse tasks.\n\n>  **Q3: What is the cost for manual design of SGGs? Can SGG also be constructed automatically by the LLMs?**\n\nThank you for the comment. The manual design of SCGs requires minimal human effort, typically less than 10 minutes for a domain expert familiar with the task and data. Since EGO-Prompt is capable of automatically refining and correcting SCGs during the optimization process, the initial SCG does not need to be perfectly accurate (see our response to W1). This substantially reduces the cost and burden of manual design.\n\n SCGs can also be constructed automatically by LLMs. Please refer to our response to W1 for details on SCG generation starting from a null initialization.\n\n \n## References\n[1] Cobbe, K., et al. Training verifiers to solve math word problems. arXiv, 2021.\n\n[2] Suzgun, M. et al. Challenging big-bench tasks and whether chain-of-thought can solve them. Findings of ACL. 2022.\n\n[3] Yuksekgonul, M., et al. Optimizing generative ai by backpropagating language model feedback. Nature. 2025.\n\n[4] U.S. Department of Transportation, Federal Highway Administration. Highway Safety Information System."}]

Author rebuttal:

Meta-review: This paper introduces EGO-Prompt, a novel framework for improving the reasoning capabilities of Large Language Models (LLMs) on domain-specific tasks. The core contribution is a method that iteratively co-optimizes both a human-initialized Semantic Causal Graph (SCG) and the system prompts used to guide the LLM. 

The core idea of jointly optimizing a knowledge graph and system prompts in an evolutionary manner is novel, the paper is easy to follow, and the results are strong. There were initial concerns raised by the reviewers, but authors managed to address most concerns in the rebuttal phase. Therefore, we have decided to accept this paper.
Decision: Accept (poster)