AI-for-Science Low-code Platform with Bayesian Adversarial Multi-Agent Framework
Authors: Zihang Zeng
Jiaquan Zhang
Pengze Li
Yuan Qi
Xi Chen
Venue: ICLR 2026 Poster (2026)
Keywords: Multi-agent framework
Bayesian inference
LLM
AI-for-Science
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Abstract: Large Language Models (LLMs) demonstrate potentials for automating scientific code generation but face challenges in reliability, error propagation in multi-agent workflows, and evaluation in domains with ill-defined success metrics. We present a Bayesian adversarial multi-agent framework specifically designed for AI for Science (AI4S) tasks in the form of a Low-code Platform (LCP). Three LLM-based agents are coordinated under the Bayesian framework: a Task Manager that structures user inputs into actionable plans and adaptive test cases, a Code Generator that produces candidate solutions, and an Evaluator providing comprehensive feedback. The framework employs an adversarial loop where the Task Manager iteratively refines test cases to challenge the Code Generator, while prompt distributions are dynamically updated using Bayesian principles by integrating code quality metrics: functional correctness, structural alignment, and static analysis. This co-optimization of tests and code reduces dependence on LLM reliability and addresses evaluation uncertainty inherent to scientific tasks. LCP also streamlines human-AI collaboration by translating non-expert prompts into domain-specific requirements, bypassing the need for manual prompt engineering by practitioners without coding backgrounds. Benchmark evaluations demonstrate LCP’s effectiveness in generating robust code while minimizing error propagation. The proposed platform is also tested on an Earth Science cross-disciplinary task and demonstrates strong reliability, outperforming competing models.

Published as a conference paper at ICLR 2026
AI-FOR-SCIENCE LOW-CODE PLATFORM WITH BAYESIAN ADVERSARIAL MULTI-AGENT FRAME- WORK
Zihang Zengl.2#, Jiaquan Zhang'.2# Pengze Li', Yuan Qit.3, Xi Chen
Artificial Intelligence Innovation and Incubation Institute, Fudan University
2 Shanghai Innovation Institute
3 Shanghai Academy of AI for Science
&_chene Fudan. edu.cn
ABSTRACT
Multi-agent systems leveraging Large Language Models (LLMs)) show immense potential for solving complex scientific problems. However, their reliability is undermined by the probabilistic nature of LLMs, which can produce hallucinations in both generated code and its corresponding test cases. In a multi-agent archi- tecture, these errors can propagate and compound, leading to flawed final outputs. To overcome these core limitations, we introduce a novel Bayesian Adversarial Multi-agent Framework for Al for Science (AI4S). Delivered as a Low-code Plat- form (LCP), our framework enhances the coding capability for scientific tasks across a wide range of base models, from 1.7B open-source L.LMs to up-to-date commercial ones. Our framework employs three agents in recursive loop that ad- versarially co-optimizes the generated solutions, the test cases used for evaluation, and the prompts driving generation. This process is governed by non-LLM-basse Bayesian updating rule, which systematically reduces evaluation uncertainty and mitigates the system's 's dependence on any single LLM's reliability. Furthermore, the LCP empowers domain experts by translating high-level natural language prompts into executable, comain-specifff requirements, eliminating the need for intricate prompt engineering. Extensive experiments confirm that our framework generates robust solutions while effectively minimizing error propagation. On a complex, cross-disciplinar Earth Science benchmark, our platform demonstrates superior reliability and outperforms state-of-the-an models, where a 32B open- source model can beat the performance of 235B model in the ScienceCode benchmark with our framework.
INTRODUCTION
Large Language Models (LLMs) transforming AI for Science (AI4S) research paradigm by automating complex scientific code generation for simulations, data analysis, and related science tasks (Chowdhery al. [003]] Nijkamp al. 2022). While models such Codex, AlphaCode,
and Cama effectively lower technical barriers for researchers (Chen et al., 2021 Li et al.)
2022. Roziere al. 2023). several challenges hinder their reliable application in A14S research.
These include: (I) potentially unclear prompt descriptions from domain scientists without computer science backgrounds, (2) complex execution pipelines for scientific tasks, and (3) the need maintain adherence to physical laws and Nomain-specinf constraints. Standard prompting and self-refinement techniques (Wei et al. 2022, Chen al.) 2023) Olausson et al., (2033) are often inadequate for handling the subtle error patterns in complex scientific workflows.
Crucially, we lack strong empirical evidence to fully trust L.LMs' capabilities in deep understanding and complex reasoning, particularly for professional scientific research tasks (Ridnik et al. 2024). Their decision decision-making processes remain opaque, and while their outputs often appear plausible, they may contain subtle inaccuracies or conceptual misunderstandings These limitations fundamentally
# Equal contribution. 'Corresponding author.

---

Published as a conference paper at ICLR 2026
Single LLM
Multi-agent Role Playing
Bayesian Adversarial Multi-agent
Manager
Developer
Assessor
Domain Knowledge
T
Task
Code
Test Cases
BO
LLM Agents
Figure I:
Comparison between three code generation paradigms: Single LLM generator, multi-agent role playing and the proposed Bayesian adversarial multi-agent framework.
constrain the performance ceiling of LLM-based coding platforms, as their capabilities are inherently bounded by the underlying LLM's intelligence level. Such inherent uncertainty demands the develop- ment of frameworks that operate without requiring absolute confidence in the LLM's intelligence level.
As illustrated in Figure T recent advances in LLM-based multi-agent systems attempt address these limitations through distributed reasoning and specialized agent roles, where different LLM agents focus on specific sub-tasks while coordinating through structured communication protocols and/or a master agent (the green ellipse). However, while such multi-agent architectures (Hong et al., 2023; Wu al.] 2023) can effectively distribute computational complexity and domain expertise across components of the underlying domain task, they introduce new challenges in propagation and validation. The system's overall reliability becomes constrained by its weakest agent, as flawed intermediate outputs from one of the agents
be uncritically accepted by downstream agents (Huang et al. 2024), potentially amplifying rather than mitigating the above limitations individual LLMs. Furthermore, evaluating the code of domain-specific tasks is often difficult. Standard unit tests may miss critical scientific constraints, theoretical foundations, or domain-specine limitations. This evaluation gap stems from three key issues: (1) scientific correctness often requires deeper domain knowledge than standard unit tests can verify; (2) comprehensive evaluation metrics may be prohibitively expensive or fundamentally intractable define; and (3) LLM-generater tests may inherit the same reliability issue as the code they aim to validate (Zhou et al., 2023). Given these, one must pay equal attention to both L.LM--geertted code and the test cases used to assess it.
This fundamental insight motivates our core design philosophy: an adversarial co-evolution frame- work where test case generation and code improvement mutually refine each other through competitive optimization, replacing traditional static verification approaches. The proposed framework structures agent interactions and evolves prompt distributions using Bayes' Theorem, reducing dependence on the base LLM's inherent capabilities. The framework comprises three specialized agents: Task Manager (TM) serving as Challenger, a Solution Generator (SG) as Solver, and an Evaluator for comprehensive assessment. Unlike conventional multi-agent code generation systems that depend entirely on L--ed evaluation and decision-making QQian et al. 2023 Hong et al. (202)), our approach introduces an adversarial dynamic between TM and SG. The TM actively constructs and refines test cases to probe the SG's current limitations, while the SG iteratively improves its code generation based on Evaluator feedback meet these evolving challenges. As shown in Figure orange ellipse, by continuously probing and validating solutions against dynamically refined test cases, our framework not only overcomes these evaluation barriers but also progressively converges on solutions that satisfy both explicit requirements from domain experts and implicit domain constraints from the specific domain or application scenarios.
The proposed framework also enhances Human-Al collaboration in A14S community Yamada et al.) 2025 Zheng et al., 2025 Babaei Giglou et al., 2024). Outside the Machine Learning community, we cannot expect an average scientist to be aware of, let alone skilled in, the extensive list of prompt engineering techniques. A typical domain researcher's 's prompt might be vague, assume implicit do- main knowledge, one, may not fully grasp. These domain gaps may lead to misinterprctations, suboptimal outputs, complete system failures. To bridge this gap, our framework incorporates specialized scheme
specialized terminology and abbreviations that an LLM, especially smaller
2

---

Published as a conference paper at ICLR 2026
within TM agent that actively structures raw user requests, resolves ambiguities through interactive clarification, and transforms potentially vague prompts into precise task plans and scientifically valid initial test cases. It maintains accessibility for non-technical domain experts while fully leveraging their domain expertise without requiring any computer science or professional prompt engineering skills. The main contributions of this work are threefold:
A Novel AI4S Low-Code Platform with Bayesian Adversarial Framework: We introduce a multi-agen ageet framework that employs a Bayesian recursive co-updating strategy to iteratively refine generated code and test cases using non 1ii--LLM-baase adversarial score. This method significantly enhances scientific coding performance across a spectrum of base models (from open-source to commercial) and allows smaller LLMs to achieve results competitive with larger counterparts.
Bayesian Optimization for code performance estimation: We proposed Bayesian Optimization method estimate the performance of given code based on its structure similarity with the tested codes, which enables the framework to handle and evaluate complicated code.
Domain Knowledge Refinement for scientific tasks:
The LCP facilitates scientific ex- ploration for non-coding professionals by enabling the generated code to better reflect domain knowledge and constraints through iteratively refining, adding and updating domain knowledge in the specially structured prompt. Our Earth Science case study exemplifies this, where the generated machine learning model not only produced superior predictions but also demonstrated minimal deviation from established ocean dynamics, ensuring scientific consistency.
In the rest of this paper, we introduce the main methodology and models in Section 2 followed by experimental setup and numerical results in Section B We conclude the work and discuss its future work and limitations in Section 4
2
METHOD
2.1
OVERVIEW
We propose Bayesian adversarial multi-agent framework designed for AI4S tasks, incorporating subjective prior knowledge and addressing complex task abilities. The framework comprises three core component agents: a Task Manager (TM), a Solution Generator (SG), and an Evaluator(Eval), Within this structure, code generation becomes dynamic interaction, primarily between the Task Manager (acting as a Challenger) and the SG agent (acting as a Solver), with the Evaluator providing the performance metrics that guide learning and adaptation. The game concludes when the SG agent produces code that successfully passes all defined validation tests.
The process initiates prior knowledge P with task description (provided by scientist user) and relevant subject materials (e.g.. prior domain knowledge, including reference code samples). P is the initial input to the TM agent, which develops structured plan of the scientific task, decomposing the main task into an ordered set of Sub-tasks. This plan is iteratively refined based on users' feedback F and refinements until user's approval and denoted 2, Subsequently, the TMM agent generates an initial set of test cases (Test Caseg) corresponding to these sub-tasks and other criteria derived from prior knowledge. These initial test cases, along with user-provided reference code base are serving as initial sample codes (Sample Codeg). Sample Code, followed by the user approved plan (Plans) to form the initial prompt:
Plans, indicated by Loop 1 of Figure
Prompto Plans @ Test Casen @ Sample Code
where @ is the direct concatenation operator. Both the test cases and the sample codes can be independently updated in subsequent iterations. This update mechanism, guided by Bayesian principles, leverages the performance of candidate codes generated previously and the effectiveness of past test cases. The objective is to iteratively refine the prompt to guide the SG agent towards producing solution that meets all test criteria and user requirements. The core Bayesian update rule for selecting specific test case i and sample code j for the prompt at iteration t + is: pl Promppt 1S58) (S$) x p(Sg(rrompt ) (Prompt..) ). This iterative refinement continues until the SG
3

---

Published as a conference paper at ICLR 2026
Bayesian Adversarial
S1
Multi-agent Framework
Loop 2
Feedback
IRL Training
Initiate
Assessed
Reference Codes
P Planning
S3
Initiate
Domian knowledge
Loop
T
Assessed
1
Loop 3
S2
Final Code
Bavesian recursive updates
Output
Task Manager
Code Generator
Prompts
Evaluator
Code
Test Cases
Demain Expert
Plan
Figure 2: Overview of the Bayesian adversarial multi-agent framework. The three red arrows indicate fusion of the user-approved plan, test cases, and codes into prompts, the distribution of which is recursively updated under the Bayesian framework. S1, S2 and S3 are the scores computed in equation [ equation 33 and equation [ Loop 1-3 indicate three iterative updating processes for plan, test cases, and codes, respectively. The dashed arrows indicate latent relationships (e.g.. Sg likelihood score) steps conducted before after the main algorithm execution.
agent achieves satisfactory success on the test cases. The pseudo-code the proposed method is described in Algorithm T
Algorithm Bayesian Adversarial Multi-Agent Framework
1: Input: Task description and domain knowledge P, reference code base, max iterations Tmax
2: // Planning till User' approval
3: Generate Plans = TM(P)
4: while NOT User Approval do
5: Plans + TM(P: F)) iteratively updates the plan given user feedback.
6: // Make initial prompts
7: Generate Test Caség == ((Sanity Checks), (Sub-tasks}) + TM(Plans)
8: Form Sample Code, [Test Caseg, Reference Code)
9: Generate Promply according to equation
// Code gene ion and eva 1 vation
11: t+0
Initialize case scores Vi (L...., (1))" +
13: while Not Code pass all checks A t < Tmaz do:
++++1
15:
Generate code batch {C,),, + SG(Pooppt, f=
// Score Calculation & Updates
16:
Compute Code Scores S2(j)" using S according to Eq. equation
17:
Update Test Case Scores (i)) using S2) according to Eq. equation
18:
Compute Prompt Score Sj using S according Eq. equation 4
19:
Chest = arg max, S2(j)"
// Bayesian updates for next iteration
20:
Test Case; + (TeTest Sa)
> Refine test cases based on St
21:
Prompt, p(Prompt,(S))
P Update prompt based S
return Chest
4

---

Published as a conference paper at ICLR 2026
2.2 PLANNING AND INITIAL CODE GENERATION
As briefly described in the above overview, the TM agent engages in a comprchensive planning phase. In particular, this involves: 1. Decomposing the primary task into sub-tasks. 2. Posing Sanity Checks for data structure and range. 3. Reasoning about the logical workflow and dependencies between sub-tasks. 4. Formulating strategic advice for the generation of effective test cases. This detailed plan is presented to the user in natural language for review and potential refinement. This interactive feedback loop continues until the user approves the plan. Once the plan is finalized, the TM agent generates an imitial set of test cases. These test cases are designed to cover the specified sub-tasks and incorporate domain-specific prior knowledge, such as sanity checks (e.g., for expected data ranges) and out-of-range detection.
Following the planning phase, the system constructs the initial prompt Prompt, as defined equa- tion [ The code generation agent then uses this prompt to produce N candidate solutions (codes). For each candidate, the system automatically generates comprehensive documentation containing: algorithm explanations, execution instructions, and detailed specifications for all functions, variables, and parameters.
2.3 PRIORT ESTIMATION WITH BAYESIAN OPTIMIZATION
We noticed that executing all the generated codes for testing can be computationally expensive. To address this issue, as well as to leverage both the accuracy of evaluation and the range of exploration in the solution space, Bayesian Optimization method is employed to estimate the performance score relates to the structural difference from all the tested codes (as in Loop 3 of Figure 2
As an initiation, all the generated code in the first iteration {Code,]isist get tested against the initial test cases, where all test cases start from the same initial difficulty score (S) (i) 1). We store the test results as a score vector S2 (will be explained in details equation (3). We then embed each C'od; to vector through structural embedding that captures features from its Abstract Syntax Tree (AST) and code embedding vectors. We then use Bayesian optimization process to predict the code's performance based on its structural similarity with the tested code, which is detailed explained in the appendix. This Bayesian Optimization approach allows the system to intelligently explore the vast solution space, prioritizing the most promising candidates for expensive testing and efficiently converging towards high-quality solution. It supports the evolution of the distribution of prompt, thus the co-evolution of codes and test cases.
2.4 EVALUATION AND FEEDBACK
The Evaluator agent is responsible for assessing the candidate codes, the effectiveness of the test cases, and the overall quality of the prompts used in each iteration.
Test Case Score (S1): This score quantifies the "True Hardness" of the jth test case Test Casej- representing its capacity to be challenging yet ultimately solvable. An effective test case should successfully discriminate between code solutions of varying quality (as in loop of Figure 2)
Σ S2(1')
Σ
S2(it)
(((1)(1 (1))) -a.
j' pass [[Code
st tail
[[Coee,t H
where S₁ (i)* is set by default for t 0, and is a hyperparameter to control the momentum of updating, which in experiments we set a = 0.8.
Code Score (S2): Each generated code C; factors:
S N receives composite based several
S2(i)* - Σ, passes T,) (11)
(3)
S1)
5

---

Published as a conference paper at ICLR 2026
Prompt Score (Sy): The overall score for a prompt used in iteration is function of the perfor- mance of the codes and test cases generated by this prompt:
M
1 M
1 N Σ S2(1)
S3¹
Si(1)
(4)
multiple prompt configurations are tested within a single logical iteration, the iteration's representa- tive prompt score might be the highest achieved. For the Bayesian update, we are interested in the score of a specific prompt configuration {Promptiy) J6 is then denoted (Sa)igho.
2.5 ITERATIVE REFINEMENT: ADVERSARIAL. DYNAMICS AND BAYESTAN PROMPT UPDATES
The core of the framework's 's learning capability lies in its iterative refinement loop, characterized by an adversarial dynamic between the TM agent (Challenger) and the SG agent (Solver), and guided by Bayesian updates for prompt composition.
The TM agent's role evolves to that of a Challenger. Based on the 'True hardness', which is measured by S1, the TM adapts its weights for future evaluation and selects test cases for subsequent prompts. It aims to create test suites that are optimally challenging for the SG's current learned capabilities-diffiie yet generally solvable to provide positive learning signal. The SG agent, as the Solver, implicitly adapts by producing code in response to these evolving challenges. Its success or failure provides the feedback signal that shapes the TM's subsequent challenging strategy.
Adversarial interaction:
enough to drive further learning and expose weaknesses,
Bayesian prompt updates: The selection of which specific test cases (indexed by and sample codes (indexed by j) to include in the prompt for the next iteration (Prompt)41) is governed by a Bayesian update rule(as indicated by the combination of Loop 2 and 3 in [
(Prompttt*1S$) x p( Prompt (p(Prompt'))
(5)
Here:
p (Prompt),) ) is the prior probability of selecting the pair (Test Case, Sample Code,) for the prompt. This prior can be uniform initially and can adapt over time based on the historical effectiveness of these components.
p(S)) Prompti,) is the likelihood of observing the score S given that the prompt was formed using Test Case; and Sample Code This term captures how well this specific combination performed. A potential formulation to ensure -oggatity and reflect that better-than-expected performance is more likely could be:
p( (S)) Promptt)) ) X exp (E(SS"-'1(i,)
where [i,) is the expected score for the generated code with Test, Code, in the prompt based on past performance or baseline. This implies that a prompt configuration performing significantly better than its historical average for that pair (Tes! Case, Sample Code,) will have higher likelihood.
The underlying intuition is to identify and prioritize "teacher-subject" pairs-specific combinations of sample code and test cases that consistently yield high-scoring prompts. This approach effectively learns which forms of guidance produce optimal results for different types of coding challenge.
Sample code pool management: The pool of available sample codes (Sample Code) is not static, but recursively updated as illustrated Loop 3 of Figure 2 Initially, it contains user-provided reference codes. As the SG agent generates new codes c those that achieve high S2 can be added to the Sample Code pool. The selection of Sample Code, for prompt can then be influenced not only by its initial status (as reference) but also by an evolving measure of its "guidance quality." learned from its impact on past prompt scores when it was included.
Final results: Within each round, if there exists a code that can pass all the test cases, the System will output it as the final result to the user. Otherwise, the System will keep using the Bayesian Adversarial method recursively till generation of a satisfying code or reaching the maximum round of iterations, which is chosen by the user in the beginning and by default set to by experience(See Section

---

Published as a conference paper at ICLR 2026
3 EXPERIMENTS
3.1 EXPERIMENTAL SETUP
Benchmarks To ensure a thorough evaluation, we utilize diverse set of benchmarks. For general code generation, we use Human Evaal, 1. HumanEva 1 EET, MBPP, MBPP- et al. 2021
Dong ct al., 2025 Hendrycks et al. 202T), and the more challenging Hendryyss et al. 2021 benchmark. For AI for Science tasks, we use the domain-specific SciCode (ecoodd(Tiaa (Tian al. 2024 and ScienceAgent BeleencekgeetBBenhhCCCCCe BenchhChehee et al. 2024) benchmarks.
Base Models Our framework is designed to be mrodel-agnostic To demonstrate this, we integrate several backbone large language models (LLMs), including the Owen series (ranging from 1.7B
to ) (Yan et al., 2025), which has versatile sizes, strong reasoning, it can demonstrate if our framework is also effective on the latest models. Beside, we also choose Deepseek-val 3
al. 2025), Claude seeeeesoneettteeeeee (Anthropic) (0225) GPT-3. 1-w.333--uuuooooo turwwooBBrow et al. 2020), and GPT GPT-4o(Hurs et al. 2024). This allows us to as- sess the performance gains attributable our framework across a spectrum model capabilities.
al. 2024), Deepseek- (Go
Compared Methods We compare our framework against several state-of-the-art baselines. These include foundational strategies like Few-Shot prompting and Chain-of-Thought (CoT), as well as other prominent agent-based systems. From the table, the competing agen- tic frameworks and prompting strategies include ReAct, Reflexion Sel DellfDebuggiin
Se1 f-Col 1 aab rat i on, MetaGPT, MapCoder, AgentCoder, and CodeCoR (Brown et al.
2020 Huang et al. 2023b Pan et al. 2025 Yao et al. 2023b; [Shinn et al. 2023 Yao et al. 2023a
Hao et al. 2223 Z Zhang ct al. [203 Jiang et al. 2024 Chen ct al. 2023 Dong et al. 2024
Wang
et al.
2023
Li et
al. 2025
Huang et al. 2023a Islam et al. 2024).
Evaluation Metries Following standard practice, we use the pass @k metric evaluate code genera- tion performance, where a solution is considered correct if it passes a set of umit tests. We primarily report pass @1 scores(Chen et al. 2021 Austin et al. 2021 Dong et al. 2025).
Parameter Setting For all experiments, we consistently applied an identical parameter set unless otherwise noted. The number of initial test cases set to 15, and the number of distinct code snippets generated in each round was 20. We maintained a minimum pool of 20 test cases; if filtering processes reduced the number of test cases below this threshold (e.g.. due to low scores), additional test cases were generated to meet this minimum. For iterative refinement, the number of codes chosen by acquisition function for further evaluation was set 5.
3.2 EFFECTIVENESS IN AI FOR SCIENCE TASKS
We established our framework's s general proficiency, which can be found in detail in the appendix, and can assert its up-to-SOTA level performance in general coding tasks. We can now investigate the framework's 's ability in scientific tasks by evaluating our framework's performance in the special- ized and demanding domain of scientific code generation. We use two scientific code generation benchmarks to demonstrate its capabilities.
First, we assess our framework on the SciCode benchmark across a wide spectrum of base models, from the 1.7B parameter Qwen3 to powerful proprietary models like Claude-sonnet-4, The results, presented in Table D show that our framework provides substantial and consistent performance uplift in all configurations. The gains are particularly striking for open-source models, with relative improvements of up to 87.1 % (for Qwen3-8b). Our framework enables smaller models to match the performance of significantly larger ones. For instance, in the "Without Knowledge" case, Qwen3- 1bb with our framework achieves a 30.6 Resolve Rate on Subproblems, equaling the baseline of the model over 16 times its size.
Owen3-235B-Al 1-335--222--aee A22b-Insttrutt25007
Second, to evaluate our framework on more complex, agentic workflows, we test it on the ScienceAgent.ence Bench, which involves more complex, multi-step scientific workflows, and we using GPT-4o as the base model. As shown in Table 2 our LCP framework achieves new ot the-art (SOTA) performance, particularly in the Valid Execution Rate (VER), where it scores 90.2 % (without knowledge) and 87.3% (with knowledge). far surpassing all other methods. This exceptional execution success rate is critical for scientific applications, as it directly validates our framework's
7

---

Published as a conference paper at ICLR 2026
strength in producing robust, executable scientific code across diverse and complex application domains. This result, combined with leading scores in Success Rate (SR) and Code- Based Score (CBS), confirms system's effectiveness in orchestrating the complex reasoning and execution steps essential for impactful AI4S applications.
Table I: Model Performance Comparison SciCode with various backbone models
Walhout Kaowlsige
With Knnwlelge
Withor Knnwkolgee
With Knowieige
Model
Method
(S)
Main
Seb-(%) (%)
Main
Metood
Subr
Main (8)
Sub (9k
(9))
Qwend-Sh
24.7887116)
27.4638.45)
Baseline
37.2/54.31))
40.6(20.49)
Qward-14h
Baseline
30.072.998
(0,499
Qweet
Bascline
33.0/79.366)
18.4
36.1(1.89)
Baseline
41,0138.550
43.1(34.0%)
37.8
10.8
Qwen3 next 80b
Albiiniiuct
Baseline
37.5074.485)
21.5
5.1
18.5(18.1%)
32.6
Chucl-semee-44
Ours
Baseline
42. 7196
51,3
7.2
43.8612886)
28.8
138
10.3
Qwend 2358 A226
Daseline
30,6
4.6
31.2
10.3
Instrict
Ours
38.927.199)
9.2
41.0010.25)
10.3
Table 2: Results on ScienceAgemtBench using GPT-4o as base model with/without prior knowledge, compare with two different agent frameworks and baseline.
Method
SR(w/o)
CBS(w/o)
VER(w/o)
SR(w/)
CBS(w/)
VER(w/)
Direct
11.8
82.6
52.9
10.8
83.8
41,2
OpenHands CodeAct
19.6
83.1
78.4
27.5
86.3
73.5
Self-Debug
22.6
84.4
83.3
23.5
85.6
71.6
LCP(Ours)
26.5
85.1
90.2
27.5
86.4
87.3
3.3 ANALYSIS OF BAYESIAN RECURSIVE CO-UPDATING
In this section, we test our framework's 's core Bayesian iterative co-updating mechanism, validating that the Bayesian recursive co-updating strategy is effective iteratively refining solutions. As illustrated across both general and scientific benchmarks in Figure 3 performance consistently and monotonically improves with an increasing number of iterations. On the general benchmarks (Figure 3a), the Pass@1 I scores on HumanEval and MBPP show substantial gains in the first three iterations, with performance beginning to converge around the fourth fifth iteration. This suggests an optimal balance between performance and computational cost. This same powerful trend is mirrored on the specialized SeiCocs benchmark (Figure where the performance score climbs steadily from 27.1 to 37.2 after five iterations, demonstrating the broad applicability and success of our iterative refinement process.
We further analyze the components of this process by conducting an ablation study on the role of Ad- versarial Test Cases (ATC) within our LCP framework, as shown in Figure While the performance with and without ATC is comparable in the initial iterations, a clear divergence emerges from the third iteration onwards. The LCP framework augmented with ATC (dash-dot lines) consistently achieves higher Pass 1 accuracy across all metrics, underscoring the critical role of ATC. By dynamically challenging the generated code with difficult edge cases, the ATC mechanism compels the system produce robust and reliable solutions, validating it as key driver of the performance gains observed in our co-updating loop.
3.4 ROBUSTNESS FOR
USERS
Finally, we address robustness and accessibility to non-Al-professionn science researcher by evalu- ating our framework's accessibilityy and effectiveness for users who may domain experts but are not specialists in prompt engineering. To simulate this scenario. we compare the performance of both the baseline models and our framework under two conditions: one with a basic, un-optimized prompt ('Without Knowledge') and one with expert-crafted prompt containing detailed domain knowledge (' ('With Knowledge'). The goal to measure how sensitive each approach is to the quality of the initial prompt.
8

---

Published as a conference paper at ICLR 2026
Performance subproblems of SciCode
40
36.5
36..
Pass@)) (%)
80
32.3
33.7
30.6
Performance
70
20
60
Wit
ATC
Suacline
WILATE ATC- NESST
ATC- HE-ET
WAT AIC NOMP
10
50
Wi ATC NIFP-ETT
No ATC NEFPET
Iterations
without knowledge
with knowledge
(a) Pass@1 of LCP with different iterations (GPT- 3.5-turbo). Dash-dot and solid lines are LCP with and without ATC, respectively.
(b) Performance on SciCode Benchmark (GPT-4o) with different iteration numbers.
Figure 3: Illustration of LCP performance over: (a) different iteration number with and without ATC component on general code benchmark; (b) difficulty iteration number on the SciCode benchmark
The results, presented in Figure A clearly demonstrate our framework's 's superior robustness. The baseline models exhibit large performance gap between the two conditions (represented by the shaded red area, "Area (Baseline)"), indicating a strong dependency on expert prompting. In contrast, our framework significantly narrows this performance gap across the entire spectrum of models (represented by the much smaller shaded blue area, 'Area (Ours)'). This shows that our multi-agent system internally elaborate and refine basic instructions, compensating for the lack of initial detail. Most strikingly, a non-professional user with our framework ("Ours- Without Knowledge') consistently and substantially outperforms expert user with the baseline model alone ('Baseline- With Knowledge').
Model Performance Comparison
40
Without Knowledge
With Knowledge
30
Baseline Without Knowledge
c
20
Baseline with Knowledge
10
(Ours)
Area (Baseline)
Owen3-1.7b
Owen3-80
Owen3-14h 32bb
Qwen3-322 Owen3-next-80p.
GPT-4o
Owen3-2358
Deepseek V3
Deepseek-R sonnet--
Claude-sonnet.
Figure 4: Model performance with basic vs. expert-crafted prompts. Our framework (blue/green lines) is significantly more robust prompt quality than the baseline (red lines), showing much smaller performance gap (shaded area) and achieving superior results even without expert knowledge.
4 CONCLUSIONS AND DISCUSSIONS
We propose Bayesian adversarial multi-agent framework for Al-for-Science (AI4S) code genera- tion that achieves sfate-of-the-s performance by iteratively refining prompt components through Bayesian updates. This approach mitigates cumulative error by treating tests and code with equiv- alent confidence, while an adversarial process guides Task Manager (TM) agent to challenge Solution Generator (SG) agent with progressively evolving tests. The framework's 's interactive planning scheme enables non-experts to translate vague prompts into validated workflows, effectively bridging the gap between Al-generated code and domain- needs. As demonstrated in Earth Science applications, our method helps democratize LLM tools for researchers without a technical background.
However, the framework has several limitations. First, its performance is dependent on the quality of the initial reference code, and it struggles enforce implicit physical laws, which may require

---

Published as a conference paper at ICLR 2026
future integration with symbolic verifiers. Second, the iterative adversarial -anesarral--nd---e anneeyesalll refinement process consumes more tokens than one-shot/zero-shee prompting because it repeatedly generates and evaluates test cases, code candidates, and updated prompts. While this increased token budget can be a practical constraint for long pipelines, in reliabilit--eritical scientific applications, we prioritize executable correctness and robustness over marginal token savings. Furthermore, evaluating the generated machine learning or deep learning models can be highly resource-intensive, and performance variability due training dynamics and data poses additional challenge the update mechanism.
Future work will focus on extending the Bayesian updates to handle equations and diagrams, and optimizing the iteration protocols for large-scale scientific simulations.
inputs, such as
ACKNOWLEDGMENTS
The computations in this research were performed using the CFFF platform of Fudan University.
REFERENCES
Anthropic. Claude sonnet 2025. htt.htttps///w..ggttooooo..o/ooo
Jacob Austin, Augustus Odena, Maxwell Nye. Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoe Le, al. Program synthesis with large language models. arXiv preprint arXiv:2108.077 2021.
Hamed Babaei Giglou, Jennifer D' D'uuza and Sören Auer. Lims4synthesis: Leveraging large language models for scientific synthesis. In Proceedings of the 24th ACM/IEEE Joint Conference on Digital Libraries, pp. 1-12, 2024.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et Language models are few-shot learners. Advances in Neural Information Processing Systems(NeurlPP5) 33:1877-911, 2020.
Mark Chen. Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph. Greg Brockman, et al. Evaluating large language models trained code. arXiv preprint arXiv:2107.0333 00334734..... 2021.
Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching large language models to self-debug. arXiv preprint arXiv:2304.05/22 2023.
Ziru Chen, Shijie Chen, Yuting Ning, Qianheng Zhang, Boshi Wang, Botao Yu, Yifei Li, Zeyi Liao, Chen Wei, Zitong Lu, et al. Scienceagentbench: Toward rigorous assessment of language agents for data-driven scientific discovery. arXiv preprint arXiv:2410.05000, 2024.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1-113, 2023.
Yihong Dong, Xue Jiang, Zhi Jin, and Ge Li. Self-collaboration code generation via chatgpt. ACM Transactions Software Engineering and Methodology, 33(7): 2024.
Yihong Dong, Jiazheng Ding, Xue Jiang, Ge Li, Zhuo Li, and Zhi Jin. Codescore: Evaluating code generation by learning code execution. ACM Transactions on Software Engineering and Methodology, 34(3):1-22 2025.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501. /20.0.....99...... 2025.
10

---

Published as a conference paper at ICLR 2026
Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305. 14992, 2023.
Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, et al. Measuring coding challenge competence with apps. arXiv preprint 1rXiv:2105009938 2021.
Sirui Hong, Xiawu Zheng. Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.008:
3(4):6, 2023.
Dong Huang, Qingwen Bu, and Heming Cui. Codecot and beyond: Learning to program and test like developer. arXiv preprint arXiv:2308.. 08784, 23, 2023a.
Dong Huang, Jie M Zhang, Michael Luck, Qingwen Bu, Yuhao Qing, and Heming Cui. Agent- coder: Multi-agent-ban based code generation with iterative testing and optimisation. arXiv preprint arXiv:2312.13010, 2023b.
Jen-tse Huang. Jiaxu Zhou, Tailin Jin, Xuhui Zhou, Zixi Chen, Wenxuan Wang, Youliang Yuan, Michael R Lyu, and Maarten Sap. On the resilience of llm-based multi-agent collaboration with faulty agents. arXiv preprint arXiv:2408..00 009889, 2024.
Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.212 2024.
Md Ashraful Islam, Mohammed Eunus Ali, and Md Rizwan Parvez. Mapcoder: Multi-agent code generation for competitive problem solving. arXiv preprint arXiv:2405. /1403, 2024.
Xue Jiang. Yihong Dong, Lecheng Wang, Zheng Fang, Qiwei Shang, Ge Li, Zhi Jin, and Wenpin Jiao. Self-planning code generation with large language models. ACM Transactions Software Engineering and Methodology, 33(7):1-30, 2024.
Jia Li, Ge Li, Yongmin Li, and Zhi Jin. Structured thain-of-thought prompting for code generation. ACM Transactions on Software Engineering and Methodology, 34(2):1-23 2025.
Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. Competition-leve code generation with alphacode. Science, 10866624) 1097, 2022.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprini 17X14:2412.19437, 2024.
Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. Codegen: An open large language model for code with multi-turn program synthesis. arXiv preprint arXiv:2203. 13474, 2022.
Theo X Olausson, Jeevana Priya Inala, Chenglong Wang, Jianfeng Gao, and Armando Solar-Lezzaa. Is self-repair a silver bullet for code generation? arXiv preprint arXiv:2306.00 09896, 2023.
Ruwei Pan, Hongyu Zhang, and Chao Liu. Codecor: An 1lm-based self-reflective multi agent framework for code generation. arXiv preprint arXiv:2501.078// 2025.
Chen Qian, Xin Cong, Cheng Yang, Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, and Maosong Sun. Communicative agents for software development. arXiv preprint arXiv:2307.0792. 6(3):1, 2023.
Tal Ridnik, Dedy Kredo, and Itamar Friedman. Code generation with alphacodium: From prompt engineering to flow engineering. arXiv preprint arXiv:2401.08500 2024.
Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi. Jingyu Liu, Romain Sauvestre, Tal Remez, et al. Code llama: Open foundation models for code. arXiv preprint ar 2308 12950, 2023.
11

---

Published as a conference paper at ICLR 2026
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems/ 36:8634-86522 2023.
Minyang Tian, Luyu Gao, Shizhuo Zhang, Xinan Chen, Cunwei Fan, Xuefei Guo, Roland Haas, Pan Ji, Kittithat Krongchon, Yao Li, et al. Scicode: A research coding benchmark curated by scientists. Advances in Neural Information Processing Systems (NeurIPS). 37:30624-3000 -06600, 2024.
Hanbin Wang, Zhenghao Liu, Shuo Wang, Ganqu Cui, Ning Ding, Zhiyuan Liu, and Ge Yu. Inter-
venor: Prompt the coding ability of large language models with the interactive chain of repairing. Consortium for Reliability and Reproducibility (CoRR), 2023.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems (NeurIPS), 35:24824-24837, 2022.
Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Autogen: Enabling next-gen Ilm applications via multi-agent
Shaokun Zhang. Jiale Liu, conversation. arXiv preprint arXiv:2330..08/55, 2023.
Yutaro Yamada, Robert Tjarko Lange, Cong Lu, Shengran Hu, Chris Lu, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist-v2: Workshop-level automated scientific discovery via agentic tree search. arXiv preprint arXiv:2504 000666 2025.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems (NeurIPS). 36:1 1809-1882222 2023a.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023b.
Kechi Zhang, Zhuo Li. Jia Li, Ge Li, and Zhi Jin. Self-edit: Fault-aware code editor for code generation. arXiv preprint arXiv:2305.0m 040877 2023.
Boyuan Zheng, Zerui Fang, Zhe Xu, Rui Wang, Yiwen Chen, Cunshi Wang, Mengwei Qu, Lei Lei, Zhen Feng, Yan Liu, et al. Agent4s: The transformation of research paradigms from the perspective of large language models. arXiv preprint arXiv:2506.23692, 2025.
Andy Zhou, Kai Yan, Michal Shlapentokh-Rottinn Rothmmn,, Haohan Wang, and Yu-Xiong Wang. Language agent search unifies reasoning acting and planning in language models. arXiv preprint arX1v:2310.04406, 2023.
12

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS (LLMS)
We confirm that LLMs were used for writing assistance and polishing of the manuseript, as well as editing Figure 2 based our handmade version. They were employed in the design of methods, implementation of experiments, or analysis of results.
B BAYESIAN OPTIMIZATION FOR CODE PERFORMANCE PREDICTION
For code embeddings, we use OpenAl's text sexxt-embedding 3-r----lrrge model in the surrogate modeling pipeline. This embedding model used map generated code candidates into semantic- structural vector space, after which the GP surrogate estimates candidate quality before expensive full execution.
In this embedding space, we can further obtain the pair-wise similarity between all the code embed- dings. This similarity is computed using a squared exponential kernel:
d(x;, X;)
k(x;, x3) exp (-
212
where d(x;, xz) ) is the distance between the two code embeddings and is length-scale parameter. These pairwise similarities xi)x form the kernel matrix K.
With the scores S2 and the kernel matrix K, we follow standard Bayesian Optimization practice and Gaussian Processes (GP) model. This model allows us to estimate the score of any new, untested code x. without running the full evaluation. This also gives the 'likelihood' to guide the Bayesian update. For each new code, the trained GP provides a predictive distribution for its score, which is characterized by
the mean function ((x.) is the expected score of the new code conditioned on the tested codes:
((x.) = k. (K +
The variance function o² (x(x.) represents our uncertainty about that predicted score:
o² xx(xx) k(x,, k""(K + I 'k
where k, is the vector of kernel similarities between the new code x, and all previously tested codes, and of is the noise term.
To decide which untested code to evaluate next, employ standard acquisition function that balances exploiting codes with high expected scores (exploitation) and exploring codes where the model is uncertain (exploration). We the Upper Confidence Bound (UCB) acquisition function:
UCB(x.) μ(x.) (x.))..
The parameter controls the trade-off between exploitation and exploration. The next code selected for full evaluation is the one that maximizes this UCB score:
Xnext = arg max UCB(x.). X.
C PERFORMANCE ON GENERAL CODE GENERATION
To test our framework of general code generation, we establish our framework's proficiency on foundational code generation tasks. As detailed in Table 3 our (ramework(LCPP) demonstrates significant performance uplift across the HumanEval, HumanEval-ET, MBPP, and MBPP-ET benchmarks. When using GPT-3.5-Ture on HumanEval and 91.1 % on MBPP, representing substantial relative improvements of 54.3% and 74.5 % over the zero-shot baseline. This superior performance holds when using the more powerful
backbone, LCP achieves pass @ scores of 88.4%
13

---

Published as a conference paper at ICLR 2026
GPT-4 model, where LCP reaches 96.95% HumanEval, proving that our framework effectively enhances even the most capable foundation models. The consistent gains across all tests, especially the extended ('EET) versions, validate the robustness and general applicability of our approach compared to other state-of-the-ar agentic strategies.
Table 3: Pass @
score comparison of various competing methods
Models
HumanEval
Hum-FExal-ET
MBPP
MBPP-ET
Foundation Models (Zero (Zero-Shot)
Incoder (6.7B)
15.2
11.6
17.6
14.3
Codel.lama (34R)
51.8
69.3
GPT-3,5-turo
57.3
42.7
36.8
Claude-instant-1
31.1
28.1
26.9
19.9
GPT-4
67.6
50.6
68.3
52.2
Agentic Prompting Strategies (GPT-3.5-turbo)
Fow-Shot
67.7 (18.2%)
54,9 (28.6%%
(26.1%)
48.3 (31.2%)
CoT
44.6 (-22..2))
37.2 (-12.9%)
46.1 (-11.798)
34.8 (-5,4%)
ReAct
56.9 (-0.7%)
49.4 (15.7%)
(28.4%)
45.9 (24.7%)
Reflexion
68.1 8%)
50.6 (18.5%)
70.0 (34.18)
47.5 (29.1%)
MapCoder
80,3(40.5%%
77.4(81.3%)
78.9(51.18)
54.44478%)
AgentCoder
79.9(39 446
77.4881 36%)
89.9(72.29)
(9.1(14..11)
CodeCoR
86.6(51.1%)
80.5(88.555 5%)
79.2(51.7%)
65 65,2(77 (26
LCP (Ours)
88.4(54.3%)
84.1(97.0%%
91.1(74.5%)
86.4(134.8%)
Agootic and Prompting Stratogics (GPT-4)
Redlexion
(34.6%)
77.1 (12.9%)
Self Debugging
80.6 (18.0%)
Self Sall-Callaorrtionn
90.2 (33.4))))
70.7 (39.7%)
78.9 (15.5%)
62.1 (19.0%)
MetaGPT
85.9 (27)()))))))
87.7 (28.4%)
AgentCoder
96.3(42.55))
86.0070.0%%
91.8(34.4%)
91.8(75.9%)
CodeCoR
34.5(39.856)
83,5(65.066 0%)
(Ours)
96.95(43:4%)
88.41(74.77%
92.51(35.4%)
89.70(71.8%)
Furthermore, test its capabilities more complex problems, evaluate framework the APPS benchmark against baseline and other two reference methods (LDB, LPW), categorized by task difficulty levels: Introductory, Interview, and Competition, using GPT-4o the LLM backbone. Following existing literature, we switch our base model in the difficulty test for fair comparison. The results, illustrated in Figure 4 demonstrate the robust capabilities of our LCP framework. LCP consistently achieves the highest Pass 1 accuracy across all difficulty tiers, scoring 92.1 1 on Introductory tasks, 77.5% on Interview tasks, tasks. This consistent superiority across varying complexities underscores the effectiveness of the LCP framework in generating correct solutions for a wide spectrum of programming challenges.
leading 38.0% on the challenging Competition
Table 4: Pass @ accuracy of multi-agent framework (LPW) compared with baseline and LDB on APPS benchmark across different difficulty levels using GPT-4o as the LLM backbone.
Difficulty Level
Baseline
LDB
LPW
LCP (Ours)
Introductory
63.8
78.7
87.2
92.1
Interview
43.5
52.2
65.2
77.5
Competition
17.4
28.3
34.8
38.0
D ADDITIONAL CASE STUDY DETAILS
This section provides supplementary information for the case studies presented in the main paper, including detailed experimental setups, prompts, generated code, and further results. To guarantee fair and interpretable comparison, we use GPT-4o as the unified base model across Cursor, Windsurf, and our framework during the case studies.
D.1 CASE STUDY 1: BEACH PROFILE PREDICTION
D.1.1 EXPERIMENTAL. SETUP
The dataset utilized for this beach profile prediction study is organized into distinct training and testing sets, containing 536 and 242 rows respectively. Each data row represents measurement point along
14

---

Published as a conference paper at ICLR 2026
beach profile, characterized by several key features: numerical coordinate denoting the distance from the profile's origin, serving as primary input; a numerical value representing the elevation at that distance, which is the target variable for prediction; and categorical feature, "Dominant Wave Direction" (e.g.. "ENE", "E"), necessitating encoding for model integration. Additional numerical columns present, representing other relevant physical environmental parameters that can be incorporated as supplementary features to enhance the predictive model's performance.
D.1.2 USER'S PROMPT FOR BEACH PROFILE PREDICTION
Please refer to the Bruun model and the Dean model to build a mathematical model to discuss the change of level height with respect to the distance from the starting point, and build a deep learning model based on our data. The following is our data path and structure
Data File Paths:
Training Data: " beachh_ aeeach_rrofile_tt profile_date processed _dtaa beachdata_ _rrii xlsx" Test Data: beeach_profile______________ data processed data/ beachdata test xlsx Data Structure Insights
The training and testing datasets contain several columns. Key columns include:
A numerical column representing the distance from profile 's origin; this is primary imput for predicting y: A numerical column representing the elevation; this the target variable for prediction
Dominant Wave Direction: This is a categorical (string) type column (e.g., values like "ENE", "E", "NE"). This column will require appropriate encoding.
All other relevant columns you might select as features are expected to be numerical integer or float) ) format.
Create the code directly and The script should not the main training evaluation logic directly when the script file is executed. Instead, it should define all necessary functions with main function main() that be WITHOUT ANY input. DO NOT USE tensorflow. Use provided data path in your code.
Listing Prompt used for beach profile prediction.
1.1.3 REFINED TASK DESCRIPTION BY LLM-TM AGENT
Please refer to the Bruun model and the Dean model to build mathematical model to discuss the change sea level height with respect to the distance from the starting point, and build deep learning model based on our data. The following is our data path and structure
Data File Paths:
Training Data: teeach_roffle_ddee processeedddate / beachdata_train xlsx
Test Data: 'beach profile data processed data/ beachdata test xlsx"
Data Structure Insights
The training and testing datasets contain several columns. Key columns include:
x: A numerical column representing the distance from a profile 's origin; this is primary input for predicting y. y: A numerical column representing the elevation; this the target variable for prediction.
Dominant Wave Direction: This is a categorical (string) type column (e.g., values like
"ENE". "E". "NE"). This column will require appropriate encoding.
15

---

Published as a conference paper at ICLR 2026
All other relevant columns you might select features expected to be in numerical integer or float) format.
create the code directly and The seript should not run the main training or evaluation logic directly when the script file is executed. Instead, it should define all necessary functions.
Extra information
The Bruun model conceptual framework used in coastal engineering to predict shoreline retreat in response to sea-level rise. It assumes balance of sediment in the cross-shore profile, resulting both erosion and deposition occurring simultaneously as sea levels change. The formula often associated with the Bruun model states that the retreat distance (R) is proportional to the rise in sea level (S) divided by the height of the active profile (a):
S x (d+ B) a B
where:- is the sea-level rise.- is the depth of closure.- B is the berm height.- a is the active height. The Dean model focuses on sediment equilibrium profiles, often fitting the profile shape with the formula:
y = 2/2
where:- A is parameter related sediment characteristics. Both models to provide theoretical insights into the interaction of sea-level changes with coastal profiles.
Feature analysis
The dataset contains various features, including:
**x**: Distance from profile 's origin, serving as a primary input.
**y**: Elevation, the target variable for prediction
*=Dominant Wave Direction*s; A categorical feature indicating wave direction. requiring encoding.
** Annuull Mean Period, Annual Mean Spring Tidal Range, Annual Mean Tidal Range*#; Numerical features capturing tidal metries.
** **Breaker Wave Height Hb, Deep Water Wave Height Hd, Mean Wave Height Provide information about wave conditions. **Dimensionless Settling Velocity, High Tide Sediment Settling of sediment transport characteristics
Velocity **: Indicative
** Latitude, Longitude=:; Geographical coordinates potentially used for spatial analysis
**Mean Grain Size, Mean Grain Size (Mz)*:: Deseribe sediment characteristics
important for erosion /sediment transport modeling.
Skewness,, Kurtosis, Sorting Coefficient *e: Statistical measures about sediment distribution
Extra Advice ===
To effectively use the provided features for the task, consider the following recommendations:
1. **Feature Engineering
Encode categorical variables such "Dominant Wave Direction" using one-hot encoding or ordinal encoding.
Calculate additional derived features if necessary, such interaction terms
between wave heights and sediment properties
Normalize standardize numerical features to ensure they contribute equally to model optimization.
16

---

analysis or Published as a conference paper at ICLR 2026
2. Incorporating Domain Knowledges:;
Use the Bruum and Dean models to inform feature selection or crafting Variables related to tide, wave height, and sediment characteristics are crucial Integrate expert knowledge about the relationships between features to inform model architecture in the neural network.
3. **Data Splitting and Validation **:
Ensure that the dataset is split appropriately into training and validation sets to evaluate model performance accurately.
**Model Architecture **:
Consider building neural network that incorporates multiple hidden layers to capture complex interactions...
Use activation functions suitable for regression tasks, such ReLU for hidden layers and linear activation for the output layer.
5. **Handling Geographical Variations **;
If spatial data relationships significant consider spatial - regression...
Listing 2: Refined task description by LLM-TM agent based on given prompt (The highlight part is the additional prior knowledge generated by agent)
D.1.4 CODE GENERATED BY CURSOR (ONLY DATA PROCESS PART)
def load ppac_and_procees__aat (filepath, scaler=None, labeL_encoder=Nons,
fit_scaler-faise, df read_exel (filepath)
Select features
feature_cols [col for in colunlumns col not ['y[]]
x df[feature_colls_ copp ()
= afl'y' paaaluss.sstype (np.float32)
Encode Dominant Have Directicn
if Dominant Wave Direction' c.columns:
10
if label_encoder is None:
11
LabelEncoder ()
if Eit_label_encoderr
Dominnnttt Wave Direction'] label_encodes firm (X[
nant Wave Direction") else:
14
15
XI Dominant Mave Direction') | label aareeeecccee...asstorre
Dominant Nave Direction }] else:
16
17
label_encoder None
18
Standardize numerical features
num cols select dtypes (include= [ wcolumn..
21
if scaler None:
22
scaler StandardScaler
23
if fit_scaler:
X[num_cols]
else:
26
[[num_cols] scaler. celler.rrasform [num_cols]]
27
28
return X. v.values.astyp fl) scaler, Label_ df 'x
..vauue.ttolist)) dt 'y' www.ues.
Listing 3: Code for beach profile prediction generated by Cursor.
17

---

Published as a conference paper at ICLR 2026
D.1.5 CODE GENERATED BY WINDSURF (ONLY DATA PROCESS PART)
Data leading and Preprocessing def load_data (train_path, test_path):
train_df pd. read_excel strain_path)
test Wk.read_exces (teet__teetteeeeeee $ Select features
feature_cor cols = [col for col in frain_df.ee collumms if col not in [y target_col 'y'
Handle categorical encoding and scaling
scaler StandardScaler
10
label_encoders ()
train_dataset BeadhProfileDatassttt (train_df feature_cols,
target_col, scaler-None, Label_enoodrr--lbbl_eeeeee fit_scaler- True)
12
test_cataset = BeachProfiLeCataset (test_df, feature_cole target_col, htw.:essstaan_dttsttt..oooo
label_ fit_scaler-=alase
13
return train_dataset, test_dataset, feature_cols, label_encoders
Listing 4: Code for beach profile prediction generated by Windsurf.
D.1.6 CODE GENERATED BY OUR FRAMEWORK (COMPLETE CODB)
Define the function to run the entire process def www.bbeehhprofle_nnee
(train_data_pattl test_data_path)= # Component 1: Data Loader
def data latta_loderrttee (train path, test_path): Load datasets
train_df pd. reae_ (train_path))
test_df = pd. recel (test_path)
One-hot encode categorical features
10
encoder OneHotEncoder (sparse=False)
11
- encoder. htt.aeattttttttfffmmtrrroto [['
Dominant Nave Direction'll I1)
encoded direction_enccdedd.ee Wave Direction') [])
Dominant
13
14
$ Drop the original categerical column and add the encoded columns
train df train drop (ctpppccoolmmmmm Nave Direction')
16
test_ds test df. drop wc..cooooccoo.mmmmsm//oooaaeen Wave Direction'] ]
17
train_df = train train (ieeteiio _ncooded _trin, col
Dominant Wave Direction! )
test_df test_ddf.join act..edd...ooooooooaaaaaaaaaaaaa dereeoooooddddeeeeeeeeeeeddddddd Dommonnnnt Wave Direction' 1))
19
20
Normalize numerical features numerical_en
wtt.ffffeeeett__yypeeeeee (include- int64',
22
Exclude variable 'y' and 'x'
23
numerical_co.. cols = cuunerical_cossddeee drop (1'y', 'x" l)
24
scaler StandardScaler
[rainaaoffumeriialccol scaler, oiceet..ii..ttnofffmmttrr..... numerical_cols))
26
test__df[numerical.cc 1)
(testfooottttteettdt (test_ [mw.eeacaco_mmmrcccccccc......
train_df, test_df
29
30
Component 2: Theoretical Model Integration
def innrgaigattieeiiitntnttiiiee i derived feature using Dean's model
test_df):
train_df [''DeennKeetture" train_df[ [xx] (2 3)
18

---

Published as a conference paper at ICLR 2026
Best_dfl' Feature'] test df
35
36
I Potential Bruun influence -> Using given example formula might
require additional variables
(ttrain_f[('"" Wave Height<])) (train_df['Xean Grain Size(l)) test_df[' Bruun B
(train_df MeMnn
38
((test_df [f__df[[y ['y'] (test_df) Mean Wave
Heigat' / (test_df. (' Mean Grain Size" ]
39
return train_df, test_df
42
Component 3: Deep learning Model def deep_learning_modee (X_train, y_train):
43
# Define the model architecture class SimpleNN (nn.Module):
46
def _init faat__ssef,, input_sizel,
47
super (SimplcNN, . solf.layezs nn. Ln.eir (input_sizc, 128)
48
sclf.layer2 Lnnein (128, 64)
self.layer3 Ln.Linear (64, 32)
51
self.output Linein (32, 1) self.relu nn.RelU()
52
53
def forward(self, x)
55
x self. Layee (x))
56
X self. (self. (sylr.
57
X self. rel. (
self. (x)
60
61
Initialize and compile the model model = SimpleNN(X_train, (X_train, shape (1])
62
return model
64
65
Component: 4: Model Training and Rva avaluation def htt__trrinnnneevalueee in (madel, train_df, test_df): # Split the data into features and target
66
X_train train_df .eff.dropffe Turns ").t)_nutyttt ()
69
y y_train train d'l'y' I.te numpy
70
X_test test df drop (celumns=('y' } to numpy ()
y_test test_df['y' tto_numpy()
73
74
Convert arrays to PyTorch tensors
75
X_train_tensor torch. tensor (X (X_train, dtype=torch. float322 y_train_tensor (-1, 1)
(y_train, ttypestorch. .vie
77
78
X_test_tensor y_test_tenser torch. (y_ttt,, syype=torch.tloatt22 view (-1, 1)
Stype=torch.:llatt22
81
Create dataloaders dataset TensorDataset Kx.train tensor, y=ttoon_ttnsorr
82
train_ = DataLoader datsetd batch satch_siie==22
shuffle=
True)
85
Define loss function and optimizer
86
criterion nn MMELLsss ()
optimizer optim.Adan Adam (madel, parameters (), 1r=0.001)
89
Training 1oop model.train() for epoch in range (100):
90
for batch X, patch in train Loader:
19

---

Published as a conference paper at ICLR 2026
optimizer. zero grad() zer__gradd))
94
outputs model (batch_x)
95
loss = criterion (ontrrinnccuuuttt (cutputs, batch_y)
96
bess.bacwaadd optimizer.stepl
99
Evaluation metrics
100
model.eval
with torch.no_gtet
predictions model (X [X_tess_tensor)
mse criterion (predictions y_test_tensor) .item
104
rmae np. sqrt (mse)
105
mae - np. mean (np.abs (predictions. numpy () y_test_tensor.
numpy ()))
=_squared
(np.
numpy
y #__ess__eenso numpy )))) ap.sum atg.ssttttgg..pprraree..eeeeeeee numpy () np
.maan (y_ttyy_tetesstooo .unpy ()))))
return (' RMSE'; rmsc, 'MAE'; mac, 'R2'; r_squared]
110
Execute the process
111
train_df, test_df data Lcader (trainddttttidttt___ttttt test_data_pathi train di, test_df df.
theoretical model interraiinnn dł, _f) train = wtw.itaataaannneeeeeeeee dropna
test test of dropna ()
115
model - deep_leerriigg_mode (train_df.r drop (columss-- 1) t>tto_uumpyyyl (),
evaluaticn_results model_trainnnngeewalttte (model, train_df,
test_df)
118
return evaluation_resulie
Listing 5: Code for beach profile prediction generated by Our Framework.
D.1.7 RESULTS AND DISCUSSION
In this case study focused on beach profile prediction, the primary objective was to integrate estab- lished theoretical models -specfiiipy the Bruun and Dean models wiit a deep learning methodol- ogy, per the user's explicit requirement. An examination of the approaches reveals that while both the Cursor and Windsurf frameworks implemented standard data preprocessing techniques such as numerical feature standardization and one-hot encoding for categorical data, they did not incorporate the specified theoretical models. In contrast, our LCP framework successfully addressed the user's need by calculating additional derived features based on the Bruun and Dean models. This direct integration of domain-specific theoretical knowledge into the feature set represents a key differentiator in our approach.
The impact of this tailored feature engineering is reflected in the prediction performance, as illustrated 5n The results indicate that the LCP framework yielded predictions superior those generated by Cursor. Furthermore, LCP's predictive accuracy was observed to be closely comparable to the results from Windsurf. This suggests that the inclusion of theoretically-derie
features not only fulfilled critical user requirement but also contributed positively to the model's 's ability to accurately predict beach profile changes, positioning LCP as a more comprehensive solution for this specific task.
D.2 CASE STUDY 2: BRAIN MRI SEGMENTATION
D.2.1 EXPERIMENTAL SETUP (BRAIN MRI)
This study utilizes subset of the "LGG MRI Segmentation" dataset, which contains brain Magnetic Resonance Images (MRI) and corresponding manual FLAIR abnormality segmentation masks for patients with Lower Grade Glioma (LGG). The original dataset, sourced from The Cancer Imaging Archive (TCIA), includes data from 110 patients. For this experiment, data from 30% of these patients was selected. This selected patient data, comprising MRI slices (typically 256x256 pixels)
20

---

Published as a conference paper at ICLR 2026
Y Inrevatioo/Prffille Value)
Ground Truth
Baseline
Our framework
Windsurf
cursor
-1
-3
40
60 100
120
140
160
X (Distance along profile)
Figure 5: Beach profile prediction results comparison.
and their associated segmentation masks, was then partitioned into training, validation, and testing sets with a 7:1.5:1.5 ratio, respectively, to develop and evaluate deep learning model for brain tumor segmentation.
D.2.2 PROMPT FOR BRAIN MRI SEGMENTATION
Develop image segmentation model to accurately and efficiently identify and segment FLAIR abnormalities in brain MR images from the LGG Segmentation Dataset.
Dataset Format & Structure:
Base Path: r r'E:lpython Brrpllpooopeeoirre newl braii_caaaeeaasse_eee_eaeeeeeeee _ase__stsdyy_____s llgg-mmi-seeementttinaascsssses study"
Organization:
The base path contains train val /, and test subfolders.
Each of these subfolders contains two further subfolders images/ and masks/.
Image Files (images/):
Format: 3-channel tif files
Naming: TCGA catpriiiio--oodee_pptttttttt-tie potttiin-idbbclll---mmeeeeree
Mask Files (masks/):
Format: Binary, I-channel tif files
Naming: TCGA_ seapiuution-code__cptteeeiee peaaienniit__slleennbbeee_msssss
Listing 6: User's Prompt used for Brain MRI segmentation.
D.2.3
REFINED PROMPT BY LLM-TM AGENT
Role
Extra Advice
** Data Preprocessing :*e Normalize image intensity values (min-max or z-score normalization) ). Ensure all images resized /cropped to the same shape (commonly
21

---

Published as a conference paper at ICLR 2026
256x256 or appropriate for U-Net input). If channel information is redundant, you could reduce to one channel for efficiency
*eLabel Preparation *** Ensure mask values binary (0 and 1). Double-check that mask and image align correctly for every slice
**Data Augmentation:* To combat overfitting and improve generalization apply transformations such as flipping rotation scaling, and intensity shifts ensuring they are applied equally to images and masks.
**Model Choice:** Use U-Net (or lightweight modification thereof) due to its strong performance in medical image segmentation with relatively small datasets
** Function:*e Use combination of Binary Cross Entropy and Dice Loss to penalize both pixel-wise errors and poor overlap.
#sBachinng:*s Due to likely high resolution and small dataset, use small batch sizes and possibly gradient accumulation.
** Evaluatiionn Report Dice the test set (already specified ). Consider visualizing a few results for qualitative assessment, e.g., overlay masks or compare ground truth to prediction
*e Visualization :* Save (or plot) a grid of test images, ground truth, and predictions for several slices to inspect the model's performance.
By thoroughly preparing the dataset, using appropriate augmentations, and selecting suitable architecture and losses, you can build robust FLAIR abnormality segmentation model for the LGG dataset. Ensure reprodueibility by setting random seeds and keeping all data splits strictly separate during model development and evaluation.
Listing 7: Partial Refined Prompt for Brain MRI segmentation, generated by LLM-TM agent
D.2.4 CODE GENERATED BY CURSOR (ONLY MAIN TRAINING L.OP))
Training Loop
def train_cne__ee eeeeechhheepocchhheeee (model, Loader, eptimizer, critericn, device) moasl.train()
epoch_loss
for imgs, masks in tqdm (loader, descc'Train', leave-False)) imgs, masks imgs.to (device), masks. to (device) optimizer.
grad
outputs model(imgn)
10
loss = criterion (outputs, masks)
11
loss bassbacckard
optimizer.stept
13
epoch_loss loss loss. (tem() ings.sies (0)
14
return epoch_loss len (loader (loader,dataset)
15
Main Function 16
def main():
18
device
cuda' corch.euda is available else 'epu')
19
Datasets
20
Model
22
model
() to (device)
23
optimizer opt Adam modee..araaeterss. (), 1r=1e-3)
24
criterion = nn. nn. BCELoss ()
# Training
best_val
27
for epoch (1nge 11): # epochs train_loss Crain_one__epchh (model, Loade,, optimizer, criterion, device)
28
val_dice evaluate (model, val loa_er, device)
30
if val val_dice > best_val:
22

---

Published as a conference paper at ICLR 2026
best_ val dice
32
best_model model state_state_dict
33
Test
34
model. www.l_aaateeedeeeeeoeca dict (best_medel)
rest_cice evaluate (model, test_loader, device) return model, test_loader, device, test_dice
37
Listing 8: Code for Brain MRI segmentation generated by Cursor.
D.2.5 CODE GENERATED WINDSURF (ONLY MAIN TRAINING LOOP)
def train_con one _tt_eecneeeeee_eee_eeeeeeeeeeeee (model, loader, optimizer, criterica)
epoch_loss
for imgs, masks in loader:
imgs, masks imgs. (DEVICE), masks. (DEVICE)
outputs = podel((igg))
loss criterion (outputs, masks)
10
Loss.backward()
11
optimizer.stepl
12
epoch_ += loss.item() size (0)
return epoch_loss 1en (loader (loader.dataset)
15
def (
16
base_ path
model UNet () to (DEVICE) optimizer opt imcimm. Adam (model pericl.ppare parameters 1r=1c-3)
19
20
criterion = nn. nn.BCELoss ()
best_val_dice =
for epoch in range (1, 11): # 10 epochs train_loss
@eain_one_epoch (model, train_loader, optimizer,
criterion)
24
val_dice evaluate (model, val_ loader) it val dice > _
25
best_ vis__al__iie dice val dice
27
best_model_state model.state__iit ()
28
model.lcade leaa__oottsttt_ediceeo (best_mcdel_statee test_éice evaluate (model, test_loader)
29
return model, test dice
Listing 9: Code for Brain MRI segmentation generated by Windsurf.
D.2.6 CODE GENERATED OUR FRAMEWORK (ONLY MAIN TRAINING LOOP)
class DiceLoss (nn. Module)
def __iiiiittts____ (self, eps=le-7):
super () ()___nii_t_) self eps
def forward logits, targets):
sigmodd targets = (targets > 0.5) fllatt)
inputs
intersection = (inputs ttrrgets. (dim=
unionset inputs. sum (2, targets. sumtssssum(e (dim= (2, 3))
11
dice
intersection self.eps) eps) (unionset self. eps)
12
loss = dice return loss.mean ()
13
15
Train and Evaluate
23

---

Published as a conference paper at ICLR 2026
def [
17
model,
18
dataloader_dict,
19
device,
n_epochs-10,
1r=le-3,
22
patience=2,
25
24
optimizer torch. Adam (model parameters (), ir=ir)
boe loss wcw/ithhoogitsssssssss ()
dice_less Dicelloss
27
best_state copy. sepydeepcopy (model.. state_dict ())
28
best_val_dice -np.inf
epochs no. for ep in range (n_eocchs)))
Becthsss___imprree__ee_ improve
31
model.train()
32
train_loss []
33
train_dice []
for in dataloader_dict/ ('train") ];
(device)
36
y y. y. (device)
37
38
pred = model
lcss 0.5+
Loss (pred, y) 0.5xdice loss (pred, y)
40
loss. backward())
41
pptimizer.step)
42
train_loss. (loss. (t)m(m))
train_dice. ddiee_coeficiien (pred, y)
Validation
45
46
Early stopping
47
model.lcad_stast dict (best_state)
89
return model
50
51
MAIN PIPELINE
def main (): ==*
54
Loads data, trains segmentation model, returns model and average Dice
on test set.
Settings (tuned for quick/robust test)
57
DATA_ PATH
58
59
Model
model = UNet UNetLite (in_ch-33 out_ch=1) to (device)
Train
62
mogel train_model
63
model,
64
cataloader_dict,
device,
1_epochs-N_MPOCHS,
67
1r-uR, patience=PATTENCE
68
Evaluate test
71
test_áicc wraluate_modes (medel, decalodder_dicc((tett ), device)
72
return model, test_dice
Listing 10: Code for Brain MRI segmentation generated by Our Framework.
24

---

Published as a conference paper at ICLR 2026
D.2.7 RESULTS AND DISCUSSION (BRAIN MRI)
In this case study, the objective was to develop model for accurate and efficient identification and segmentation of abnormalities in brain MR images, based relatively open-ended description that primarily specified the data format structure. While both the Cursor and Windsurf frameworks opted for traditional U-Net architecture and utilized BCELoss for model training, our LCP framework adopted different strategy. LCP generated code implementing a "Lite U-Net." more streamlined architecture designed for faster computation. Furthermore, for the training process. ICP combined nn.BCEWithl LogitsLoss() with custom Dicelooss)..
The practical outcomes of these differing approaches are evident in the performance metrics presented in 5 Most notably, the LCP framework demonstrated a significant advantage in computational efficiency, with its generated code requiring only approximately one-quarter of training time compared to the solutions from Cursor and Windsurf. In terms of segmentation accuracy, the LCP framework achieved a Dice score on the test dataset that surpassed Cursor's results and was only marginally lower than that of Windsurf. This indicates that LCP's choice of lighter model and compound loss function provided a highly efficient solution that maintained a competitive level of accuracy, effectively addressing the user's call for both efficiency and accuracy in complex image segmentation task.
Framework
Cursor
Windsurf
LCP (ours)
Dice Score
0.6627
0.7232 131.6s
0.7185
Training time
127.9s
36.2s
Table 5: Comparison of Dice scores on test data with code generated by different frameworks.
Ground
Prediction
Image
Image
Figure 6: Brain MRI Segmentation by Cursor.
25

---

Published as a conference paper at ICLR 2026
Prodiction
Ground trch
Image
Ground Trun
Predictioo
Figure Brain MRI Segmentation by Windsurf.
Image
Grund Tuth
Reficion
rrpul Imag
Goard Tuth
hediciin
Input Image
Figure 8: Brain MRI Segmentation by LCP.
26

---

Published as a conference paper at ICLR 2026
D.3 EXPERT ADVICE FILTERING AND LEAKAGE PREVENTION
In our framework, expert advice is restricted to domain knowledge such as physical formulas, data constraints, and high-level scientific logic (e.g., guidance like "use the Bruun model equation"), rather than implementation-levà solution code. In our experiments, this advice was strictly filtered to avoid ground code leakage. In addition, except SciCode (With Knowledge) and the targeted case-study settings, we did not inject expert knowledge, reducing leakage risk and preserving fair comparisons.
D.4 TRAINING PERFORMANCE
To train our model, we collected 120 unique answers for specific LeetCode problem. This dataset was then divided, with 60 answers designated for training. The remaining 60 answers from the target problem were combined with 60 code samples from different LectCode problems to form our test dataset.
The training process demonstrated efficient learning, as illustrated in which shows the training loss plotted against epochs. The loss converged rapidly, showing a significant decrease from the start to the 10th epoch.
Upon evaluating the trained model on the test dataset, using classification threshold of O, we achieved promising results. The model demonstrated Accuracy of 0.8487, ROC AUC score of 0.9322, and Precision, Recall, and F1-score all at 0.8475.
Train ng Loss per Epoch for Multiple Experiments
1.0
Expenment
Experiment
experimen:
Expenment
Expenment
Experiment
0.6
1035
0.2
0.0
7.5
12.5
Eaoch
Figure 9: Training loss against epoch for the IRL experiment.
D.5 STATE AND ACTION SPACE DEFINITION EXAMPLE
Vocabulary:
staate ><UNK>: 0
Name_U1:
Name <UNKNOWN>: 11
arg U1: 12
arg <UNKNOWN>: 22
Assign_U1: 23
Assign <UNKNOWNN: 33
<tate Assigns Assign U1): 34
state > Attribute Attribute 35
27

---

Published as a conference paper at ICLR 2026
<state DlasssDeCCCassssee 36
<state >functionDeffuunciioDDeef 37
<state >Impor((math): 38
<state State>>/pporFrrooos::ppppo 39
<ttate>arg(rrg_U) U1): 40
Add: 43
BinOp: 44
Call sqrt ) 45
Constant: 46
Load: 47
Module: 48
Mult: 49
Listing 11: State and Action Space Definition.
Based on the State and Action Space defined above, we will transfer code into state-action trajectory, for example:
Code:
import math from os import path math. sqrt (16)
It would be converted into:
State
[[38), [38, 39], [34, 35, 38, 39]]
Action:
[[38], [39]. [34, 35, 45, 46, 50, 51]]
Listing 12: Single example about code processing
E PROMPTS USED FOR LARGE LANGUAGE MODELS (LLMs)
This section details the specific prompts provided to the Large Language Models (LLMs) for various sub-tasks within our framework. These prompts are displayed to mimic structured textual input.
E.I LLM-TM (TASK MANAGER) PROMPTS
E.1.1 PROMPT FOR DATA ANALYSIS AND PRIOR KNOWLEDGE REFINEMENT
I am working on a code generation task:
task_ jess_desrription_ss
task_ task_description }
task description end>
need you to analyze the data will be working with and add extra information about
the task escription Please structure your response as follows:
<extra_iie eextra_infrrmation
Based on the task description, provide any additional information or context that might be relevant to the task. For example, adding mathematical formulas, domain-specific knowledge, or any other relevant information that can aid in the code generation process.
28

---

Published as a conference paper at ICLR 2026
</ extra information
feature __aalysss
Provide a brief introduction to the features present in the data. If specific feature information is not available please describe the overall data structure you would expect or require for this task. </ feature_anale aealure_analyiis
<advice>
Based on the task description, extra knowledge and the data features (or expected data structure )
provide advice on how these features can be effectively used to accomplish the project, and how to incorporate the additional information into the project.
Suggest potential data transformations, feature engineering steps, or specific ways to leverage features and extra domain-specific knowledge in the code generation process. </advice>
Listing 13: LLM-TM: Prompt for data analysis.
have an initial task description and some analysis regarding the data and approach. need you to synthesize all this information into a single, clear, and more detailed refined task description...
The refined task description should incorporate insights from the feature analysis and the advice provided, making it more actionable and comprehensive for a code generation system. should clearly state the goal, the expected inputs (data/ features ), and any key steps considerations mentioned in the advice.
Here is the information:
original_task..o batiaal__as__essrppiioo sesecrritttiiosstrrtt
original_task__onne tasc_descripttionnnnnncccoa }
original task dewcription end?
extra_ detra_informatio__so start _tart >
extra_information
sexura_infrrmtion_enn _ndd>
feature_analysis_stn feature anallysis }
feature_amn llll__aalllllslleee senysssssnnd<>> >
advice_start> [ <advice_end>
Now, please provide the refined task description Output only the refined task description itself, without any extra conversational text or tags. Refined Task Description
Listing 14: LLM-TM: Prompt for prior knowledge refinement.
1.2 PROMPT FOR TASK DECOMPOSITION
29

---

Published as a conference paper at ICLR 2026
You expert agent specialized in decomposing code generation tasks into structured, detailed, and clear subtasks and then give a detailed overall plan based on your defined subtasks. Given simple high-level task deseription, your job is to break it down into logical subtasks that clearly illustrate the workflow and ensure easy understanding and execution.
Each decomposed subtask should aim create function or class as reusable component contributing to the overall task. If the provided task is too simple or atomic to require multiple components, your decomposition should only contain a single component.
Your output must strictly follow the format below:
<components>
"component_ 1":
" step_task tass_sssdsspptionnnn ": str.
"input_formatt ": [[ [[typ shape or null ]].
output format": tyyy shape null ]l.
"work flow": [sr ].
cest_case_genereeee advise ": [srr]
"component_2"
step_tas taa__tsk__d_aa descrippiiinnnn ": str.
"input_ffrman "; [[" type", shape or null l]. " "output "ormat": "outputtformat".. [[" type", shape or null Il.
"work flow": [ str
test case peasee_tntrttion advise ": [str]
</component>>
<overall >
Cimput_frrmal ": ty"t,,"". shape or null ]].
": [[" type", shape or null ]}.
"components": [tr ].
"plan" ": [str]] J.
test cest_case_eenie cat_ecaaeeeattioonnnmmmmmo advise ": [ str)
</ overall_plan>
Here are additional detailed explanations of each field
For <components>:
*scomponent Xw#: The key represents the subtask name, it should be replaced by the actual class function name of the component (e.g., "merge_arrays" "calculate_ median "). *s ttep_task__ teep_task_escrpppppee description **: Provide clear and concise description of exactly what this subtask aims to achieve, specifically mentioning the intended functionality or role of the created component (function/ class). format
aaddulate_mediin
"eeinput_format== Describe the format of each input argument required for this subtask. It is list of lists where each inner list has two elements:
The first element indicates the data type (e.g., " list ", "dict", NumPy array, torch. Tensor). DO make sure the data type is string
The second element indicates the fixed shape if applicable; otherwise, it is null.
**output_ +soutputtformatee: format**: Describe the format of each output argument generated by this subtask. It follows the same list structure as 'input_format " note that it has to be list of lists
30

---

Published as a conference paper at ICLR 2026
*=work flowe=; Provide detailed step-by-step plan that outlines the workflow of how the component functions to achieve the subtask.
test cas__cas____ crase_generation advise Provide list of detailed guidelines suggestions aimed at generating diverse and comprehensive test cases, explicitly mentioning potential edge cases and critical scenarios that need coverage.
For <overall_plan,
+4: *sinput_format=:: Describe the format of the input arguments required for the overall format' in the component section.
task follows the same structure as **output *=outputt format=== Describe the format of the output arguments generated by the overall task. It follows the same structure as 'ourput_format" in the component section. #scomponentses: List the components in the order.
*splan=*: Provide a detailed step-by-step plan that outlines the workflow of how the components interact with each other to achieve the overall task. This should be a high-level description of the process.
** test test_case__eeteeeeeeeeeeeeeesss_ caase_aenerattonnn advise Provide list of detailed guidelines suggestions aimed at generating diverse and comprehensive test cases for the overall task. explicitly mentioning potential edge cases and critical scenarios that need coverage.
Your decomposition should strive for clarity correctness, modularity. and ensure each step can be tested independently. Now, given the following simple task description
"( (TTASS_DESRRPPH
Use < to indicate both start and end of the component part and the overall plan. Ensure that the components and the overall plan are clearly separated.
Please provide your structured decomposition according to the instructions above.
Listing 15: LLM-TM: Prompt for task decomposition.
You expert agent specialized in refining and improving code generation plans through iterative feedback. Given task description previous decomposition output, and user feedback, your job is to critically analyze the existing plan and modify it accordingly while maintaining the required output format.
Carefully review the previous components and overall plan, then:
1. Preserve correct valid elements that don't conflict with the feedback
2. Make targeted modifications based on the user's s specific advice
Ensure consistency between components and overall plan
4. Verify input / output formats and workflow logic
5. Check for any introduced errors during modification
The input consists three elements: Original Task Description: "((TASK DESCRRPTION Previous Decomposition Output:
(PREVIOUS SPRRVIOOSSOOUTPUTI]
User Feedback: "[(USER_ADVICH)
Your output must STRICTLY follow the original format with these sections campoonetss..c/coompeeeet
<overall_plan >..</ overall_plane
Follow these guidelines
Explicitly address all points in the user feedback
Clearly document any changes made from previous version
Preserve JSON structure and formatting requirements
If feedback contradicts original requirements, prioritize feedback
31

---

Published as a conference paper at ICLR 2026
Again, user feedback is: "[(USER [[USER_ADVICE)]
Provide your refined decomposition with clear explanations of changes in the component descriptions and overall plan. Ensure modularity, testability and coverage of edge cases mentioned in feedback.
Listing 16: LLM-TM: Prompt for plan refinement based feedback.
E.1.3 PROMPT FOR TEST CASES GENERATION
You are a test case generation agent. Your task is to create Python test functions to validate code generation task based on the provided specifications Follow these instructions carefully:
### Input Specifications
TasTask Description **;
task Book_deser_sst
** *=Inuut Formats#:
sappu_descrrstte
** Output Formatæ#:
output_descrt _srrr
*=Cooponents Used##; [components_set }
**Plan**;
plan_str) str)
Test Case Advise#s:
advisory_liss
### Requirements:
1. ** Test Function Structure #*:
Each test function must accept *oonly the function under test its parameter (e.g gg-.... "def test_case (func) :...)
Return "True" if the test passes, "False" otherwise. Do not assertions please return boolean value.
Include input generation, runtime checks, code inspection, result validation within the function.
2. ** Test Types** (use one of these for indicating the test_type)) ):
correctness Validate output against expected results for specific inputs _ddee_case": Test inputs like empty lists extreme values, or invalid data. "runtime" : Mcasure execution time (c.g., ensure it's below a threshold). 'component <aroonent_chckk'; Verify the function 's code uses specified components (e.g., via string inspection). )
eerror_ error_handling" :: Check if errors are raised for invalid inputs.
3. *esest Case Diversity
Cover all provided advisories
Include least one test per advisory and one for each test type where applicable
### Output Format:
For each test case, you need to firstly define the Test Types to indicate what type of test case you are going to create and then give the reasoning and explanation of the test case. After that, generate the test function based on the your reasoning.
For each test function, return with following structure
<Type>
32

---

Published as a conference paper at ICLR 2026
Pick one of correctness ledge_cett case rruntime componennt checklerror handling </Type>
<Planning>
Introduce how would you design the test function. Specify the purpose of the test function and the reasoning behind it. Explain step by step why your test case is correct and what is the expected output.
</Planning>
<Code>
def test_case (func):
# Your test function code here
return True or False as test result, and message </Code>
If you going create multiple test cases, please separate them with <sparatorr> tag.
(example_text]
Generate test cases that rigorously validate the function 's behavior, code structure
and performance.
You MUST strictly follow the output format and structure The generated test functions MUST be rumnable function that use another python function as its parameter and it should output both the Test result (True or False) and message to give extra information about the test result - (or example, f"Test failed expected X but got Y" or "Test failed output with shape [xl, y1] but got [x2, y2J", where the X, and shapes need to be replaced by the actual output and expected output in test function).
Listing 17: LLM-TM: Prompt for test case generation.
E.2 LLM-CG (CODE GENERATION) PROMPTS
E.2. PROMPT FOR CODE GENERATION
Role =
You are a highly skilled coding assistant designed to generate clear, efficient and correct code based structured task descriptions and detailed plans provided by the user. Your responses must precisely follow the instructions formats, and constraints given by the user, and you must strictly adhere onput-output formats, workflows, and specific guidelines outlined.
Task Description task description }
Components === ceapoonent__esccrpiitttt
Overall Plan plan_text]
Test Cases ==
campleed_testccaate
Instructions
Generate the COMPLETE code based on the components and plan above.
DO MAKE SURE the complete code a runnable function, all components are correctly integrated with in this function.
The complete function should take the input arguments as specified in the overall plan and return the output specified.
33

---

Published as a conference paper at ICLR 2026
Please add much comments as possible to your code to explain the logic and any critical steps.
Structure your response follows:
<Code>
Your code here. DO make the output is single function that integrates all components.
</Code>
<Planning>
detailed btep-by-step explanation of the code's workflow. </ </Planning>
<Main Function Name>
The name of the main function that integrates all components.
</Main Function Name>
Provide the code with the same indicator and structure shown in Instructions. DO NOT return any test cases or example usages your code!
Listing 18: LLM-CG: Prompt for code generation.
E.2.2 PROMPT FOR CODE REFINEMENT
Role
You are a code refinement specialist designed to improve existing implementations based on specific feedback. Analyze the provided feedback, identify areas for improvement. and modify the code while strictly maintaining the required input output formats and component specifications.
Task Description
task descripttti }
Components
[oompnennss_ _errrinptdddereitioono
== Overall Plan ===
plan_text]
Test Cases
sampled_test_caset
User Feedback [sser feedback)
Previous Best Code Generation isampled_code foodes_with_emmeee
Refinement Requirements
Before refining the code, tell me the reason why the last code failed to pass the test function, and how would you improve the code.
Instructions
Generate the COMPLETE code based on the components and plan above.
DO MAKE SURE the complete code a runnable function, all components are correctly integrated with in this function.
The complete function should take the input arguments as specified in the overall plan and return the output specified Please add as much comments as possible to your code to explain the logic and any critical steps.
34

---

Published as a conference paper at ICLR 2026
Structure your response follows:
<Think>
Explain why does last code failed to pass the Test Function, and how to fix it </Think>
<Code>
Your code here. DO make sure the output is single function that integrates all
components.
</Code>
<Planning>
A detailed step-by-step explanation of the code's workflow.
</Planning>
<Main Function Name>
The name the main function that integrates all components.
</Main Function Name>
Provide the code with the same indicator and structure as shown in Instructions. DO NOT return any test cases example usages in your code!
Listing 19: LLM-CG: Prompt for code refinement.
F SAMPLE TM-GENERATED TEST CASES
This section provides concrete examples of test cases generated by the TM agent (SciCode example; first subproblem of problem 4, using Qwen3-32B), following the rebuttal supplementary material format.
<separator>
<Type>
correctness
</Type>
<Planning>
This validates the correctness of the incomplete Cholesky factorization function using a small symmetric positive definite matrix with known structure and sparsity The input matrix is:
S$
A (bbmati
2
0 & 5 & 0 WI
2 &
lndd bmatrix S$
The expected incomplete Cholesky factor $K$ should be sparse, preserving the zero pattern from SAS. The exact Cholesky decomposition would give:
s$
L= Vbegin Dregin(bmatie [mmatrix]
0
0 & \sqt (5) & 0 W
& \sqrt (2)
bbmatrix)
S$
Since $A [13) = -A 2 'ineq OS, the corresponding entries in SK$ are non-zero.
35

---

Published as a conference paper at ICLR 2026
We compute the first
-$L_(111 >sqrt (4] = 2$
-Since $A [21) = =0S, $L [21] 0$
[31] (2 - 0)/2 = 1$
Second row: $L_ Asqrt {5}$ $3_ = OS, so SL_ (32) = oS
Third row: Compute diagonal: (33) ssrr (6 [__(31122 -sqrt (6 1] -sqrtt (5}S
So the expected $K$ is:
S$
K \begin (bmatrix)
2 & & O WI
0 & \sqrt (5) O
& & \sqrt (5)
lend(mman (mmtrrx
S$
We will use this to validate the output.
</Planning>>
<Code>
def test_case (func):
import numpy np
# Define the input matrix A
= up..rray: ([
[4, 0, 2].
[0, 5, 0].
[2, 0,
J. dtype= float )
# Expected K based on reasoning expected np. array ([
[2, 0, 0],
[0, np. sqrt (5), 0}.
[1, 0, np. sqrt (5)]
I)
# Call the function under test
try:
result K = func(A)
except Exception as return False, f"Test failed: exception raised during execution (e)"
# Check if result is NumPy array if not isinstance (result_K, nd.rrayyyy))
return False, "Test failed output is not NumPy array"
# Check shape matches input if result_K shape != A.shape: return False, f"Test failed: output shape (result K. shape) does not match input shape [A.shape]"
# Check for lower triangular structure
if not np. allelose (result K [[np. triu indices from (result K, k=1)], O):
36

---

Published as a conference paper at ICLR 2026
return False,
"Test
failed
upper
triangle contains non-zero values"
# Check sparsity pattern matches A's lower triangle
if not np. allclose Tresult_K[ng] [np.wwheee(A = 0) 0):
return False, "Test failed non-zero entries in positions where input was zero"
# Compare result with expected using tolerance due floating point precision if not np. allclose (result_K, expected K, atol=le-6): return False, TeTesst failed: expected fexpected K). got Desull_K])
return True, "Test passed: correct incomplete Cholesky factor computed" </Code>
<separator>
<Type>
edge_case
</Type>
<Planning>
This case checks how the function handles an empty matrix input. Incomplete Cholesky factorization is undefined for empty matrices. We expect the function to either raise an error or return an appropriately sized empty matrix. This tests the robustness of the implementation when given invalid or edge-case inputs.
</Planning>
<Code>
def test_case (func): import numpy np
# Input is empty matrix
= in.arrra (I), dtype= float). reshape ((0, 0))
try:
result K = func(A) except Exception as return False, f"Test failed: exception raised during execution - (e)
# Expecting an empty matrix of the same shape if result K. shape != (0, 0):
return False, f"Test failed expected empty matrix but got shape
[eesult_Ksshap
result K.ii shhape))
return True, "Test passed: function correctly handled empty matrix input"
</Code>
<separator>
<Type>
edge_case
</Type>
<Planning>
This test case evaluates the function 's behavior when provided with singular matrix (not positive definite ). Incomplete Cholesky factorization requires the matrix to be symmetric and positive definite A singular matrix (e.g., with determinant zero) will cause the square root of a negative number or division by zero the algorithm. The test expects the function handle such cases gracefully either by raising error or returning an appropriate message.
</Planning>
<Code>
def test_cast (func):
37

---

Published as a conference paper at ICLR 2026
import numpy np
# Singular matrix that is symmetric
= n (I
[1, 2},
[2, 4]
dtype= float) )
try:
result_K = func(A)
except ValueError: return True, "Test passed: function correctly raised an error for singular
matrix"
except Exception
return False, T"Test failed unexpected exception raised (e)"
# If no exception is raised, check for NaNs or invalid outputs if np. up.any(np. isnan (result_K))> return True, "Test passed: function returned NaNs for singular matrix"
return False </Code>
"Test failed: function did not detect singular matrix"
Listing 20: TM agent outputs with test case type, planning rationale, and executable test function.
38


Reviews:
[{"review_id": "iMWJOSalfu", "reviewer": "Reviewer_G9ac", "summary": "The paper proposes a multi-agent framework for AI for science code generation, which is trainable via Bayesian adversarial learning. The multi-agent framework enables a logical breakdown with a task manager, solution generator and an evaluator, where each agent has dedicated capabilites. The key idea is that the task manager iteratively modifies the prompt and test cases to different difficulty levels conditioned on the previous attempt by the solution generator. This naturally fits the notion of adversarial learning where two different modules compete against each other to achieve an equilibrium.", "questions": "Address weaknesses above.\n\nAdditional comments: \n1. What is S(j+) in equation 2? \n2. Provide some outputs of TM agent with sample test cases. \n1. Can you upload the entire code snippet in D.1.6? \n2. You use different icons for plan and codes. Actually, the legend in Fig 2 maybe wrong -- the icon used to denote N parallel codes is marked as plan.\n3. The distinction between loop 2 and loop 3 in Fig 2 should be explicitly discussed somewhere in Sec 2. \n\nMinor nitpickings:\n1. Typos in line 15 algorithm 1: \"Equation equation 234\", \"Test sase\"\n2. Typo line 230: \"in Loop 3 of fig:Diagram\",", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. Very neat idea that can be plugged into any existing agentic framework and LLMs.\n2. By design, the framework allows for curriculum learning (although this is not explicitly mentioned by the authors) -- I think this is afFigure 5 strength where the difficulty level can be adjusted over iterations.", "weaknesses": "1. Line 276: \"TM adapts its weights for future evaluation\" -- what weights are being updated here. Make it clear that the Bayesian updates are over the choice of prompts/codes, and not over tokens. Also, sub-agents are not trained, so there are no weight updates to the LLMs. \n2. Are the test cases part of the plan that goes through loop 1 in Fig 2? Ideally, the domain expert should also provide some feedback on test cases during the planning phase. It is possible that the LLM generating test cases only generate dummy tests especially if the task is really difficult.\n3. Based on the example in D.2.3, the extra advice provided by the domain experts can be very specific modeling design ideas. How to ensure the extra advice do not divulge the ground-truth best solution?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VezEuUpmov", "reviewer": "Reviewer_FjrG", "summary": "The paper proposed a multi-agent framework for robust code generation. The robustness in the generated code is ensured by assigning roles to each agent in the framework, one agent sets increasingly difficult tests for the code while one aims to generate code to pass them. A third permits preemptive evaluation of generated code by predicting performance (by generalizing from existing data) rather than having to execute the code, saving time and computation resources.", "questions": "Does one of the agents iteratively generate test cases over iterations? If so, how are the weights over existing test cases, denoted by $\\lambda$ in Algorithm 1, used?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "- The underlying idea of gradually increasing the difficulty of tests that generated output must pass, where the update is done on the basis of actual external evaluation and not on the basis of potentially untrustworthy LLM assessment, is a very good one. And has been implemented well: being resource-efficient by evaluating only a subset of code generations.\n- Strong empirical performance of the proposed framework, across a diverse range of testbeds and use-cases.", "weaknesses": "- It is very unclear *where* the “bayesian update rule” is used. As in Algorithm 1, it suggests that it is used to re-weight existing test cases to present increasingly challenging ones as iterations proceed—that is, for the final prompt curation as each iteration. As per Appendix B, the rule is used for selecting a candidate code for evaluation (hence, by the evaluator agent).\n- Unclear algorithmic steps: Line 12 says the $\\lambda$'s are the test case weights, however they never get used in subsequent steps. The text suggests that they are updated based on $S_3^t$, however, the distribution over **existing** prompts (used interchangeably with test cases, seemingly) gets updated.\n- Missing details: In Appendix D.1, a case study of the proposed method vs Cursor and Windsurf is presented. But it is unclear what underlying base models are being used, which as per the results in Table 2, has a strong bearing on the final performance. Further, the framework will have *three* (same or different) base models while Cursor and Windsurf presumably work with just one. Which models are used by all these methods add important insight on the downstream performance. It is difficult to interpret the results and determine the effectiveness of the framework without these details.\n- Appendix B talking about “code embeddings” and a general setup for how GPs work. Details about how the code is embedded are lacking? What model is used to encode the code into embeddings?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oRq91MVyW7", "reviewer": "Reviewer_vXXH", "summary": "This paper introduces a novel AI4S (AI for Science) Low-Code Platform powered by a Bayesian Adversarial Framework. The goal is to empower domain experts by translating high-level natural language prompts into executable, domain-specific requirements, eliminating the need for intricate prompt engineering. The framework contains three parts, including a Task Manager, a Solution Generator, and an Evaluator. The framework iteratively refines the prompt through Bayesian update rule. Results show that the framework works well when the user prompt is written by non-experts and it also works well on general code generation questions.", "questions": "1. What would the token budget–performance trade-off look like compared to other methods?\n2. Could you provide more insights into the other modules? For example, how important is it to design the TM prompts as shown in the appendix? Which pieces of information are most important?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is clearly written and the ideas are easy to follow\n2. The code generation framework is novel and achieves SOTA performance on both AI4S benchmarks and code generation benchmarks\n3. Extensive experiments have been conducted to show more in-depth details about the framework", "weaknesses": "1. For the SciCode benchmark, the baseline is somewhat unclear, and comparing it to only a single baseline seems insufficient.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)