DARE-bench: Evaluating Modeling and Instruction Fidelity of LLMs in Data Science
Authors: Fan Shu
Yite Wang
Ruofan Wu
Boyi Liu
Zhewei Yao
Yuxiong He
Feng Yan
Venue: ICLR 2026 Poster (2026)
Keywords: LLM
Dataset
Benchmark
Data Science
Primary area: datasets and benchmarks

Abstract: The fast-growing demands in using Large Language Models (LLMs) to tackle complex multi-step data science tasks create a emergent need for accurate benchmarking. There are two major gaps in existing benchmarks: (i) the lack of standardized, process-aware evaluation that captures instruction adherence and process fidelity, and (ii) the scarcity of accurately labeled training data. To bridge these gaps, we introduce DARE-bench, a benchmark designed for machine learning modeling and data science instruction following. Unlike many existing benchmarks that rely on human- or model-based judges, all tasks in DARE-bench have verifiable ground truth, ensuring objective and reproducible evaluation. To cover a broad range of tasks and support agentic tools, DARE-bench consists of 6,300 Kaggle-derived tasks and provides both large-scale training data and evaluation sets. Extensive evaluations show that even highly capable models such as gpt-o4-mini struggle to achieve good performance, especially in machine learning modeling tasks. Using DARE-bench training tasks for fine-tuning can substantially improve model performance. For example, supervised fine-tuning boosts Qwen3-32B’s accuracy by 1.83× and reinforcement learning boosts Qwen3-4B’s accuracy by more than 8×. These significant improvements verify the importance of DARE-bench both as an accurate evaluation benchmark and critical training data.

Published as a conference paper at ICLR 2026
DARE-BENCH: EVALUATING MODELING AND INSTRUCTION FIDELITY OF LLMs IN DATA SCIENCE
Fan Shu
Yite Wang
Ruofan Wu'
Boyi Liu?
Zhewei Yao?
Yuxiong He²
Feng Yan
University of Houston
Snnofllke Al Research
ABSTRACT
The fast-growing demands in using Large Language Models (LLMs) to tackle complex multi-step data science tasks create an emergent need for accurate bench- marking. There are two major gaps in existing benchmarks: (i) the lack of standardized, process-aware evaluation that captures instruction adherence and process fidelity, and (ii) the scarcity of accurately labeled training data. To bridge these gaps, we introduce DARE-bench, a benchmark designed for ma- chine learning modeling and data science instruction following. Unlike many existing benchmarks that rely on human- or model-based judges, all tasks DARE-bench have verifiable ground truth, ensuring objective and reproducible evaluation. To cover a broad range of tasks and support agentic tools, DARE- bench consists of 6,300 Kaggle-derived tasks and provides both large-scale train- ing data and evaluation sets. Extensive evaluations show that even highly ca- pable models such as gpt-o4-mini struggle to achieve good performance, espe- cially in machine learning modeling tasks. Using DARE-bench training tasks for fine-tuning can substantially improve model performance. For example, supervised fine-tuning boosts Qwen3-32B's accuracy by 1.83 and reinforce- ment learning boosts Qwen3-4B's accuracy by more than 8x. These signifi- cant improvements verify the importance of DARE-bench both as an accurate evaluation benchmark and critical training data. Our data will be released at https: /tte:://giituuu wtm.au..ooooo//oooo Snwwf//Sowwwlaa lake Labs Maaeebbech
INTRODUCTION
OpenAl 2025a c Yang et al, 2025) are in- creasingly employed data-science (DS) agents to perform data reading, transformation, and mod- eling through tool-augmented code execution. Such a rapid adoption demands rigorous benchmarks evaluate and enhance the effectiveness and reliability in performing these complex, multi-step workflows. However, due to the cost and complexity of evaluation, existing benchmarks can only evaluate final-answer accuracy, and leaving other valuable metrics such as process fidelity and producibility largely unmeasured (Zhang et al., 2024 JJing et al. (0044). Meanwhile, many existing works (Guo et al. 2024, Zhang al.) 2023) Hong et al. 2024) in this area focus on using prompt engineering and workflow design to improve model performance.
Large language models (LLMs) (Anthropic 2025a b
Question
+ prediction.co wsvv
target Varisble, perform forecatinn...
D ground truth
Files
eetaadata
LLM Agent
Sandbox
+ eval results
Figure 1: DARE-bench defines each task by providing natural-language question and structured files (metadata and train/test splits). An LLM agent executes code within a sandbox to generate predictions, which are compared against ground truth for automatic and reproducible evaluation.
"Equal contribution. PPooject lead. Work done during Fan Shu's internship Snowflake Al Research.

---

Published as a conference paper at ICLR 2026
Table Comparison between DARE-bench and existing benchmarks.
Data File
Inst-
Time Series
Train Tasks
Benchmark
Domain
Verifiable
Tasks
follow
Meng (Huang
2023
Deep Learning
13
MLE-bench (Chan
Deep Learning
75
SWE-bench Umnnnzz
2024
Software Eng.
V
21,294
DS-1000 (Lai al. 2023
Data Science
X
X
1,000
Arcade Yun 2022
Science
X
1,082
Spider2 Ca0 2024
Data Science
X
V
494
DSEval Zhang
Data Science
X
V
825
DSBench Jing
2074
Data Science
X
V
S40
DA-Code (Huang
2024
Data Science
X
V
X
500
DataSeiBeneh [Zhhang al.
2025
Data Science
X
X
X
222
DABstep (Egg ct al.
20254
Data Science
X
V
X
450
DSBC KKdiyylla el
2025
Data Science
X
V
303
DARE-hench (Ours)
Data Science
V
V
6,300
We complement these works by taking benchmark approach to train LLM agents with high fidelity data and sophisticated yet reproducible evaluation to better acquire domain-specific skills in DS workflows.
Creating benchmarks that capture process fidelity for both training and evaluation is significantly challenging. The main challenge comes from two-fold. First, the sources for crafting training data (e.g., expert-level, executable DS process traces) are scarce and prohibitively expensive to acquire. Existing benchmarks largely rely on -uman--rocessed data and often center on Kaggle competi- tions, creating major data bottleneck. Second, evaluating "process fidelity" is highly non-trivial as randomness and environmental effects confound behavior, and verifying that an agent follows permissible DS practices requires controlled, instrumented harness. These challenges limit the data quality and evaluation scope of existing benchmarks, and thus miss the opportunities to better release the full potential of models.
To address the challenge of data quality and scarcity, we leverage LLMs to process auxiliary con- tent, such task descriptions, metadata normalization, rule extraction, instead of heavily relying on human involvement so that the data generation scalable with quality. We further improve Kaggle competitions to
the data quality with better diversity by pivoting from leaderboard-orient the broader pool of Kaggle datasets, yielding a more diverse and representative problem set such time-series domains. To address the evaluation challenge, seeds, reproducible environments) so that process fidelity is enabled by an outcome-based, verifi- able reward-enabling reinforcement learning (RLVR) instead of anman-involved reward. These approaches work coherently to construct a large-scale, trainable benchmark for data science that measures modeling performance and process fidelity, and boosts training performance.
engineer determinism (e.g., fixed
To this end, we introduce Datascience Agentic REasoning bench (DARE-bench), a training-focused DS agent benchmark featuring two verifiable task families: process-aware instruction-folloiinn tasks with ground truth from executing reference solutions that strictly follow the task instruction; and (ii) ML modeling tasks evaluated against the dataset's original ground truth under reproducible metrics. Our design for the instruction-followinn tasks leverages key advantage of data science: high degree of reproducibility. We find that by controlling the randomness and providing explicit instructions. procedurally faithful execution to robustly and automatically evaluate process fidelity by verifying the agent's 's final answer against the ground truth. As shown in Figure I for both task families, each task provides natural nntural-lannuage question and structured files. The LLMSs execute code within a sandbox to generate predictions, which is checked automatically for scoring. In Table we compare DARE-bench against existing benchmarks in terms of the task coverage, verifiability, training task support, and number of tasks to demonstrate DARE-bench's significant advancements.
produce deterministic outcome. This allows us
We conduct extensive evaluation on both strong general-purpose and code-centric LLMs. The eval- uation results reveal that many LLMs without task-aligned training fail miserably due process deviations, runtime errors, and metric mis missspccification For instance, Qwen3-32B baseline only
2

---

Published as a conference paper at ICLR 2026
Table 2: Overview of DARE-bench benchmark composition and the primary capabilities evaluated by each task type. Variants are denoted as IF = Instruction Following, MM = ML Modeling, XF = eXogenous Features, CF Canonical Forecasting.
Task Type
Train Tasks
Test Tasks
Capability Assessed
Classification-IF
1160
74
Instruction following
Classification-MM
1160
74
ML Modeling
Regression-IF
899
45
Instruction following
Regression-Me MM
899
45
ML Modeling
Time-series XF
915
57
Predictive ML, forecasting
Time-series-Cr
915
57
Predictive ML, forecasting
achieves a total score of 23.25, while the smaller Qwen3-4B baseline performs even worse which scores 4.39. By contrast, DARE-bench bridges this gap by providing training-focused benchmark with verifiable large-scale training data and useful and sophisticated reproducible evaluation. Su- pervised fine-tuning yields absolute gains of nearly 20 points, while reinforcement learning boosts Qwen3-4B from 4.39 to 37.40. Overall, DARE-bench significantly improve success rates, process adherence, predictive performance, and robustness across a variety practical data science tasks.
RELATED WORK
LLM Agents. Research into Agentic LLMs focuses on their ability as independent agents through planning, tool calling, and memory capabilities. The integration of reasoning with actions or APIs occurs through ReAct (Yao et al. 2023) and Toolformer (Schick al., 2023) frameworks as re- searchers work multi-agent collaboration and autonomous tool-augmented systems. Applying these to real-world data science remains difficult because current benchmarks lack adequate training
resources and often omit critical domains such time-series forecasting the distinction between open-ended problem solving and strict instruction caaruction-folooinggg
LLMs for Coding and Data Science Benchmarks. The advancement of coding benchmarks de- pends on the use of testable pass/fail signals. The HumanEval (Chen et al. 2021) and MBPP Austin et al., 202T) provided short self-contained functions with hidden unit tests while SWE:-benh (Jienne et al. (0024) tests models on actual GitHub issues that need multiple file modifications and complete project testing. The community now performs end-to-end data science (DS) tasks as its new approach to this paradigm. The DS-1000 (Lai et al. (023) teaches NumPy/Pandas program- ming but DSBench (Jing et al.) 2024) and MLE-bench (Chan et al. 2024) use Kaggle competition problems which require multi-step analytics. The DABstep (Egg et al. 2025b) dataset contains 450 financial tasks from real-world applications and DataSeilBnnch (Zhang al. 2025) uses Task- Function-Code (TFC) to evaluate programs which are then verified by human evaluators. DSBC (Kadiyala et al. 2025) addresses private datasets via structured metadata. The research uses Chen et al. (2024) to evaluate visualization skills and Bendinelli et al. (2025) to assess data cleaning abil- ities and Kaggle leaderboards (Grosnit et al., 2024; Chan et al. (024) to measure performance. The benchmarks show a sequential development from basic unit testing code sophisticated tool-based agents which perform complete DS workflows and produce quantifiable results.
Reinforcement Learning with Verifiable Rewards. The implementation of verifiable program- matic signals in reinforcement learning enables model training at scale without requiring preference data. The automatic checking system consists of unit tests and solvers and execution traces for math and code verification. GRPO (Shao et al. 2024) achieves lcarning stability through its relative rollout feedback system which DeepSeek-RI (Guo et al.) (025) and GPT o-series (OpenAl (025d) extend by verifier-enhanced objectives. The methods combine symbolic proofs with coding tests and retrieval/search execution graphs improve reward-as checkerr for both correct answers and verifiable reasoning trace generation.
3

---

Published as a conference paper at ICLR 2026
3 DARE-BENCH
DARE-bench consists of three data science task-families classification, regression and time-series forecasting, each with two variants that probe distinct agent capabilities. For clarity, we denote these variants using intuitive abbreviations: IF (Instruction Following) and MM (ML Modeling) for classification and regression; XF (eXogenous Features) and CF (Canonical Forecasting) for time- series forecasting. In classification and regression, the IF variant emphasizes by requiring LLM to faithfully reproduce reference workflows, whereas the MM variant targets ML modeling with outcome-based evaluation. These variants capture complementary real-world needs. IF simulates workflow where an agent must strictly execute senior scientist's detailed design. Conversely, MM reflects an silcome-drivenn scenario where customers only care about the final accuracy, granting full freedom to the LLM. For time-series forecasting, the distinction between the two variants more nuanced: in the XF variant, we retain not only the timestamp and entity identification columns but also all exogenous features from the original dataset; in the CF variant, however, while exogenous features remain available for training, the test set constrained only the timestamp and entity columns, making it closer to a classical forecasting setup. We partition our collection of 6,300 tasks into an approximately 95/5 train/test split, designating the most recently updated tasks as the test set. Table aable]ssumarizer the dataset scale and the primary capability assessed in each task type. Tool schema and task examples are shown in Appendix K
cnteruction-oolowiing
3.1 DATASET CURATION
Data Scourcing
@ Task Design
3 S Post-Process
4
Finalization
Kaggle API + Y License Size Filter
Identify targets Feasibiltyy check
Train split Extra preparation
@ Sandhor varification @ Final package
Figure 2: Automated pipeline of DARE-bench. The construction process consists of four stages: (1) Dataset Sourcing, where Kaggle datasets are filtered by tags, license, size, and metadata; (2) Task Design, where schema summaries, targets, features, and feasibility
analyzed with the help of LLM; (3) Post-Process, including splitting, noise injection for IF tasks or resampling or entity checks for Lime-series-CF tasks; and (4) Finalization, which validates solvability in a sandbox for IF tasks and produces standardized benchmark artifacts.
To construct DARE-bench, we design an automated data curation pipeline that systematically trans- forms raw Kaggle datasets into standardized machine learning tasks. Unlike prior benchmarks which rely mainly on manual curation, our approach integrates web crawling, L----bseed task formulation, controlled data transformations, and sandbox verification to ensure both quality and scale, Shown in Figure 2 the pipeline consists of four stages. Detailed prompts are shown in Appendix I
Dataset Soureing with Augmented Metadata. We selected Kaggle as the primary data source due to its breadth of real- world, user-contributed datasets. The official API of Kaggle retrieves candidate datasets that meet specific criteria including tabular format and valid open license. Additionally, develop lightweight web crawler to extract additional data from webpage descriptions that were present in the dataset, providing additional metadata elements to the LLM through column previews and natural-language deseriptions which help the model understand the context of the task formulation.
LLM-Assisted Task Design and Feasibility Analysis. For each sourced candidate dataset, we employ an LLM to assess whether it can support a well-posed predictive task. The model receives both the dataset preview and the detailed description The LLM detects a target column which can be either categorical or continuous for classification and regression tasks along with structured features and their corresponding data types. For time-series forecasting tasks, the model detects timestamp columns and numerical targets that evolve through time and exogenous features addition to identifying the temporal frequency of the data. Only datasets deemed feasible by this automated analysis proceed to the next stage.
duplicate expert assessment on large scale.
Post- Proc-Pr Feasible datasets are then transformed into uniform benchmarking tasks. The data is split randomly into training and testing sets. For instruction-follovii tasks, controlled noise is injected into roughly twenty percent of the training data, which simulates real-world data quality
4

---

Published as a conference paper at ICLR 2026
issues through numerical values that exceed valid ranges and unexpected categorical entries, and the testing set serves as the clean reference data. The chronological split method is used for time-series forecasting to preserve the natural order of time in the data. LLM then detects entity identifiers to stop data leakage between groups and it performs automatic resampling of irregular time series data to uniform intervals through an aggregation method suggested by the model.
Finalization. After the post-process step, for instruction-followiii tasks, the validation process for each task runs independently in a sandbox environment by executing the reference solution code sequence including data loading, preprocessing, training, and prediction generation. Since these tasks rely on reference outputs rather than fixed ground truth values, the sandbox ensures that the instructions can be faithfully executed and the generated predictions are fully reproducible under the same random seed. contrast, MI. modeling tasks directly use ground-truth values (e.g., class labels or numerical targets) for evaluation and do not require sandbox execution. Finally, the task is packaged into a standardized format that includes training and testing data, metadata describing the dataset and task, the natural language task description, and the corresponding reference.
3.2 TASK FORMULATION
Input and output. Suppose we have the task description Q. an accompanying dataset description testing set without target values Deest = {x: ) and access to code execution tool T. The tool T enforces maximum wall-clock runtime T max. while the agent is subject to an interaction budget of K turns. Given these inputs and constraints, g produces executable code C, which is run within T on Duain to fit model and subsequently on Drest to generate predictions y, i.e., ý = G(Q Dimin: Drest, M, (TTmax: K)).
M. training set Puain {(x,, y:) Mimis
Evaluation metrics. We evaluate models differently depending the task type. For instruction- following tasks (i.e., Classification-IF and Regression-IF), we compare the model's generated pre- diction y against the simulated reference output Yief obtained from the reference solution code Cress and assign a score of y = Yver and 0 otherwise. For ML modeling tasks, including Classification- MM, Regression- MMM, and both Time-series-Xe XF and Time-series-CC -C, we directly compare the model predictions ý against the masked ground -ruth values Ygt- Specifically, we adopt the macro-Fi score for classification- MM tasks to account for class imbalance, and use the clipped coefficient of deter- mination for regression and time-series forecasting, defined as clip( (²²) min [1, max (0, R²)) }. For tasks with multiple prediction targets, the evaluation metrie is computed by averaging targets. Details of our reference solution code can be found in Appendix J Jand calculation of macro- Fl and R2 can be found in Appendix D
all
3.3 FEATURES OF DARE-BENCH
DARE-bench introduces several key features that distinguish it from prior benchmarks in data sci- ence and machine learning:
ML Modeling and Instruction Following, DARE-bench differs from other existing benchmarks because assesses two fundamental data science capabilities which are essential for weal-world ap- plications: ML modeling and task instruction following for data processing and model development.
Verifiable Ground Truth. The evaluation process of DARE-bench depends on actual labels and simulated reference solution outputs to produce results that can be replicated. The system design removes all dependencies on human judgment and model-based assessments that enables evaluation metrics to directly assess task performance. This design is similar to coding benchmarks such as SWE-bench (Jimenez et al. (2024) and math benchmarks like AIME (Balunović et al.) (2025), making extremely suitable for supervised fine-tuning (SFT) and reinforcement learning with verifiable rewards (RLVR).
Dual Role as Evaluation and Training Resource. The benchmark offers a training dataset which enables users to perform model fine-tuning and aligmment. As we will demonstrate in Section 5 the models trained on DARE-bench achieve better results than their baselines, which proves that the dataset serves as a benchmark and a resource to improve data science L.Mss.
Diversity, Realism, and Practical Constraints. Our datasets are created from Kaggle sources, making them naturally diverse, multilingual, and spanning various domains while capturing real-
5

---

Published as a conference paper at ICLR 2026
Table 3: Distribution of task domains across the DARE-bench train and test sets.
Dataset
Finance
Health
Business
Technology
Autemotive
Education
Environment
Others
Train
16.9%
10.2% 8.4%
7.3% 8.2%
4.0% 5.6%
4.5% 3,3%
2.8% 3.1%
6.8% 2.4%
47.5%
Test
17.1%
51.9%
Table 4: Hyperparameter sensitivity analysis for 04-mini across different turns and sandbox maxi- mum execution time limit configurations.
turss
time
class-II
class-MM
reg- II
reg-
time-XF
300
37.16
55.44
29.71
51.69
37.99
200
67.56
57.89
53.62
57.60
42.29
9.67
180
73.42
61.07
63.76
60.92
41.59
9.79
120
73.87
61.42
65.21
61.05
42.11
8.82
100
75.22
63.36
62.31
62.07
42.03
10,97
100
76.80
65.88
66.66
62.41
40.03
9.92
world challenges such class imbalance, missing values, and noise. As illustrated in Table quantitative analysis confirms this broad coverage, showing that DARE-bench spans a wide spec- trum of real-world verticals across both training and test sets. Details in categorization can found in[Appendix M| In addition, enforced constraints-sut as a 10-minute execution limit and bounded tool invocation turns reflect realistic user expectations for efficient, accurate solutions. See more details on Appendix H
4 EVALUATION
In this section, we present the experimental results and analysis of several LLMs evaluated using DARE-bench.
4.1 EXPERIMENT SETTINGS
We experiment with state-of-the LLMs from open-source ones such as Qwen3-32B and Qwen3- 4B (Yang et al., 2025). to proprietary models such as gpt-o4-mini (OpenAI, (025d), gpt-4o, gpt-
4.1 AT 2025a), gpt-5 (OpenAT 2025b), Claude-Sonnet-3.7 (Anthropic 2025a), and Claude-
Sonnet-4 Anthrooicc 20255).
For all the experiments, we employ greedy decoding strategy whenever applicable, along with sandbox (ByteDance-Seed Foundation Code Team (2024) for code execution. To reduce random- ness, each task repeated three times and we report the average score. We evaluate all tasks using either accuracy or the macro-FIle F1/clippe R score. The "classification-I" and 'regression-IP met- rics are measured using strict, binary (0/1) accuracy. 'clssification-MMMM is measured using graded (0.0-1.0) macro-Fl score. The remaining metrics, 'regression-MM'' 'time-series-XF"; and 'tieesseriss-CF are all evaluated using the clipped R²2 score.
We conduct our evaluation in two stages. First, we perform sensitivity analysis on the key hy- perparameters for our evaluation framework using one of the most advanced models, gpt-o4-mini, specifically turns and sandbox maximum execution time. These limits are set simulate real-world applications, as user would not wait infinite time for an agent to complete a task. Our goal to find balanced configuration. Second, with this configuration, conduct comprehensive comparison of several leading LLMs on our benchmark.
4.2 HYPERPARAMETER SENSITIVITY ANALYSIS
The results, shown Table 4 clearly indicate clear trend emerges: performance generally im- proves with a higher number of interactive turns. We observe a dramatic leap in performance when moving from 3-turn configurations to 5-turn configurations. For example, the classification-II score jumps from 37.16 (at 3 turns, 300 s) to 67.56 (at 5 turns, 200 s). This suggests that allowing the agent more opportunities iterate and refine approach is crucial.

---

Published as a conference paper at ICLR 2026
Table 5: Main evaluation results benchmark (test tasks) under the configuration where turns
set as S and sandbox maximum execution time set as 200 The best score in each column is bolded
Model
class-IF
class-MM
reg.
reg. reg-MM
time-Xt
time-C2 CF
gpt-4o
32.88
40.45
20.28
40.60
35.54
4.77
gpt-4.1
55.82
57.83
52.17
58.62
40,78
6.60
gpt-5
69.81
43,40
57.24
56,29
36.83
10.13
gpt-o4-mini
67.56
57.89
53.62
57.60
42.29
9.67
Claude-Sommet-3.7
61.48
61.03
46,37
63.20
49,88
13,70
Claude-Somnet-4
16.21
18.27
15.21
11.33
4,80
0.01
Qwen3-32B
17.11
30.71
15.21
35,86
26.96
0.00
Qwen3-49
3.60
5.23
0.72
3.29
6.97
0.00
The highest performance on classification-IF (76.80) was achieved at the (15 turns, 100 s) setting. However, for our main model comparison, sought balance between performance and computa- tional efficiency (i.e., cost and latency). We selected the (5 turns, 200 s) configuration as our standard setting. This configuration (5 turns, 200 s) serves robust and practical baseline; it significantly outperforms 3-turn setups and achieves strong, representative scores across all metrics (c.g., 67.56 on classification-IF. 53.62 on regression-IF, and 42.29 on time-series-XFF within practical time constraint, i.e., approximately 1000 user wait time in total.
4.3 MODEL COMPARISON
Based on our sensitivity analysis, we adopt the (5 turns, 200 s) configuration for a comprehensive comparison of all models. The main results are presented in Table Statistics the average token usage and the number of tool invocations are listed in Appendix K
In this standardized setting, Claude-Sommet-3/. emerges the top por-ffoonnnng model. It achieves the highest scores on four of the six evaluation metrics: "classification-IF" (69.81), "classification-MM (61.03). 'regression-MM' (63.20), 'time-series-XF" (49.88) and 'time-seriesCFF" (13.70), demon- strating its strong overall capabilities for this benchmark. gpt-5 leads the two IF columns, achieving the highest "classification-IF" (69.81) and 'regression-IF" (57.24).
The results also reveal marked disparities between model generations. Claude-Sonnet-4 under- performs significantly compared across all metrics. A key reason is that grained substeps, executing almost every small operation separately. As result, completing single benchmark task often requires very large number of steps, and the model nearly always exceeds the allowed step limit, leading to premature failures. Meanwhile, among the open-source models, Qwen3-32B and Qwen3-4B perform far below the proprictary models, struggling in all categories and failing entirely on Lime-series-CE This highlights that complex, multi-step data analysis sand- boxed environments remains considerable challenge for current open- LLMs.
predecessor Claude-Sonnet-3.7, with notably weaker scores tends to decompose tasks into very fine-
Moving beyond the quantitative scores in Table understand why models fail on our benchmark, conducted systematic qualitative analysis of failed trajectories. Our goal is to identify the primary bottlenecks and limitations of current SOTA agents.
Incorrect Tool Argument Passing. A fundamental failure mode observed that LLMs often failed to correctly interface with the code-execution tool. While the generated Python code was logically correct, they frequently mismatched tool parameters (e.g., forgetting to pass filenames), causing execution to fail before code could run. Definition of our tool can be found in Appendix K
Instruction Following Failures. LLMMs often ignored explicit constraints: processing steps in the wrong order, skipping required transformations, or omitting critical function arguments (Figure These emrrrs show weak adherence task specifications.
Flawed Reasoning in Open-Ended Tasks. More subtle problems came from brittle reasoning. Common issues included misuse of metadata (hard-coding values), risky preprocessing (e.g.. naive
Due to licensing restrictions on specific source datasets, this table reports the performance on henchmark. We provide the performance on the strictly open-sourceable subset in Appendix F.

---

Published as a conference paper at ICLR 2026
Task
Python Code
Task Question
Use tran porquzt and ricodatt bd to noure spect sis daly tetween 10 a NLPChasse makel
stauture ady [...
sepectal ranges (TTia, spect Alsza's 10.0, 10.403
random
geneinte
('mpuiie's Stmplalopeees Excreeigg
(acalar', Standon dicaled ())
Train.cs csV
Test.cov
progrosmoon Colamntronnieraee
traatforeeen (1 noa traaataeeee EEA, feacoree) 3>
Alrie
Carmonommnn prepresesse))
('lassttiarr' (CCllanttee))
Medadata. Ixt
Taa dinwe of
Evaluation
Pradiction
Grondd Trush
eegeet henomallyy browet
100 oriust uuldie wanatr
Crrowert
Cirovt
Fall
Ockure desoppees
Vilseen
Rlaven
Figure 3:
Example of an instruction-folloiing task where the agent fails respect explicit con- straints. Despite being asked to fix the random seed, the model omitted the required argument, leading to incorrect predictions and an evaluation failure.
Table 6: Fine-tuning and RL improve performance over baselines. gains compared to the baseline of the same model.
Superscripts denote absolute
Model- Perf
Model
Setting
class-IF
class class-MM
reg-IF
reg-MM MM
time-XF
time-CF
Total
Qwen3-32B
Baseline
17.11
30.71
15.21
35,86
26.96
0.00
23.25
65.03
Qwen3-32B
SFT-FV
40.54-2145
44.71-1389
42.75-m0:5
49.21-13335
39.95+1299
0.07+9.0
42.42-19.7
72.32
Qwen3-32B
SFT-AV
40.54:2143
47.20 16.49
42.02
36.1
55.56 19.70
33.56 '6.6
0.00
42.91
19.72
70.27
Qwen3-32B
SFT-BV
38.06 2095
48.91 18.20
42.75
27.44
$4.55-18.0
35,91 18.95
0.00
42.83 19:58
15.01
Qwen3-32B
SFT-DV
38.55 38.58-21447 43.82-13.1
39.13*
$1.00*
15
38,92 96
0.00
41.12-17:08
71.68
Qwen3-4B
Baseline
3,60
5.23
0.72
3.29
6.97
0.00
4.39
54,18
Qwen3-4B
RI.
38, 38.96 35.36
39.44"34.11
31.88
37.02133.7
32.28
2.28*222
37.40
62.55
label encoding, mishandling NaNs), and unreliable type inference. Such shorteuts led fragile pipelines and frequent errors.
Time-Series Task Failures. Performance 'time-series-CF was especially poor. Reflecting lack of exposure to complex time-series reasoning. LLMs often failed to produce valid output formats or relied on trivial heuristics (last value, mean), resulting in near-zero predictive accuracy.
This qualitative analysis reveals that current agent failures are multi-faceted. They range from ba- sic API misuse and poor instruction following to, most critically, a lack of robust, generalizable reasoning for complex tasks. The widespread use of brittle preprocessing and the near-total failure on complex t time- seriess formatting suggest that current agents, while proficient at simple code generation, still lack the deep, domain-spectnn reasoning required for autonomous data science.
5 FINE-TUNING LLMS WITH DARE-BENCH
To further strengthen the performance of foundation LLMs on DARE-bench, we explore two com- plementary training paradigms: supervised fine-tuning (SFT) and reinforcement learning (RL). SFT leverages curated supervision from rejection-sampli traces to align models more closely with task requirements, while RL directly optimizes models with verifiable outcome rewards. The following subsections detail each approach, their implementation, and the improvements they yield.
Rejection Sampling and Supervised Fine-tuning. To obtain high-quality supervision signals, we egjectionssaple traces generated across multiple runs, using task-specific filtering strategies.
We generate data for supervised fine-tuning through rejection sampling using task-independent fil- ters that evaluate trajectories for validity, quality, and speed. A trajectory is valid if it achieves exact match for IF tasks or exceeds type-specific score threshold for predictive tasks. A task is con- sidered diverse if its sampled runs contain both successes and failures (IF) or if the variance of its
8

---

Published as a conference paper at ICLR 2026
Table 7: Ablation study the impact of Instruction Following (IF) and MI. Modeling (MM) data with SFT-DV rejection sampling data.
Train Data
class-IF
class-MM
reg-IF
reg-MM
time-XF
time-CF
haseline
17.11
30.71
15.21
35,86
26.96
0.00
IF
40.99
22.38
47.8223261
27.85 801
23.83 9.13
0.00
MM
11.71-540
45. 45.69-14.98
18.84-166
45.38+9.52
34.12*7.16
0.00
IF+M
38.58
43.82 11..1
39.13
51.00
38.92 -1.92-11.6
0.00
Table 8: External validation DSBench (Jing al. 2024) after converting tasks into our format. Superscripts denote absolute gains over the baseline of the same model.
Model
Setting
Competitioo--ever Accuracy
Qwen3-32B
Baseline
32.38
Qwen3.322 32B
SFT-FV
37.82*5-44
Qwen3-32B
SFT-AV
41.08 +8.70
Qwen3-32R
SFT-VV
40.06 7
Qwen3-32B
SFT-DV
42.41 -1008
Qwen3-4B
Baseline
18.23
Qwen3 4B
RL
40.00-21.27
predictive scores exceeds threshold. We study four strategies: FV (Fastest- which keeps the quickest valid trace for each task; AV (All- Valid), which retains all valid traces; BV (Best- Valid), which for diverse tasks selects the single best valid trace; and DV (Duo-Valid), which for diverse tasks retains the top-2 valid traces (fastest for IF, highest-scoring above the mean for predictive). Both IF and predictive tasks their natural evaluation metrics (exact match or macro-F1 clipped R²) to define validity and rank trajectories. More details are provided in Appendix L
Reinforcement Learning. We perform reinforcement learning with GRPO (Shao al. 2024) on Qwen3-4B (Yang et al. 2025) using the DARE-Bench training tasks with the verl (Sheng et al. 2025) framework. During training, we found that the group normalization used in GRPO intro- duces training instability. Therefore, we chose to remove the normalization component similar to DrGRPO (Liu et al., 2025), which mitigates the training stability issue. Moreover, we use sequence- level aggregation as in the original GRPO, rather that token-level aggregation used by DAPO Yu ct al. 2025). Additional training details can be found in Appendix G
Fine-tuning Results. Table 6 summarizes results for both SFT (Qwen3-32B) and RL (Qwen3- 4B). Specifically, Model-Perf measures the quality of the model's predictions by focusing solely on successful attempts for MM tasks. This metric isolates the quality dimension from the validity dimension, confirming that fine-tuning improves the model's 's actual data science proficiency, not just its adherence syntax rules. Across IF and MM tasks, fine-tuning yields substantial improvements over the baseline, with absolute gains of nearly 1.83x in total score and 10% in ModelPerf. Different strategies bring complementary benefits: AV yields the strongest overall improvements for MM tasks, while FV favors IF tasks. Reinforcement learning on Qwen3-4B provides even larger relative gains, boosting the total score from 4.39 to 37. 57.40 and ModelPerf from 54.18 62.55. These results confirm that DARE-bench not only improves instruction following but also translates into better downstream modeling accuracy once correct predictions are generated.
Impact of Data Composition. As shown in Table 7 we use SFT-DV to further investigate the spe- cific contributions of IF and MM data through an ablation study. Training exclusively on MM data boosts predictive modeling performance but degrades instruction adherence, while training solely on IF data shows the inverse. Only the combined approach successfully integrates both capabili- ties, achieving robust balance. This confirms that process- -oriented and outcome-oriented tasks are complementary and essential for a comprehensive data science agent.
Failure Analysis. Shown in Table 9 we categorized incorrect trajectories to identify specific rea- soning bottlenecks. Proprietary models mainly face problems with Code Errors, while open-source baselines frequently exceed execution limits because of inefficient exploration. Training DARE- bench effectively mitigates these issues; notably, RL on Qwen3-4B reduces code errors by 48 percent

---

Published as a conference paper at ICLR 2026
Table 9: Failure mode analysis across different models.
Model
Inst Adhere
Code Firor
Code Exec Limit
Max Taken Limit
gpt-5
126
333
3.7
158
262
Owen3-328
106
257
Qwen3 32B SFT-DV
4.3
80
236
256
Qwen3-4B
79
174
102
Qwen334B RT.
49
91
331
119
Table 10: Comparison between Native Function Cal and DataWiseAgent on DARE-bench and DSBench.
Framework
Model
class-IF
class-MM
reg-IF
reg-MM
time-XP
time-CF
DSBench
Native Function Call
Qwen3-32B
17.11
30.71
15.21
35.86
26.96
0.0
32.38
Native Function Call Qwen3-32B SFT-DV
38.58
43.82
39.13
51.00
38.92
0.0
42.41
Data AnseWitttgent
Qwen3-32B
21.62
29.63
34.78
34.40
30.45
0.0
29.17
and halves code execution limit errors, demonstrating that our supervision significantly enhances both code correctness and efficiency.
Case Studies of Fine-tuning Effects. To further illustrate the benefits of fine-tuning, we highlight two representative failure modes that were substantially reduced. First, common pre-fine-tuning er- ror occurred when L.M provided tool with incorrectly generated tool arguments. The code executor tool requires three explicit arguments including code, input file and output file. However, LLMs fre- quently generated correct Python code that opened files but failed to pass the filename into the tool's file.to.load argument, causing sandbox execution to fail. After fine-tuning, the frequency of such mismatches decreased remarkably. Second, the baseline models tried to use natural-language column names from the task description without checking the provided me tadata txt which led to KeyErrors. The first step of the fine-tuned models involved examining the metadata file for references actual column identifiers which led to the development of reliable executable solutions.
External Validation and Comparison. To further assess generalization and compare with state- of-the-art specialized agents, we adapt data modeling tasks from DSBench (Jing et al.) 2024) into the DARE-bench task format. As shown in Table g all fine-tuned versions outperform the original baseline, proving that DARE-bench enhances performance beyond in-domain tasks. Specifically, inclusive sampling methods (AV and DV) yield the most significant improvements by leveraging wider range of valid traces compared stricter filtering
(FV and BV). Furthermore, we compare our fine-tuned models with DataWise (You et al. 2025) under identical settings. As detailed Table TO our model compare favorably to DataWisee Agent, achieving a score 42.41 compared to 29.17. This demonstrates that framework offers competitive adaptability and robustness in diverse data science workflows compared to existing specialized agents.
CONCLUSION AND FUTURE WORKS
We present DARE-bench, a training- focuuee benchmark for DS agents which enables executable evaluation and trainable supervision through two verifiable task families: (i) process- instruc- tion following with reference-code ground truths, and (ii) ML modeling with dataset ground truths. The 6,300 Kaggle-derived tasks show poor performance from strong general-purpose LLMs until they receive task-specific data but fine-tuning on DARE-bench artifacts produce reliable and repeat- able enhancements in process fidelity and predictive performance and execution failure reduction. Our design uses the exccutable-benchmark. approach which software engineering professionals have adopted solve DS-specific problems that recent evaluations have identified.
We will expand our task type coverage (figuree/seecche//cuuteenen straints and verifier-basel objectives, and add anomaly detection tracks (tabular and seress with appropriate event/segment-level metrics and weak/unsupervised scoring protocols.
strengthen procedural con-
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work is partially supported by NSF CAREER-2305491, We would like to thank Jeff Rasley for his help with the open-source release. We would like to thank the Area Chair and reviewers for their valuable feedback and suggestions.

---

Published as a conference paper at ICLR 2026
REFERENCES
Anthropic. Claude sonnet 3.7.
https://cloud..
generative iineraativeaa/doos caclossppattnn pesaaterrmmdeess/llee
2025a. Extended thinking mode,
step-by-step reasoning capability.
Anthropic.
Introducing claude sonnet http::/www.attioppc..ooom
2025b. Improved coding and reasoning, available via APT Bedrock Vertex.
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang. Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint (rXi::2108.07732 2021.
Mislav Balunović, Jasper Dekoninck, Ivo Petrov, Nikola Jovanović, and Martin Vechev. Matharena: Evaluating ilms uncontaminated math competitions. arXiv preprint wwxiv:2505.23285 2025.
Tommaso Bendinelli, Artur Dox, and Christian Holz. Exploring IIm agents for cleaning tabular machine learning datasets. arXiv preprint arXiv:2503.0666 2025.
ByteDance-Seed Foundation Code Team. Sandhoxfusion: A multi-language code sandhox exe- cution tool for evaluating code generation models. https://githai: com/bytedanceelleeeccoccccaaoace SandboxF usion 2024. Used in FullStack Bench: Evaluating LLMs as Full Stack Coders.
Ruisheng Cao, Fangyu Lei, Haoyuan Wu, Jixuan Chen, Yeqiao Fu, Hongcheng Gao, Xinzhuang Xiong, Hanchong Zhang, Wenjing Hu, Yuchen Mao, et al. Spider2-v: How far are multimodal agents from automating data science and engineering workflows? Advances in Neural Informa- tion Processing Systems, 37:107703-107744, 2024.
Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung. Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, et al. Mle-bench: Evaluating machine learn- ing agents on machine learning engineering. arXiv preprint arXiv:2410.0709, 2024.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 2021.
Nan Chen, Yuge Zhang, Jiahang Xu, Kan Ren, and Yuqing Yang. Viseval: A benchmark for data visualization in the era of large language models. IEEE Transactions on Visualization and Com- puter Graphics, 2024.
Alex Egg, Martin Iglesias Goyanes, Friso Kingma, Andreu Mora, Leandro von Werra, and Thomas Wolf. Dabstep: Data agent benchmark for multi-step reasoning. arXiv preprint arXiv:2506.23719, 2025a.
Alex Egg, Martin Iglesias Goyanes, Friso Kingma, Andreu Mora, Leandro von Werra, and Thomas Wolf. Dabstep: Data agent benchmark for multi-step reasoning, 2025b. URL https:
//arxiv..rg//bs//5555 2ttp///2556.2211999
Antoine Grosnit, Alexandre Maraval, James Doran, Giuseppe Paolo, Albert Thomas, Refinath Shahul Hameed Nabeezath Beevi, Jonas Gonzalez, Khyati Khandelwal, Ignacio lacobacci, Ab- delhakim Benechehab, al. Large language models orchestrating structured reasoning achieve kaggle grandmaster level. arXiv preprint S0Miv::2111.33566 2024.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in lims via reinforcement learning. arXiv preprint arXiv:2501. 1244, 2025.
Siyuan Guo, Cheng Deng, Ying Wen, Hechang Chen, Yi Chang, and Jun Wang. Ds-agent: Au- tomated data science by empowering large language models with case-based reasoning. arXiv preprint 4r21v:2402.17453, 2024.
Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Chenxing Wei, Danyang Li, Jiaqi Chen, Jiayi Zhang, et al. Data interpreter: An Il agent for data science. arXiv preprint 4002 10..224222..97... 18679, 2024.
12

---

Published as a conference paper at ICLR 2026
Qian Huang, Jian Vora, Percy Liang, and Jure Leskovec. Mlagentbench: Evaluating language agents on machine learning experimentation. arXiv preprint arXiv:2310.03302, 2023.
Yiming Huang, Jianwen Luo, Yan Yu, Yitong Zhang, Fangyu Lei, Yifan Wei, Shizhu He, Lifu Huang, Xiao Liu, Jun Zhao, al. Da-code: Agent data science code generation benchmark for large language models. arXiv preprint arXiv:2410.07331, 2024.
Carlos E. Jimenez, John Yang, Alexander Wettig,
Shunyu Yao, Kexin Pei, Ofrr Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues?, 2024. URL
hittps://ariiss wttp://.rggbbbb////// 066770
Liqiang Jing, Zhehui Huang, Xiaoyang Wang, Wenlin Yao, Wenhao Yu, Kaixin Ma, Hongming Zhang, Xinya Du, and Dong Yu. Dsbench: How far are data science agents from becoming data science experts? arXiv preprint arXiv:2409.07703, 2024.
Ram Mohan Rao Kadiyala, Siddhant Gupta, Jebish Purbey, Giulio Martini, Suman Debnath, and Hamza Farooq, Dsbc: Data science task benchmarking with context engincering, arXiv preprint
2rXiv:2507.23336, 2025.
Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang. Ruiqi Zhong, Luke Zettlemoyer, Wen-tau Yih, Daniel Fried, Sida Wang, and Tao Yu. Ds-1000: A natural and reliable benchmark for data science code generation. In International Conference on Machine Learning, pp. 18319-18345. PMLR, 2023.
Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding rl-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783, 2025.
OpenAI. Introducing GPT-4.1 in the api. htttps://peeaai com/c com/innex/gptt44--1 April 2025a. Lower-latency, cheaper GPT model family optimized for instruction following and long context.
OpenAl. Introducing GPT-5,
Au-
gust 2025b. Latest flagship model with improved code and agentic performance.
OpenAl. Introducing openai o3 and 04-mini. April 2025c.
http:://openal.com/nnnc
OpenAl.
Openai o3 and 04-mini system
card.
https://cdn:
openai. Sreemai.com//pff/22213333 02dc 4789 800b 77758f3722c1
a-3-and-2n miniini---- weaiiniissttmmmmmmccm card. pdf April 2025d. System card describing capa-
bilities, evaluations, and safety considerations.
Timo Schick, Jane Dwivedi-YY Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36:68539- 68551, 2023.
Zhihong Shao, Peiyi Wang. Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models. arXiv preprint arXiv:2402.033300 2024.
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, pp. 1279-1297, 2025.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui. Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
13

---

Published as a conference paper at ICLR 2026
Pengcheng Yin, Wen- Li, Kefan Xiao, Abhishek Rao, Yeming Wen, Kensen Shi, Joshua How- land, Paige Bailey, Michele Catasta, Henryk Michalewski, et al. Natural language to code gener- ation in interactive data science notebooks. arXiv preprint arXiv:2212.09245: 2022.
Ziming You, Yumiao Zhang, Dexuan Xu, Yiwei Lou, Yandong Yan, Wei Wang, Huaming Zhang, and Yu Huang. Datawiseagent: A notebook-centric Ilm agent framework for automated data science. arXiv preprint arXiv:2503.07044 2025.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint arXiv:2503. 14476, 2025.
Dan Zhang, Sining Zhoubian, Min Cai, Fengzu Li, Lekang Yang, Wei Wang, Tianjiao Dong, Ziniu Hu, Jie Tang, and Yisong Yue. Datascibench: An Ilm agent benchmark for data science. arXiv preprint arXiv:2502. 0225502.13997 13897, 2025.
Lei Zhang, Yuge Zhang, Kan Ren, Dongsheng Li, and Yuqing Yang. Micopilot: Unleash- ing the power large language models in solving machine learning tasks. arXiv preprint arXiv:2304.1497 2023.
Yuge Zhang, Qiyang Jiang, Xingyu Han, Nan Chen, Yuqing Yang, and Kan Ren. Benchmarking data science agents. arXiv preprint vrXiv:2402.1716, 2024.
Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Livia Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, al. Sglang: Efficient execution of structured language model programs, Advances in neural information processing systems, 37: 62557-62583, 2024.
14

---

Published as a conference paper at ICLR 2026
OVERVIEW OF THE APPENDIX
The Appendix is organized follows:
Appendix
contains reprodueibility statement.
Appendix
contains the use of LLMs this work.
Appendix
provides the limitation of the work.
Appendix
contains the explanation of the evaluation metrics used in this work.
Appendix
provides the performance on the strictly open-sourceable subset.
Appendix models and
provides average number of tokens and tool calls for completions of different prompts.
Appendix G provides training details of the RI. experiments this work.
Appendix contains detailed description of DARE-benu features.
Appendix provides example prompt of the preprocessing steps of this work, including column inference and task identification.
Appendix
contains reference code for instruction cestruction-followingg tasks in this work.
Appendix K
contains tool schema used in our experiments and some task examples.
Appendix
provides details of the rejection sampling implementation of this work.
Appendix M provides details on how we make the use of LLM to classify tasks.
A REPRODUCIBILITY STATEMENT
We have attached the subet of our test set in the supplementary materials. Once accepted, we will release the full test set of our benchmark. The training set and model checkpoints will also be provided upon request, and we plan release them publicly depending from the research community. Also, a detailed description of our data processing procedure is included in subseettio 3.1] These resources are intended to facilitate reproducibility and allow future researchers to build upon our work.
the feedback receive
B THE USE OF LARGE LANGUAGE MODELS (LLMs)
In this project, LLMs were used as assistive tools. Specifically, we used LLMs to polish the writing of the paper and to assist in finding related works. In addition, LLMs were used during the data pro- cessing stage, for tasks such as data filtering, question rewriting, and identifying task targets. Beyond these uses, the research ideas, experimental design, and analyses were developed independently by the authors. The authors take full responsibility for all content presented this paper.
C LIMITATIONS
While DARE-bench provides a large-scale, verifiable, and trainable benchmark, several limitations remain. First, the current tasks primarily tabular based, the benchmark does not yet cover multimodal inputs such as text-image combinations or diderdiagrmm interactions. Second, the cost of generating large numbers of executable traces can be high, and the rejection sampling strategies, while effective, may introduce biases toward shorter trajectories.
D EVALUATION METRICS
We report results using two standard metrics for classification and regression tasks: macro-F1 and R²
15

---

Published as a conference paper at ICLR 2026
Macro-F1. For classification task with C' classes, let TPc. FPc. and FN, denote the number of true positives, false positives, and false negatives for class c, respectively. The precision and recall for class are defined
TP.
TPc
Precision
Recall
TP FP.
TP. FN:
The Fi-score for class e is
2 Precision. Recall
F1 Precision, + Recalle
The macro-F1 is then the unweighted mean across all classes:
Macro-F1
Fle-
C
c=1
R (Coefficient of Determination). For regression/time-ssries tasks with ground-truth values {ux ) and predictions {ūr (ütHe define the mean of ground-truth values as y H Mi. The Rt2 metric
2 (yi 0:)²
1
1 7)2
An R² value close to indicates strong predictive performance, while values close to 0 or nega- tive indicate weak or wonse-than-baseling performance. Since H² can negative when the model performs worse than predicting the mean, adopt a clipped R22 defined as
Relipeed = max(R², .0),).
to ensure that regression scores remain in [0. and are comparable to classification metrics.
E PERFORMANCE THE STRICTLY OPEN-SOURCEABLE SUBSET
To ensure broad applicability and adherence to strict compliance standards, constructed a strictly open-sourceable subset of our benchmark. While the full benchmark aggregates data from diverse sources maximize coverage, certain sources impose licensing constraints (e.g., ShareAlike Non-Commercial clauses) or lack explicit licensing information, which may limit their utility in proprietary model development.
The strictly open-sourceable subset explicitly excludes data sources governed by restrictive licenses, including Creative Commons ShareAlike (SA), Non-CCmmerriie (NC), and sources with Unknown or custom restrictive terms. This subset is composed exclusively of data distributed under permissive licenses, such MIT, Apache-2.0 CCO, and CC-BY-4.0. This ensures that the subset can freely used, modified, and integrated into downstream applications without "viral" licensing obligations.
Table 11: Performance on the Strictly Open-Sourceable Subset
Model
class- IF
class-MM
reg-IF
reg. 1eg-MM
time-Xt XF
time-
gpt-4o
31.86
41.10
20.63
39.74
37.09
5.24
gpt-4.1
55.39
57.75
50.00
57.68
6.81
gpt-5
70.10
43.18
55.56
$5,21
36.78
7.84
gpt-o4-mini
68. 1-4
59.14
51.59
57.48
42.15
8.15
Claude- 3ondee--onnet33.7
61.52
61.22
46.03
61.36
51.21
12.08
Claude-Sonnet-4
17.70
14.29
0.02
Qwen3-32B
16.67
30.92
15.08
35,42
27.26
0.00
Qwen3-4h
3.43
4,99
0.79
2.28
7.00
0.00
Table 11 presents the evaluation results on this filtered subset using the same experimental configu-
ration as the main evaluation turns, 200 s sandbox time).
We observe that the performance trends on this subset are largely consistent with the full benchmark, indicating that the permissive subset retains sufficient difficulty and representativeness to serve as reliable proxy for the full evaluation.
16

---

Published as a conference paper at ICLR 2026
F AVERAGE NUMBER OF TOKENS AND TOOL CALLS
Detailed statistics on the average token counts and tool invocations are provided in Table 12 All token metrics are standardized using the Qwen3 tokenizer.
Table 12: Average number tokens and tool calls for completions of different models and prompts. All token counts calculated using the Qwen3 tokenizer.
Overall Tool
Model
IF Tokens
IF Tool Calls
MM Tokens
MM Tool Calls
Overall Tokens
Calls
prompt
$96.7
224.6
350.7
apt-5
609.5
2.2
582.5
2.4
TlandeSSonet-33
675.2
3.6
894.3
4.8
R30.0
Qwen3-322 32B
2093.3
3.1
1693.0
3.5
1816.8
Owent-32B-SFF-DD
1778.3
3.3
1572.0
3.6
1638.7
3.5
Qwen 4B
1691.1
3.7
1151.7
3.9
1328.1
Owen3-1B-RL
1549.4
3.7
1140.1
3.7
1277.99
3.7
G REINFORCEMENT LEARNING TRAINING DETAILS
G.1 REWARD DESIGN
Instruction following tasks. For instruction following tasks including Classification-II and Regression-IF tasks. We have reference solution code Cief with corresponding simulated predic- tion for data Dust as Yrer (ee(D))))) Given the model prediction ý G(Q, Dusin D Pret. M, T) and simulated ground truth Vret. we use the following reward:
0.1, ý exists,
1.1, ý Yief:
0,
otherwise.
Note that LLMs may be unable to generate a prediction.co cav file due to the max turns or sandbox execution time limit.
Predictive ML tasks. For other tasks, including classification--MM regression-PM, time-series- XF, and time-series-CH have masked ground-truth data Ygt- Given the prediction provided by LLM y, we define the reward as
0.1 d(),
exists,
0
otherwise,
where d X' y + [0, 1] denotes the distance measure between the prediction and the target. For classification tasks, we adopt the macro-FI score to account for class imbalance. For regression and time-series tasks, we use the clipped coefficient of determination, defined as
RR)p(R)) = min max (0, R²)}
If there multiple prediction targets, compute the distance by taking the average of them.
G.2 OTHER TRAINING PARAMETERS
Reinforcement learning. We summarize our RL training dyper-parameee in Table 13
17

---

Published as a conference paper at ICLR 2026
Hyper-parameter
Value
RL algorithm
GRPO
Shao et al. 2024
Loss aggregation
Sequence level
Group normalization
Fa se
Learning rate
10-4
Training mini-batch size
64
KL regularization
False
Rollout batch size Number of rollouts per question Rollout backend
8
SGl.ang (Zheng et al.
2024
Rollout temperature
1.0
top P
0.95
top_k
50
Model sequence length
32,768
Table 13: Hyper-parametee used for reinforcement learning experiments.
H DETAILED DESCRIPTION OF OTHER DARE-BENCE FEATURES
Automated and Scalable Curation. The task generation process in DARE-bench uses a defined approach which collects data from Kaggle and incorporates web-scraped content before LLMs ver- ify the tasks and produce standardized definitions. The automated pipeline generates authentic work assignments large scale across multiple fields through an approach that needs minimal human involvement.
Diverse and Realistic Coverage. The benchmark contains 6,300 tasks which cover multiple do- mains and languages, including tabular classification and regression well advanced time-series forecasting. By drawing directly from real-world Kaggle datasets, naturally incorporates common data challenges such class imbalance, missing values, noise, and temporal irregularities, providing a more faithful simulation of practical data science scenarios.
Time and interaction constraints. DARE-bench implements realistic usage scenarios through its requirement for both time-limited wall-clock operation and restricted interaction turn counts. In practice, end users are unlikely wait hours for a model train full pipeline; hence, we cap execution time to 10 minutes for fast-response settings. The system limits the total number of agent-environment dialogues which forces models to find efficient solutions instead performing endless cxploration. The established limitations in this benchmark create testing environment which mirrors actual operational conditions for interactive data science agents.
18

---

Published as a conference paper at ICLR 2026
I EXAMPLE PROMPT FOR COLUMN INFERENCE AND TASK IDENTIHICATION
The following prompt guides the model to check task suitability and identify prediction target and relevant features from the provided dataset description and data information.
Task Target and Feature Identification Prompt
You are given dataset along with description. Your tasks are as follows:
Task 1: Assess Logistic/Lineet Lineer Regression Suitability Determine whether logistic regression (for classification) or linear regression (for regression) can be appropriately used to model this dataset. Use this strict checklist:
Classification: target must be categorical; features structured; manageable missing values.
Regression: target must be numeric and continuous; features structured; managcable missing values; categorical targets allowed.
all conditions met, the method is appropriate. Otherwise, is not. When uncertain, output False.
Task 2: Identify Task Type, Target Column, and Feature Columns You must select column names only from the list below inside Column list:, avoid using names from Context description:.
Column list: (all_columns]
Context / description:
[flltered_metadttte (scraper)
Infer:
The task type (classification or regression)
A list (S 3) of candidate target columns
The best set of feature columns
Task 3: Column Type Inference
For each column in the list, classify it as:
"numerical"; meaningful arithmetic operations
"categorical": groups/codes, arithmetic not meaningful
Instructions:
Return a Python dictionary with every column as a key
Value must be either "numerical" "categorical"
Use dataset description to guide decisions
Final Output Format
Output exactly 5 lines, in LaTeX-boxed format:
1. Method suitability + (brxed(True) or \boxed/False)
Task type - \bexed clasllficaaiitc ion or \boxed(zogression)
Target column candidates + \boxed ["target1", "target2"
4. Feature columms - boxed[["cool]]] "co12", ...]}
5. Column types + \boxed (("co1 11; "numerical", "co12"; "categorical })
19

---

Published as a conference paper at ICLR 2026
The following prompt reformulates the user question into precise and well-structured instruction.
Rewrite Question Prompt
You are given machine learning task described final _uealliuustooo and dictionary of column metadata in
sitiii....o.. Your job to rewrite the final_ques ion in fluent natural language, making it easier to read while keeping the meaning and structure intact. Here's what you must do:
Replace column names feature names in final _uuaatt on with their natural lan- guage descriptions from metadata deweeriptserrrrrrrrr_e Preserve the original ordering of features in lists.
If column feature name is not present metadate declaatttddecriitiinnnn rewrite into natural-sounding phrase using best judgment.
3. Rewrite structured formats (lists, dicts) into natural language paragraphs, while retaining original item order.
4. Keep existing natural language unchanged.
Keep all file paths unchanged.
6. File names paths must be wrapped in backticks.
Target column names must wrapped backticks.
8. Final output must be clear instruction in natural language.
If string None appears value ranges, treat it as categorical value Nonc.
10. Do not include headings, markdown, or extra explanations retur only the rewritten ques- tion.
11. Use only standard English characters.
12. Explicitly preserve ordering requirements in the rewritten question.
Here the final_gquss final_question: {question)
Here is the metadata thentaadata_desccppiioo (description]
Now return only the rewritten version the question, using natural language descriptions where pos- sible. Preserve file paths, model names, and categorical values exactly as given.
20

---

Published as a conference paper at ICLR 2026
The following prompt determines whether the dataset is time-series and infers the appropriate tem- poral type information.
Time-Series Identification and Typing Prompt
You are given datasct and its description.
Task Assess Suitability
Check Time Series Forecasting applies. Conditions:
1. Must have clear timestamp column (e.g., 'date", 'time").
Must have target variable changing over time (e.g., sales, temperature).
3. Observations should be sequential and time-dependee denedeeendent.
Time interval must be regular or resample-able (e.g., daily, hourly).
If all met, output True; otherwise Falsc
Task 2:
Identify Key Columns From the list below, infer:
Best timestamp column
Best target column
Optional exogenous feature columns
Column list: (all_columns)
Context: fialtered_metadatte (scraper)
Preview (first 50 rows): [df_preview)
Task 3: Column Typing
For each column, classify "timestamp", "numerical", "categorical", "other". timestamp exists, also infer its format (Python strftime).
Final Output Format (6 lines, LaTeX-boxed):
Suitability + (brxcd( or (Faxec(Faasc)
2. Timestamp column \boxed (column_name) or \boxed (ambi guous)
Target column + bboxdd ( coluun_nammemeeeeee or \boxed (ambi guous)
4. Exogenous features + \boxed ["coli", ...]) \boxed [[]
5. Column types + \ooxed ("col1"; "timestamp", "col2"; "pumerical"!>
6. Time format + \boxed ( & -m-dd &H:&M:8 &S) \boxed (amb guous)
21

---

Published as a conference paper at ICLR 2026
The following prompt identifies grouping entities (e.g., users, products, regions) that structure the dataset for time-CF tasks.
Entity (Group) Identification Prompt
You are given dataset for time series forecasting. The datasct includes timestamp column, tar- get column be predieted, and possibly multiple other columns representing categorical numeric features. Your task is identify which column(s) represent the entity group ID) that the column(s) that differentiate multiple independent time series within the dataset. Please analyze the column names and a sample of the data (including at least the first few rows), and answer the following:
Which column(s) should
to distinguish different time series entities?
2. Briefly explain why those column(s) were selected entity identifiers.
If entity column needed because the dataset represents single time series, say so explicitly.
Dataset Description
[description]
Additional identificattion suggestions (optional)
(on.ttyy_
Sample Data (first 30 rows)
(sample_str)
Column statistics (distinct counts, top value frequency, example value patterns)
moolumm_state_ee
Output format (exactly lines, LaTeX-boxed. nothing else):
1. Entity Columns + \boxed [ ("col_namel", "col_name2", ...]} or \boxed [[] if none
Justification > \boxed <Your explanation here>)
22

---

Published as a conference paper at ICLR 2026
The following prompt decides whether resampling is needed for the dataset and, if so, specifies the appropriate resampling strategy.
Resampling Decision Prompt
You are expert data scientist assisting with time series preprocessing. Your goal is to decide whether given time series dataset needs resampling (i.c., converting irregular overly fine-gre mne-grained timestamps fixed frequency like daily/hourly..
Task Description
You given:
A brief description the dataset task.
2. The first 30 rows of the dataset (including timestamps and relevant columns).
each time value, up 5 rows are shown.
time value had more than 5 rows, is marked with comment.
3. The target column for forecasting or analysis: [target.col} }.
Please analyze:
Whether the time column appears irregular, too granular, or dense.
Whether each row represents meaningful unit (c.g., per-day summary) or low-level log (c.g., events).
Whether resampling could make the series easier model.
If resampling is needed, recommend:
The resampling rule (c.g., 1min, Smin, 1H, 1D).
The aggregation function for the target column ([target col)): choose from "mean" " "sum", "count" ", other common aggregations.
If resampling is not needed, it may be because the data is evenly spaced or each row meaningful as-is.
Dataset Description
(description)
Sample Data (first 30 rows) (sample_str)
Output format (exactly lines, LaTeX-boxed, nothing else):
1. Should resample + \boxed (True) (FalsssFalee)
Reason + \boxed <One-sentence explanation
3. Suggested rule \boxed <1mmn//mmn////////1 mull>}
4. Target aggregation + \boxed <mean/ wtt http:///scmcootn...... for [target_col/">]
23

---

Published as a conference paper at ICLR 2026
J REFERENCE CODE FOR INSTRICTION-FOLLOWING EVALUATION
Below we include the reference implementation used to evaluate instruction-folloviin tasks in our benchmark.
Reference Code for InstructionnFoolowinng Evaluation
import jaon inport 08
import sqlite3
Inport pandas
inport cumрy as
from sideam..ncieLeeeeetiio. import testn_test_eplit
import standardScaler, Onakotzncoder
skclearn, imputa import s'mpls mimplsmmputer From sklearn. compose import ColumiTrans
from dklearrn.pppeling import Fipe' ine
Linear_____dde import Logist iotegresion,, LinearRegression, Ridge,
from sklearn. tree impozt Decleion
DecieionnceeeRegeesee
cellearn.neighher. import Kbeiihhsorsclassifiee,, Keeighhorstegrezzor
from sileaan,neerre comrmnneual_aaen network neatrnnnoorln import MLPClagsifier, MLPRegraaser from sklearn.natee ve_baye Import GausalanNS
from sklearn. import LinearSVC
from skl punttttptt import Mult:outputtlass fler
sklearn.metrics import acouracy_score, classificationneepoe mean_squazed_eerror =2_score
Funct to load and join tables from sorite fila def poge_aantLoonn/eciteeeeee p
sqlite3. conneecttontttttcc (sqlite_peth)
tables = po. qurn__ai__uery(meme
name FROM sqlice_maater WHERE tapee'table'/
conn)
df None
dr_tab re.dead_sql [ar___ll__eeeeee (f FROM "[Sbble)"*, conn] ic 'zw_iid not in dl_Lab.colunes: continue
if d$ is Nena:
df df_tab
else: df df.merge (df_tab, row_rdv hogsrinner') cons.close() retura d2
def train trwin prannppreddcr_dellllllo itrain_ds df, feature_cols, model_type,
colum target_cols=1 answer' inputer_typer sepeee_ttp===mmp mopte="eee_ffeeuen problom_cypes crrolln_yype=sssaaaa assi candom_atate==22) print TRARKTNG PTPRITNE")) print (*=" 6U)
check if target column train_df.colume: for target_col in ta-get_colas
training data
targes_cclums (f"Targt column " [target_columm]] not found in training data!")
(r"vall"lllll columna in train_df: [List (traant_tf..oouussss return None
Check if all feature columna exist in both datasets mlasing__eaturn paatures_rrain [col for col in feature_cols if col not in trair_df. columns] m1 missing "eatures_evaa [col for col in feature_ools if col not in eval_df.columm
if (f"Missing feature columns in train_df:
print
return None
missing_ fostures_etrrllll it"issiMMising feature columns in eval_dii [mtssiing_tatuure___v]]]]> return None
formstted targeat = ".join("\[] ("."""")""ann.n..n"."". format (col) for sol in target_colsi (t colurn [formatted_ taatgtedtta found in training data") print (I" Training dalaset shape: (Lzain_dd.shepe shape print Evaluation
shape: [eval_ shape))
Prepare training festures and target
24

---

Published as a conference paper at ICLR 2026
X_rrain train_df (festufffeeeeeeeeee zopy
Yutroin train_df [target_colaa.cog copyy
Prepare evaluation Iestures (end Larget if it exists! X_eval eval_ad (feature_cols oopy ooppy
Check if target column exists eval_ for evalustion has_eval_targen True for target_columm in ta"get_cola:
Larges_column in eval_dr.columee tas_eval_ierge: Palse
has_oval_sarge::
y_eval eva__d[(terget_eall ee
print (E"Maagez column found in evaluation data will compute metrics")
clae:
y_eval None
print predictions") in training data
found evaluation will only
Check for mull y_train. () sum() ()um() use two sum to get the total mull mill_argget__rriin null number if tull_targets_traat > 0:
print 1E" Found faull_targets_tret
targets in training removing
these rows")
valsd_indices "y_train.'smul: any (axis=1) make suse target row K_train "Ltrein(vaci__indicees
y_train = y_ttrain
print (I" Pinal training data: (X_train, rоws*) print Final evaluation
(KLevaL.shape zows")
Separate numertc and categorical features
numeric_foatures categorics seeegorisal_featuree _feeturre = (
For col in feature_cols:
type training
if column_type__inee wty.p_ennereenceleccco [col]. (ome)) == "numericol":
(co.)
elif gellmm_typo_innfronncce [col; ; lewer() "eatogorical";
lttegorical_featrr appln
print (f" Numeric features (1len (aumeria (eaturess) cuureric_ffatures//
print Categoricsi festures ((1en [Ilenceatteerricccleeeeueeee :): [cataporical__eatursee )*)
assert len (nuteric_features)
len lentesteeprriial_eeetvee
len (Ceature_colls
Create preproceasing pipeline transformers
rumaric_ features:
apppdd ('mum', Pipeline(l
(1 impater', SimgieInputer (ssuaalegy-immutere.bpe)))))
(sscllrr, StandardScaler (1)
1b sumeric_featuzes]]
if aetaoorical_featuree
transformers.aggent caatt', Pipeline
(! imputer', SimpleImputer (' M'enehot', GnedoeEnnooder unandle__nknonnr/iig igneree, sperse__outpt--Palse)
1), catagoricol_faatures))
preprcresscr Columniransfommn (ksmefforeeeetttteooomeee
Choose model based problem type
if problem_Lype,
lower Lower iE model_type "Logissickegsession" modal Logiatiikegressionn if len (target_cooi)
"glassificatiio":
milesonrpurclaassiieeer
elff model_type =s "Decis onT aass
model DecieionnreeC..ee asaifier trantoo_etaceerraddoddaaaa
elif medel_type "GaussianNB":
model SaussianNS
if len (target_colls)
model Mult:ourgprcllsssiieee (model)
elif model_type "LinearSve*;
25

---

Published as a conference paper at ICLR 2026
model Araears ornarassetttteeeeeeeeerooooooood stateeranndo_statee if len sttarge__col_
model outt:oottttlaaelcci (model)
elif model_type - "XIFClassifLer":
modal MiPClass
else:
raise (F model type: [yoeel_
print Uning (modei_type) for claasificationn)
probien__ypee.owerr! "regzession"; if model_type - "LineaRegzessionn"
modal Limearkegressionn)
elif medel_type "BecissonnZzeRRegresssus
model hrt.a//oo..eeeeeeeeeeeeeeooeeeee
elif model_type "Ridge":
model htt.:r/mmmmmmmtttteerddddooooo
elif model_type "Lasao";
model medel_type "KIPSegrasser":
model oerrrssooorrrnooosssssteeeeeooo
else:
raise ValueFrro" if Tnvaa model type: [model__ypel>
print 1E" Using (madei_type) for regression")
Create full pipeline ml mi_piani = Pipeline([
I'preprocessor', preprocessor), l'model', model)
1)
Train the print (F"TRAINIG NODEL...") try:
ml_pipeline, iitraniaiin y_train) print 1E" Model trsined successfully")
except Exception e: print 1E" Error during training: (el*)
return Mone
Make predictions
print (I"MAKING PREDICTIONS..."
y_pred pipppelinn.prreeic (X_eval)
if"Preoi intraccciioons completed")
except Exception as e:
print If"Bercr during prediction: (e)")
return None
Evaluate model (only if we have evaluation targets) hog_oval_sarger and y_eval print iETHonEs, EVALUATTON")
None:
print ("=* 30)
$ Remove zows with sull Largets in evaluation for metrics valid_eval y_eval_clean y_evel [valid_eval_masi y_pred_alean y_pred (val'd_va__al___
y_evallnottas))
Lenly_evalssees 0:
print(" valid evaluation targets found skipping evaluation metrics")
else:
if sroobee_tyyp..owee. accuracy = accuracy_ (y_evllclenn,, c-pred_clean) print (t* Accuracy: (acouracy:.4f)) print [["Classification Report:") print (classification (y_eval cleran)
"classification":
+ Show sample predieriens print (f*sample psedictions:")
for in range (m1n len len|y_erss_ollarr] actual predicsed clreed[ scatus "right" if actual predicted else "wrong" print (E" [st[sta)ss
Aatual= Predicteds
[redicctdd=ppredccteed" (predicted]";
elsa:
26

---

Published as a conference paper at ICLR 2026
mean__quuredd_rrco (y_eeelleeeleeea y-pred_clean)
r2 222_scoreee r2__scortty_eval lyy_eee___eeeory y_prad_clean) np.agrt
print (f R112] Scores (121.41)))
peint (I* RMSE: (=mee:.417")
print (5" MSE: (mse:.4E:")
$ Show nample predictlors print (r"Sanple PsecictLons=")
for i in range (min Lealy_evai_clea:)) actual seeevvl_cleannilee..
predicted y=pred_clean(A]
diff abs (actual predicted)
Actual= Predicted- (predi/ prodictodd-prrodtetaaa print (iff=(dlffs.sr)))
1EEALIIE"OALLUUTIOO SKIPPED No target zolumn in eveluation data") print Generated predi predictions";
y_pred_dd = pd. (y preé, columnsny colums)) __pred_df.1neee 'row_id', eval_df ("row_1dt
return {
"pipeline': pl_pipeline,
'predictions': y_pred
'eval_indices': X_eval index,
'probien_type': preblem_type,
'X_train': X_=roin,
"y_train' y_srain,
'Xeval': X_eval,
"y_eval'; y_evai has_eval_target
None,
"haa_eval_target'' pas__val__aareee
*numeria_"ee "enerree"eatuues":: nume-i e_featurea,
"atepericol_Featuress"" cateppricol_foatreee
festuze_cols $leature_cols
model_type "Smodel_=ype"
$columm_typoiinnfrrmnm
target_cols starget_cole
inputer_type "Gimpuzer_type"
problem_type "áprobien_typpe
zandom_etate Scandom_sLale
fave_file_type "$save_file_type"
1f save_file_typp as 'sqlite':
con: sglite3.conned
train_df rd.resd_sql ("SELBCT FROM train_set", coant train_dr train_dd.. ([None: rp.nar)) conn.close()
eval_df laad_and_join ("al vil.qiite""
eval_dt eval_af. ([Noee: np.nan))
elif save_file_type 'esv'; train_df oeb.reaLccc/trriii_Looeeee.e
kesp_default_ne-eaaae as_values
-[*"]]
eval_df pww.cee.eewwwwaalllolaaoo
keep se_values=("")
olif save_file_type == 'parquet';
train_df pd. ([None: sp.nar))
train_d
eval_df ad.read_parquet ( ("ve__ll.prquuue
eval_df eval_df. (None: ng.nan))
train_predics_nodell
train_af train_df,
eval_df eval_of,
[eature_cols [eature_cols,
model_type model_type,
taxget_ccolprarrsssccol.
impiter..yossiipoee__.po
prablom_ttpa-prccheer..tt Eanoom_stateeraaddon_ttdtt
rasult poalt('pradictions" _to_esv simmmllaedddeee. prad_looddd_ cavv, indox=False)
27

---

Published as a conference paper at ICLR 2026
K
TOOL SCHEMA AND TASK EXAMPLES
The following schema defines the details of our code executor tool.
Tool Schema: python executor
"Lype": "Lunction",
"name"; "python_executt
"description"s "Execute Python script an isolaced BTTP sandbox wita 230-second time limit. Each cun is single shot and stateless Ino REPL, persistent envircrment between runs). You may upload inpat files via "iles_to_leadd and retrieve resulta via "files_to_son ' The maximum file size for both upload and download la 200 MB. The tool returna the full program output, including both siderr.
explicit 'print (...)" statements capture values in output. This tcol can be invoked up to times conversation.",
"pazametess";
"type": "object",
"properties";
*coda":
*Lype*; "string", "description"; "The Python execute."
3,
siles_to_loed
"type"; "array",
"itama": "bype*; *string"
'description"; "List input paths to upload prior execution (e.g. DX" input..covi", "'config.joonn")) "]"""
3r Biles_tt to__avet___ ttpe" ; "array", iitems:
"Lype": "string"
daccription": *List of output file patha to download after execution (a.g- (\* results.cav\", tlogg.txt""))""
}, "requized";
"code",
"filea_to_load",
*<tlea_to_save*
1,
BaddltiionaPPrperrra fal.e
"strict":
28

---

Published as a conference paper at ICLR 2026
The following provides a example of IF task.
Instruction Following Task Example
Question
Please complete the task as described below without asking any follow-up questions or requesting ad- ditional information. Proceed under the assumption that all required information is provided. You are given access to training Parquet file named train_vl- parquet, which contains single table, and metadata file metadata txt that descrihes the original dataset and each of its columns. Your task is to perform classification using this data and predict the target column Pers corsonall ty validation set located at val vl. parquet. Unless stated otherwise, you should use default parame- ters for all steps including model training and preprocessing. First, load the training file directly. Then, filter the training dataset using the expected ranges while retaining any the relevant columns, excluding only those rows where non-missing value violates its expected range. The expected ranges are as follows the specified order: number close friends must between 0.0 and 15.0; presence of stage fright must be either "No" or "Yes"; social media post frequency must be between 0.0 and 10.0: frequency of going outside must be between 0.0 and 7.0: feeling drained after socializing must be either "No" "Yes": frequency of social events must be between 0.0 and 10.0; hours spent alone daily must be between 0.0 and 11.0. After filtering, select only the features listed in their original order: number of close friends, frequency of social events, presence of stage fright, feeling drained after socializing, and hours spent alone daily. The numeric features, to be used in the specified order, number of close friends, frequency of social events, and hours spent alone daily, and the categorical features, also be used in the specified order, presence of stage fright and feel- ing drained after socializing. Handle missing values by imputing numeric features with mean and categorical features with the most frequent value. Prepricess the data by applying a standard scaler to the numeric features and one-hot encoding to the categorical features with handle unknown set to to to False. Train single Logist www.reeeeessse.eeeeeee
that have missing values in
ignore and sparse_cutpor scikit-learn with random.state=86. Finally, make predictions on validation set and save results to a CSV file at predict redictiion... . cav, including the column row.id i as provided the original val_vl. parquet and the corresponding predictions aligned with each row_id id so that performance can be computed correctly.
model using
Metadata
Overview
Dive into the Extrovert vs. Introvert Personality Traits Dataset, rich collection of behavioral and social data designed to explore the spectrum human personality. This dataset captures key indicators of extroversion and introversion, making it valuable resource for psychologists, data scientists, and researchers studying social behavior, personality prediction, data preprocessing techniques.
Context
Personality traits like extroversion and introversion shape how individuals interact with their social environments. This dataset provides insights into behaviors such as time spent alone, social event attendance, and social media engagement. enabling applications in psychology, sociology. marketing. and machine learning. Whether you't 'e predicting personality types or analyzing social patterns, this dataset your gateway uncovering fascinating insights.
Dataset Details
Size: The dataset contains 2,900 and 8 columns.
Features:
Time_spent.Alone. Hours spent alone daily (0-11).
S Stageffar:: Presence of stage fright (Yes/No).
Social.eveent.atteeaaan Frequency of social events (0-10).
Going.outsids: Frquuuuu of going outside (0-7).
Draineddaffer..occiiiiiii. Feeling drained after socializing (Yes/No).
Pricnds.circll.sizz. Number of close friends (0- 15).
Post Post.irequency: Social media post frequency (0-10).
Personality: Target variable (Exttrover((ntrooerrt))
Data Quality: Includes some missing values, ideal for practicing imputation and preprocessing.
Format: Single CSV file, compatible with Python, R, and other tools.
Data Quality Notes
29

---

Published as a conference paper at ICLR 2026
Contains missing values in columns like ime_spenn_AAooee and Going.outsi.ee side, of- fering opportunities for data cleaning practice.
Balanced classes ensure robust model training.
Binary categorical variables simplify encoding tasks.
Potential Use Cases
Build machine learning models to predict personality types.
Analyze correlations between social behaviors and personality traits.
Explore social media engagement patterns.
Practice data preprocessing techniques like imputation and encoding.
Create visualizations to uncover behavioral trends.
About this file
About this file This dataset contains 2,900 entries with features related to social behavior and per- sonality traits, designed to explore and classify individuals as Extroverts or Introverts.
30

---

Published as a conference paper at ICLR 2026
The following provides example of ML Modeling task.
ML Modeling Task Example
Question
Please complete the task as deseribed below without asking any follow-up questions or requesting additional information. Your task is to achieve good performance while balancing training time and accuracy in the sandbox. You provided with processed dataset, along with metadata file metadata .xxt that describes the original dataset and each of its columns. Load data in train parquet as your full training set. Once that's done, using only CPU resources, train a classification model on this training data. Then use val v2 parquet to generate a prediction for Internet A for each zow_id. The model should output a file named predict ct ion. csv. The file must contain the column row_id (as provided in the original val.v2 parquet and the corresponding predictions. Each prediction should be aligned with its row jid that performance be computed correctly.
Metadata
AI Tool Usage by Indian College Students 2025
This unique dataset, collected via a May 2025 survey, captures how 496 Indian college students use Al tools (e.g., ChatGPT, Gemini, Copilot) in academics. It includes 16 attributes like Al tool usage, trust, impact grades, and internet access, ideal for education analytics and machine learning. Columns
Student_Names Anonymized student name.
College_Nanl _aae: College attended.
Stream: Academic discipline (e.g., Engineering, Arts).
Year_of_študy: Year of study (1-4).
A.I AI_Tools_Used: Tools used (c.g., ChatGPT, Gemini).
Daily.Usage.HHure Hours spent daily Al tools.
Use Purposes (e.g.. Assignments, Exam Prep).
Truus__in.AI.Tooll:: Trust level (1-5).
Impact.on.Gradcs:: Grade impact (-3 to +3).
1o.F.oteesorr_AAloo..tte:: Professor approval (Yes/No).
PreierredAAI.oool: Preferred tool.
Awareness.Level1 awareness (1-10).
Wi..ing.to.Pa..fr..cccoo Acces:: Willingness to pay (Yes/No).
State: Indian state.
Device_Used: Device (c.g., Laptop, Mobile).
Enternet.Access Access quality (Poor/Meiium/High))
Use Cases Predict academic performance using Al tool usage. Analyze trust in Al across streams regions. Cluster students by usage patterns. Study digital divide via 'InternetAAcces'..
Source: Collected via Google Forms survey May 2025, ensuring diverse representation India.
Note: First dataset of its kind Kaggle!
31

---

Published as a conference paper at ICLR 2026
The following provides example of Time Series Canonical Forecasting Task.
Time Series Canonical Forecasting Task Example
Question
Please complete the task as deseribed below without asking any follow-up questions or requesting additional information. Your task is to achieve good performance while balancing time and accu- racy in the sandbox environment. You are provided with processed dataset, along with a meta- data file metadata. txt that describes the original dataset and each of its columns. Load the file train. cav as your complete training data. This file contains the raw, non-resampled time series data. Once that's done, using only CPU resources, train time series analysis model(s) on the training data. Then, based on the file val_v2.csv, forecast the target column Close. Generate predictions exactly for each row .id present in the validation data, Output file named preaictien file must contain the column row.id id provided in the original val _v2 c&v and the corresponding predictions. Each prediction should aligned with row so that performance can computed correctly.
The
Metadata
Here's to the crazy ones ones-the the data dreamers, the analysts, the visionaries who believe that a handful of numbers reveal the DNA of innovation, This dataset is than collection of Apple Inc.'s historical stock prices; it's chronicle invention, perseverance, and thinking differently.
What's Inside
Time Span: Daily stock price data for Apple Inc. over multiple years Features: DDate:: The day of the record "Close": Price at market close Format: CSV, clean and ready for analysis
Why This Matters
Apple is not just company, it's a movement. Its stock price reflects not only financial performance, but the world's response innovation launche, leadership changes, economic cycles, and the occa- sional "one more thing."
Possibilities
Visualize long-term growth and volatility Model trends, moving averages, momentum Forecast future prices with machine learning Detect the impact major product launches events Explore relationships between volume and price action
Inspiration
As you explore this data, don't Ion't just look for patterns-look for stories. See how moments of genius and risk-taking ripple through the numbers. Use this dataset to inspire your own creativity, your own analysis, your own 'insanely great' discoveries.
Whether you'le re here to build a predictive model, craft beautiful visualizations, or simply marvel at the journey, remember: The people who crazy enough to think they change the world with data...
are ones who
About this filc About this This file contains historical daily stock price Apple Inc. Each row represents one trading day and includes key financial metrics that track Apple's performance on the stock market.
Columns descriptions
Date: The calendar date for the trading record (format: YYYY-MM-DD), Close: The price of Apple's stock the end of the trading day.
32

---

Published as a conference paper at ICLR 2026
L REJECTION SAMPLING IMPLEMENTATION DETAILS
We sample up to K candidate trajectories per task. Each trajectory records: (i) Final_scors and (ii) end-to-end wall-clock time. For IF tasks, final_score is exact match E {0, 1]; for predictive tasks, final_score is normalized metric such as macro-FI or clipped It2.
L.1
VALIDITY AND DIVERSITY CONDITIONS
Validity.
A trajectory is considered valid if:
For IF tasks: final_score=
For predictive tasks: final 2 type-specific threshold:
class-MM: 0.8, reg-MM: 0.7, time-XF: 0.6, time-CF: 0.3.
Diversity.
A task is considered diverse if:
For IF tasks: among the K trials, least one final.score and at least one f iinal_score= = 0.
For predictive tasks: the variance of the K scores satisfies
Var(S:) 2 threshold, class-MM/reg- 0.15, time-XF: 0.15, time-CF: 0.1.
L.2 REJECTION SAMPLING STRATEGIES
FV (Fastest- Valid).
For every task that has least valid trajectory:
IF tasks: keep the single fastest valid trajectory.
Predictive tasks: keep the trajectory with the highest final.score.
AV Vall-Valid). For every task:
Keep all valid trajectories (as defined above).
BV (Best- Valid). For every diverse task:
tasks: keep the single fastest valid trajectory.
Predictive tasks: keep the trajectory with the highest final
Thus BV applies the same selection rule FV, but restricted to diverse tasks only.
DV (Duo-Vali))) For every diverse task:
tasks: keep the two fastest valid trajectories (or one fewer exist).
Predictive tasks: keep the top-2 trajectories by score, restricted to those with s(t) 8; (above mean).
NOTES
FV applies to all tasks with valid traces; BV and DV apply only to diverse tasks.
AV the only strategy that may return multiple valid trajectories even for non-diverse tasks.
FV/BV always select at most one trajectory; DV at most two; AV can return more.
This design ensures balance between efficiency (FV), diversity (AV), quality (BV), and complementary coverage (DV).
33

---

Published as a conference paper at ICLR 2026
M TASK DOMAIN CLASSIFICATION METHODOLOGY
To assess the diversity of DARE-bench and verify its coverage across real-world scenarios, we clas- sified each task into a primary domain (e.g., Finance, Health, Technology). Given the scale of the benchmark (6,300 tasks), manual classification was infeasible. Therefore, we employed an auto- mated LLM-based classification pipeline utilizing the rich metadata associated with each Kaggle- derived dataset.
Metadata Usage. The classification relies on four key metadata fields:
Title: The official name of the dataset.
Subtitle: A short phrase summarizing the dataset content.
Description: The full natural language description of the dataset context.
Keywords: User-provided tags from the original Kaggle source.
Classification Taxonomy. To ensuring consistency, we defined controlled vocabulary of allowed domains based on common industry verticals: finance, health, business, technology. automotive, education, environment, and others.
Prompt Design. We constructed strict prompt to instruct the LLM to identify the single best domain label. The prompt enforces a hierarchical reasoning logie: prioritizes explicit domain terms found in the user-provided keywords before inferring the domain from the title or description. The full prompt template provided below:
Domain Classification Prompt
Instruction: Identify the single best domain for task using the provided metadata.
Input Data:
Title: [title}
Subtitle: {subtitle)
Description: [description}
Keywords: (keywords)
Example Domains: [agriculture, finance, health, business, technology, automotive, education, envi- ronment, other]
Reasoning Steps:
Keyword Check: First, strictly check the provided 'keywords' list for any explicit domain word (e.g.. 'finance', 'health'). match found from the allowed list, select immedi- ately.
2. Inference: If no direct domain appears in the keywords, infer the most appropriate domain based the semantic context of the 'title', 'subtitle", and 'description"
3. Output Formatting: Output exactly ONE lowercase word from the allowed domains list. Do not output punctuation, explanations, or spaces. If the domain is uncertain does not fit the specific categories, output 'other'.
Output:
34


Reviews:
[{"review_id": "50eXqQ8XLH", "reviewer": "Reviewer_pBVz", "summary": "Large Language Models (LLMs) are increasingly adopted for complex multi-step data science (DS) tasks, yet existing benchmarks suffer from two critical gaps: a lack of process-aware evaluation (e.g., instruction adherence and process fidelity) and scarce high-quality labeled training data. To address these, this paper introduces DARE-BENCH, a training-focused benchmark for evaluating LLMs’ DS capabilities, encompassing both machine learning (ML) modeling and instruction following.\n\nDerived from Kaggle datasets, DARE-BENCH includes classification (Instruction Following/IF, ML Modeling/MM), regression (IF, MM), and time-series (eXogenous Features/XF, Canonical Forecasting/CF) tasks, split into 95% training and 5% test sets. Unlike benchmarks relying on human/model judges, all tasks have verifiable ground truth (reference outputs for IF tasks, original dataset labels for MM tasks), ensuring objective, reproducible evaluation via a sandboxed code execution environment.\n\nExtensive evaluations show that even advanced LLMs (e.g., gpt-4o-mini, Qwen3-32B) perform poorly on baseline tests, especially in time-series tasks. However, fine-tuning with DARE-BENCH yields significant improvements: supervised fine-tuning (SFT) increases Qwen3-32B’s accuracy by 1.83×, while reinforcement learning (RL) boosts Qwen3-4B’s accuracy by over 8×. External validation on DSBench further confirms generalization.", "questions": "1. How do you validate the quality of your generated dataset? \n2. Do you have qualitative and quantitative analysis?\n3. What's the detailed dataset statistics for your dataset, e.g., information like how many tool calls, how many tokens are your prompt or your completion.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "DARE-BENCH has several strengths against previous work.  \nUnlike counterparts that only assess final-answer accuracy, DARE-BENCH uniquely evaluates both ML modeling performance and instruction fidelity, filling the void of process-aware assessment. It also provides 6,300 Kaggle-derived tasks with verifiable ground truth (reference outputs for IF tasks, original labels for MM tasks). The training data seems to be valuable. \nIn addition, its four-stage pipeline minimizes human effort, enabling large-scale task generation (6,300 tasks) while ensuring realism—e.g., 20% noise injection in IF tasks to simulate real-world data issues.", "weaknesses": "The task diversity is limited. It exclusively covers tabular data, lacking support for multimodal DS tasks (e.g., text-image fusion, speech-data analysis), restricting applicability to broader DS scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cwzdg2wwRz", "reviewer": "Reviewer_NfgX", "summary": "This paper introduces DARE-bench, a benchmark for evaluating and training LLMs on data science (DS) tasks. Considering that existing benchmarks lack of process-aware evaluation and scarce labeled data, the DARE-bench offers 6,300 Kaggle-derived tasks (covering classification, regression, time-series forecasting) with verifiable ground truth. Tasks include two variants: Instruction Following (IF, testing workflow adherence) and ML Modeling (MM, testing outcome accuracy). Experiments show baseline LLMs perform poorly, but SFT and RL using DARE-bench data drastically improve performance.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper is good-writing and easy to follow.  The benchmark provides comprehensive evaluation scope, specifically, it covers diverse DS tasks (including underrepresented time-series forecasting) and enforces real-world constraints (execution time, interaction turns), enhancing practical relevance.\n2. DARE-BENCH serves both as an evaluation tool and a large-scale training resource, with proven effectiveness in improving LLM performance via SFT/RL.", "weaknesses": "1. Lack of Comparison with Specialized DS Agents. The paper evaluates general-purpose and code-centric LLMs  but omits comparisons with specialized data science agents, which are explicitly designed for multi-step DS workflows. This gap makes it hard to contextualize DARE-bench’s utility. It is unclear whether the benchmark’s gains (via fine-tuning) can match or surpass the performance of purpose-built DS agents,\n2. Provide more explanations about the Instruction Following (IF) and ML Modeling (MM) metrics. The paper frames IF (workflow adherence) and MM (outcome accuracy) as two core DS capabilities to evaluate together, but fails to justify their joint necessity. Especially given Table 4’s results showing no clear correlation between the two metrics. For example, GPT-5 scores highest in classification-IF (69.81) but ranks mid-tier in classification-MM (43.40); Claude-Sonnet-3.7 excels in MM tasks (e.g., regression-MM: 63.20) but lags GPT-5 in IF.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EfIwi7NXdv", "reviewer": "Reviewer_BggK", "summary": "This paper introduces DARE-bench, a benchmark designed to evaluate and train LLMs for data science tasks, addressing two critical gaps in existing benchmarks: (i) the absence of process-aware, verifiable evaluation (e.g., measuring instruction adherence) and (ii) scarcity of high-quality labeled training data. Derived from 6,300 Kaggle datasets, DARE-bench includes two task families—process-aware instruction-following (with reference-code ground truth) and ML modeling (with dataset ground truth)—covering classification, regression, and time-series forecasting. Key features include verifiable ground truth (enabling objective, human-judge-free evaluation) and a dual role as both an evaluation tool and training resource. Evaluations show strong LLMs (e.g., Qwen3-32B, gpt-4o-mini) perform poorly on baseline tests, but supervised fine-tuning (SFT) and reinforcement learning (RL) using DARE-bench data yield dramatic gains: SFT improves Qwen3-32B’s accuracy by 1.83×, and RL boosts Qwen3-4B’s accuracy by over 8×.", "questions": "As listed in weakness.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. It addresses two key gaps in existing benchmarks: it enables verifiable, process-aware evaluation (relying on reference-code or dataset ground truth, no human/model judges) and provides 6,300 Kaggle-derived tasks as large-scale training data, ensuring objective, reproducible assessments . \n2. Its task coverage is comprehensive—covering classification, regression, time-series forecasting, with two variants (instruction-following/ML modeling) probing core DS capabilities, outperforming peers (e.g., DS-1000, DSBench) in time-series support and training task provision .", "weaknesses": "1. Tasks are almost exclusively tabular, excluding multimodal inputs (e.g., text-image combinations, code-diagram interactions) common in modern DS.\n2. Generating large-scale executable trajectories (for training data) is costly, and rejection sampling strategies may introduce biases toward shorter trajectories.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UwPLrptEwh", "reviewer": "Reviewer_Ffce", "summary": "The author suggests DARE-bench, designed to evaluate and train LLMs for data science workflows. It addresses two major gaps in existing benchmarks: (1) lack of process-aware evaluation capturing instruction adherence and modeling fidelity, and (2) lack of accurately labeled training data. DARE-bench includes 6,300 verifiable, executable Kaggle-derived tasks divided into two families. The benchmark enables reproducible automatic evaluation via sandboxed code execution. Experiments show that general-purpose LLMs perform poorly on these tasks without domain-specific training, but fine-tuning on DARE-bench data dramatically improves both process fidelity and prediction accuracy. The paper demonstrates up to 8× improvement in smaller models after reinforcement learning fine-tuning, and also validates generalization improvements on external datasets like DSBench.", "questions": "1. How does DARE-bench handle tasks involving stochastic algorithms (e.g., random forest, neural nets) while maintaining deterministic evaluation?\n\n2. Are the reference “ground truth” codes verified manually, or automatically synthesized and how is correctness guaranteed?\n\n3. Does reinforcement learning with DARE-bench tasks risk overfitting to procedural templates rather than improving general data reasoning?\n\n4. Can this benchmark be extended beyond Kaggle-style tabular tasks, e.g., to unstructured text or multimodal datasets?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Introduces a large-scale, executable, and verifiable benchmark that measures both process fidelity and predictive accuracy, filling a key gap in LLM evaluation for data science.\n\n2. Provides a rich dataset derived from Kaggle that supports both evaluation and supervised/RL-based training, improving reproducibility and scalability.\n\n3. Demonstrates concrete, measurable performance improvements and reduced execution failures after fine-tuning, validating benchmark utility for both evaluation and model training.\n\n4. Clearly integrates determinism and sandbox execution to ensure fair, reproducible comparison across models, a major methodological advantage.", "weaknesses": "1. Heavy reliance on Kaggle-derived data may bias task diversity toward structured tabular and forecasting problems, limiting broader domain generalization.\n\n2. The benchmark’s focus on reproducibility via deterministic setups may underrepresent realistic data science variability or stochastic modeling behavior.\n\n3. The paper lacks ablation or error analysis to quantify which aspects of fine-tuning drive improvements most.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)