Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
Authors: Bohan Lyu
Yadi Cao
Duncan Watson-Parris
Leon Bergen
Taylor Berg-Kirkpatrick
Rose Yu
Venue: ICML 2025 poster (2025)
Keywords: LLM Alignment
AI for Science
Primary area: deep_learning->large_language_models
TL;DR: This paper proposes AWL, a method where LLMs first learn scientific knowledge via tool interaction and then adapt by solving easy problems directly and hard ones with tools.

Abstract: Large Language Models (LLMs) demonstrate promising capabilities in solving scientific problems but often suffer from the issue of hallucination. While integrating LLMs with tools can mitigate this issue, models fine-tuned on tool usage become overreliant on them and incur unnecessary costs.
    Inspired by how human experts assess problem complexity before selecting solutions, we propose a novel two-component fine-tuning method, *Adapting while Learning* (AWL). In the first component *World Knowledge Learning* (WKL), LLMs internalize scientific knowledge by learning from tool-generated solutions. In the second component *Tool Usage Adaptation* (TUA), we categorize problems as easy or hard based on the model's accuracy, and train it to maintain direct reasoning for easy problems while switching to tools for hard ones.
    We validate our method on 6 scientific benchmark datasets across climate science, epidemiology, physics, and other domains. Compared to the original instruct model (8B), models post-trained with AWL achieve 29.11\% higher answer accuracy and 12.72\% better tool usage accuracy, even surpassing state-of-the-art models including GPT-4o and Claude-3.5 on 4 custom-created datasets. Our code is open-source at \url{https://github.com/Rose-STL-Lab/Adapting-While-Learning}.

Adapting While Learning:
Grounding LLMs for Scientific Problems with Tool Usage Adaptation
Bohan Lyu" Yadi Cao 2
Duncan Watson-Parris'
Leon Bergen" Taylor Berg- Rose Yu*
Abstract
of large language models (LLMs) in solving scientific prob- lems, from answering general questions (Lu et al., 2022; Zhang et al., 2024b) to contributing to scientific discover- ies (Ma et al., 2024; Kumar et al., 2023; Liu et al., 2022). However, except for the largest models like ChatGPT-ol and DeepSeek-v3. the abilities of LLMs for scientific reasoning are still typically limited to high school levels (Rein et al., 2024; Cobbe et al., 2021; Hendrycks al., 2024). LLMs have the innate behavior of hallucination (Farquhar et al., 2024) and can produce scientifically invalid outputs.
Large Language Models (LLMs) demonstrate promising capabilitiiss in solving scientific problems but often suffer from the issue of hallucination. While integrating LLMs with tools can mitigate this issue, models finetuned on tool usage become overreliant on them and incur unnecessary costs. Inspired by how human experts assess problem complexity before selecting solutions, we propose a novel two- component fine-tuning method, Adapting While Learning (AWL). In the first component World Knowledge Learning (WKL), LLMs internalize scientific knowledge by learning from tool- generated solutions. In the second component Tool Usage Adaptation (TUA), we categorize problems easy or hard based the model's s accuracy, and train it to maintain direct reasoning for easy problems while switching to tools for hard ones. We validate our method on scientific benchmark datasets across climate science, epidemiology, physics, and other domains. Com- pared to the original instruct model (8B), models post-trained with AWL achieve 29.11 11% higher answer accuracy and 12.72% better tool usage accuracy, even surpassing state-of-the-ar models including GPT-4o and Claude-3.5 on 4 custom- created datasets. Our code is open-source at https: / htt.bbttthhtt.tggggg/egggg. com Rose-STT--Lab. Adapting- While -ealee-LLLnngg..
Recent studies have shown that LLMs can mitigate hallu- cination by accessing general-purposs tools (Schick et al., 2023; Tang al., 2023: Patil al., 2023; Qin al., 2023; Wang al., 2024b). Naturally, incorporating specialized scientific tools, such as physics-based simulators, presents solution to complex scientific problems (Schick et al., 2023;
Ma et al., 2024; Liu et al., 2022). However, recent studies also indicate that LLMs lack the ability to make adaptive decisions about tool use ( YYu et al., 2024; Huang et al., 2023): for hard problems, LLMs may not know when or how to use tools, resulting hallucinatory responses; conversely, for easy problems, LLMs may become over-reliant tools, resulting in unnecessary computational cost overheads.
We observe that human scientists often first gauge the dif- ficulty a problem before deciding whether to direct rea- son or employ external tools (Payne et al., 1993; Kruger Dunning, 1999). Hence, we seek instill similar adaptive capabilities in LLMs to achieve balance between accuracy and cost when solving scientific problems.
To this end, we propose a novel training paradigm, Adapting While Learning (AWL), which consists of two components. The first component, World Knowledge Learning (WKL), uses supervised fine-tuning and preference learning to align pre-trained LLM with highly accurate solutions generated using information from external tools, thereby internalizing scientific knowledge. In the second component, Tool Usage Adaptation (TUA), we evaluate the LLM's direct answer- ing ability and classify questions as easy or hard based on the model's accuracy. While maintaining the same align- ment target for easy questions, we train the model to follow external tool-usage traces for hard questions, enabling intel- ligent selection based on problem complexity.
1. Introduction
To realize the ultimate dream of building an Al scientist, numerous works have explored the impressive capabilities
'Equal contribution 'Tsinghua University. Work partially done during Bohan Lyu's visit to UC San Diego. Uniivrriiyy of California, San Diego. Correspondence to: Rose Yu <roseyu ucsd.edu>.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
(a) World Knowledge Learning
(b) Tool Usage Adaptation
(c) Model Performance
Frocze
Supervised Fine- one-Toning
1. Model Evalocdonn Wineg
Original
LLMs interact with tools generare solutions
SLLM
As
Train
All Questions
LLM
Solution
Simutators [Tooks]
Qe A2
Fine questions and solutionn
0
BLLL
{..oy
}
Proforence Optimization
2. Taal-Ciiiu Trace Deneration
Worid Knooweddge Learning
Solutions
Solution Sampling
Q1
&LLM
Open-Ended Questions
LELMM
Trace1
Noe Use
3. Training with Adaotative Solutions
Tool Usage Adaptation
Tool
Q Trace
{....
f....
Problems
2. Solutin Rankiigg and Sorchut
Easy
DPO with Preferencs
LLM
Prontems
Figure 1. Pipeline of Adapting While Learning. (a) World Knowledge I. Learning: L.LMs undergo supervised fine-tuning for questions and preference learning for open-ended questions; (b) Tool Usage Adaptation: Questions classified as easy/hard based on LLM's s direct-ansee answer accuracy. For easy questions, training targets remain unchanged as the solutions; for hard questions, targets are modified to tool-usage traces. (c) Model improvement visualization: Leftward movement of the vertical dashed line indicates more questions can be solved internally; Movements of horizontal lines for easy/hard questions, respectively, show more intelligent tool usage decisions.
We empirically evaluated our model on diverse range of scientific datasets, from college-level math and physics to research frontiers like climate science and epidemiology. The experimental results show significant improvements for pre-trained base model (8B) after post-training with our method. The post-trained model even surpasses frontier closed models taining challenging and specialized questions that frontier LLMs had not encountered during their pre-training.
(RL) (Rafailov et al., 2024; Meng et al., 2024; Ouyang et al., 2022; Lee et al., 2023; Bai et al., 2022). Direct Preference Optimization (DPO) (Rafailov et al., 2024) is a special re- placement to RL that utilizes designed preference between pairwise data for aligmment, which makes particularly suitable for data collection for post-training.
newly created custom datasets con-
In our work, we employ SFT for all questions and addi- tionally utilize DPO to Iearn preferences between different proposals for open-ended questions.
Our contributions are summarized as follows:
Training LLMs for Scientific Problems. Previous work has sought to ground LLMs using domain-specific knowl- edge in various scientific fields: climate science (Thulke et al., 2024), biomedical science (Luo et al., 2022), molec- ular science (Chithrananda et al., 2020), and general sci- ence (Zhang et al., 2024b; Taylor et al., 2022). Most of these approaches heavily rely on expert annotations or distil- lation from stronger models and face scalability limitations due to computational and expert labor costs.
We introduce a novel two-component training paradigm, Adapting While Learning (AWL), which enables LLMs to efficiently solve real-world scientific problems of varying complexity.
We construct 4 new datasets spanning various scientific domains: epidemiology, climate, Mojuco, and PDEs, to facilitate future research in this direction.
These limitations highlight the need integrate scientific tools into both data generation and training processes.
Compared to the pretrained base model, our post- training achieves an average improvement of 29. 10% in answer accuracy and a 12.72% increase tool us- age accuracy across all datasets. On our newly created datasets, the post-trained model even surpasses state- of-the-art closed models like GPT-4o and Claude-3.5.
LLM Tool Usage. LLMs have demonstrated impressive performance in using external tools (Yao et al., 2023; Schick et al., 2023; Patil et al., 2023; Qin et al., 2023), such as web interfaces and shopping platforms (Yao et al., 2022; Cheng et al., 2024), code interpreters (Ma et al., 2024; Cai et al., 2023), scientific simulators (Kumar et al., 2023; Liu et al., 2022; Bran et al., 2023), and scientific knowledge bases (Kraus et al., 2023; Thulke et al., 2024).
2. Related Work
LLM Alignment. Alignment techniques aim to make LLMs behave in accordance with human values, using meth- ods such as supervised fine-tuning (SFT) (Zhang et al., 2024a; Scheurer et al., 2023; Dong et al., 2023; Yuan et al., 2023; Song et al., 2024) and reinforcement learning
Among these tools, scientific tools (simulators and knowl- edge bases) provide consistent results that could potentially internalized by the model, yet prior works have not ex-
2

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
(a) Supervised Fine-tuning
1. Retrieve Date: 2051, Place: Ribeira Brava, Setting: ssp337
Cuuti-Choice Question
21.89'C
Solution
How much will the temperature of
LLM
Ribeira Brave in 2051 under ssp337
2. Emulate Scienario Time: 2051, 1, Place: Ribeira Brava, Setting: sxp337, Detta CUty 0.25
LLM
increase by 22 12 21-21.100 0.23 "C The answer C.
change incressee by 25% ? A. 0.05 B. 0.12 C, 0.25 D.0.30
CD2
Tools
22.12'C
(b) Preference Optimization
Rank
Temperature: 6.3424'C
Open-Ended Question
Design new maritime route between
Busan and Margaret River that would increase BC levels slong the routa by 3.0%, minimiize the globall average tampanture increas sxp585.
LLM
Tools
Temperature: 6.3449'C
under
y/ Invalid: Interact with the land.
y>
Figure 2. WKL training strategies: (a) For both determinate and open-ended questions, we first train the L.LM to directly generate solutions (obtained from tool interactions during the solution generation phase). (b) For open-ended questions, additionally, we sample an ensemble of trial proposals, rank the proposals using predefined metrics, and convert the rankings into preference pairs for DPO training.
plored this opportunity. Furthermore, existing studies have not addressed training LLMs to make adaptive decisions about tool usage based on problem complexity, often result- ing in over-reliance on the tools covered during training.
trace t: {In).. Both the solution y and the tool trace I are the labels in our dataset, which can as training targets, respectively, depending on the difficulty of the question. The process can be formalized as:
These limitations highlight the need for a training approach that enables LLMs to use tools adaptively and reach a bal- ance between answering with internal knowledge and seek- ing help from external tools.
y~ g~ m(. xr (IR),
(I)
3.2. World Knowledge Learning (WKL)
In WKL, we finetune a pre-trained model Tg. where 0 rep- resents the trainable parameters for finetuning, generate solutions y directly without tool usage. The no-tool-use restriction is specified in the prompt Pn. This process is formalized as:
3. Methodology
As shown in Figure 1, our pipeline first generates solutions through tool interactions for each question (Section 3.1). The training process consists of two components: World Knowledge Learning (WKL), where the model is trained to internalize the knowledge directly (Section 3.2), and Tool Usage Adaptation (TUA), which classifies questions easy or difficult based on the precision of the model's 's direct re- sponse without tools. We maintain direct-answer targets for easy questions, while changing the training targets to tool traces for hard questions (Section 3.3). To ensure the consistency of knowledge between components, we design combined loss across WKL and TUA (Section 3.4). Fi- nally, we extend the framework open-ended questions by incorporating preference optimization (Section 3.5).
JDireca (0, P)
(2)
E D.w (J))..)) [log Toly xx P)],
where D represents the training dataset. The loss for WKL is then:
JWKL (0, D) = (0, (0, P.).
(3)
While WKLaims to internalize knowledge for direct prob- lem solving, certain complex problems are still too challeng- ing to learn. Therefore, we follow with TUA to train the model to intelligently switch to tools for these problems.
3.1. Generating Solutions and Tool Traces
3.3. Tool Usage Adaptation (TUA)
As shown in Figure 2, we developed automated solution generation pipeline that produces both direct responses and tool-use traces. The LLM ग receives access to scientific tools E (e.g. numerical simulators) via system prompts. Given the context of the question z with a labeled, correct tool trace f, we force the LLM to use the tools through by prompt Py- The LLM then generates solution y by combining the context with the returned information from
TUA begins with partitioning the dataset into easy/hard subsets by evaluating the model after it has been trained with WKL for an epoch. For each question, we sample an ensemble of directly generated answers to calculate the accuracy rate. If the accuracy is higher than a predefined threshold, the question classified as easy, resulting in two subsets: Deasy for problems the LLM can solve directly, and Drand for the remaining ones. We now relax the constraint
3

---

Adapting While Learning:
Grounding LLMs for Scientific Problems with Tool Usage Adaptation
(a) Question Generation Pipeline
(C) Open-Ended Questions
Expert
Question Templates
Multi-Choice Questions with Answers
Climate
Consultation
How much will the temperatuore of [[iitt]] in Hyear]] under Hscenarioo] change the emission CH4 is (increase/becreesse)) by [ppercenagel]%?
Design a maritime route from Singapore Dubai increases SO, levels by 3. slong the route. The proposal should have the lnast impact on global temparature projections for 2033 under SSP128
Tool Functionality
Open-Ended Questions
(b) Multi-Choice Questions
Mujoco
PDE
Epidemiology
In a physics lalooratory, double pendulum experiment is set up with the tollowing parameters: [panameters]] How does the posltion of the first pendulum change over 1-second obsarvation period?
In 2D population spread process with [initial condition] aud [bonndary confitionn what the population at (xy)-139,2) km after years? (D=0.86, Lxm86 Lyma km
You are tasked with managing Califeriia' response during pandemic. Given fsomario description) [initial states], With a 2.9- unit budget adjust omega home interventions (max 0.3 per county), how should you allocate across Californie's 's counties to minimize the peak Home Severe incidence during the pandemic?
Climate
Epidemiology
How will the temperature
Given fscenarii description) [initial states) On which day will the number of hospitalized cases in California reach its maximum?
Gikongore in 2018 under sep585 change emissiio CO2 is decressed -25%?
Figure (a) Question generation pipeline using templates. Selected demo (b) multi-choice and (c) open-ended questions from our custom-created datasets.
on tool usage and let the model choose whether to use tools based on the question's 's difficulty (with prompt denoted as P.). The success of making the correct binary decision is achieved implicitly by maintaining different alignment targets for easy/hard subsets: For easy problems Deasy, we keep the alignment target as direct answering; however, for hard problems Daard we switch the alignment target to the augmented solution with tool usage trace and guide the LLM to follow the trace outside of the tool set E. For Daard the alignment loss reads:
We attribute this performance drop to the loss of consistency between different prompts: As noted above, during deploy- ment, only the prompt P. will be used enable the adaptive switch to tools for more difficult problems; however, the internalization of knowledge for easy questions was only optimized using the prompt P.. Recent work (Zeng et al., 2024) indicates that knowledge acquired under one prompt may not transfer well to another, potentially leading to per- formance degradation.
To address this problem, we propose a combined loss term that simultaneously optimizes the alignment for both com- ponents, eliminating the need for alternation between com- ponents:
Jrace (0, D)
(4)
(...D,-a( log (t E).
The combined training loss considering both easy and hard questions in the whole dataset reads:
JMx (8, D, Dewyy Drana) JWKI D) + JreA (0, Deesy, Drunt).
(6)
JTUA (0, Deasy, Dhard) (Oreaa(O, Deasy, P.) + JTrace Dhard P.).
(5)
Due this combined loss term and elimination of the alter- nation, we conduct the partition (easy/hard) at the beginning of each epoch (instead of between epochs). Importantly, this mixing strategy differs from simply re- re-weighting terms in (5), as explicitly contains both prompts P, and P..
We note again that we apply the same prompt P. for all questions during TUA. This consistency is important for deployment, when we do not have labels for new questions, requiring the I.M to implicitly decide on its own whether to use tools or not (i.e., we must use P.).
3.5. Extension to Open-ended Questions with DPO
Real-world scientific applications often include open-ended problems such as design, planning, and optimization. These tasks present distinct challenges that require modifications to our pipeline:
3.4. Knowledge Consistency Across Components
A naive approach to combining the two components would be to alternate the training loss terms between adjacent epochs. However, in our preliminary experiments, we ob- served significant performance drops for easy problems using that approach.
Instead of fixed ground-truth answers, these problems require evaluating, comparing, and ranking different

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
Table Answer Accuracy (%) across different datasets and models. All baselines prompt Pa (no tool usage). Our haseline model evaluated with both P. (intelligent tool usage) and Py (forced tool usage). We report metrics for the model after AWL using Ps. and P. as Py forces tool usage and leads to no difference. highlight results ranked first and second.
Models
Mujoco
PDE
Climate
Epidemiology
MATH
SciBench
Average
Llama3.1-70B
46.79 79
55.83
37.50
30.83
73.53
45.00
48.25
GPT4o
52.86
69.17
35.83
32.50
82.94
71.67
57.50
GPT4o-mini
51.79
70.83
30.00
35.83
75.29
68.33
55.34
Claude3.5-Somee
48.57
65.83
32.50
35.00
77.65
67.50
54.51
Llama3. 1-8 (Basc)-P, P.
26.76
20.83
39.17
18.89
54.71
20.83
30.20
Llama3.1 1.0.3....5000 1-8B (Basc)-P,
57.14
59.17
76.67
58.89
55.89
29.17
56.16
LIama3. 1-8B (Base)-P,
59.32
61.67
77.50
57.78
57.64
31.67
57.60
LJama3. 1 1-8B (Base)-ICL
61.43
59.17
76.67
58.89
49.41
26.67
55.37
Llama3. 101----AWIE AWL-P.
56.07
75.00
81.67
51.11
61.18
30.83
59.31
64.17
78.33
83.33
74.44
62.35
34.17
66.13
proposals using domain-specific metrics, necessitating a modified dataset generation approach.
the L.M can generate successful plans with a higher-than- threshold probability, while hard problems are those where successful generation is less probable.
Tool verification (e.g., experiments or simulations) is often expensive, requiring models to develop strong internal knowledge to efficiently generate proposals with higher success rates. We address this through a modified WKL.
Modified TUA. For harder problems where single-shot proposals are likely to fail, the model is prompted to fol- low the expanded trace t', i.e., generating an ensemble of proposals within a certain resource budget, followed by rigorous evaluation of every proposal, ranking them, and finally selecting the optimal solution.
In some applications like aircraft design, the design is hard, and failures can be catastrophic. The model must therefore still recognize the necessity of external verification, if needed, despite its high cost. We achieve this through modified TUA.
4. Experiments
4.1. Dataset
Modified Data Generation. For each task, we generate an ensemble of trial proposals using the LLM. These pro- posals are evaluated using domain-speciff tools (c.g. neural climate simulators that output the future temperature map), with task-specific metrics L post-processed from the tool outputs (c.g., averaging the temperature map difference to obtain the average temperature rise). The metrics enable for ranking and pairing preference formation among proposals. The expanded tool trace l' now encompasses: ensemble generation, proposal evaluation and ranking, and then the final optimal proposal selection.
We employ two public benchmark datasets, MATH (Hendrycks et al., 2024) and SciBench (Wang et al., 2024a), and construct four new scientific datasets for our experiments: Mujoco, Partial Differential Equations (PDEs), Climate Science, and Epidemiology. Detailed descriptions, statistics, and demo questions of all datasets are presented in Appendix A.
As shown in Figure 3, our custom dataset construction fol- lows a systematic pipeline. First, we design domain- specific question templates based on both the expert consultation and the simulator functionality. We then generate individ- ual questions by sampling parameters within scientifically valid ranges. Finally, for multi-choice questions, we use the simulator to precompute the correct answers, while for open-ended questions, we design metrics to evaluate both the validity and quantitative aspects of model- gener-eeee solutions. We present some demo questions for our custom- created datasets in Figure 3. (b) and (c).
Modified WKL. We augment the standard SFT loss as in (6) with standard DPO loss term (Rafailov et al., 2024) using pairwise preferences derived from the ensemble of proposals. This helps the model learn from the relative outcomes of different proposals and increases the probability of generating a proposal that meets the requirement.
Modified Easy/Hard Questions Partition. As there are no longer "golden answers" in open-ended questions, we replace the "accuracy rate" with the "success rate"- the pro- portion of proposals within the ensemble that meet prede- fined requirements (e.g., temperature rise below a specified limit). In this framework, easy problems are those where
The Mujoco dataset involves problems in rigid- and soft- body dynamics, integrating real-world complexities such as stiffness, damping, and friction based on the Mujoco physics engine (Todorov et al., 2012). The PDEs dataset focuses on solving 1D and 2D partial differential equations
5

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
Table 2. The Accuracy of Tool Usage. The models after AWT. demonstrate remarkable accuracy across all datasets. In contrast, most other
models show accuracy around 50% which indicates an inability to make intelligent decisions on tool usage.
Models
Mujoco
PDE
Climate
Epidemiology
MATH
SciBench
Average
Llama3.1-70B
49.66
50.00
48.67
48.94
56.09
50.93
50.71
GPT4o
50.30
52.41
48.70
50.57
43.73
50.00
49.28
GPT4o-mini
50.34
52.35
48.81
61.84
46.39
68.36
54.68
Claude3.5-Someet
50.39
51.27
49.38
54.95
49.96
54.37
51.72
L-ama3.1-8B (Base)
51.61
49.05
48.32
48.63
50.09
59.58
51.21
Llama3.1-88 1-B (Base)-ICL
54.08
50.00
50.96
48.63
53.19
55.09
51.99
Llama3.1-88 -WLL
61.60
66.67
63.45
67.00
62.09
62.75
63.93
in areas like heat transfer and population dynamies using in-house numerical solvers. The Climate Science dataset leverages neural surrogate model (Niu et al., 2024) to gen- erate questions based on different places, climate scenarios (e.g..ssp122 ssp126, ssp245), greenhouse gas emissions, etc. The Epidemiology dataset is built using surrogate model (Wu et al., 2023) that predicts cpidemiological states based on multi-dimensional input features.
AWL training. More details on the training data and training process can be found in Appendix A.2 and Appendix D.
Tools. We employed different tools for each dataset. For Mujoco, we designed custom scenarios 9 (such as double pendulum system and friction tests), where each scenario is wrapped in a corresponding API. For PDEs, we developed in-house numerical solvers for different scenarios (such as transient and steady-state heat transfer for ID and 2D domains) and provided their APIs, respectively. For the Climate and Epidemiology datasets, we employed APIs that encapsulated the respective neural surrogate models of these dynamics. For MATH and SciBench, of related libraries (e.g.. SymPy and NumPy) as tools and let the LLM generate scripts to use these tools. The details related to open-ended questions, such as the thresholds and trial budgets, are provided in Appendix A.3
All data sets comprise questions with definite answers. In our custom-created datasets, these questions are in the form of multiple-choid questions (MCQs), while public datasets contain only questions with numerical answers. In addition, the Climate and Epidemiology data sets include open-ended questions (e.g., policy proposals for climate change miti- gation). As these questions lack definitive golden answers, they require an improved pipeline learn the preference between different proposals (see Section 3.5).
treated the APIs
4.3. Evaluation Metries
4.2. Experiment Setup
We primarily evaluate two types of accuracy: Answer Accu- racy and Tool Usage Accuracy.
Models. We used Ll ama- 3. 10-1--85 -----nnnttttt (Dubey al., 2024) as the base model. For performance comparison to the base model, we consider two variants of prompts (no tool use Pn and force tool use Py), as the base model has not been trained on tool selection. For our post-trained model, we consider two prompt variants (Px and intelligent tool use P.). Furthermore, we include baseline that enhances P. with in-context learning (ICL), where a few examples with correct tool-use decisions are provided in the prompt as context. For frontier models, we consider four other open and closed source state-of-the-ar (SOTA) models, namely GPT4o, GPT4c-mini, Claude-3.5-Sonnee and Llama-3. ther closed-source or too computationally expensive for implementing our post-training approach.
Answer Accuracy. Answer accuracy quantifies the pro- portion of correct answers provided by the models. For multiple-choice questions (MCQs) in our custom-created datasets, assign binary scores based whether the model selects the correct choice. For numerical answers, the MATH dataset uses prior math-specific evaluation method (Yang et al., 2024), while the SciBench dataset follows the official evaluation approach in its paper, where answers are correct if they fall within +55% of the true value.
These models are ei-
Tool Usage Accuracy. Tool usage accuracy assesses the model's ability to make intelligent decisions about tool us- age: using tools for difficult questions while answering directly for easier ones. Questions are classified as easy (E) hard (H) based on the model's accuracy without tools (P.). With the P: prompt, the model decides whether to use tools (T) or not (N) for each question. Our tool usage accuracy is defined as x ( EN
Training. For our custom datasets, we constructed a col- lection of questions and randomly split them into training and test data sets. We utilized the standard dataset configura- tion for MATH. Since SciBench does not provide a training set, we randomly split it into training and test data sets. In the main experiments, we performed two iterations of
EN+ET + HN+HHT ). where a value
HT
close to 100% indicates ideal tool usage decisions, while 50% suggests random Decision-making We note that alter-
6

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
1.0
Proportion
0.
04
Easy-Noc Choosing
02
Easy-Taol
Hard-Tool
Clande Level
Claude AWL Level
Clande Level
Claude Level
ClandeAWL Level
Hard-Not Choosing Tool
Figure 4. Tool usage decision of different models on MATH dataset of 5 difficulty levels. Investigated models are Claude-3.5 3.5-onnntttt Llama-3.1-800 1-8B-Base, and
native definitions may be more suitable for specific cases and provide additional metries in Appendix E.1.
Tool Usage Accuracy. We present the tool usage accuracy in Table 2. Overall, trained model achieves the best tool usage accuracy across all datasets, except SciBench, where it ranks second, demonstrating the ability to make intelligent decisions on tool usage. In contrast, other models exhibit accuracy around 50%, indicating two typical cases: either overreliance on tools never attempting to use them (more empirical evidence is presented in Appendix E.3).
5. Results
5.1. Overall Performance
Answer Accuracy. We report the comparison of answer accuracy in all data sets using different models in Table I, both under tool-free (Px) and tool-using (P./P)) settings.
Furthermore, investigate the tool decisions in the MATH dataset, which provides prior labels of difficulty lev- els, as illustrated in Figure 4. Our trained model exhibits a reasonable increase in tool usage as the difficulty of the ques- tion increases, while the base model shows an overreliance on tools regardless of difficulty. A notable exception among the baselines is Claude-3.5, which demonstrates greater con- fidence in answering questions directly for both easy and hard questions, possibly because MATH is public dataset and has been covered during the pretraining phase.
On public datasets, our model surpasses the base model after training. However, it falls short of closed models, probably because tasks within open datasets were extensively covered during the pre-training of these models (Anthropic, 2024; Achiam et al., 2023; Dubey et al., 2024).
Another contributing factor is the relatively small size of our base model compared closed-source frontier mod- els, which limits its reasoning capabilities complex benchmarks. To verify this hypothesis and the scala- bility of our approach, we post-trained larger model (Qwen2.
with AWLon PDE, Mu- joco, MATH, and SciBench. As shown in Table 3, the larger post-trained model consistently demonstrates improved an- swer accuracy and tool usage accuracy, narrowing the per- formance gap these open-source datasets and empirically validating our method's scalability. Notably, on the MATH dataset, the larger post-trained model achieves performance approaching that of state-of-the-art closed-source models.
Incorrect
Correct
Base-Pa
Base-Pr
AWL-P,
AWL-P.
Figure 5. Different models' performance on the PDE Dataset: com- paring pre- and post-training, with and without tool usage.
Component-wise Performance Analysis and Visualiza- tion. We visualize the progressive performance improve- ment of the model on the PDE dataset in Figure 5, showing transitions across different prompt strategies and training components of AWL The first transition, from Base-P, to Base-Py shows natural improvements by using tools. The transition from Base- to AWL-P,, demonstrates nearly identical performance, confirming successful internaliza- tion of knowledge through WKL. The subsequent transition from AWL-P, to AWL-P, shows how questions that are too challenging to internalize are effectively solved by intel- ligently switching to tools, resulting in further performance gains.
Table Answer Accuracy (%) under Pa. and P., and Tool Usage Accuracy (%), of Qween_.511B-Instrrue before and after training.
PDE
Mujoco
MATH
SciBench
Ans Acc. Base-P.
61.67
54.28
74.12
17.50
Ans Acc. AWL-P,
78.33
60.00
81.18
56.67
Ans Acc. Base- P.
69.17
44.28
79.41
46.67
Ans Acc. AWL-P:
80.00
62.85
82.35
65.83
Tool Usage Acc. Base
48.91
50.00
48.45
48.84
Tool Usage Acc. AWL
63.58
54.16
54.69
58.54

---

Adapting While Learning:
Grounding LLMs for Scientific Problems with Tool Usage Adaptation
Miscellancous Analysis on Tool Usage. For the sake of conciseness in the main text, include additional miscel- laneous analysis on tool usage in Appendix E. Specifically, Appendix E. provides additional metrics for analysis; Ap- pendix F.2 shows the evolution of tool usage decisions over training epochs; and Appendix E.3 compares the tool us- age decisions our method and baseline methods on our custom-creates and open datasets, respectively.
approach in different numbers of evaluation samples (k: 3, and 5) assess its variation.
Climate
PDE
1,00
Proportion
0.75
0.50
0.25
Base
wie WKL
w/o TUA
AWL
Bese
wie W.L.
w/o TLA
AWI
Table Percentage of responses that satisfy the constraints and meet pre-established quality threshold.
Etsy-Not Choosing Tool
Hard-Tool
Essy-Tool
Hard-Not Choosing
Dataset
Base
Base-P;
AWL
AWL.-P
Climate
31.82
29.09
37,50
40.17
Epidemiology
17.50
22.50
33.75
53.75
Figure 6. Composition of Tool Usage Decisions: Impact of indi- vidual training components in ablation study.
Dataset
AWL-RRI
AWI-RII--1
GPT4o
Claude3.5
Climate
47.50
49.16
34.17
31.51
Epidemiology
41.25
56.25
43.75
36.25
Functionality Sub-components. Figure presents an ablation study the functionality of WKL and TUA by evaluating the proportion of the four tool use decisions (EN, ET, HN, HT). We observe that omitting either component leads to tool over-reliance. Moreover, without WKL, the model exhibits the lowest answer accuracy, as it is never trained on distilled knowledge directly.
Table Tool Usage Accuracy (1; first line) and Tool Usage Rate (J. second line) across different models, respectively.
Dataset
Tool Usage Metrics
GPT4o
Clande3.5
Base
AWL-RL
Climate
Accuracy (%)
50.00
50.00
49.37
56.57
Rate (56)
92.50
100.00
Accuracy (
50.00
50.00
44.16
55.76
Epidem.
Rate (%)
100.00
100.00
88.75
26.25
This ablation shows the necessity of both components in our approach: WKL for knowledge internalization and TUA for intelligently switching to tool usage.
5.2. Open-ended Questions
Climate Dotaset
We evaluated approach open-ended questions of cli- mate and epidemiology datasets, comparing performance across several baselines. Table 4 reports the percentage of proposals that meet predefined requirements. The results show that models trained with preference learning (denoted as "AWL-RL") show significant improvements over those trained with only our original approach ("AWL"), highlight- ing the benefits of incorporating both preference learning and the TUA component.
OUT
0.79
AWAL
3.4
0.2
24
40
50
106
PDU Dotset
SFT
AWA
Table 5 shows the accuracy of tool usage in all models. Compared to both the base model and closed-source fron- tier models, our trained models achieve better tool usage accuracy while reducing the overall frequency of tool calls. Notably, this improved efficiency is substantial in saving computational cost: our method reduces the average call to tools per question from 7.21 to 2.70 in Climate tasks and from 2.80 to 0.42 in Epidemiology tasks, all without com- promising output quality. We provide additional win-rate comparisons across all models in Appendix F.
0.6
E:55
24
40
10
166
Noise (%)
Figure 7. Model Performance vs. Noise Level: Comparison be- tween our two-component method and SFT-only approaches on Climate and PDE datasets.
Robustness Against Noisy Data. Generating solutions via LLMSs human expert annotation inevitably introduces noise. Since such noisy training data can degrade model performance, we examine how our method's robustness compares to model trained with only SFT under increasing noise levels. The results are shown in Figure 7.
5.3. A Ablation Study
We chose the Climate and PDE datasets to perform ablation studies on the functionality of WKL and TUA, respectively, as well as the impact of noise level on the performance of our method. We also examined the robustness of our
8

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
The performance of the WKL-only model degrades dras- tically with increasing levels of noise, as the underlying distribution becomes polluted. However, this does not sig- nificantly impact the trained model with P.. The model judges these polluted questions as harder and opts to use tools to ensure accuracy, demonstrating the robustness of our approach. As noise levels increase, the performance of the SFT-only method declines, while models trained with our method demonstrate robust performance.
that the workflow complexity exceeded the base model's capabilities. The remaining incorrect answers were further categorized into (a) calculation errors, (b) reasoning errors, and (c) knowledge gaps.
We utilized GPT-4o to annotate these error types by pro- viding it with conversation logs and ground-truth answers. Table presents the proportional distribution of each error type before and after training. Additional details on the annotation methodology, absolute error counts, and exam- ples of each enror type are provided in Appendix G. Notably, AWL reduced the absolute number of errors in all categories.
Table Sensitivity analysis on the problem difficulty threshold.
Answer Accuracy (%)
Tool Usage Accuracy (%)
This analysis reveals two key findings. First, there is a de- crease in the proportion of reasoning errors and knowledge gaps after training, suggesting a successful internalization of scientific reasoning processes and domain-specifie knowl- edge, primarily through the WKLcomponent. Second, the predominant remaining errors after training involve agent limitations, indicating that using stronger base models with enhanced capabilities for longer or multihop tool interac- tions could further improve performance.
MATH
SeiBench
MATH
SciBench
Base
Ours
Base
Ours
Basc
Ours
Base
Ours
54.71
62.09
17.50
30.83
50.09
62.09
60.22
62.75
65.88
72.35
30.00
54.16
57.73
64.37
52.38
58.74
74.11
75.88
37.50
55.83
62.16
65.36
52.22
58.27
Table 6 reports both answer accuracy and tool usage accuracy for the base and trained models under each k. We observe that our method consis- tently improves performance in all values of k. Although absolute accuracy increases with larger l: for both base and post- trained models, the relative improvements of our method remain stable. This confirms that our approach is ro- bust to variations in the difficulty threshold and generalizes well across different partitioning strategies.
Sensitivity to Sample Size.
6. Conclusion and Future Work
We introduced novel two-component post-training ap- proach to enhance Large Language Models (LLMs) in solv- ing scientific problems of varying complexity. Our approach equips LLMs with the ability to intelligently choose be- tween using appropriate tools or conducting basic reasoning by assessing the difficulty of a problem, resembling the adaptive problem-solving strategy of human experts. Exper- iments across diverse datasets demonstrate that our method significantly improves the performance of smaller base model: on average, our fine-tuned models achieve 29.11% increase in answer accuracy and a 12.72% improvement in tool usage accuracy across all datasets, even enabling them to surpass larger frontier models such as GPT-4o and Claude-3.5 on newly created custom datasets.
Table 7. Distribution of error types (in percentages) across datasets for base and trained models. Eroor Type
Mujaco
PDB
Climate
Epidem.
Base Model
Agent Limitation
35.29
61.05
27.40
36.99
Calculation Mistakes
0.00
0.00
0.00
0.00
Reasoning Errors
45.10
26.32
24.66
23.29
Knowledge Gaps
19.61
12.63
47.95
39.73
AWL
Agent Limitation
47.97
80.00
40.91
45.45
Our method strikes balance between reliability and cost, and we expect it to serve as a foundation for building re- liable, cost-effective scientific assistants. We note several promising directions for future research. First, our cur- rent approach requires domain-specifie fine-tuning; future work could explore unified training across related scientific domains. Second, while our method uses binary classi- fication of problems easy or hard, real-world problems often involve a spectrum of complexity, suggesting the need for more granular difficulty evaluation and tool selection strategies. Additionally, stepwise adaptive tool utilization could further enhance automation in multi-step scientific workflows, where different tools are required for various steps. Finally, expanding our method to handle multi-modal inputs and outputs would broaden its applicability.
Calculation Mistakes
3.25
3.33
4.55
0.00
Reasoming Errors
39.02
16.67
13.64
22.73
Knowledge Gaps
9.76
0.00
40.91
31.82
5.4. Failure Mode Analysis
To better understand the impact of our method and identify areas for future improvement, we performed failure mode analysis of model outputs under the tool-free setting (Pr).
For questions that our post-trained model incorrectly swered, we first examined whether each question could be solved by giving the model access to tools (using Py). If the problem remained unsolvable, for instance, due to requiring complex sequences of multiple tool interactions, we clas- sified it as failing due to "agent limitation", which means

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
Acknowledgment
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. Training verifiers to solve math word problems. arXiv preprint 11Xiv2110014100 2021.
This work was supported in part by the U.S. Army Research Office under Army- -CASEE award W91 INF-07-R-0000--00 the U.S. Department Of Energy, Office of Science, IARPA HAYSTAC Program, and NSF Grants #2205093, #2146343, #2134274, CDC-REAA-FT-23-0069, DARPA AIE FoundSci and DARPA YFA.
Dong, H., Xiong, W., Goyal, D., Zhang, Y., Chow, W., Pan, R., Diao, S., Zhang. J., Shum, K., and Zhang, T. Raft: Re- ward ranked finetuning for generative foundation model alignment. arXiv preprint urXiv:2304.06767. 2023.
Impact Statement
This paper presents work whose goal advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang. A., Fan, A., and others, The llama 3 herd of models. arXiv preprint arXiv:2407.2178. 2024.
Farquhar, S., Kossen, J.. Kuhn, L., and Gal, Y. Detecting hallucinations in large language models using semantic entropy. Nature, 630(8017) 2024.
References
Achiam, J., Adler, S., Agarwal, S., Ahmad, L.,, Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., and others, Gpt-4 technical report. arXiv preprint #rXiv23303.08774 2023.
Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardt, J. Measuring math- ematical problem solving with the math dataset, 2021. URL https:/aaxxiv. arg/abs/2103.035
Anthropic The claude model family: Opus, sonnet, haiku. ht:ps://www-coe http:://wwww.cch..hho......... anthropic com/ 8857627
2024.
Huang, Y., Shi, J., Li, Y., Fan, C., Wu, S.. Zhang, Q. Liu, Y., Zhou, P., Wan, Y., Gong, N. Z., and others, Meta- tool benchmark for large language models: Deciding whether to use tools and which to use. arXiv preprint arXi::2310.03128, 2023.
Model_Caree @card_claude__ pdf, March 2024. Ac-
cessed: 2025-01-30.
Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., McK- innon, C., and others, Constitutional ai: Harmless- ness from ai feedback. arXiv preprint arXiv:2212.08073, 2022.
Kraus, M., Bingler, J., Leippold, M., Schimanski, T., Senni, C. C., Stammbach, D., Vaghefi, S., and Webersinke, N. Enhancing large language models with climate resources. Technical report, Swiss Finance Institute, 2023.
Bran, A. M., Cox, S., Schilter, O., Baldassari, C., White, A. D., and Schwaller, Chemcrow: Augmenting large- language models with chemistry tools. arXiv preprint arXiv:2304.05376, 2023.
Kruger, J. and Dunning, D. Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self assessmenss. Journal of personality and social psychology, 77(6):1121, 1999.
Cai, T., Wang, X., Ma, T. Chen, X., and Zhou, D. Large language models as tool makers. arXiv preprint arXiv:2305. 17/26, 2023.
Kumar, V. V., Gleyzer, L., Kahana, A., Shukla, K., and Karniadakis, E. Mycrunchgpt: A chatgpt assisted framework for scientific machine learning. Journal of Machine Learning for Modeling and Computing. 2023.
Cheng, K., Sun, Q. Chu, Y., Xu, F., YanTao, L., Zhang, J., and Wu, SeeClick: Harnessing GUI grounding for advanced visual GUI agents. In Proceedings of the 62nd Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), pp- 9313-9332, Bangkok, Thailand, August 2024. Asso- ciation for Computational Linguistics. URL https: //aclanthology wrt.:/rrg////22 conelllong.... 505.
Lee, H., Phatale, S., Mansoor, H., Mesnard, T., Ferret, J., Lu, K., Bishop, C., Hall, E., Carbune, V., Rastogi, A., and others, Rlaif: Scaling reinforcement learning from human feedback with ai feedback. arXiv preprint MwXiv:2309.0022 2023.
Liu, R., Wei, J., Gu, S. S., Wu, T., Vosoughi, S., Cui, C., Zhou, D., and Dai, A. M. Mind's eye: Grounded language model reasoning through simulation. arXiv preprint arXiv:2210.05352 2022.
Chithrananda, S., Grand, G., and Ramsundar, B. Chemberta: large-scale self-super pretraining for molecular prop- erty prediction. arXiv preprint (iiv:20/0.00885/ 2020.
10

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
Lu, P., Mishra, S., Xia, T., Qiu, L., Chang, K., Zhu, S., Tafjord, O., Clark, P., and Kalyan, A. Learn to explain: Multimodal reasoning via thought chains for science ques- tion answering. In The 36th Conference on Neural Infor- mation Processing Systems (NeurIPS), 2022.
Rein, D., Hou, B. L., Stickland, A. C., Petty, J., Pang, R. Y., Dirani, J., Michael, J., and Bowman, S. R. GPQA: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024.
Scheurer, J., Campos, J. A., Korbak, T., Chan, S., Chen, A., Cho. K., and Perez, E. Training language mod- els with language feedback at scale. arXiv preprint arXiv:2303. 16755, 2023.
Luo, R., Sun, L., Xia, Y., Qin, T., Zhang, S., Poon, H.,, and Liu, T. Biogpt: generative pre-trained transformer for biomedical text generation and mining. Briefings in bioinformatics, 23(6):bbac409, 2022.
Schick, T., Dwivedi- Yu, J.. Dessi, R., Raileanu, R., Lomeli, M., Hambro, E., Zeurleooerr I.., Cancedda, N., and Scialom, T. Toolformer: Language models can teach themselves to use tools. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
Ma, P., Wang. T., Guo, M., Sun, 7.,, Tenenbaum, J. B., Rus, D., Gan, C., and Matusik, W. Lim and simulation bilevel optimizers: A new paradigm to advance physi- cal scientific discovery. In International Conference on Machine Learning. PMLR, 2024.
Song, F., Yu, B., Li, M., Yu, H., Huang, F,, Li, Y., and Wang, H. Preference ranking optimization for human alignment. Proceedings of the AAAI Conference on Artificial In- telligence, volume 38, pp. 18990-18998, 2024.
Meng, Y. Xia, M., and Chen, D. Simpo: Simple preference optimization with reference-free reward. arXiv preprint arXiv:2405. 147344 2024.
Niu, R., Wu, D., Kim, K., Ma, Y., Watson-Parris, D., and Yu, R. Multi-fidelit residual neural processes for scal- able surrogate modeling. In International Conference Machine Learning, ICML 2024, Proceedings of Machine Learning Research, 2024.
Tang, Q. Deng, Z., Lin, H., Han, X., Liang, Q., and Sun, L. Toolalpaca: Generalized tool learning for language models with 3000 simulated cases, 2023.
Taylor, R., Kardas, M., Cucurull, G., Scialom, T., Hartshorn, A., Saravia, E., Poulton, A., Kerkez, V., and Stojnic, R. Galactica: A large language model for science. arXiv preprint arXiv:221 1.09085 2022.
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., and others, Training language models to follow instructions with human feedback. Advances in neural 2022. Patil, S. G., Zhang. T., Wang. X., and Gonzalez, J. E. Gorilla: Large language model connected with massive apis, 2023.
Thulke, D., Gao, Y., Pelser, P., Brune, R., Jalota, R., Fok, F., Ramos, M., Wyk, I., Nasir, A., Goldstein, H., and others, Climategpt: Towards synthesizing interdis- ciplinary research on climate change. arXiv preprint arXiv:2401.09644 2024.
information processing systems,
Payne, J. W., Bettman, J. R., and Johnson, J. The Adaptive Decision Maker. Cambridge University Press, New York, 1993. doi: 10. 10.101//88978 117/C889981199113333 139173933.
Todorov, E., Erez, T., and Tassa, Y. Mujoco: A physics engine for model-based control. In 2012 IEEF/RSI Inter- national Conference on Intelligent Robots and Systems, pp. 5026-5033 IEEE, 2012. doi: 10. N0.1109/RRS..20 2012 6386109.
Qian, C., Han, C., Fung, Y., Qin, Y., Liu, Z., and Ji, H. CREATOR: Tool creation for disentangling ab- stract and concrete reasoning of large language mod- els. In Bouamor, H., Pino, J., and Bali, K. (eds.), Find- ings of the Association for Computational Linguistics: EMNLP 2023, pp. 6922-6939, Singapore, December 2023. Association for Computational Linguistics. doi: 10. 00.18633/v1/0022 1/03./////.....ndiinngsss-//// findings- findings-emnlp4622
Wang, X., Hu, Z., Lu, P., Zhu, Y., Zhang, J., Subramaniam, S., Loomba, A. R., Zhang, S., Sun, Y., and Wang, W. Scißench: Evaluating College-l Scientific Problem- Solving Abilities of Large Language Models. In Pro- ceedings of the Forty-First International Conference on Machine Learning, 2024a.
Qin, Y., Liang, S., Ye, Y., Zhu, K., Yan, L., Lu, Y., Lin, Y., Cong, X., Tang, X., Qian, B., and others, Toolllm: Facilitating large language models to master 16000+ real- world apis. arxiv preprint arXiv:2307. 2023.
Wang. Z., Cheng, Z., Zhu, H., Fried, D., and Neubig, G. What are tools anyway? a survey from the language model perspective. arXiv preprint arXiv:2403. 2403.15452. 2024b.
Rafailov, R., Sharma, A., Mitchell, E.,, Manning, C. D. Er- mon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. Ad- vances in Neural Information Processing Systems, 36, 2024.
Wu, D., Niu, R., Chinazzi, M., Vespignani, A., Ma, Y., and Yu, R. Deep bayesian active learning for accelerating stochastic simulation. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 2023.

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., Huang, F., and others, Qwen2 technical report. arXiv preprint arXiv:2407.1067 2024.
Yao, S., Chen, H., Yang, J., and Narasimhan, Web- shop: Towards scalable real-world web interaction with grounded language agents. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems, vol- ume 35, pp. 20744-20757. Curran Associates, Inc., 2022.
Yao, S., Zhao, J.. Yu, D., Du, N., Shafran, Narasimhan, K., and Cao, Y. React: Synergizing reasoning and act- ing in language models. In International Conference Learning Representations (ICLR), 2023.
Yu, B., Baker, F. N., Chen, Z., Herb, G., Gou, B., Adu- Ampratwum, D., Ning, X., and Sun, H. Tooling not tooling? the impact of tools on language agents for chem- istry problem solving, arXiv preprint 2XXi:2411.007223 2024.
Yuan, H., Yuan, Z., Tan, C., Wang, W., Huang, S., and Huang, F. RRHF: Rank responses to align language mod- cls with human feedback. In Seirth-seventh Conference on Neural Information Processing Systems, 2023.
Zeng, A., Liu, M., Lu, R., Wang, B., Liu, X., Dong, Y., and Tang, J. AgentTuning: Enabling generalized agent abilities for LLMs. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Findings of the Association for Computational Linguistics ACL 2024, pp. 3053- 3077, Bangkok, Thailand and virtual meeting, August 2024. Association for Computational Linguistics. doi: 10. 10.18653/v1/004..mmee 181.
Zhang, B., Liu, Z., Cherry, C., and Firat, O. When seal- ing meets IIm finetuning: The effect of data, model and finetuning method. arXiv preprint 1rX11:2402.17193, 2024a.
Zhang, D., Hu, Z., Zhoubian, S., Du, Z., Yang, K., Wang, Z., Yue, Y., Dong, Y., and Tang, J. Sciglm: Training scientific language models with self-reflective instruction annotation and tuning, 2024b.
Zheng, Y., Zhang, R., Zhang, J., Ye, Y., Luo, Z., Feng, 7.,, and Ma, Y. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting ofthe the Association for Computational Lin- guistics (Volume 3: System Demonstrations), Bangkok, Thailand, 2024. Association for Computational Linguis- tics.
12

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
A. Dataset Details
We utilize two existing public datasets, MATH and SciBench, alongside four custom scientific datasets that we developed: Mujoco, Partial Differential Equations (PDEs), Climate Science, and Epidemiology. Below, we provide detailed descriptions of the datasets, along with the tools employed construct and evaluate them.
A.1. Overview
A.1.1. MUJOCO
We developed the Mujoco dataset to address problems in rigid- and soft-body dynamics. This dataset based on the Mujoco physics engine (Todorov et al., 2012), which simulates realistic physics scenarios. Previous work introduced dataset comprising 39 qualitative questions and trained LLMS to solve them using MuloCo simulations. However, this benchmark has proven to be too simplistic for current models, which can achieve 100% accuracy with case. To address this limitation, we have developed new dataset consisting of 8 distinct scenarios of different complexity based on a public turtorial '. Each scenario contains an average of 14.5 adjustable parameters, including variables such as the initial position and velocity of objects, time constants, damping ratios, friction coefficients, and the gravitational acceleration of the environment.
A.1.2. PDE (PARTIAL DIFFERENTIAL EQUATION)
The PDE dataset focuses on solving partial differential equations in fields such as heat transfer, chemical engineering, and population dynamics. We wrote 1-D and 2-D partial differential equation solvers for the diffusion process, which can be set with different variables like diffusion coefficient and size of the field, and different kinds of initial situations and boundary situations with different parameters.
A.1.3. CLIMATE
The Climate Science dataset comprises problems related to predicting earth surface temperature changes based on climate scenarios. The dataset is built using a neural surrogate model (Niu et al., 2024) that integrates data across multiple fidelity levels for robust climate modeling. The model utilizes 12 climate driver variables as input, encompassing total emissions of greenhouse gases (CO2, CH+) and the first five principal components of global aerosol gas (BC, SO2) distributions, derived from 72x96 global grid. The output predicts air temperature 2 meters above the Earth's 's surface scale. The model spans historical data from 1850-2015 and projects future scenarios from 2015 to 2 2100 under four Shared Socioeconomic Pathways (SSPs): ssp126, ssp245, ssp370, and ssp585. These scenarios range from sustainable development with low challenges to mitigation and adaptation (ssp126) to fossil-fueled development with high challenges to mitigation and adaptation (ssp585), representing a spectrum of potential future climate states and associated societal responses.
global
A.1.4. EPIDEMIOLOGY
The Epidemiology dataset focuses on simulating disease spread and predicting epidemiological states time. This dataset is based on state-of-the-art surrogate model (Wu et al., 2023) that predicts disease progression using multi-dimensional input features. For the California epidemic scenario, the input consists of two components: 1. county-level data for 58 counties, including 24 features per county per day over 28 days, 2. 10 initial state-level features. The output predicts 10 state-level features for each of the next 28 days.
A.1.5. MATH
MATH (Hendrycks et al., 2024) is a widely used benchmark that consists of high-school-level mathematics competition problems. The dataset covers various topics such algebra, geometry, and number theory, and is divided into five difficulty levels. It remains challenging compared with another renowned math dataset GSM8K (Cobbe et al., 2021), where current 7B LLMs already achieve over 80% accuracy. Following previous work (Qian et al., 2023), we utilize problems from the MATH test set with definite numerical answers to evaluate our methods.
'https://pab4.. gi thub. to/mujocopy.hor 1
13

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
A.1.6. SCIBENCH
SciBench (Wang al., 2024a) collegiate-Ievel benchmark that includes scientific problems in fields such Mathematics, Physics, and Chemistry. Like MATH, the problems numerical and focus real-world scientific applications. We use the SciBench dataset to evaluate models complex numerical problems.
A.2. Statistics
Table 8 shows the statistics of the seven datasets used in our experiments. For custom datasets (Mujoco, PDE, Climate, and Epidemiology), show the number of scenarios and question templates used to generate the problems. The existing datasets (MATH and SciBench) are from established benchmarks that do not provide information about scenarios and templates.
Table Statistics of the datasets: number of questions in training and test sets, and number of scenarios and question templates where applicable. MATH and SciBench from existing benchmarks that do not provide information about scenarios and templates.
Dataset
Train Questions
Test Questions
Scenario
Templates
Multi Multi-Choice Questions
Mujoco
960
280
9
53
PDE
1627
120
36
5
Climate
640
120
5
19
Epidemiology
1720
90
1
Numerical Questions
MATH
1600
170
SciBench
266
120
Open- Ended Questions
Climate
582
120
1
1
Epidemiology
493
80
1
1
A.3. Details in Open-Ended Problems
Thresholds. In evaluating open-ended questions. we employ quantitative thresholds as acceptance criteria. For climate questions, a proposed maritime route is deemed acceptable if its implementation contributes a global mean temperature increase not exceeding C. In epidemiological questions, policy interventions considered successful when the resultant indicator falls below the critical threshold of 0.1 in the specified measurement framework.
Budgets. For climate questions, the validation of proposal components is constrained to maximum of uses of the corresponding tool, while the quantitative assessment tool is limited to 3 applications. For epidemiological questions, single integrated tool is utilized to simultaneously evaluate both validity and quantitative metrics, with its usage capped at 3 instances.
Easy/Hard Problem Partition. If the model's answers meet the thresholds in least 4 out of 5 attempts under P., the question is classified as an easy problem. Otherwise, the question classified hard problem.
A.4. Question Examples
We provide question examples in our custom-created datasets with different scenarios and question templates.
14

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
In physics laboratory, double pendulum experiment is up with the following parameters:
Gravitational acceleration: -9.61 m/s?
Mass of first pendulum rod: 0.1 kg
Mass of first pendulum bob: 0.07 kg
Mass of second pendulum rod: 0.17 kg
Mass of second pendulum bob: 0.2 kg
Sliding friction coefficient: 0.11
Torsional friction coefficient: 0.68
Rolling friction coefficient: 0.21
Initial angle of the first pendulum: 0.98 radians
Initial angular velocity of the first pendulum: 0.86 rad/s
Initial angle of the second pendulum: 2.21 radians
Initial angular velocity of the second pendulum: -0.87 rad/s
The pendulum released and its motion is observed for seconds.
How does the position of the second pendulum change over the 5-second observation period?
(A) Stable
(B) Steady increase by 14.4%
(C) Fluctuating, decrease by 40.3%
(D) Fluctuating, overall stable
In physics laboratory, rolling ball experiment is set up with the following parameters:
Gravitational acceleration: 9.27 m/s?
Initial position: 0.79 meters
Radius the ball: 0.12 meters
Mass of the ball: 2.78 kg
Sliding friction coefficient: 0.58
Torsional friction coefficient: 0.35
Rolling friction coefficient: 0.2
Initial velocity: -1.15 m/s (X), 4.01 m/s (Z)
Initial angular velocity: 1.27 rad/s (Y)
- Damping coefficient: 0.23
The ball rolled and its motion is observed for seconds.
What the range positions (in meters) that the ball occupies during its motion in the 1-second observation period?
(A) [-0.36, -0.27]
(B) [-0.27, -0.18]
(C) [-0.18, -0.09]
(D) [-0.09, -0.00]
Figure 8. Example questions in the Mujoco Dataset.
15

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
What is the average temperature of Palenga in 1869?
(A) 21 21.084519988496
(B) 23.720084953308
(C) 26.355649948120
(D) 28.991214942932
What is the temperature of Toumoukro in 2035 under ssp370 if the emission of CH4 is increased by 25%?
(A) 22.50640016655777
(B) 25.3197080612183
(C) 28.133008956902
(D) 40.9463098526001
How much will the temperature of Hamalah in 2047 under ssp126 change the emission of CH4 is decreased
-00%
(A) -0.02068302
(B) -0.00741459
(C) -0.01771736
(D) -0.02278250
What is the range of temperature of Soweto in 2063 under different climate settings?
(A) [21 1083115777999961, 22.5793609619
(B) [22.579366961111406 21.350406666722552
(C) 23.33040064662285,, 24. 24112153331142997
(D) (24.12145233154297, 148922298013377421
For Ebreichsdorf, Gleisdorf, Perchtoldsdorf, Voitsberg, which city has the lowest temperature in 2058 under
ssp245?
(A) Perchtoldsdorf
(B) Ebreichsdorf
(C) Gleisdorf
(D) Voitsberg
What is the minimum level of agreement we should support if we want to control the temperature of Rocha
2083 under 18. 18.7069814300537133
(A) ssp370
(B) ssp245
(C) ssp585
(D) ssp126
Figure Example multi-choice questions in the Climate Dataset.
Design new maritime route between Singapore (lon: 103.8, lat: 1.3) and Dubai (lon: 55.2972, lat: 25.2631) that would increase SO2 levels along the route by 3.0%. Propose a route that would minimize the global average temperature increase 2033 under ssp126. Present your answer as a list of coordinates (longitude, latitude) representing key points along the route. Format your response as follows: [(longitude_1, latitude_ (longitude 2, latitude.2). ... (longitude_n, latitude .n)] Include at least the starting point, endpoint, and any significant waypoints. Ensure that the distance between any consecutive points in your list no less than 2 degrees in either latitude longitude. Note that for straight segments of the route, you only need provide the coordinates for the start and end of that segment, without listing all points along the straight line. The route will be automatically connected based on the nodes you provide.
Figure 10. Example open-ended question in the Climate Dataset.
16

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
In ID chemical diffusion experiment, the initial concentration is uniformly at 28 mol/I.. Dirichlet boundary conditions are applied, with the concentration fixed at 13 mol/L at 0 and 6 mol/L at r L, where L = 4 em. The diffusion coefficient is D 0.0007 cm²/s. After 253 seconds, what is the maximum concentration (mol/L)?
(A) 19.502
(B) 22.288
(C) 25.074
(D) 27.86
In ID population spread process, the initial population density is 60 individuals/km' for x < L/2 and 30 individuals/km' for 2 L/2, with Neumann boundary conditions (zero flux at the boundaries). The domain length is = 44 km and the diffusion coefficient D 0.68 km²/year. What is the maximum population density (individuals/km²) after years?
(A) 60.0
(B) 66.0
(C) 72.0
(D) 78.0
In 2D heat conduction experiment, the initial temperature follows checkerboard pattern with alternating
regions of 100 9C and 0 "C Dirichlet boundary conditions are applied with temperatures of "C, 14 C, 73 "C, and 21 "C at the left, right, bottom, and top boundaries, respectively. The domain dimensions are L,, 65
em and Ly = 6 cm, and the diffusion coefficient is D = 0.21 cm²/s. After 356 seconds, what is the minimum temperature (?C)?
(A) 8.0
(B) 8.9
(C) 7.1
(D) 10.4
In a 2D chemical diffusion experiment, the initial concentration follows checkerboard pattern with alternating regions of 100 mol/I. and mol/I.. Neumann boundary conditions (zero flux at the boundaries) used, with the domain dimensions
L., 1 and Ly 10 em. The diffusion coefficient is D 0.0006 cs After 1000 seconds, what is the maximum concentration (mol/L)?
(A) 1:3.50492315544777733, 20. 20.00626361945248)
(B) [20.00626331945524,, 37.7411505592285544 (C) [37.741540559285945, 82.61383728899433)
(D) (82.61383728899933,, 97.38889999110777)
In a ID chemical diffusion experiment, the initial concentration is set at 75 mol/1.. Dirichlet boundary conditions are applied, with the concentration fixed at 88 mol/L at x and mol/L x = L, where = 4 cm. The diffusion coefficient is = 0.0009 cs After 50 seconds, what the maximum gradient of concentration (mol/L per cm)?
(A) 144.82
(B) 159.302
(C) 173.784
(D) 188.266
Figure 11. Example questions in the PDE Dataset.
17

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
In an epidemiological study simulating the spread of disease across California, daily data from 58 counties over 28 days is used to model disease transmission dynamics. Each county has 10 imput features per day, and the model is initialized with 24 state-level features.
The 10 county-level features are:
O: seasonality min
1: omega community interventions
2: omega work interventions
3: omega school interventions
4: omega home interventions
5: alpha school interventions
6: transit commute interventions
7: international travel interventions
8: domestic travel interventions
9: RO
The 24 state-level features are:
0: prevalence CA state total Latent
1: prevalence CA state total Infectious symptomatic
2: prevalence CA state total Infectious asymptomatic
3: prevalence CA state total Hospitalized
4: prevalence CA state total ICU
5: prevalence CA state total Removed asymptomatic
6: prevalence CA state total Removed symptomatic
7: prevalence CA state total Home asymptomatic
8: prevalence CA state total Home mild
9: prevalence CA state total Home severe
10: prevalence CA state total Removed hospitalized
11: prevalence CA state total Deaths hospitalized
12: incidence CA state total Latent
13: incidence CA state total Infectious symptomatic
14: incidence CA state total Infectious asymptomatic
15: incidence CA state total Hospitalized
16: incidence CA state total ICU
17: incidence CA state total Removed asymptomatic
18: incidence CA state total Removed symptomatic
19: incidence CA state total Home asymptomatic
20: incidence CA state total Home mild
21: incidence CA state total Home severe
22: incidence CA state total Removed hospitalized
23: incidence CA state total Deaths hospitalized
In this simulation, the following data is provided: { siimulation parameters}
Given the data, on which day does the maximum value of incidence CA state total ICU occur?
(A) Day? 7
(B) Day 1
(C) Day 25
(D) Day 9
Figure Example multi-choice question in the Epidemiology Dataset.
18

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
You are the leader of California, tasked with managing the state during pandemic. Your role is to formulate effective policies to mitigate its impact by allocating resources and making key decisions. You will be provided with specific details of the pandemic, including its severity, affected regions, and other relevant data. Based on this information, you will be asked to design a policy.
Given the following information: ((information of county names and feature n
{parameters of the original scenario))
You are given total budget of 2.9 to adjust 'omega home interventions' across all counties. Each county's adjustment cannot exceed 0.5. How would you allocate this budget to minimize the peak value of pprevalence total Death Hospitalized"?
Figure 13. Example open-ended question in the Epidemiology Dataset.
19

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
B. Prompt Examples
Pn for our custom-created datasets:
Answer the following question. Your answer should be in the following format: Solution: <Your solution process> Answer: <Your answer, one of A/B/C/D>
Question: ((question))
P; for our custom-created datasets:
Given the Following functicns, please respond with JSON for function call with its proper arguments that best answers the given prompt.
Respond in the format ("namc": function nanc, "parameters"; dictionary of argument namc and its value). Do not use variables.
(functions))
If you don't know the answer, you can use the tool to help you. If you can answer the problem without the tool, answer the proplem directly.
Question: ((question))
Pn for SciBench and MATH:
Answer the following question. Your answer should be in the following format: Solution: <Your solution process>
Answer: <Your answer, pure number>
Question: ((question))
P; for SciBench and MATH:
Flease answer the following question. You can write code to solve the problem or give the answer directly. When answering, you should first give the Solution then give the Answer The answer should be pure number without LaTeX or unit signs. Zach time, you should either write code or answer the question. Your final answer should in one of the following formats:
If you want to write code, your should be in the following format:
Thought: <Your thought>
Action: wrice_and_run_com
Code:
'pyyhhn
<Your code>
If you want to answer the question, you should answer in the following format:
Thought: <Your thought>
Action: answer_questton
Solution: kYour solution>
Answer: <Your answer>
Question: | [Iquestion]
20

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
For Py. remove descriptions about intelligent tool usage in the above prompts, requiring the of tools. If the model directly answers the question, will ask the model to use tools before answering.
In scenarios involving tool usage (Py and P,), to ensure consistency in the format of the model's 's responses, we design an "answer question" tool. If the model intends to answer question, it will invoke this tool and return the answer within the tool's parameters.
Following are 2 examples of tool descriptions. The first one a climate simulator:
"type"; "function",
"function;
"name": "ra__ggreenhuse",
"description": "Predict the temperature of place in the future under a speci climate scenario with DIY change of CO2 and CH4 based the original setting.", "parameters";
"cypo": "object",
"properties":
"longitude";
"type": "number",
"descriptica": "The longitude of the place you would check the temperature for, a float from -180 to 180."
"latitude":
"type": "number",
"description" "The latitude of the place you would check the temperature for, a float from -90 to 90."
"setting":
"type": "string",
"enum"; "ssp126",
"ssp245",
"ssp370",
"ssp585"
I, "descriptica": "Future climate scenarios, string from ssp126, ssp245, ssp370, s9p585."
"year": {
"type": "number",
"descripticn": "The year you would check the temperature for, an integer from 2015 2100." "
"delta_co2":
"type": "number",
"descriptica": "The change of C02 you would like make, float. co2_after refore (1 delta_co2)."
"delta_CH4"; "type": "number",
"descriptica"; "The change of CH4 you would like to make, float. CH4 after CH4 before
(1 + selta_CH4).
"thought":
"type": "string", "description": "Your internal reasening thoughts why you call this function."
"required"; "Longitude", "latitude",
21

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
"sstting", "year", "delta_co2" "delta_CH$
The second one is the "answer question" function introduced above.
"type": "function",
"function":
"name": hanswer_question
"description"; "Answer the Question.",
"parameters":
"type": "object",
"properties":
"thought":
"type": "string",
"description": "Internal reascning and thoughts of why you call this function."
"answer"; "type": "string", "enum" "A", "B",
"C",
"D"
},
"description": "Your answer to this question." "
}, "sloving_processs" {
"type": "string", "descripticn": "Detailed list how do you solve this question, step by step
"requi irequired!. red": [ "thought", "answer", proceving_proces"" process"
22

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
C. Solution Examples
C.1. Example from the Climate Dataset
Problem:
Question: How much will the temperature Ewo in 2068 under ssp126 change the emission of CO2 is
decreased by- -25%?
(A) -0. 10.1301711
(B) -0. 169864666
(C) -0.09639389
(D) -0.05745936
Correct Answer: B
Problem-solving without tool usage:
answer this question, would need to know the specifics of the climate model used (e.g., CMIP6 model) and the scenario used (ssp126). However, since this information is not provided, I'll provide general framework for how might approach this problem.
1.
Choose climate model: Select suitable climate model that has been run under the ssp126 scenario.
2. Select location: Identify the location of Ewo (presumably small town in the Democratic Republic of Congo).
3. Run the climate model: Run the climate model for the year 2068 under the ssp 126 scenario with -25% reduction in CO2 emissions.
4. Extract the temperature change: Extract the temperature change at the location of Ewo for the year 2068 from the climate model output.
5. Compare with options: Compare the extracted temperature change with the provided options select the correct answer.
However, without access to the climate model output or the specific details of the scenario, I'll provide a placeholder answer.
Solution: Run climate model for the year 2068 under the 2sp126 scenario with -25% reduction CO2 emissions and extract the temperature change the location of Ewo. Therefore, the answer is D.
Problem-solving with tool usage.
Assistant: Call query lat and lon with parameters ('city name': 'Ewo'}
Feedback from query and lon: latitude: -0.8742, longitude: 14.8167.
Assistant: Call diy greenhouse with parameters { ('longitude': 14.8167, 'latitude'; -0.8742, 'setting': 'ssp126',
'year': 2068, 'delta_CO2': -0.25, 'delta.CH4': 0}
Feedback from diy greenhouse: The temperature is 27.52265167236328
Assistant: Call retuerrtemprrature with parameters ('longitude": 14.8167, 'latitude": -0.8742, 'year': 2068.
'setting': 'ssp126''}
Feedback from future taaurettemerature. The temperature is 27. 27.69251632669A42
Assistant: Call answer questtin with parameters ('thought': 'The temperature for Ewo 2068 under the ssp126 scenario with no changes emissions is 37.6925166326044222
With a 25% decrease in CO2 emissions, The difference is 27.6 77.692516326904297 27.52255167236228
the temperature is 27.5226516723622
0.1 1116866655441015562 'answer': 'B', 'sloving "sloving_proces:" process': '1. Retrieved the latitude and longitude of Ewo. 2. Predicted the temperature for 2068 under the ssp126 scenario with 25% decrease in CO2 emissions. 3. Retrieved the temperature for 2068 under the ssp126 scenario with changes in emissions. 4. Calculated the difference between the two temperatures to find the change."}
The solution generated based on the information from tool usage, which then used for World Knowledge Learning training:
Direct answer under P; prompt:
23

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
To calculate the temperature change in Ewo by 2068 under the ssp126 scenario with a 25% decrease in CO2 emissions, first identify Ewo's s location latitude -0.8742 and longitude 14.8167.
The temperature for Ewo in 2068 under the ssp126 scenario with changes in emissions is determined to be 27.6925 C. With 25% decrease in CO2 emissions, the temperature is calculated to be 27.5227
The difference between the two temperatures is:
27.6925 C 27.5227 = 0.1699
This matches option B from the given choices. Therefore, the answer is B.
{ ('name": aanswerqqeestiont "parameters': ('thought': 'I can answer the problem directly.', "answer': " To calculate the temperature change in Ewo by 2068 under the ssp126 scenario with a 25% decrease in CO2 emissions, we first identify Ewo's location at latitude -0.8742 and longitude 14.8167. The temperature for Ewo in 2068 under the ssp126 scenario with no changes in emissions is determined to be 27 C. With 25% decrease CO2 emissions, the temperature calculated to be 27.5227 difference between the two temperatures is:27.6925 C 0.1699 C. This matches option B from the given choices. Therefore, the answer is }}")}
C.2. Example of Open-Ended Question
Problem:
You are the leader of California, tasked with managing the state during pandemic. Your role is to formulate effective policies to mitigate its impact by allocating resources and making key decisions. You will be provided with specific details of the pandemic, including its severity, affected regions, and other relevant data. Based on this information, you will be asked to design policy.
Question:
Given the following information: ((information of county names and feature names)
[parameters of the original scenario)
You are given a total budget of 2.9 to adjust 'omega home interventions' across all countics. Each county's s adjustment cannot exceed 0.5. How would you allocate this budget to minimize the peak value of 'prevalence total Death Hospitalized"?
Problem-solving without tool usage:
Solution: Based on the data provided, I'll prioritize allocating resources to larger population centers and areas showing higher hospitalization trends. Los Angeles, San Diego, and other major metropolitan areas should receive more substantial adjustments to their work interventions, while maintaining smaller but strategic allocations to mid-sized counties.
Answer: { "Los Angeles": 0.2, "San Diego": 0.2, "Orange": 0.2, "Riverside"; 0.15, "San Bernardino": 0.15,
"Santa Clara": 0.1, "Alameda": 0.1, "Sacramento": 0.1, "Contra Costa": 0.05, "Fresno": 0.05, "San Francisco":
0.05, "Ventura": 0.05, "San Mateo": 0.05, "San Joaquin": 0.05
Problem-solving with tool usage.
24

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
Assistant: Action:
Simulate Thought: To optimize the peak value of 'incidence CA state total Hospitalized by the last day, will allocate the budget to the counties with the highest impact on the incidence of hospitalizations. will start by identifying the counties with the highest incidence of hospitalizations Day 0 and allocate higher portion of the budget to these counties. Data: The county with the highest incidence of hospitalizations Day 0 is Los Angeles, followed by San Francisco, Fresno, Orange, San Bernardino, and San Diego.
Answer: { (oLos : 0.2675, "San Francisco": 0.2675, "Fresno": 0.2675, "San Bernardino": 0.2675, "San
Diego": 0.2675
Feedback: The peak value of "incidence CA state total Hospitalized" is 0.280065
Assistant: Action: Simulate
Thought: Based on the feedback from the simulator, realize that the initial allocation was optimal. will re-evaluate the data and notice that the counties with the highest incidence of hospitalizations Day 0 are indeed Los Angeles, San Francisco, Fresno, San Bernardino, and San Diego, but the peak value of 'incidence CA state total Hospitalized is not as high as expected. This suggests that the allocation was not optimal. will try to allocate the budget more evenly among the counties with the highest impact on the incidence of hospitalizations. Data: The counties with highest incidence of hospitalizations on Day are Los Angeles, San Francisco, Fresno, San Bernardino, and San Diego.
Answer: "Los Angeles": 0.2000, "San Francisco": 0.2000, "Fresno": 0.2000, "San Bernardino"; 0.2000, "San
Diego": 0.2000, "Orange": 0.0675, "Alameda": 0.0675 }
Feedback: The peak value of 'incidence CA state total Hospitalized" is 0.27842450141906744
Assistant: Action: Answer
Thought: Based on the feedback from the simulator, realize that the second allocation was not significantly diffcrent from the first one, resulting in a similar peak value of 'incidence CA state total Hospitalized', will re-evaluate the data and notice that the counties with the highest incidence of hospitalizations on Day 0 are Los Angeles, San Francisco, Fresno, San Bernardino, and San Diego However, I will allocate the budget more efficiently by assigning the maximum allowed adjustment to the county with the highest incidence, Los Angeles, and the remaining budget to the other counties in proportion their incidence rates. Data: The county with the highest incidence of hospitalizations on Day 0 is Los Angeles, followed by San Francisco, Fresno, San Bernardino, and San Diego.
Answer: "Los Angeles"; 0.2675, "San Francisco": 0.1446, "Fresno": 0.0675, "San Bernardino": 0.0675, "San
Diego": 0.0675, "Orange": 0.0, "Alameda": 0.0}
25

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
C.3. Example from the SciBench
Problem:
Question: A 10.0 g block with charge of +8.00 x 10~5C is placed in an electric field E 600)) What is the magnitude of the electrostatic force the block? The unit of the answer is N.
Correct Answer: 0.245
Problem-solving without tool usage:
Solution:
The force on the block is the product of the electric field E and the charge g of the block:
gE.
Calculations: We can find F with the equation for the magnitude of vector and the given values of E and q:
F |gEi = 1 + E3 2
(3000 N N 6-600 C/
99.00 10t N²/C² 3.60 10S /C2
9.36 x 10t N2 /²/C== v9 x 10 N2 /2222
3.06 10 N/C.
Answer: 3060000 (x)
Problem-solving with tool usage:
26

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
Assistant:
Thought: To find the magnitude of the electrostatic force on the block, we will use the formula F = q where is the charge of the block and E' is the magnitude of the electric field. Since the electric field is a vector, we first need to find its magnitude using the Pythagorean theorem. Action: write and run code Code:
# Given data
q 8.00e-5 charge in
E_x 3000 electric field N/C
E_y -500 # electric field in N/C
Magnicude of the electric field (F_x**2 f_y+*2)**0..
print (MMagniiuue of the electric field (E): (EI N/C")
# Electrostatic force
print rostacctr force (F) (F) N*)
Tool Feedback: Magnitude of the electric field (E): 3059.41170815555 N/C Electrostatic force (F): 0.2447529366524537 N
Assistant:
Thought: The magnitude of the electric field was calculated using the Pythagorean theorem, and the electrostatic force was then found by multiplying the charge of the block by the magnitude of the electric field. The calculated electrostatic force is 0.244759936524537 N. Action: answer_question Solution: First, find the magnitude of the electric field E' using the Pythagorean theorem:
E =
F2 + F
Given: E, 3/00 E, N/C Substitute the values:
E = v (3000N/C)² (-600 N/C)2
E v 9000000 360000
E 9360000
E = 3059. ...41 111177115677 N/C
Next, find the electrostatic force using the formula:
F F=
Given: Charge of the block q 8.00 10-$C C
Substitute the values:
F = (8.00 x 10 C) x (30599411708155671 N/C)
0.244772933632253377
Therefore, the magnitude of the electrostatic force on the block is 0.24475293665245377 N. Answer: 0.244753 (v)
27

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
Solution generated based the information from tool usage, which is then used for World Knowledge Learning training:
Solution: We aim determine the magnitude of the electrostatic force 10.0 g block with charge of +8.00 10-5 C placed in an electric field (3000i 600ÿ) N/C.
Given Data:
Charge of the Block (g): +8.00 10-5 C
Electric Field (E): (3000) 600}) N/C
Analysis:
1. Magnitude of the Electric Field (E):
To find the magnitude of the electric field, we use the Pythagorean theorem:
E = F + F
Given Values:
E, 3000 N/C
Fy. = 600 N/C
Substituting Values:
= /3000 N/C)2 + 600 N/C)²
v 99000000 360000
E 99360000
E 3059 41598411701155677 N/C
2. Electrostatic Force (F):
The electrostatic force on the block is given by the formula:
F = gE
Given Values:
8.00 x 10 C
Substituting Values:
== (8.00 x 10 5 C) X (30599411108811567) N/C)
F 0.2447559300524537 N
Conclusion:
The magnitude of the electrostatic force the block is 0.244753 N. Answer: 0.244753 N
28

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
D. Fintune Backbone Details
For training, we employ Llama-Factory (Zheng et al., 2024) as the LLM training platform. Table 9 shows our training hyperparameters for both supervised fine-tuning and DPO training. For the preference optimization training Section 5.2, we first perform supervised fine-tuning using the preferred answers from the preference dataset, then apply LoRA for DPO training ensure model robustness. All training is performed the L40S and A100 servers.
For inference, we deploy open-source models internally on our server and utilize the APIs of proprietary models, respectively.
Table 9. Hyperparameters for supervised fine-tuning and DPO training with LoRA.
Full-purameter Supervised Fine-tuning
Parameter
Value
Train batch size
Learning rate
1.0e-5
Number of epochs
3.0
LR scheduler
cosine
Warmup ratio
0.1
Precision
b
DPO Training with LoRA
Parameter
Value
LoRA target
all
LoRA rank
8
DPO beta
0.1
Train batch size
32
Learning rate
5.0e-6
Number of epochs
3.0
LR scheduler
cosine
Warmup ratio
0.1
Precision
bfl6
29

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
E. Additional Analysis of Tool Usage Accuracy
E.1. Other Metrics for Analysis
Here we provide detailed analysis of tool usage accuracy across various models and datasets. We first elucidate the categorization of tool usage decisions Table 10. In the table, we categorize decisions into four types based on problem difficulty (Basy Hard) and tool usage choice (Tool Not Choosing Tool). Easy problems correctly without using tools, while Hard problems assistance. The Tool Not Choosing Tool distinction represents the model's decision to use not use tools when given the option. Therefore, EN (Easy problems solved without tools) and HT (Hard problems solved with tools) are expected in the aspect of intelligent tool usage.
those that the model those that the model cannot answer correctly without
Table 10, Explanation Tool Usage Decision, where v indicates the expected decisions: not choosing tools for casy problems (EN) and using tools for hard problems (HT').
Tool (T)
Not Choosing Tool (N)
Easy (E) Hard (H) HT (V)
ET
EN (V)
HN
The following tables present various metries evaluate tool usage across different models and datasets. Table 11 employs EN balanced mcasure of tool usage accuracy, computed as 1 ( EN+ + HT #N+HTT giving equal weight to performance both problem types to address potential dataset imbalances. Tables 12 and 13 disaggregate this metric into easy and hard HT+HN respectively. These assess the models' ability to recognize when tool usage unnecessary for simpler tasks and beneficial for complex problems. Table 14 measures the difference in tool usage rates between hard and easy problems, computed as HT+HN ET+EN Higher values indicate better selectivity, with tools used more for hard problems and avoided for easy ones, while lower values suggest over-reliance on tools. Table 15 presents the accuracy of tool usage decisions without accounting for potential class imbalances, computed ET HT HNN with
problem categories, measured by EN+ET and MT
EN
ET
PEN+ET: HNN Finally, Table 16 quantifies the proportion of total tool usage, calculated EN+ lower values indicating more selective tool use.
EN+ HT
Table 11. The Accuracy of Tool Usage, measured with
ENTET
HN
Models
Mujaco
PDE
Climate
Epidemiology
MATH
SciBench
Average
Llama3.
49.66
50.00
48.67
48.94
56.09
50.93
50.71
GPT4o
50.30
52.41
48.70
50.57
43.73
50.00
49.28
GPT4o-mini
50.34
52.35
48.81
61.84
46.39
68.36
54.68
Claude3.5-Sonne.
50.39
51.27
49.38
54.95
49.96
54.37
51.72
Llama3.1-88 (Base)
51.61
49.05
48.32
48.63
50.09
59.58
51.21
Llamaš. 1-88---WII
61.60
66.67
63.45
67.00
62.09
62.75
63.93
EN Table 12. The Accuracy of Tool Usage for easy problems, measured with EN+
Models
Mujoco
PDE
Climate
Epidemiology
MATH
SciBench
Average
Llama3.1-70B
0.00
0.00
0.00
2.70
94.40
85.19
30.38
GPT4o
1.35
4.82
0.00
30.77
70.21
0.00
17.86
GPT40-mini
0.69
4.71
0.00
41.86
54.69
68.29
28.37
Staude3.5-Somee
1.47
2.53
0.00
38. 38.10
89.39
72.84
34.06
Llama3.1-8 1-BB (Base)
6.58
16.00
2.13
0.00
5.38
36.00
11.01
Llama3. 1-8B-AWL
48 41
86.67
63.27
52.17
71.15
35.14
59.47
E.2. The Evolution of Tool Usage Accuracy with Training Epochs
Figure 14 illustrates the evolution of our model's performance in the form of different solution types (EN, ET, HN, HT) on the Climate dataset at different training epochs.
As training progresses, we observe a significant increase in the proportion of correct direct answers (blue bars), indicating
30

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
Table 13. The Accuracy of Tool Usage for hard problems, measured with
RT
HT-MN'
Models
Mujoco
PDE
Climate
Epidemiology
MATH
SciBench
Average
Llama3.1170B
99.33
100.00
97.33
95.18
17.78
16.67
71.05
GPT4o
99.24
100.00
97.40
70.37
17.24
100.00
80.71
GPT4o-mini
100.00
100.00
97.62
81.82
38.10
68.42
80.99
Claude3.5 Souude..5-Soree S-nnnett
99.31
100.00
98.77 77
71.79
10.53
35.90
69.38
Ilama3.1 1-88 (Base)
96.63
82.11
94.52
97.26
94.81
83.16
91.41
Llama3. 1 1-8B-AWL
74.80
46.67
63.64
81.82
53.03
90.36
68.38
Table 14. Difference of Tool Usage Rate between Hard and Easy problems, measured with BT
HT
ET BT+EN
Models
Mujoco
PDE
Climate
Epidemiology
MATH
SciBench
Average
Llama3.1-70B
01.67
0.00
2..7
-2.12
12.18
1.85
1.43
GPT4o
0.59
4.82
-2.6
1.14
-12.55
0.00
-1.43
GPT4o-mini
0.69
4.71
-238
23.68
-7.22
36.71
9.36
Claude3,5-Somes
0.78
2.53
1.23
9.89
-0.08
8.74
3.44
Llama3.1 (Base)
3.21
-1.99
3.35
-2.74
0.18
19.16
2.43
Llama3.1 1-8B-AWL
23.20
33.33
26.90
33.99
24.18
25.50
27.85
Table 15. The Accuracy of Tool Usage, measured with
EN+82 HT
EN4 #T
Models
Mujoco
PDE
Climate
Epidemiology
MATH
SeiBench
Average
Llama3.
52.86
44.17
60.83
66.67
74.12
47.50
57.69
GPT4o
47.50
34.17
62.50
57.50
61.18
28.33
48.53
GPT4o-mini
48.57
32.50
68 68.33
67.50
50.59
68.33
55.97
Souuee.5-Sonna.
51.79
35.83
66.67
60.00
71.76
60.83
57.81
I.amm3.1 (Base)
72.54
68.33
58.33
78.89
45.88
73.33
66.22
Llama3. --BL-AWL
60.00
76.67
63.33
66.67
64.12
73.33
67.35
Table 16. The Proportion of Tool Usage (J), measured with
ET TT
Models
Mujoco
PDE
Climate
Epidemiology
MATH
SciBench
Average
Llama3311-7B
99.64
100.00
98.33
95.83
8.82
15.83
69.71
GPT4o
98.93
96.67
98.33
70.00
27.65
100.00
81.93
GPT4o-mini
99.64
96.67
98.33
73.33
43.53
43.33
75.81
Clanne3.5-Sonnet
98.93
98.33
99.17
68.33
10.59
30.00
67.56
Llama3.1 1-B (Base)
95.77
82.50
95.83
97.78
94.71
79.17
90.96
Llama3. 1-8B-AWL
61.79
21.67
41.67
64.44
38.24
82.50
51.72
3.0
0.8
Proportion
0.6
0.
Easy-Not Choosing Toul
Easy-Tool
Hard-Taa
Hard-Not Choosing Tool
Base
Epoch
Epoch 2 Epoch 3 Epoch 4
Epoch
Figure 14, Composition of Tool Usage Decisions in Climate Dataset Training: Evolution growing momentum training terms.
31

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
successful knowledge internalization. Additionally, there is notable decrease in tool over-reliance (initially, orange and gray bars dominate nearly 100%) and an increase in tool usage for hard questions (orange bar). This demonstrates the effectiveness of our training approach in intelligently switching to tool usage only when question is hard.
E.3. Composition of Tool Usage Decisions across Open and Custom Datasets
Figure 15 illustrates the composition of tool usage decisions for different models both custom and public datasets. We observe that for custom datasets, the closed models tend to over-rely tools, whereas for open datasets, they tend to provide direct answers. This empirically supports our hypothesis that closed models have encountered similar questions in open datasets and are familiar with the answers.
Easy-Not Choosing Tool
Easy-Tool
Hard-Tool
Hard-Not Choosing Tool
MATH
0.8
Proportion
0.4
0.2
0.0
0.0
Claude3
Llama3.1 1 -70B
GPT4o-mini
GP
Base
AWL
Claude3.5
Llama3. I 1-70B
GPT miiii
GPT4o
Base
AWL
Figure 15. Composition Tool Usage Decisions Different Models on Both Custom Public Datasets.
32

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
F. Pairwise Win Rate Comparison for Open-ended Questions
Figure shows the win rate comparisons between different models on open-ended problems. For the climate dataset, our AWL-RL-P; model achieves win rates of upproximately 70% against base models and 59% against closed models. The epidemiology dataset shows stronger performance, with win rates of over 80% against base models and 65-80% against closed models. These results validate method's s effectiveness in handling complex open-ended scientific problems.
Climate Dataset
Base
50.0
45.0
43.3
34.2
31.7
29.2
42.5
61.7
Base-Pr-
55.0
50.0
45.8
40.8
38.3
30.8
43.3
69.2
Winning Method
AWL
56.7
54.2
50.0
55.0
48.3
45.8
50.8
77.5
AWT.Pi
65.8
59.2
45.0
50.0
48.3
47.5
57.5
77.5
90
AWL-RL
80
AWL-RL-Pi-
70.8
69.2
54.2
52.5
51.7
50.0
65.00
80.8
GPT4p
57.5
56.7
49.2
42.5
35.0
35.0
50.0
67.5
70
Claude3.5
38.3
30.8
22.5
22.5
20.8
19.2
32.5
50.0
-60
Epidemiology Dataset
Win Rate (%)
Base
50.0
55.0
47.5
15.0
42.5
17.5
15.0
15.0
-50
Base-Pi-
45.0
50.0
51.2
20.0
46.2
21.2
23.8
16.2
Winning Method
AWL
400
AWL-Pi
85.0
80.0
67.5
50.0
61.3
38.8
57.5
46.2
330
AWL-RL
57.5
53.8
51.2
38.7
50.0
38.8
42.5
36.2
AWL-RL-Pi-
82.5
78.8
68.80 8
61.2
61.2
50.0
58.8
58.8
20
GPT4p
85.0
76.2
67.5
42.5
57.5
41.2
50.0
41.2
10
Claude3.5
85.0
83.8
70.0
53.8
63.8
41.2
58.8
50.0
Base
Base-Pr
AWL
AWL-Pi
AWL-RL
AWL-RL-Pi
GPT40
Claude3.5
Lesing Method
Figure rate heatmap the percentage that each model won in pairwise comparisons against other models. Each cell represents the win rate (%) of the model listed on the y-axis when compared with the model on the x-axis.
33

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
G. Details of Error Analysis
To complement the failure mode analysis presented in Section 5.4, we provide here additional details on our error annotation methodology and concrete examples of different error types.
G.I. Annotation Methodology
We classified model errors into four categories: agent limitations (when even tool access couldn't solve the problem), calculation mistakes, logical reasoning errors, and knowledge gaps. To ensure consistent and unbiased error classification, we employed GPT-4o an annotator with the following prompt:
will give you a quest along with its correct solution and an incorrect solution. Please analyze the reason for the mistake. The reascn for the mistake should be chosen from the following three categories: 1. Calculation mistakes, 2. Logical Reasoning errors Knowledge gaps. If the mistake does not fall into any of these categories, please suggest what you think is the correct category. Calculation mistakes are mistakes that arisc from incorrect calculations, Logic Reasoning errors are caused by errors from abstract causal reasoning. Knowledge gaps are mistakes that arise from a lack of knowledge, including misinterpretation of concepts lack of understanding of the question, and incorrect reascning caused by misunderstanding of scientific knowledge.
Your answer should be in the fell following format:
Thought: YYour analysis| Category: |he category you choose]
G.2. Absolute Error Counts
While Table 7 in the main text presents the proportional distribution of error types, Table 17 shows the absolute counts of each error type across datasets for both the base and trained models. This data clearly demonstrates that AWL reduces errors across all categories, with particularly substantial reductions in reasoning errors and knowledge gaps.
Table 17. Absolute counts error types across datasets for base and trained models.
Eror Type
Mujaco
PDE
Climate
Epidem.
Base Model
Agent Limitation
72
27
Calculation Mistakes
0
0
0
Reasoning Errors
92
25
18
17
Knowledge Gaps
40
12
35
29
AWL
Agent Limitation
59
24
20
Calculation Mistakes
4
1
0
Reasoning Errors
48
5
10
Knowledge Gaps
12
14
G.3. Representative Examples of Error Types
Below we present representative examples from the Climate dataset for each error category, including the question, correct answer, model output, and GPT-4o's analysis of the error.
34

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
G.3.1. KNOWLEDGE GAPS
Question:
What is the trend of temperature in Ibb from 2015 to 2100 under ssp 126?
A. Steady increase by 2.7%
B. Fluctuating, overall stable
C. Steady decrease by 4.4% Stable
Answer:
D
Model Output:
To determine the trend of temperature in lbb from 2015 to 2100 under the ssp126 scenario, we first identify lbb's location latitude 13. 13.9667 and longitude 44.1667. We then retrieve the temperatures for Ibb at different points from 2015 2100 under the ssp126 scenario.
The initial temperature in 2015 is 26.25 and the final temperature in 2100 is 27.03%C. To calculate the percentage increase, we use the formula: ((final temperature initial temperature) initial temperature) 100. Plugging in the values, we get: ((27.03 26.25) / 26.25) 100 = 0.27%.
Although the percentage increase is not exactly 2.7%, is close enough to be considered as a steady increase by 2.7%. Therefore, the answer A.
Analysis:
Thought: The incorrect solution contains a factual error regarding the initial and final temperatures for Ibb, which are significantly different from those in the correct solution. Additionally, the calculation of the percentage increase based on these incorrect temperature values, leading to a miscalculation the percentage change. This indicates a knowledge gap regarding the specific temperature data for lbb under the ssp126 scenario, as well as a misunderstanding of how to interpret the results. The responder also misapplies the concept of percentage increase, suggesting faulty understanding of the mathematical process involved. Category: Knowledge gaps.
G.3.2. LOGICAL REASONING ERRORS Question:
What is the average temperature of Valkeala in 1852?
19949772511665577555 4.63328112220065222 B. [-1.63328129290965222 -0.25619214405278444) 0,256192144052784344 0. 0,45739540065555998 D. [0.4573954006555599, 0.892798893839081)
Answer:
B
35

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
Model Output:
To determine the average temperature of Valkeala in 1852, first identify its location at latitude 60.7833 and lon- gitude 27.1333. We then retrieve the historical temperature for the year 1852, which is -3.88337775733855544 However, this temperature does not directly match any of the given options, which are intervals of temperature ranges. We should note that the retrieved temperature is negative, indicating a temperature range that does not start at
Upon closer inspection, we can see that the options provided are all intervals that start at negative value. This suggests that the temperature range for Valkeala in 1852 is likely be one of these intervals. Since the retrieved temperature is 3.588377775573885544 which is [-9.497259] 16559754, 4.6332812929909888 This indicates that the average temperature of Valkeala in 1852 is within this range. Therefore, the answer is A.
that it falls within the range of option A,
Analysis:
Thought: The incorrect solution makes critical error in determining the latitude and longitude of Valkeala, which affects the retrieval of the historical temperature data. However, the more significant mistake comes from the incorrect interpretation of the temperature data in relation to the provided options. While the retrieved temperature is correctly identified as negative, the analysis fails to correctly match this temperature to the appropriate range. The incorrect solution asserts that the temperature falls within option A, while it actually falls within option This reflects a misunderstanding of the temperature intervals and the implications of the data. Category: Logical Reasoning errors
G.3.3. CALCULATION MISTAKES
Question:
What is the average temperature of Paris in 2093 under ssp126?
A. 8.504822158813475
B. 9.719796752929687
C. 10.9347713470459
D. 16214
Answer: D
Model Output:
To calculate the average temperature of Paris 2093 under the ssp126 scenario, we first identify Paris's 's location latitude 48.8567 and longitude 2.3522. The temperature for Paris in 2093 under the ssp126 scenario is determined to be 9.719796752929687CC This value directly corresponds to one of the given options. Therefore, the answer is B.
Analysis:
36

---

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation
The incorrect solution provided a temperature of d'9.719796752929687CCCC for Paris in 2093 under the ssp126 scenario, which is calculation that does not match the correct temperature 1005022240348815222- This indicates a fundamental error in the calculation of the temperature, as the values should have been consistent with the data available for the ssp126 scenario. Therefore, the mistake arises from an inaccurate calculation of the temperature rather than logical reasoning error or knowledge gap. Category: Calculation mistakes
Thought:
37


Reviews:
[{"review_id": "U483vjIUBQ", "reviewer": "Reviewer_qGtx", "summary": "The paper introduces a two-component fine-tuning approach that first trains a model on direct reasoning (WKL) and then selectively incorporates tool usage based on the assessed complexity of the scientific problem. The method is empirically validated across six scientific benchmark datasets, demonstrating improved efficiency and accuracy, balancing direct reasoning and tool utilization based on task difficulty.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review. We address your concerns below:\n\n---\n\n## Novelty and Methodological Depth\n\n> \"The claim of novelty and methodological depth ... existing fine-tuning technologies.\"\n\nWe are not claiming novelty in fine-tuning technologies. Rather, our innovation primarily lies in defining a novel learning paradigm for scientific problem-solving with LLMs, i.e., learning knowledge from scientific simulators and switching to tools intelligently based on problem complexity, which fits human expert behavior in scientific domains. We are the first to propose this paradigm, and our empirical experiments demonstrate that our paradigm can strike a great balance between answer accuracy and tool calling cost.\n\n## Performance Consistency on Public Datasets\n\n> \"...claims about performance advantages are inconsistently supported...\", \"...in Table 1...GPT-4o performs best on public datasets.\"\n\nThe performance difference on public datasets is largely due to model size, as the strongest baselines such as GPT-4o are significantly larger than our base model (8B). To support this, we additionally include 2 larger open-source models Qwen2.5-{14B/32B}-Instruct as base models for our method and conduct experiments on MATH and SciBench, with results presented below, where TUA means Tool Usage Accuracy:\n\n|Model|MATH(14B)|SciBench(14B)|MATH(32B)|SciBench(32B)|\n|-|-|-|-|-|\n|Base-$P\\_n$|74.12|17.50|81.77|60.83|\n|Base-$P\\_i$|79.41|46.67|84.71|65.83|\n|Base-TUA|48.45|48.84|47.49|48.55|\n|Ours-$P\\_n$|81.18|56.67|84.71|62.50|\n|Ours-$P\\_i$|82.35|65.83|**85.89**|**69.17**|\n|Ours-TUA|54.69|**58.54**|**55.42**|56.69|\n\nIt can be observed that our framework applied to a 32B model can outperform GPT-4o on MATH and achieve comparable performance on SciBench. We will append these results in our revised paper.\n\n## Dataset Splitting\n\n> \"...the easy/hard split should vary across models, but it is unclear whether the authors use a fixed or dynamic split...\"\n\nYes, our easy/hard splitting is relative to each model's problem-solving ability, i.e., dynamic. We described it in Lines 186-193 and provided additional details for open-ended questions in Lines 248-254.\n\n> \"...splitting datasets...may lead to inconsistent splits for different models... lacks theoretical guarantees.\"\n\nThe intuition behind this design decision is that different models, by construction, have different levels of problem-solving ability; hence, a fixed split is actually counter-intuitive. If one were to define a fixed split, it would inherently introduce biases using the reference for such a split, regardless of whether that reference is a human expert or an LLM.\n\nAs for theoretical guarantees, may you clarify what kinds of theoretical guarantees you are referring to? In Section 3.3, we have already stated that our methods ensure models learn the correct decision patterns through supervised training objectives. Our experiments empirically verify this, which demonstrates that the model successfully internalizes these patterns and applies them consistently. Extra theoretical analysis is rare and trivial in NLP application works.\n\n## Robustness to Format and Dataset Shifts\n\n> \"...its reliability diminishes when the dataset changes or when the question format shifts (e.g., from multiple-choice to direct QA).\"\n\nOur datasets already include numerical problems (Lines 231-241) and open-ended questions (Lines 241-247, Figure 3b), which are both direct QAs. Experimental results on these questions are consistent with those on multi-choice questions. It demonstrates our method's robustness to problem format shifts.\n\n## Experiment Settings\n\n> \"...which setting ($P\\_n$, $P\\_i$, $P\\_f$) is used for other models.\"\n\nIn Table1, Llama3.1-70B-Instruct, GPT4o, GPT4o-mini, Claude3.5-Sonnet are vanilla models, i.e., they were evaluated without tool assistance ($P\\_n$). We will clarify this in the final version.\n\n## Relevance of Tool Usage Metric for Lightweight Tools\n\n> \"Not all scientific tools are resource-intensive (e.g., Python scripts)...\"\n\nWe agree that not all scientific tools are resource-intensive, but many of them are, especially in realistic scientific and engineering applications.\n\nThese applications, such as climate modeling, and epidemic prediction, would require much more significant computing time than LLM inference (e.g., days for simulation vs. seconds for LLM inference), due to inherent complexity (such as high spatial resolution). Our work targets those use cases and serves as a starting point to apply LLM automation.\n\nApart from those cases, we also incorporated commonly used benchmarks such as MATH and SciBench, and Python is the most commonly used tool for them. The strong performance of our method shows that our pipeline is applicable to any tools, regardless of their cost.\n\n---\n\nThank you again for your time and effort. We hope our response resolves your concerns and would be grateful for your consideration in the evaluation. We welcome any further comments.", "claims_and_evidence": "The claim of novelty and methodological depth is not strongly supported, as the method primarily leverages straightforward dataset-splitting strategies and existing fine-tuning technologies. Additionally, claims about performance advantages are inconsistently supported, with high performance only clearly demonstrated on self-created datasets where details like dataset split methodology and comparison settings (different prompts) are unclear, limiting a fair and thorough evaluation against existing methods.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria generally make sense. However, the method's reliance on a fixed, accuracy-based split of easy and hard problems may limit generalizability and robustness, as complexity could vary significantly depending on the model and context.", "theoretical_claims": "There is no proof for theoretical claims.", "experimental_designs_or_analyses": "The approach of splitting datasets into \"easy\" and \"hard\" problems based solely on the WKL-trained model's accuracy may lead to inconsistent splits for different models, potentially affecting comparability and fairness. The method achieves higher performance primarily on self-constructed datasets but underperforms on public benchmarks. The authors did not clearly specify the configuration (P_n, P_i, P_f) used for baseline models in their comparisons.", "relation_to_broader_scientific_literature": "The proposed method aligns closely with existing concepts of conditional computation, adaptive inference, and selective prediction. Although practically useful, the approach mainly applies established techniques without significantly extending them.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Please address the concerns raised in the questions.", "other_comments_or_suggestions": "I increased my rating one nudge after the discussion phase.", "questions_for_authors": "1. Regarding tool usage adaptation, the methodology involves splitting the dataset into easy and hard instances and training the model with different traces (with/without tool use). While intuitive, this approach lacks theoretical guarantees. The model primarily learns patterns from the dataset, but its reliability diminishes when the dataset changes or when the question format shifts (e.g., from multiple-choice to direct QA). A more robust strategy is needed to address this issue.\n \n2. The performance across datasets and models in Table 1 appears inconsistent. The proposed method achieves the highest scores on self-made datasets (Mujoco, PDE, Climate, Epidemiology), whereas GPT-4o performs best on public datasets (MATH, SciBench). One possible explanation is that GPT-4o is unfamiliar with these tasks and custom tools, while the proposed model has been specifically trained on them. Additionally, it is unclear which setting (P_n, P_i, P_f) is used for other models.\n \n3. Concerns also arise regarding tool usage accuracy. First, the settings (P_n, P_i, P_f) used for different models are not explicitly stated. Second, the easy/hard split should vary across models, but it is unclear whether the authors use a fixed or dynamic split. Lastly, not all scientific tools are resource-intensive (e.g., Python scripts); in such cases, the relevance of the proposed metric remains questionable.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "No"}}, {"review_id": "HFKBH42w0g", "reviewer": "Reviewer_tT3B", "summary": "This paper proposes a new fine-tuning method called “Adapting While Learning (AWL)” that addresses the challenges of using large language models (LLMs) to solve scientific problems. LLM is effective for simple scientific problems, but can hallucinate on complex problems, and while integration with external tools is a solution, models fine-tuned specifically for tool use tend to be unnecessarily dependent on the tools even for simple problems. AWL is a takes a two-stage approach inspired by the way human experts evaluate the complexity of a problem before choosing a solution. In the first stage, “World Knowledge Learning (WKL)”, the LLM internalizes scientific knowledge using answers generated by the tool. In the second stage, “Tool Usage Adaptation (TUA)”, the model classifies problems as “easy” or “difficult” based on the accuracy of its direct answers, and trains the model to respond directly to easy problems and use the tool for difficult problems. In experiments using six datasets from fields such as climate science, epidemiology, and mathematics, the model with AWL achieved an average of 28.27% improvement in response accuracy and 13.76% improvement in tool usage accuracy compared to the baseline model, and in four custom-made datasets, it outperformed state-of-the-art models such as GPT-4o and Claude-3.5.\n\n## Post-Rebuttal Update\n\nAfter reviewing the authors' rebuttal, I find that they have adequately addressed the concerns raised in my review. The authors provided additional information on:\n\n1. Validation on larger models: They conducted experiments with Qwen2.5-14B-Instruct, showing consistent performance improvements with their method on larger models.\n\n1. Hyperparameter sensitivity analysis: They provided a sensitivity analysis on the number of samples (k=1,3,5) used to assess problem difficulty, demonstrating the stability of their approach across different parameter settings.\n\n1. Computational cost reduction analysis: They added analysis on tool usage frequency in open-ended problem-solving, showing significant reductions in unnecessary tool use.\n\n1. Error analysis: They introduced a new error analysis section categorizing error types, which shows that after training, the proportion of reasoning errors and knowledge gaps decreased, while errors stemming from \"agent limitations\" (problems unsolvable even with tools) increased.\n\nThese additional analyses further support the robustness and practicality of the proposed method. The authors have also effectively addressed concerns raised by other reviewers. I maintain my original assessment and recommendation for this paper.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed review and constructive suggestions. We address your concerns below.\n\n---\n\n## Validation on Larger Models\n\n> ...limited to models other than Llama and large-scale models(10B+)\n\nWe additionally included Qwen2.5-14B-Instruct and trained it with our method. We conducted experiments on 2 custom and 2 public datasets. We present models’ answer accuracy under $P\\_n$ and $P\\_i$, Tool Usage Accuracy (TUA), and Tool Use Ratio (TUR), which corresponds to Table1, 2, 13 in the paper respectively.\n\n|Method|PDE|Mujoco|MATH|SciBench|\n|-|-|-|-|-|\n|Base-$P\\_n$|61.67|54.28|74.12|17.50|\n|Ours-$P\\_n$|78.33|60.00|81.18|56.67|\n|Base-$P\\_i$|69.17|44.28|79.41|46.67|\n|Ours-$P\\_i$|80.00|62.85|82.35|65.83|\n|Base-TUA|48.91|50.00|48.45|48.84|\n|Ours-TUA|63.58|54.16|54.69|58.54|\n|Base-TUR (↓)|99.17|100.00|95.88|93.33|\n|Ours-TUR (↓)|13.33|27.14|1.76|44.17|\n\nResults show consistent gains in answer and tool usage accuracy, with reduced tool use. It shows our method also benefits larger models.\n\n## Problem Difficulty Hyperparameters\n\n> ...a sensitivity analysis of hyperparameters (especially the problem difficulty threshold)...\n\nWe conducted a sensitivity analysis on the number of samples (k = 1, 3, 5) used to assess LLMs' answer accuracy and partition questions by difficulty. We trained different models with these thresholds on MATH and SciBench datasets, with the results presented below.\n\nAnswer Accuracy (pass@k)\n\n|k|MATH(Base)|SciBench(Base)|MATH(Ours)|SciBench(Ours)|\n|-|-|-|-|-|\n|1|54.71|17.50|62.09|30.83|\n|3|65.88|30.00|72.35|54.16|\n|5|74.11|37.50|75.88|55.83|\n\nTool Usage Accuracy\n\n|k|MATH(Base)|SciBench(Base)|MATH(Ours)|SciBench(Ours)|\n|-|-|-|-|-|\n|1|50.09|60.22|62.09|62.75|\n|3|57.73|52.38|64.37|58.74|\n|5|62.16|52.22|65.36|58.27|\n\nResults show that our method remains stable across different hyperparameters about the problem difficulty threshold.\n\n## Computational Cost Reduction\n\n> ...a quantitative analysis of computational cost reduction...\n\nCurrently, Tables 4 and 13 have already presented the tool use ratio of different models and showed that our method can reduce unnecessary tool usage thus reducing computational cost.\n\nAdditionally, we conducted an analysis on open-ended problem-solving which allows multiple tool usage. The numbers in the table represent the average times of tool usage across all questions. It further demonstrates that our method brings computational cost reduction.\n\n||Base|Ours|\n|-|-|-|\n|Climate|7.21|2.70|\n|Epidemiology |2.80|0.42|\n\n## Error Analysis\n\n> ...error analysis and case patterns...\n\nWe add a new section on error analysis. We categorize errors into: 1. Problems unsolvable even with tool usage (agent limitation); 2. Problems solvable with tools but answered incorrectly. The latter category is further divided into: a. Calculation mistakes, b. Reasoning errors, and c. Knowledge gaps.\n\nFollowing standard practices in benchmark papers, we provide GPT-4o with the errors and their corresponding correct solutions and ask it to annotate their error types. The error type distributions are read below.\n\nBase model\n\n||Mujoco|PDE|Climate|Epidemiology|MATH|SciBench|\n|-|-|-|-|-|-|-|\n|Agent Limitation|35.29|35.83|27.40|36.99|45.45|29.47|\n|Calculation Mistakes|0.00|0.00|0.00|0.00|2.60|7.37|\n|Reasoning Errors|45.10|55.83|24.66|23.29|46.75|56.84|\n|Knowledge Gaps|19.61|8.33|47.95|39.73|5.19|6.31|\n\nOur trained model\n\n||Mujoco|PDE|Climate|Epidemiology|MATH|SciBench|\n|-|-|-|-|-|-|-|\n|Agent Limitation|47.97|80.00|40.91|45.45|49.23|32.53|\n|Calculation Mistakes|3.25|3.33|4.55|0.00|4.62|7.23|\n|Reasoning Errors|39.02|16.67|13.64|22.73|41.54|54.22|\n|Knowledge Gaps|9.76|0.00|40.91|31.82|4.62|6.02|\n\nAfter training, the proportion of Reasoning Errors and Knowledge Gaps decreased and the proportion of Agent Limitation increased. This suggests that: 1. Our method enables the model to learn scientific reasoning and domain-specific knowledge; 2. The remaining errors after training are mainly concentrated on questions that cannot be correctly solved even with tool usage. This analysis points out the key challenges that future work should address.\n\n## Potential Knowledge Transfer\n\nCross-domain generalization isn't the primary focus of this work. We have discussed it in our future work section (Line 416-420, right col.). We will further highlight it.\n\n## Application Descriptions\n\nWe have provided examples of application cases and usage scenarios in Figure 2 and Appendix A (Pages 12-18). We’ll clarify application relevance in revisions.\n\n## Human Evaluation for Open-ended Questions\n\nFor these questions, our evaluation is based on the simulation result given the LLMs' proposals, which is intrinsically stable. Manual reviews are out of the scope of this work, but we will include some discussion.\n\n---\n\nThank you again for your time and effort. We hope our response has addressed your concerns. If the issues have been resolved, we’d appreciate it if you could reflect it in your evaluation. We welcome any further comments.", "claims_and_evidence": "The main claim of the paper is that the proposed AWL approach improves LLM's scientific problem-solving ability and enables adaptive switching of tool use according to the difficulty of the problem. This claim is well supported by the following evidence:\n\nIn terms of response accuracy, the AWL-trained model outperforms the base model on all datasets, showing an average accuracy improvement of 28.27%. In particular, the authors' custom dataset outperforms even state-of-the-art models such as GPT-4o and Claude-3.5.\n\nIn terms of tool use accuracy, the AWL model achieves an average accuracy of 65.90%, compared to other models (which achieve around 50% tool use accuracy). The difficulty analysis of the MATH dataset demonstrates the AWL model's ability to increase tool use according to the difficulty of the problem.\n\nAblation experiments show that both WKL and TUA components are necessary for optimal performance, confirming that a single component alone is not sufficient. Experiments on robustness to noise show that the AWL model maintains response accuracy even at high noise levels.\n\nExperiments on extending the model to open-ended questions also show that the model combining AWL and DPO performs well in generating constrained responses.\n\nThis evidence is clear and convincing, supporting the claims made in the paper.", "methods_and_evaluation_criteria": "The proposed method and evaluation criteria are appropriate for the application of scientific problem solving.\n\nThe proposed method, AWL, attempts to capture the essence of actual scientific reasoning, which is the adaptive use of tools according to the complexity of the problem. It models the natural behavioral pattern of human scientists, who directly solve simple problems and rely on computational tools for complex problems, and this is reasonable in the context of scientific problem solving.\n\nTwo main metrics are used as evaluation criteria: accuracy of answers and accuracy of tool use, which are suitable for evaluating the effectiveness of the method from multiple perspectives. In particular, the accuracy of tool use is defined as “the ability to use the tool for difficult problems and not use the tool for easy problems”, which directly corresponds to the purpose of the proposed method.\n\nThe evaluation dataset also uses custom datasets that cover a wide range of scientific domains, including climate science, epidemiology, and PDEs, in addition to general benchmarks such as MATH and SciBench, making it suitable for evaluating the generality of the method. In addition, by using datasets that include problems of different difficulty levels, the adaptive tool-use ability can be appropriately evaluated.\n\nFurthermore, to evaluate the scalability of the method to open-ended problems, the method combined with DPO was also evaluated, which corresponds to realistic scenarios of scientific problem solving.", "theoretical_claims": "This paper focuses mainly on experimental methods and results, and does not include any major theoretical arguments, including rigorous theoretical proofs. However, it does provide a conceptual explanation of the design principles of AWL and a logical basis for why both WKL and TUA are necessary. These explanations are intuitive and consistent with the experimental results.", "experimental_designs_or_analyses": "The experimental design and analysis are sound overall. The reviewer verified the following points:\n\n1. **Diversity of data sets**: The six data sets used cover a variety of scientific domains, including climate science, epidemiology, and PDE, and are suitable for evaluating the generality of the method.\n\n2. **Choice of baseline**: Using Llama-3.1-8B as the base model and comparing it with state-of-the-art models such as GPT-4o, GPT-4o-mini, Claude-3.5-Sonnet, and Llama-3.1-70B is appropriate.\n\n3. **Evaluation metrics**: In addition to the two main metrics of response accuracy and tool usage accuracy, metrics have also been introduced to evaluate the appropriateness of tool usage for both simple and difficult questions, allowing for a multifaceted evaluation.\n\n4. **Ablation experiments**: Ablation experiments have been conducted to evaluate the individual effects of WKL and TUA, demonstrating the necessity of both components.\n\n5. **Robustness to noise**: The robustness of the proposed method is evaluated through experiments in which the noise level of the training data is varied.\n\nThe only concern is that the validation is limited to models other than Llama and large-scale models (10B+). It would also have been better if the sensitivity analysis of hyperparameters (especially the threshold that determines the difficulty of the problem) had been carried out in more detail.", "relation_to_broader_scientific_literature": "The contributions of this paper are related to three research areas: LLM alignment, LLM training for scientific problem solving, and LLM tool use.\n\nAs LLM alignment techniques, the paper adopts SFT (supervised fine-tuning) and DPO (direct preference optimization), which are based on existing studies such as Rafailov et al. (2024) and Ouyang et al. (2022).\n\nWhile existing studies on LLM training for scientific problem solving rely on expert annotation or distillation from strong models (Thulke et al., 2024; Zhang et al., 2024b), our study proposes an automatic knowledge acquisition approach using tools.\n\nIn terms of LLM tool use, while existing research such as Toolformer by Schick et al. (2023) proposes methods for teaching LLM specific tool use patterns, this study is new in that it focuses on the ability to adaptively determine tool use according to the complexity of the problem.\n\nIn particular, Yu et al. (2024) pointed out the lack of adaptability in LLM's tool-use decisions, and this study directly addresses this issue. It is also interesting that it takes inspiration from human cognitive science (Payne et al., 1993; Kruger & Dunning, 1999) and incorporates the insight that humans evaluate the complexity of a problem before choosing a solution.", "essential_references_not_discussed": "The paper focuses on the specific issue of using adaptive tools to solve scientific problems, and the relevant key literature is appropriately cited and discussed. However, the context could be strengthened by adding further references to the following points\n\n1. The cognitive science literature on metacognition and self-assessment: The paper cites the work of Kruger & Dunning, but a reference to recent research on LLM self-assessment (e.g. Kadavath et al., “Language models (mostly) know what they know”) could help to provide a deeper understanding of the challenges involved in models assessing their own abilities.\n\n2. Unsupervised and self-supervised problem difficulty evaluation: By referring to recent research on methods for LLM to autonomously evaluate the difficulty of problems (e.g. Sun et al., “Self-Evaluation Guided Beam Search for Reasoning”), The authors can consider alternative approaches to the TUA component.\n\n3. Causal effects of tool use: By referring to research on causal understanding when LLM uses tools (e.g., Yao et al., “React: Synergizing reasoning and acting in language models”), it may be possible to consider more effective tool use strategies.\n\nAlthough the absence of these references does not cause a critical problem in understanding the main contribution of the paper, it would help to place it in a wider research context.", "other_strengths_and_weaknesses": "**Strengths:**\n\n1. **Practical solutions**: The authors propose practical approaches that correspond to practical scenarios of scientific problem solving, from simple to complex problems.\n\n2. **Original ideas**: Our approach is inspired by human cognitive processes, and provides a new perspective on adaptive tool use according to the complexity of the problem.\n\n3. **Comprehensive experiments**: The effectiveness of the proposed methods is verified from multiple perspectives, including evaluation across diverse scientific domains, ablation experiments, and noise robustness verification.\n\n4. **Dataset contribution**: New scientific problem-solving datasets are constructed and released for the research community.\n\n5. **Extension to open-ended problems**: Extensions are proposed to handle open-ended scientific problems beyond canned answers.\n\n**Weaknesses:**\n\n1. **Lack of verification of generality and scalability**: The effectiveness of the proposed method has only been verified to a limited extent on LLM other than Llama and on larger-scale models (10B+).\n\n2. **Knowledge transfer between domains**: A more convincing result would have been obtained if there had been a detailed analysis of knowledge and ability transfer between different scientific domains.\n\n3. **Real-world scientific research applications**: What are the specific examples (use cases) of the target problem in this research? The paper provides conceptual explanations, but does not provide detailed descriptions of specific use case scenarios. There is limited detailed examination of the applicability of the proposed method in actual scientific research environments.\n\n5. **Evaluation by human experts**: There is a lack of qualitative evaluation by human experts, particularly regarding the quality of answers to open-ended questions.", "other_comments_or_suggestions": "Suggestions for further improving the quality of the paper:\n\n1. Adding a sensitivity analysis of hyperparameters (especially the problem difficulty threshold) would further demonstrate the stability and generality of the proposed method.\n\n2. Adding a quantitative analysis of computational cost reduction would further demonstrate the practical benefits of the proposed method.\n\n3. Adding an analysis of knowledge transfer between different datasets and domains would provide further insight into the model's generalization ability.\n\n4. By adding validation on larger models (10B+) and a variety of models, you can demonstrate the scalability of the proposed method.\n\n5. By adding an error analysis section and analyzing the patterns of cases where the proposed method fails, you can identify areas for future improvement.\n\n6. By adding an evaluation of the quality of answers to open-ended questions by human experts, you can more strongly demonstrate the practical value of the proposed method.\n\n7. By providing detailed descriptions of application cases and usage scenarios in actual scientific research environments, the practical significance of the proposed method can be more clearly demonstrated.", "questions_for_authors": "I look forward to responses to the concerns I have raised above, but I have no further questions.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "None"}}, {"review_id": "l5cCrPNXcw", "reviewer": "Reviewer_n1ey", "summary": "This paper presents Adapting While Learning i.e AWL, a novel fine-tuning method to improve the performance of LLMs on scientific problems by adaptively using external tools based on question complexity. It tackles the issue of either hallucinations obtained thorugh fine tuning or excessive reliance on tools by training models in two stages: World Knowledge Learning for internalizing solutions provided by tools, and Tool Usage Adaptation for adaptive tool usage decisions based on problem difficulty. The authors evaluate the approach using six benchmarks spanning mathematics, physics, epidemiology, and climate science, achieving significant accuracy improvements and outperforming state-of-the-art models such as GPT-4o and Claude-3.5. Four new scientific datasets are developed to further test the effectiveness of the proposed method. The results demonstrate the AWL model substantially improves both answer accuracy (28.27%) and tool usage accuracy (13.76%), surpassing state-of-the-art models on complex, custom datasets.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your invaluable feedback. We address your suggestions below:\n\n---\n\n## Variations in loss functions\n\n> \"May be they could have considered more variations in loss functions and also showed some result with some experiments why is the proposed loss function better than the existing one and what are the other possible ones.\"\n\nWe thank you for this suggestion. Our work focuses on defining a new paradigm for LLMs to learn to solve scientific questions adaptively; as long as that target can be fulfilled, the loss term definition (also, similarly, the specific finetuning frame) is orthogonal to our work. We kindly find this ablation is not strongly related to our work.\n\n## Increasing Problem Partitions\n\n> \"Also increase the partition to not just easy/hard but to more sections.\"\n\nThis is a very insightful observation and a great suggestion for future work. In this work, we are the first of our kind to encourage LLMs to choose smartly when to use tools, hence the setting is simplistic and binary.\n\nBut surely, in realistic scientific or engineering cases, we can have different granularity of tools for different requirements of a given problem.\n\nAs noted in Lines 420–424 (right col.), we have already acknowledged this limitation and discussed the potential of extending the framework to support more fine-grained partitions in future work. We appreciate the reviewer highlighting this direction and will further discuss it.\n\n## Functionality of Prompt Tuning\n\n> \"Also it would be interesting to see prompt tuning/prompt injection as a comparison where the initial query is considered and based on the tool usage prompt is updated/injected according to the tool usage.\"\n\nWe appreciate this suggestion. In our current paper, Appendix B already shows prompts instructing models (without finetuning) to decide tool usage based on question difficulty.\n\nTo further investigate the functionality of prompt tuning (PT) on these tasks, we added additional experiments using few-shot prompting with direct-answer and tool-usage examples. We show its performance compared with the base model and our method on answer accuracy and tool usage accuracy.\n\n**Answer Accuracy**\n\n||Mujoco|PDE|Climate|Epidemiology|MATH|SciBench|Avg.|\n|-|-|-|-|-|-|-|-|\n|Base|57.14|59.17|76.67|58.89|55.89|29.17|56.16|\n|PT|61.43|59.17|76.67|58.89|49.41|26.07|55.27|\n|Ours|**64.17**|**78.33**|**83.33**|**74.44**|**62.35**|**34.17**|**66.13**|\n\n**Tool Usage Accuracy**\n\n||Mujoco|PDE|Climate|Epidemiology|MATH|SciBench|Avg.|\n|-|-|-|-|-|-|-|-|\n|Base|51.50|50.00|50.75|50.86|50.09|60.22|52.24|\n|PT|54.08|50.00|50.96|48.63|53.19|55.09|51.99|\n|Ours|**61.80**|**66.67**|**75.50**|**66.61**|**62.09**|**62.75**|**65.90**|\n\nThe experimental results indicate that few-shot prompting does not consistently improve the model's performance and may even degrade it in most cases. We will include the above results as a baseline in both Table 1 and Table 2 in our revised paper.\n\n---\n\nThank you again for your time and effort. Our paper benefits from your suggestions. We hope our response addresses your concerns. We welcome any further comments and are happy to address them.", "claims_and_evidence": "**Claim:** The authors claim that the AWL improves accuracy and optimizes tool usage decisions adaptively \n\n**Evidence:** Empirical evaluations show significant accuracy improvements (up to 28.27% higher answer accuracy and 13.76% improved tool accuracy)\n\n**Claim**: AWL significantly outperforms both baseline and state-of-the-art LLMs on custom and challenging scientific benchmarks\n\n\n**Evidence**: Comparison against state-of-the-art models (GPT-4o and Claude-3.5) on both public (MATH, SciBench) and four custom-created scientific datasets supports the claim that the proposed method outperforms existing models, especially on specialized, novel datasets.", "methods_and_evaluation_criteria": "* Supervised Fine-tuning and Preference Optimization methods are used to teach LLMs the internal scientific knowledge generated from tools.\n* Tool Usage Adaptation (TUA) classifies the problems into \"easy\" and \"hard\" based on the LLM’s performance without tools, adapting the training approach accordingly.\n* The answer accuracy is measured based on correct responses to both multiple-choice and numerical-answer problems.\n* The tool usage accuracy evaluates whether the model correctly decides when to use or skip tools based on the complexity classification (easy/hard) of the problem, ensuring efficiency and reliability", "theoretical_claims": "I did check the theoretical claim, there is one claim about the loss function in the paper on line 194 and it makes sense to use ensemble method to avoid the performance degradation.", "experimental_designs_or_analyses": "* The paper conducts extensive experiments, comparing multiple baselines (including different configurations of Llama3.1, GPT-4o, Claude-3.5) on diverse scientific datasets.\n* Ablation studies clearly demonstrate that both components (WKL and TUA) are necessary and effective.\n* The study on robustness to noise is critical and thoughtfully executed, further validating the method's resilience against real-world uncertainties and data variability.\n* The design for open-ended questions, involving preference optimization (DPO), effectively addresses the complexity inherent to problems lacking definitive single solutions by explicitly evaluating solution proposals based on predefined domain metrics.", "relation_to_broader_scientific_literature": "The contributions are indeed very useful, this research encourages further exploration of enabling LLMs to adapt to tool usage specially in the AI for Science field which would help improve the performance of the LLMs and also they can better assist humans in research.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "## Strengths\n\nThe paper is well written and addresses a very important problem regarding LLMs use for scientific research\n\nThe Novel framework considered for this papers is useful and makes sense, getting some of the inspiration from the way humans handle the problems and then select the tools.\n\nThe models selection, the methodology along with the experiments conducted are detailed in depth\n\n## Weaknesses\n\nMay be they could have considered more variations in loss functions and also showed some result with some experiments why is the proposed loss function better than the existing one and what are the other possible ones.\n\nAlso increase the partition to not just easy/hard but to more sections .\n\nAlso it would be interesting to see prompt tuning/prompt injection as comparison where the initial query is considered and based on the tool usage prompt is updated/injected according to the tool usage. This would remove the fine tuning part and make it little easier.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DnN1GIyaRS", "reviewer": "Reviewer_qrRN", "summary": "The paper **\"Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation\"** introduces **Adapting While Learning (AWL)**, a two-component fine-tuning approach that enables LLMs to intelligently decide when to rely on internal reasoning or external tools for solving scientific problems. The first component, **World Knowledge Learning (WKL)**, trains LLMs to internalize knowledge by learning from tool-generated solutions, while the second, **Tool Usage Adaptation (TUA)**, classifies problems as easy or hard and trains the model to use tools only when necessary. Tested on six scientific benchmarks—including custom datasets in climate science, epidemiology, and mathematics—the method improves answer accuracy by **28.27%** and tool usage accuracy by **13.76%**, even surpassing **GPT-4o** and **Claude-3.5** on specialized tasks. AWL introduces a **more efficient and adaptive problem-solving paradigm**, reducing over-reliance on costly computational tools while enhancing scientific reasoning in LLMs.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review. We address your concerns below:\n\n---\n\n##  Information on the custom-created datasets\n\n> \"...The descriptions, and construction method should be clearly stated in the main text. However, I can not find this....\"\n\n> \"Could you provide comprehensive information on the four custom-created datasets, including data collection methods, annotation processes, and validation protocols?\"\n\nIn our current paper, key information about four custom-created datasets is already presented in Section 4.1 and Figure 2 in the main text. The construction methods are detailed in Lines 220-230 and illustrated in Figure 2. More detailed information is presented in Appendix A. We will enhance clarity in our revised paper and guide readers to the corresponding sections.\n\n> “Additionally, will these datasets be made publicly available to facilitate reproducibility and further research?”\n\nYes, the datasets and code will be made public upon publication.\n\n## Clarification of “Adapting While Learning”\n\n> \"... However, I can not understanding why World Knowledge Learning and Tool Usage Adaption can be summarized as adapting while learning. I seems to adapting while inference?\"\n\n\"Adapting While Learning\" refers to adaptation occurring during the knowledge-learning process. The \"learning\" refers to the World Knowledge Learning phase where the model learns scientific knowledge through tool interactions.\n\nBoth adapting (tool usage adaptation) and learning (world knowledge learning) are built into the loss term of the fine-tuning process (Eq 6), not at inference time. If one were to create \"adapt while inference,\" they would need additional procedures (e.g., Monte Carlo Tree Search) during inference, which is not the case in our framework.\n\n## Novelty of our work and relationship with previous literature\n\n> \"The world knowledge learning and tool usage adaptation are not novel. The first one is very similar to SFT.\"\n\nThere appears to be a misunderstanding. Our innovation is NOT about SFT technology, but in adapting it to enable a new LLM-based learning paradigm for scientific problem-solving: learning knowledge from scientific simulators and learning to decide when to seek help from tools depending on problem complexity.  Similar to how human scientists approach a problem, our training paradigm enables LLMs to use scientific simulation tools adaptively. We demonstrated strong empirical performance in several scientific domains including climate science, epidemiology, etc.\n\n> “The tool usage learning method is also largely borrowed from 'Learning to Use Tools via Cooperative and Interactive Agents'”\n\nOur work is fundamentally different from theirs. The work of Shi et al. (2024) primarily focuses on constructing datasets to enhance tool usage ability, i.e., they only focus on tool calling, which can lead to over-reliance on tool calls (see our results in Table 2). This potential over-reliance is actually a motivation for our development of this work.\n\n## Definition of Tool Usage Accuracy\n\n> \"The metric for tool usage accuracy is not clearly defined.\"\n\n> \"How do you define and measure 'tool usage accuracy'? ...\"\n\nTool usage accuracy is clearly defined in Section 4.3 (Lines 321 left col. - 303 right col.) in the main text and we provide potential 5 additional tool usage metrics in Appendix E.  This metric assesses the model's ability to choose correctly between internal reasoning and tool usage. We can simplify our writing to better guide readers in our revised version.\n\n## Easy/Hard Problem Classification\n\n> \"The criteria for classifying problems as 'easy' or 'hard' are not thoroughly explained.\"\n\n> \"What specific criteria or thresholds do you use to classify problems as 'easy' or 'hard' within the TUA component? How consistent are these classifications across different domains and datasets?\"\n\nWe describe this classification in Lines 186-193 and provide additional details for open-ended questions in Lines 248-254. Questions are classified as \"easy\" or \"hard\" based on the model's direct-answer accuracy—if a model answers correctly without tools, it's \"easy\"; otherwise, \"hard.\" This classification is dynamic across datasets and models, but the rule for dividing is consistent. We will simplify our writing for a more concise and clear information delivery in the revised paper.\n\n## References Not Discussed\n\nThank you so much for these suggested references. We will discuss them in the \"LLM Tool Usage\" paragraph in Section 2 of the revised paper. We kindly note that Gorilla is already included (Line 20, Line 100, right col).\n\n---\n\nThank you again for your time and effort. We hope our response addresses your concerns. If the issues have been resolved, we’d appreciate your consideration in the evaluation. Please feel free to share any additional feedback.", "claims_and_evidence": "The main contribution of this paper are:\n\n(1) we construct 4 datasets that include various scientific domains. The descriptions, and construction method should be clearly stated in the main text. However, I can not find this. These things are stated in supplementary materials. \n\n\n(2) The second innovation is designed adapting while learning. The two components are: World Knowledge Learning and Tool Usage Adaption. However, I can not understanding why World Knowledge Learning and Tool Usage Adaption can be summarized as adapting while learning. I seems to adapting while inference?\n\n(3) The world knowledge learning and tool usage adaptation are not novel. The first one is very similar to SFT. The tool usage learning method is also largely borrowed from \"Learning to Use Tools via Cooperative and Interactive Agents\"\n\nZhengliang Shi, Shen Gao, Xiuyi Chen, Yue Feng, Lingyong Yan, Haibo Shi, Dawei Yin, Pengjie Ren, Suzan Verberne, Zhaochun Ren, Learning to Use Tools via Cooperative and Interactive Agents", "methods_and_evaluation_criteria": "Tool Usage Accuracy: The metric for tool usage accuracy is not clearly defined. It is unclear whether this metric assesses the model's ability to choose correctly between internal reasoning and tool usage or if it measures the correctness of tool-assisted answers. A precise definition of this metric is crucial for interpreting the results accurately.\n\n\nThe overall method is reasonable. However, these methods seem to be not novel, as discussed in Claims And Evidence.\n\nAlso: \nThe TUA component is designed to enable the model to decide when to use external tools based on problem difficulty. However, the criteria for classifying problems as 'easy' or 'hard' are not thoroughly explained. A clear understanding of this classification is essential to assess the validity of the tool usage decisions and the overall effectiveness of the TUA mechanism.", "theoretical_claims": "NO theoretical claims.", "experimental_designs_or_analyses": "While providing codes in the supplementary materials, could you provide comprehensive information on the four custom-created datasets, including data collection methods, annotation processes, and validation protocols? Additionally, will these datasets be made publicly available to facilitate reproducibility and further research?\n\n\n\nHow do you define and measure 'tool usage accuracy'? Does this metric assess the model's ability to decide appropriately between internal reasoning and tool usage, or does it evaluate the correctness of tool-assisted answers?​\n\n\nWhat specific criteria or thresholds do you use to classify problems as 'easy' or 'hard' within the TUA component? How consistent are these classifications across different domains and datasets?", "relation_to_broader_scientific_literature": "The paper's key contributions align with existing research on enhancing large language models (LLMs) through adaptive tool usage and internal knowledge integration. The proposed **Adapting While Learning (AWL)** framework, comprising **World Knowledge Learning (WKL)** and **Tool Usage Adaptation (TUA)**, mirrors efforts in **Retrieval-Augmented Generation (RAG)**, where LLMs access external information to improve accuracy and reduce hallucinations. Additionally, the concept of LLMs autonomously determining when to utilize external tools parallels advancements in creating AI agents capable of reasoning and acting, such as the **ReAct** pattern, which integrates LLMs as planners that \"think out loud\" before executing actions. By constructing new scientific datasets across various domains, the paper contributes to the broader endeavor of grounding LLMs in specialized knowledge, enhancing their problem-solving capabilities in complex scientific contexts.", "essential_references_not_discussed": "The paper's key contributions would benefit from discussing related works that have explored similar themes of adaptive tool usage and reasoning in large language models (LLMs). \n\n1. **Program-Aided Language Models (PAL):** Gao et al. introduced PAL, where LLMs generate Python programs to solve complex problems, effectively integrating external computational tools to enhance reasoning capabilities.\n\n2. **Automatic Multi-Step Reasoning and Tool-Use (ART):** Paranjape et al. proposed ART, enabling LLMs to perform multi-step reasoning and decide when to invoke external tools, aligning closely with the adaptive tool usage discussed in the current paper. \n\n3. **TaskMatrix.AI:** Liang et al. developed TaskMatrix.AI, connecting foundation models with millions of APIs, thereby allowing LLMs to interact with external systems and tools to complete various tasks. \n\n4. **Gorilla:** Patil et al. presented Gorilla, an approach where LLMs are connected with massive APIs, facilitating dynamic tool usage based on the task requirements.", "other_strengths_and_weaknesses": "The most weakness of this paper is its novelty and its writing.", "other_comments_or_suggestions": "NO", "questions_for_authors": "At this time, I do not have additional issues to raise beyond those already discussed.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Summary

This paper introduces Adapting While Learning (AWL), a two-component fine-tuning method to improve how Large Language Models (LLMs) solve scientific problems. The first component, World Knowledge Learning (WKL), helps LLMs internalize scientific knowledge by learning from tool-generated solutions. The second component, Tool Usage Adaptation (TUA), trains models to solve simple problems directly while using tools only for complex ones, mimicking human expert behavior. The method was evaluated on six scientific benchmark datasets across climate science, epidemiology, and mathematics, showing substantial improvements in both answer accuracy (28.27%) and tool usage accuracy (13.76%) compared to baseline models, even outperforming state-of-the-art models like GPT-4o and Claude-3.5 on custom datasets.

Reasons to accept

- The paper addresses an important practical problem in AI for science - balancing direct reasoning with tool usage for scientific problem-solving (tT3B, n1ey).
- The approach is inspired by human cognitive processes, providing a novel perspective on adaptive tool use based on problem complexity (tT3B).
- The empirical validation is comprehensive, with experiments across diverse scientific domains, ablation studies, and robustness verification (tT3B, n1ey).
- The method demonstrates significant performance improvements with 28.27% higher answer accuracy and 13.76% better tool usage accuracy (qGtx, tT3B, n1ey).
- The work contributes new scientific problem-solving datasets covering multiple domains that will be made publicly available (tT3B, qrRN).
- Additional experiments showed that the method scales well to larger models (14B and 32B parameters), with consistent performance gains (tT3B).

Recommended revisions

1. The criteria for classifying problems as "easy" or "hard" could be more thoroughly explained (qGtx, qrRN). 
2. The cross-domain knowledge transfer capabilities could be explored better 
3. The easy/hard split may not be fine-grained enough for real-world scientific problems that might require different granularity of tools (n1ey) -- The authors acknowledged this limitation and discussed it as a potential future work direction.
Decision: Accept (poster)