Training a Scientific Reasoning Model for Chemistry
Authors: Siddharth Narayanan
James D. Braza
Ryan-Rhys Griffiths
Albert Bou
Geemi Wellawatte
Mayk Caldas Ramos
Ludovico Mitchener
Michael Martin Pieler
Samuel G Rodriques
Andrew White
Venue: NeurIPS 2025 poster ()
Keywords: Large Language Models
Reasoning
AI4Science
Primary area: machine_learning_for_sciences
TL;DR: We report a reasoning model for chemistry

Abstract: Reasoning models are large language models that use extra "thought tokens" before answering, providing both higher accuracy and explicit reasoning for their response. A major question has been whether language model reasoning generalizes beyond mathematics, programming, and logic, where most previous work has focused. We demonstrate that reasoning models can be post-trained in scientific domains without additional domain pretraining, and require substantially less data compared to contemporary domain-specific models.
We report ether0, a 24B parameter LLM (based on Mistral-Small-24B) that can reason in natural language and respond with chemical structures. This reasoning model was trained with reinforcement learning on 577,790 experimentally-grounded chemistry tasks involving synthesized organic molecules. Our model outperforms all previous general-purpose chemistry models, frontier models, and humans, and is more data efficient relative to specialized models. We anticipate that this method can be applied to train highly data-efficient language models specialized for predictive and generative tasks across a wide variety of scientific domains.

Training a Scientific Reasoning Model for Chemistry
Siddharth M. Narayanan's James Jamess D. Braza', Ryan-Rhys Griffiths', Albert Bou', Geemi P. Wellawatte', Mayk Caldas Ramos', Ludovico Mitchener', MMihhell Martin Pieler', Samuel G. Rodriques's Andrew D. White
FuuurrHouse Inc., San Francisco, CA Thesee authors jointly supervise technical work at FutureHouse. Correspondence: {sam, andrew) efuturehouse.. org
Abstract
Reasoning models are large language models that emit a long chain-of-thought before answering, providing both higher accuracy and explicit reasoning for their response. major question has been whether language model reasoning general- izes beyond mathematics, programming, and logic, where most previous work has focused. We demonstrate that reasoning models can be post-trained for chemistry without additional domain pretraining, and require substantially less data compared to contemporary domain-specific models. We report ethero, 24B parameter that reason in natural language and
LLM (based on respond with chemical structures. This reasoning model was trained with reinforce- ment learning 640,730 experimentalll-grouus tasks ranging from synthesizability, to blood-brain barrier permeability, to human receptor activity, scent. Our model exceeds general-purpose chemistry models, frontier models, and human experts on molecular design tasks. is also more data efficient relative to specialized models. We anticipate that this method can be applied to train data-efficien language models specialized for tasks across a wide variety of scientific domains.
chemistry problems across 375
Introduction
The dominant approach to improve the accuracy of large language models (LLMs)) in recent years
has been to scale pre-training corpora size and pre-training compute budget [1, 2, 3, 4]. Partly driven by the finite availability of pre-training data, however, attention has shifted towards alternative scaling dimensions. Such dimensions include strategies such as majority voting [5, 6]. "budget- forcing" [7]. and test-time training [8], which attempt to scale inference compute. Broadly, reasoning models attempt to improve performance emitting their thought process before arriving at an answer. Early approaches in this vein attempted to elicit reasoning behavior through bhain-of-thougll (CoT) prompting [9, 10]. More recently, however, reasoning behavior has been demonstrated to emerge through reinforcement learning (RI.) post-training, without the need for CoT-style prompting.
RL post-training represents shift of focus from pre-training data to problems with verifiable rewards. Solutions to such problems can be checked for correctness, allowing the model to generate new, verifiable outputs during learning, explore the space solutions, and overcome limits imposed by fixed data resources. Multiple works have demonstrated the potential of this approach, particularly in the domains mathematics and programming. These include both flosed-sourcee models [11, 12], and more recently, large number of open-source models [13, 14, 15, 16, 17, 18, 19].
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

SFT on reasoning
R1/Mistral Instruct
patterns
Base Reasoner
user:
GRPO verifiable tasks
ether0:
Inee to figure tue main product of this reaction, The which substizuted phenol, then there's C((O-]X[O:])-, which the carbonate ion (CC3"2-), The other reuctant is BrOo(-O)OOC, which ethyl bromoacelate...
Specialist 1
Specialist 2
Specialist N
lans, likely from K2C03.
Rejection Sampling
Selected Outouts
77 words skipped
Distillation
After alkylation, the G the end becomes OCH2CODEt.
Generalists
Therefore, the main productss iM C0C1-000-000--00000000000000000
All-task ORPO
ethez0
ether0:
Chemistry
Reasoning Model
SF1
Task-Specific
Distillation
All-Task
0.7
0.5
8.4
0.2
3.0
Pancttonal
Lecton
EUPAC
Netroaystharia
formula
Figure 1: An overview of the training methodology and an example reasoning trace for etherO. Training stages shown in the bottom panel where the accuracy per step scaled have the same x-axis range (see Appendix E).
Scientific domains may be particularly well suited for reasoning models because, as in mathematics and programming, it is often straightforward to assess the quality a solution, but much more difficult to generate solution. For example, we may be able to measure the solubility of given molecule, yet designing a molecule with a desired solubility can a significant challenge. These "inverse problems" are common in many areas of the physical sciences [20, 21, 22, 23, 24]. More broadly, the scientific method is grounded in structured reasoning: formulating hypothesis based on observation, evaluating the logical implications of the hypothesis based experiment, and refining the hypothesis based on analysis of the results experiment. Science often involves cognitive strategies such as breaking problems into subproblems, responding failures, reasoning backwards from desired outcomes, which are strategies also exhibited by reasoning models [25]. However, despite the conceptual alignment between science and reasoning models, there is still relatively little work scientific reasoning models, aside from benchmarks on multiple choice questions [26, 27, 28).
In this work, we focus chemistry, with tasks centered on designing, completing, modifying, or synthesizing molecules. This setting a good demonstration for scientific reasoning models. First, molecules can be represented in text in the SMILES format [29, 30, 31]. avoiding the complexities of training a modality-speciife encoder. Second, text-based representations of molecules are short relative modalities in materials science and biology such nucleotide sequences CIF files. Third, generating and editing molecules a critical application, where novel compounds may lead to meaningful clinical and commercial advancements.
We demonstrate the efficacy of reasoning models in chemical tasks by introducing etherO. novel model that reasons in natural language and outputs molecular structures as SMILES strings. On the chemical reasoning tasks under consideration, ether0 outperforms frontier LLMs, human experts, and models trained for general chemistry. Moreover, ether0 supports key stages of the drug discovery pipeline: can generate candidates during hit discovery, enables molecule editing hit-to-lead to improve potency, selectivity, physicochemical properties, and it contributes to lead optimization by

---

refining compounds to enhance efficacy, reduce toxicity, and improve ADMET profiles, all while being aware of synthesizability.
To efficiently train our model, we utilize a series of optimizations over vanilla RI., including distillation of reasoning behaviors, a dynamic curriculum, and initializing RL with distillation from expert models. We further analyze etherO's data efficiency, failure modes, and reasoning behavior to understand the utility of reasoning in solving chemistry problems.
Related Work
Reasoning Models Reasoning models are characterized by attempt to impart system 2-type decision-making [32] to LLMs. Early efforts to this affect include seain-of-thought (CoT) [9], zero- shot CoT [10], and Tree of Thought (ToT) [33] which seek to elicit reasoning by modifying LLM prompts. Later attempts make use of process-level supervision to provide feedback on individual reasoning steps [34, 35, 36.. Most recently, number of reasoning models have been released [12, 37, 13, 11, 17, 38, 39] using large-scale reinforcement learning via Group Relative Policy Optimization (GRPO) [40] inference time scaling [41, 7].
Reasoning Models in Chemistry While frontier reasoning models have been evaluated on chem- istry tasks [18, 37, 11]. the vast majority of these benchmarks have consisted of chemical "knowledge" tasks rather than chemical reasoning tasks [13]. While datasets such as GPQA-D [28], MMLU [26],
MMLU-Pro [27]. OlympiadArena [42], and Humanity's 's last exam [43] assess chemistry knowledge, they do not assess the model's ability to perform sophisticated chemical reasoning tasks such as retrosynthesis and proposing new structures. While many works have evaluated non-reasoning L.LSs on chemical reasoning tasks [44, 45, 46], used L.MSs components for chemical tasks [47, 48, 49], or investigated CoT-style prompting strategies [50, 51J, to the best our knowledge there have been attempts to directly train reasoning models perform chemical reasoning tasks using large-scale reinforcement learning. In terms of other scientific domains, OmniScience [52] targets general science applications through distillation reasoning traces. Med-RI [53] applies GRPO to medical vision-language tasks, using reinforcement learning to improve generalization and clinically grounded behavior across multi-modal diagnostic reasoning tasks. BioReason [54] integrates DNA foundation model with an LLM and combines supervised fine-tuning and GRPO to enable interpretable, multi-step genomic reasoning.
Chemical Reasoning Tasks
We construct dataset of 640,730 chemical reasoning problems, comprising 18 different tasks. Molecules are represented in the question and expected answer as SMILES, which encodes the molecular graph chemical reaction as ASCII text [55]. The answers are all either molecule or reaction. Many tasks are broken down into subtasks. For example, in the solubility editing task, one subtask is to increase solubility without changing the molecular scaffold, and another without affecting specific functional groups. Table summarizes all problems in our dataset, and Section C.2 provides full details on the dataset provenance as well as the construction of each task.
change
We strove to only use synthesized molecules when constructing our dataset, in contrast to previous work in cheminformaties based on "hypothetical" molecules [82]. Thus, all the questions and answers are based on the result of physical experiments. Full reward function implementation details are provided in Section C.3. In addition the criteria listed, tasks marked with also check that the proposed molecules are plausibly synthesizable by fragmentation into rings and local groups (details in Section C.3).
Solubility editi: Modify a given molecule to increase or decrease aqueous solubility (log S). Subtasks impose additional constraints enforcing similarity to the input molecule. The log S objective is computed using KDESol [56] and constraints are evaluated using RDKit [83] and exmol [84].
IUPAC name: Given IUPAC name of molecule, produce the corresponding SMILES string for the molecular structure. Verified with RDKit.
SMILES completion Given SMILES string of molecular fragment, provide completion that results in a valid molecule. Verified with RDKit.
Molecular formula': : Propose molecule given molecular formula in Hill notation [85]. Verified
3

---

Table Breakdown of verifiable reward training tasks. ML model verifier: trained predictive model; MCQ: multipl--coicee questions. Templates: unique phrasings category. Data source: short name (see citations for full attribution). "Not sum; multiple-choice property questions share templates.
i Also performs a "reasonable molecule" check.
Task
Subtasks
Examples
Verifier
Templates
Data source name
Solubility cdit
115977
ML model[56], code
15
ChEMBL
IUPAC name
74994
code
10
COCONUT(S8, 59]
SMILES completion
74990
code
COCONUTI58, 591
Molecular formula
18738
code?
10
COCONUT(58, 591
Functional group
74562
code
ChEMBL[57]
Elucidation Retrosynthesis
74164
code
10
COCONUT(S8, 59]
67252
model[60). Bloom
filter[61]
Reaction prediction
61205
onde
10
ORD[62, 63] [laSMol[64]
Molecule caption
54148
onde
8
Safety
11 180
5687
MCQ
8
Pubchem[65]
Scent
4240
MCQ
DyFUME[66, 67, 68,
69, 70, 71, 72, 73, 74,
751
Blood-brain barrier
2064
MCQ
[B6[
Receptor binding
150
1663
MCQ
8
EveBio [77]
ADME
1030
MCQ
Fang ADME[78]
Aqueous solubility
464
MCQ
AqSoIDB[79]
LD50
342
MCQ
8
Pubchem [65]
pKa
336
MCQ
8
8UPAC[80)
Photoswitches
23
MCQ
Photoswitches{81]
Total
375
640,730
9 81
13
with RDKit.
Propose a molecule given a molecular formula and 1-3 desired functional groups. Verified with RDKit and exmol.
Functional group":
Elucidation: Determine the chemical structure a molccule found in an organism given its molecular formula and background information the organism. Since the problem is underdetermined, consider any answer to be correct the proposed molecule has a Tanimoto similarity (ECFP4 [86]) of at least 0.7 to the ground truth. Verified with RDKit.
Provide single-step reaction to produce the given target molecule. The reactants must all be purchasable molecules (determined by manufacturer catalogs in Bloom filter [61]), and the product of the proposed reaction must match the target molecule predicted using the Molecular Transformer model [60].
Retrosynthesis:
Reaction prediction: Given a chemical reaction, predict the major product. Verify exact molecule match with RDKit.
Molecular caption: Given a textual description a molecule, produce the SMILES of the molecule. This task uses data from Yu et. al [64], which itself comes from PubChem [87, 88, 89]. Verified with RDKit.
Multiple choice questions: Predict modify properties of molecule, for which accurate oracle exists. Instead, multiple options are presented, and the model is expected to select the one that has been experimentally determined satisfy the criterion. Verified by string matching. See Section C.
Background
Supervised Fine-Tuning. As in prior work [90, 13], we use SFT to initialize policy for RL (Equation S1). ). the demonstration dataset Daemo is itself from another policy n', this can also be considered a form of expert iteration [91, 92] or knowledge distillation [93].

---

Reinforcement Learning. online reinforcement learning improve our models. In particular, we use Group Relative Policy Optimization (GRPO) [40].
While SFT can be used to warm-start the policy, we rely heavily on
Given question x from the dataset, sample G completions y1,.... yc (--22)) Each is assigned a reward .1..... TG and a corresponding advantage:
A F mean ( (ri ra}
(1)
st
Given single problem x and a group of completions {Vi}, I the per-group objective is:
J(0, Mh. ................................ yG)
clip
Ac,c -BDKa [ w.s.
(2)
1/1
Tioaaa (yi, x. si,ce)
where is the policy being optimized, Teaa is the policy from which we sampled rollouts, and Teef reference policy, clip is the standard PPO clip function [94]:
clip(r, c) = min A, max (r, 1 e), - c). A}.
The global policy objective we seek to optimize over the training set of problems D is therefore:
Janro D) Σ
x10, UG))
(4)
V....
(1x))
RED
For completeness, the GRPO algorithm is detailed in Algorithm 1.
Training
In this section, we describe method to train large language model to reason about and answer the problems detailed in Section 2. We utilize a multi-stage training procedure, consisting of alternating phases of (a) distillation [93] and (b) GRPO [40, 13]. At high level, the stages are: (1) Supervised fine-tuning on long chain-of-thought reasoning sequences; (2) Task-specific "specialist" GRPO; (3) Distillation of specialist models into all-task "generalist" model; and (4) Generalist GRPO. Using a family of task-specific reasoning models to generate synthetic data for a generalist model has been recently demonstrated to be effective strategy in other domains [95, 96].
Unless otherwise stated, our policies are trained from Mistral-Snie -mall- -44- 2---22-Insttt Sess-Innstuutt22255 [97]. To simplify formatting of the model output, introduce four new tokens to the base model's vocabulary to demarcate reasoning and answering boundarics. During distillation and RL, these tokens are used to respectively format and validate sequences with the following structure:
<| Schink_start. >> THOUGHT < tettt_hinddd
<| ansver start ANSWER < answer_ _nd
4.1 Long CoT Supervised Fine-Tuning
We warm-start our model with SFT on rejection-samin long chain-of-thought sequences to jump- start RL with a policy for which reasoning and SMILES answers already in-distribution.
The SFT sequences are first generated by prompting DeepScek-RL with a subset of the training dataset, with a maximum token budget of 8192 tokens. To remove low-quality responses, we enforce the following criteria: (1) each sequence ends with an answer enclosed XML tags; (2) the answer is valid SMILES/SMIRKS: and (3) passes an
check for relevant reasoning (Section B. considered rejecting responses with incorrect answers, but R1's success rate below 1% for many tasks. Our goal during SFT is to find good Pre-RL. initialization, not necessarily to maximize accuracy, and prior work [13, 25] suggests that SFT even on inaccurate reasoning sequences can be sufficient. Therefore, do not discard sequences that end in incorrect answers.
5

---

Early experiments showed that starting RL with long reasoning sequences was inefficient: sampling dominates training time, and the extra reasoning did not translate to higher accuracy. So instead, we prompt
to summarize R1-generated reasoning in fewer tokens (Section B.2). In total, this procedure results in 14,021 demonstration traces across all problem categories. From these traces, we extract the answer and thought (defined as all tokens except the SMILES answer) and reformat them to produce the SFT dataset.
4.2 Specialist RL
The chemistry problems we are optimizing against have varying difficulty, both across and within tasks. To address the former, we first perform GRPO on a family of policies on related problem categories. This proved to be more robust than various forms of scheduling or curriculum learning, because it enabled tuning hyperparameters independently. The following tasks are grouped together into specialists, due their relatedness: (1) molecular formula, functional group, and elucidation; (2) all multiple-choice questions. All other tasks are trained independently, resulting in seven total specialists, The reward assigned to each model response is:
r(y) Format_rrwarde
x accuracy
(5)
where f format_reward is if the format is met and O otherwise; accuracy _ewaard is if the answer satisfies the problem (Section 2) and otherwise. The only exception is the specialist trained on molecular formula, functional group, and elucidation, which uses softer accuracy_reward if the desired molecular formula is met but other constraints are not, then 0.5 returned. Note that RL allows to bootstrap new behaviors not present in the SFT traces. An example of this can be shown in Section F.I.
4.2.1 Advantage-Based Curriculum
The GRPO advantage reduces to zero groups in which all elements achieve the same reward. Besides the KI. term, these "trivial" groups do not contribute to the policy gradient, and their fraction of the batch fr can reach 90% during training. DAPO [98] tackles this by discarding trivial groups and resampling problems, requiring (1 f x as many sampling attempts per batch.
We instead use heuristic: if a problem results in a non-trivial group from the current policy, it is added to a curriculum buffer. At each training iteration, fraction (Cour) of the batch is selected from the buffer instead of the dataset. Since these problems were recently non-triee non-trivial, we expect lower fr than the rest of the dataset. If buffer problem becomes trivial, is removed from the curriculum. This method can reduce fr with no additional computational cost, demonstrated in Section F.3. A similar method has been previously employed in the offline setting, using reward variance [99].
The above curriculum algorithm will exhaust the buffer faster than can be filled if the following bound is not met: Ceur I (1 f11 12 ff), where f are the expected trivial fractions from the dataset and buffer, respectively. To use a high eour without exhausting the buffer, we seed the curriculum buffer using a union of non-trivial problems from previous experiments. This can be interpreted as using model-derived difficulty annotations.
4.2.2 Problem rewriting
The problem templates described in Section 2 vary the language by which problems are posed, but we hypothesized the model may struggle to generalize to unseen phrasings or the presence of distracting information. Therefore, some fraction of the time, we prompt Gemini 2.5 Flash rewrite the problem, while retaining all relevant information. Two prompts are used in equal proportion: one that simply asks the LLM restate the problem, and another that also directs add extrancous information (Section B.4). These rewritten problems are used both during RL and subsequent distillation.
4.3 Distillation
To merge the specialist models into final generalist model, we perform another distillation via supervised fine tuning on the base Mistral- Smmall244B 1ea---Instuut-2220 model. This can also be seen behavior cloning or expert iteration [91, 92].
Unlike previous work [17, 18, 19, 95), we do not rejection-sample model responses after training, but instead collect correct responses from the entire training run. These sequences then filtered to

---

remove those with low reasoning quality, as judged by an LLM and regex for non-English language (Section D.I). We further observed that some open-ended tasks are susceptible answers with undesirable molecular substructures; we therefore reject such responses (Section D.2). Finally, if multiple responses remain for given problem, only the two latest responses are kept.
The final distillation training set concatenates these sequences with the SFT dataset (Section 4.1).
SFT is performed upon this dataset initialize the policy for the next phase.
4.4 Generalist RL
Having distilled all tasks into single model, we perform a combined GRPO phase across all tasks. An online curriculum is used (without seeding) to encourage learning. Unlike the task- specific phase, all accuracy rewards are binary, without any partial credit assigned. However, to disincentivize undesirable substructures arising during RI. (after being rejected during distillation), assign molecule quality bonus reward during the last steps of this phase (Section D.2) As in the specialist phase, problem rewriting is enabled. We also run safety alignment procedure described in Section E.3.
5 Results
Here report the results of training outlined in Section 4. The seven specialist models were trained using 24-72 Nvidia H100 GPUs each, with varying set of hyperparameters (detailed in Section E. A total of 186,010 sequences were collected from the specialist training runs for distillation. A single SFT epoch was sufficient for distillation, with batch size of 64 and learning rate of 1.9 10-5, The all-task RL training phase was performed using 384 H100 GPUs, over 4 days; all hyperparameters are described in Section E.2. The final safety alignment phase required 104 H100 GPUs (see Section E.3).
using the procedure
We compare our etherO's performance against multiple baseline models on a set of holdout evalua- tion problems, analyze its reasoning behavior, and identify its primary failure modes. We also assess its sample efficiency and conduct ablation studies on the effect of reasoning.
5.1 Model Performance
Figure shows how each stage of the training pipeline contributes to model performance across tasks. All tasks show significant improvement during the task-specific RL phase, despite post-SFT accuracy often starting very low. Distillation successfully transfers specialist capabilities to the generalist model, though some problem categories, such solubility edit and functional group, experience drops performance. Nonetheless, the all-task RI phase is able to recover from these degradations, resulting in final performance that matches or exceeds that of the corresponding specialist models.
To contextualize etherO's capabilities, Figure 2 compares its performance against both general- purpose LLMs (e.g., Claude, ol) and chemistry-ppecffic models (ChemDFM, TxGemma). Our (OA) categories and delivers competitive
model achieves the highest accuracy on all performance on multiple-choice questions (MCQs). We hypothesize that we achieve higher margins over other methods in OA tasks because they are more amenable RL without overfitting: Firstly, we simply have more OA problems than MCQs (Table 1). Secondly, many OA tasks have non-unique answers, allowing for more exploration during training without memorization of the answer.
In Figure S2, we demonstrate that the our safety alignment procedure, which results in the ether0 refusing 80% of unsafe questions, does not meaningfully degrade capability on the measured tasks. An annotated model response is provided in Figure 4.
5.2 Data Efficiency
Prior work has suggested that training reasoning models via can be data-efficient [99], although these results are not conclusive [100]. In Section 5.1, we benchmark the performance of ether0 against other L.LMs trained with and without reinforcement learning. In this section, we now investigate the data efficiency of ether0 during both training and inference.
7

---

ether 9
oi-nini-2225-24-1
despseek-r1
Uxgerna_276, 270_chat
Hunan
gartni-2. portni-2.55pporr S-pro-prevrtew-0-
ChandFW-vt..
randon
03-2223-86-13
Sonude-----onmet--Bee
spt-4.1-2885-00-
open answer
nuttiple choice
0.8
Accuracy
0.4
0.2
5H0.S complation
toteatter Formola
Funcstonal group
PRacttion prenuction
Detr evtrosyttouas
TSPPC home
Saledy
KINS
recopte bireteep
Liba
Scaott
salentity
Figure 2: Per-task performance of our model compared to general-purpose L.MMs. For multiple choice tasks, the "random" line accounts for varying numbers of options between problems. The human bar is an average of four chemists equipped with only the molecule drawing tool ChemDraw. Humans were not evaluated on receptor binding and scent tasks, as the prructure-propert relationship mostly unknown, making these tasks essentially impossible without additional tools.
B)
(100)
Bestassir
Oenden chance
Mossto
Oherall U. acLurary on 9922 Lu3k
10006---30 20-2
Weldort
4.2
Seaction ecanples
darkigg trelniing (enocsends)
Figure 3: Data efficiency analysis. (A) Comparison of etherO to Molecular Transformer (MT) on reaction prediction: ether0 outperforms the published MT (dashed line) and shows higher data efficiency compared to retraining MT from scratch dataset (f retrained). (B) Effect of in-context learning (ICL) on multiple-choice questions (MCQs).
First, we compare etherO to a traditional model (i.e., not an LLM) trained with supervised learning. The Molecular Transformer (MT) [60] is state-of-ther the- model for chemical reaction prediction, trained on nearly 480,000 USPTO reactions [101]. When trained on our dataset of 60,000 reactions, ethero outperforms MT, even when MT is retrained on the same data (Figure 3A). On our held-out test set, ether0 achieves 70% accuracy after 46,000 examples, compared to MT's 64. 1 1% on the full USPTO dataset. We also retrained MT from scratch on our smaller dataset. The retrained versions of MT (denoted by MT!) failed to exceed 30% accuracy. a threshold surpassed by etherO after seeing only 10% of the available training data. This demonstrates that reasoning model can achieve performance competitive with a dedicated traditional model given considerably less data.
Second, we apply in-context learning (ICL) [2]) to evaluate the models' ability to leverage additional data at inference time. ICL involves providing exemplar peestion-answer pairs directly the prompt to guide the model's 's response. In our setup. we construct ICL prompts from MCQs by selecting one of the distractors (i.e., incorrect options) from the original question and appending it as a labeled example. To maintain consistent random baselines between the one-shot and zero-shot versions, we remove the selected distractor from the set of choices the actual question. Full details on the formatting and implementation of ICL are provided in Section F.4, Using this strategy, Figure 3B
8

---

Problen:
Apigenn good starting guass
The organism
Plumbogo spp. contains compound with formula C16H1207, What's Ikeyy SMILES ES based brochemisiry?
ethez8:
Okay, so need to gure our a liley SMILES rinucture for compound with the formub C1G-1207 found Bumbagg apo lants the penus Plumbage ollen produce lavonaids and coumantre The irmula
CSHB02
Quercetin is closest named molecure the model knows
First coneddrr Bacomes, which are Bheororig wrse Apseants, theecre, tas farmule cibinioss Aetdin (C16-4 methocy group. Howevrr, subtiop cal fltvonolds might be glycoeyated
addiforrl
kncolin (C1SH10CG) with methvl (OHB), Methstated Add ng another coygen is macoyy Perraps car bonyy and group (CODI) making neos does not ethert
01641206
Conseder mostcatio moose fi heelin to Crasnoos dose emarge craeeiing with cteimao77 heee. ad0ioona mnctitll grous Oucrootin
didiigg mothy group would C1HH
Que cateeaii (ctsimpo77) plux a cHd group teccmmes Eheenoox to sutstiuuing cae it with Chd greup ada
Examples of that from RL
cede 131 97
1007 gimett Merneloing degrasaiion saygen veristion)
replacing hydragen
Flavonoldss are yoosylaned Guoreetin3 H-t-methyl ester would and
This compound from nen-reasoning GPT-A.5
Altemativoly, oor sider spigerin I01 1005) plus mothyl group 016H1205 Add on aduitional carbooylc 01741207)
methysood Blaonol with C 8-811077 could be kaompterol mothyl other dfcm 013H1000 trotroon get
"methoy"" replace hystrogen
making
Finally consider querceth 10 111H00000 1007) aoding GHB 016H1207 This fix whotever orggen enated compound
Hence, jet's verity Quercstin-C. Quercetin (01G41200
Changing ane phenolic OH added a methoay. melty mading
query
Correct molecute
be accurota lkidy soucture. SSuncatted
compoundd upon quattatin stecocylatio ounding chain:
hychorpp
Figure 4: Annotated reasoning trace of the model correctly solving an unseen structure elucidation task, where o3, rl, Gemini 2.5-pro 05-07-25, and GPT-4.5 fail. The trace illustrates exploration, backtracking, and verification. The model does not know the real molecule name (azaleatin), referring to it as quercetin-C both the strengths and limitations of etherO's learned capabilities in complex, multi-step chemical tasks.
indicate quercetin with an extra methyl group. Overall, this trace highlights
demonstrates significant gain across MCQ tasks. Considering zero-shot performance, etherO shows an overall performance of 50.1% 1% in our test set, which is comparable to the 52.2% reached by 'o1-2024-12-17' However, under one-shot prompting, ether0 surpasses all evaluated frontier models, highlighting its ability to generalize from minimal context. These results illustrate that our model, despite limited training data, can further increase performance and exceed the performance of frontier LLMs when appropriately guided at inference time.
5.3 Reasoning Performance and Behavior
In Figure 4, we annotate representative completion etherO on a challenging open-answer task. The completion displays multiple lines of reasoning and verification, and additionally creates new words to help solve the problem, such as "Quercetin-C" As judged by chemistry expert evaluation (Figure S7), the reasoning is generally coherent and proceeds logically from question to answer.
To validate the hypothesis that explicit reasoning improves model performance, we compare a model trained with reasoning and model trained without reasoning under otherwise identical settings. The non-reasoning model was constructed through distillation on the distillation data used for our all-task reasoning model, but with the thoughts removed from the sequences. This procedure followed so as to control for the task distribution seen during distillation. Our results, shown in Figure 5 (left), clearly demonstrate that the reasoning model consistently outperforms the non-reasoning model across the majority of evaluated tasks.
Subsequently, we perform a more qualitative study of etherO's reasoning. Recent work [25] suggests that the prevalence of "cognitiive behaviors" (e.g. verification, backtracking) model's reasoning is linked to its capacity to solve complex problems. To confirm this observation, we use similar strategy to measure the frequency of such behaviors (behavior counts) over the course of model training (Section F.2).

---

Dassentig sblation
SOLDS orplatti
Noiet
Rashard Chiistrg
Sulgert
Figure 5: Left: Per-task performance of reasoning and non-reasoning models. Right: Evolution of model reasoning behaviors on the evaluation set throughout training. across three problem categories: functional group, reaction prediction, and SMILES completion. We track 4 reasoning behaviors: backtracking, backward chaining, subgoal setting, and verification, alongside completion length.
These behavior count metrics shown in Figure 5 (right) for three tasks (see Figure S4 and Figure S5 for all tasks). We find that task behavior during training loosely fall into three distinct patterns. Some tasks, such molecule formula and functional group, exhibit increases in both behavior counts and completion lengths, along with marked improvements when reasoning is added. Others, including IUPAC name and reaction prediction, show limited change in behavior count but clear increases in sequence length, with more modest gains from reasoning. Finally, tasks such as solubility editing and SMILES completion generally show little change in either metric and clear benefit from reasoning. These observations suggest that the emergence of cognitive behaviors is not merely a byproduct of training, but is selectively amplified in tasks where structured reasoning advantageous.
Limitations
Although etherO is trained variety of chemistry tasks, it can struggle to generalize beyond its training distribution. For example, we do not expect strong performance on inorganic chemistry, such generating crystal structures, since the model primarily trained SMILES strings of or- ganic molecules. The intensive RL. training also reduced general ffollottoon-folooiingg and chat capabilities, including multi-turn conversation. While many small-molecule design workflows rely heavily on tools, tool calling was not included in etherO's training. In our evaluation, we used MT validate predicted reactions, which may introduce limitations, and benchmarked against sfate-of-the art LLMs, though other specialized non-LLM models could perform better on specific tasks, Future work could integrate chemistry reasoning and tool-calling into single model.
7 Conclusion
In this work, we show that reasoning models, previously successful in mathematics and programming, can also solve chemical reasoning questions often unsolvable by non-reasoning models. We introduce ethero, 24B-parameter reasoning model trained on a curated dataset of challenging tasks in molecular design, completion, modification, and synthesis. We detail our training pipeline, which consists of several interleaved phases of reinforcement learning with verifiable rewards and behavior distillation. On held-out evaluation set, etherO significantly outperforms frontier LLMs, domain experts, and specialized models, particularly open- pnen-rnswer tasks. We analyze the model's reasoning behavior, failure modes, and data efficiency, highlighting where reasoning helps and how it evolves during training. Finally, we release the model weights, benchmark data, and reward functions. We believe this work demonstrates strong potential for future work on reasoning models on scientific tasks.
10

---

Acknowledgments
We acknowledge Prof. Gianni de Fabritiis for key early discussions on the impact of reasoning models in scientific domains. Early prototyping work was done with compute resources from the National AI Research Resource Pilot, including support from NVIDIA and the NVIDIA DGX Cloud. VoltageParrk was our main compute partner for the final models, donating significant computational resources and assisting with scaling. We also acknowledge all members of FutureHouse for useful research discussions, including Michael Skarlinski (early RL code prototyping). Muhammed T Razzak (ideas and discussion about advantage-basee curriculum), Jon Laurent (helping expert evaluators), and Remo Storni (inference infrastructure).

---

References
[1] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
[2] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33:1877- 1901, 2020.
[3] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001 200.::0000..0.61,, 2020.
[4] Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimat large language models. In Proceedings of the 36th International Conference on Neural Information P Processing Systems, pages 30016-30030, 2022.
[5] Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint ar Xiv: 2407.21777 2024.
[6] Siddharth Narayanan, James D Braza, Ryan-Rhys Griffiths, Manu Ponnapati, Albert Bou, Jon Laurent, Ori Kabeli, Geemi Wellawatte, Sam Cox, Samuel Rodriques, et al. Aviary: training language agents on challenging scientific tasks. arXiv preprint arXiv:2412.21154 2024.
[7] Niklas Muennighoff, Zitong Yang. Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannanch Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling. arXiv preprint arXiv:2501. 193933 2025.
[8] Ekin Akyürek, Mchul Damani, Adam Zweiger, Linlu Qiu, Han Guo, Jyothish Pari, Yoon Kim, and Jacob Andreas. The surprising effectiveness of test-time training for few-shot learning. arXiv preprint arXiv:2411.07277 2024.
[9] Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc Le, Denny Zhou, al. Chain-of-thougle prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824-24833 2022.
[10] Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in Neural Information Processing Systems, 35:22199-22213, 2022.
[11] Anthropic. Claude 3.7 Sonnet and Claude code, February 2025. Accessed: 2025-04-26.
[12] Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. OpenAl ol system card. arXiv preprint arXiv:2412.166720 2024.
[13] Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, al. DeepSeek-RI Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint 2xxXiv:2511.1944 2025.
[14] Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1.5: Scaling reinforcement learning with LLMs. arXiv preprint arXiv:2501. 20..222999.199.... 2025.
[15] Michael Luo. Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang. Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. DeepScaleR: Surpassing ol oprevvwww with a 1.5B model by scaling RL. https: https:://pretty-radiaea wwt.ooob....o.nnn..mmmmommmmmooe 0t Prreiiie--i.. 061199021660000beecca00001134443 2025. Notion Blog.
wot.ionnon... si
Modee baateelby--- -aseeellbb--saangg----g RL-
12

---

[16] Michael Luo, Sijun Tan. Roy Huang, Xiaoxiang Shi, Rachel Xin, Colin Cai, Ameen Patel, Alpay Ariyak, Qingyang Wu, Ce Zhang, Li Erran Li, Raluca Ada Popa, and lon Stoica. Deepcoder: A fully open-source 14B coder at O3-mini level. https://peettt-radiooo .notion...tee/eeeeeeee www.:/steeeeeecooeee----elll -apee---ppp------------ Source- Seve-14B-Coo -eaecee---ooee-raeeeee-e Cooder-at-03-mmn----e 1ccfB1900c660003eeeea33333332332 2025.
2025. Notion Blog.
[17] Akhiad Bercovich, Itay Levy, Izik Golan, Mohammad Dabbah, Ran El-vv,,v, Omri Puny, Ido Galil, Zach Moshe, Tomer Ronen, Najeeb Nabwani, et al. Llama-Nemotron: Efficient reasoning models. arXiv preprint arXiv:2505.00949, 2025.
[18] Syeda Nahida Akter, Shrimai Prabhumoye. Matvei Novikov, Seungju Han. Ying Lin, Evelina Bakhturi, Eric Nyberg, Yejin Choi, Mostofa Patwary, Mohammad Shoeybi, et al. Nemotron- CrossThink: Scaling self-learning beyond math reasoning. arXiv preprint arXiv:2504. 13941, 2025.
[19] Aaron Blakeman, Aarti Basant, Abhinav Khattar, Adithya Renduchintala, Akhiad Bercovich, Aleksander Ficek, Alexis Bjorlin, Ali Taghibakhshi, Amala Sanjay Deshmukh, Ameya Sunil Mahabaleshwarkar, et al. Nemotron-H: A family of accurate and efficient hybrid Mamba- transformer models. arXiv preprint arxiv25504.0362 2025.
[20] Simon Arridge, Peter Maass, Ozan Öktem, and Carola-Bibiane Schönlieb. Solving inverse problems using data-driven models. Acta Numerica, 28:1-174, 2019.
[21] Gregory Ongie, Ajil Jalal, Christopher A Metzler, Richard G Baraniuk, Alexandros Dimakis, and Rebecca Willett. Deep learning techniques for inverse problems in imaging. IEEE Journal on Selected Areas in Information Theory, 1(1):39-56, 2020.
[22] Ali Siahkoohi, Gabrio Rizzuti, Rafael Orozco, and Felix J Herrmann. Reliable amortized variational inference with physics-based latent distribution correction. Geophysics, 88(3): R322, 2023.
[23] Hongkai Zheng, Wenda Chu, Bingliang Zhang, Zihui Wu, Austin Wang, Berthy T Feng, Caifeng Zou, Sun, Nikola Kovachki, Zachary E Ross, et al. InverseBench: Benchmarking plug-and-play diffusion priors for inverse problems in physical sciences. arXiv preprint urXiv:2503. 11043, 2025.
[24] Daniel Lesnic. Invesse problems with applications in science and engineering. Chapman and Hall/CRC, 2021.
[25] Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. Cognitive behaviors that enable self- inff-iiipiiiigggg reasoners, or, four habits of highly effective stars. arXiv preprint arXiv:2503.01307 2025.
[26] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Mcasuring massive multitask language understanding. In International Conference on Learning Representations, 2021.
[27] Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj. Xuan He, Ziyan Jiang, et MMLU-Pro: A more robust and challenging multi-task language understanding benchmark. In The Thirty-Eighth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024.
[28] David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level Google-Proof Q&A benchmark. In First Conference on Language Modeling, 2024.
[29] David Weininger. SMILES, a chemical language and information system. 1. introduction to methodology and encoding rules. Journal of Chemical Information and Computer Sciences, 28(1):31-36, 1988.
[30] David Weininger, Arthur Weininger, and Joseph I Weininger. SMILES. 2. algorithm for generation of unique SMILES notation. Journal of Chemical Information and Computer Sciences, 100,), 1989.
13

---

[31] David Weininger. SMILES. 3. DEPICT. graphical depiction of chemical structures. Journal of Chemical Information and Computer Sciences, 30(3):337-243, 1990.
[32] Daniel Kahneman. Thinking, fast and slow. Farrar, Straus and Giroux, New York, 2011.
[33] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information processing systems, 36:1 1809-11822, 2023.
[34] Shibo Hao, Yi Gu, Haodi Ma, Joshua Hong, Zhen Wang, Daisy Wang, and Zhiting Hu. Reasoning with language model is planning with world model. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8154-811 8173, 2023.
[35] Zitian Gao, Boye Niu, Xuzheng He, Haotian Xu, Hongzhang Liu, Aiwei Liu, Xuming Hu, and Lijie Wen. Interpretable contrastive Monte Carlo tree search reasoning. arXiv preprini arXiv:2410.01707, 2024.
[36] Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, llya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2024.
[37] OpenAL. OpenAl o3 and o4-mini system card, April 2025. Accessed: 2025-05-08.
[38] Yiwei Qin, Xuefeng Li, Haoyang Zou, Yixiu Liu, Shijie Xia, Zhen Huang, Yixin Ye, Weizhe Yuan, Hector Liu, Yuanzhi Li, et al. 01 replication journey: A strategic progress report-part 1. arXiv preprint arXiv:2410. 18982, 2024.
[39] Zhen Huang, Haoyang Zou. Xuefeng Li, Yixiu Liu, Yuxiang Zheng, Ethan Chern Shijie Xia, Yiwei Qin, Weizhe Yuan, and Pengfei Liu. Ol replication journey-part 2: Surpass- ing O1-preview through simple distillation, big progress or bitter lesson? arXiv preprint arXiv:2411.16488 2024.
[40] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, YK Li, and Daya Guo. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXivv2402.033000 2024.
[41] Zhongzhen Huang. Gui Geng, Shengyi Hua, Zhen Huang, Haoyang Zou, Shaoting Zhang, Pengfei Liu, and Xiaofan Zhang. Ol replication coumey-part 3: Inference-ting scaling for medical reasoning. arXiv preprint arXiv:2501.06455 2025.
[42] Zhen Huang, Zengzhi Wang, Shijie Xia, Xuefeng Li, Haoyang Zou, Ruijie Xu, Run-Ze Ze Fan, Lyumanshan Ye, Ethan Chern, Yixin Ye, et al. OlympicArena: Benchmarking multi-discipline cognitive reasoning for superintelligent AI. Advances in Neural Information Processing Systems, 12099199922 2024.
[43] Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, et al. Humanity's last exam. arXiv preprint 1rXiv 2025.
[44] Taicheng Guo, Kehan Guo, Bozhao Nan, Zhenwen Liang, Zhichun Guo, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. What large language models do in chemistry? comprehensive benchmark on eight tasks. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.
[45] Tianhao Li, Jingyu Lu, Chuangxin Chu, Tianyu Zeng. Yujia Zheng, Mei Li, Haotian Huang, Bin Wu, Zuoxian Liu, Kai Ma, et al. SciSafeEval: a comprehensive benchmark for safety alignment of large language models in scientific tasks. arXiv preprint arXiv:2410.03769, 2024.
[46] Kehua Feng, Keyan Ding, Weijie Wang, Xiang Zhuang, Zeyuan Wang, Ming Qin, Yu Zhao, Jianhua Yao, Qiang Zhang, and Huajun Chen. SciKnoWEval: Evaluating multi-level scientific knowledge of large language models. arXiv preprint arXiv:2406. 09098, 2024.
[47] Daniil Boiko, Robert MngKnight, Ben Kline, and Gabe Gomes. Autonomous chemical research with large language models. Nature, 624(7992):50--578 2023.
14

---

[48] Andres M. Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D White, and Philippe Schwaller. Augmenting large language models with chemistry tools. Nature Machine Intelli- gence, 6(5):225-55) 2024.
[49] Andres M Bran, Theo A Neukomm, Daniel P Armstrong, Zlatko Jončev, and Philippe Schwaller. Chemical reasoning in LLMs unlocks steerable synthesis planning and reaction mechanism elucidation. arXiv preprint arXiv:2503.08337 2025.
[50] Sirui Ouyang. Zhuosheng Zhang, Bing Yan, Xuan Liu, Yejin Choi, Jiawei Han, and Lianhui Qin. Structured chemistry reasoning with large language models. In Proceedings of the 4 s1st International Conference on Machine Leurning, pages 38937-38952, 2024.
[51] Yunhui Jang. Jachyung Kim, and Sungsoo Ahn. Chain-of-thoughte for molecular understand- ing. arXiv preprint arXiv:2410.05610, 2024.
[52] Vignesh Prabhakar, Md Amirul Islam, Adam Atanas, Yao-Ting Wang. Joah Han, Aastha Jhun- jhunwala, Rucha Apte, Robert Clark, Kang Xu, Zihan Wang, et al. Omniscience: A domain- specialized Ilm for scientific reasoning and discovery. arXiv preprint arXiv:2503.1760) 2025.
[53] Yuxiang Lai, Jike Zhong, Ming Li, Shitian Zhao, and Xiaofeng Yang. Med-R1: Reinforcement learning for generalizable medical reasoning in vision- langunla models. arXiv preprint vrXiv22503.13933 2025.
[54] Adibvafa Fallahpour, Andrew Magnuson, Purav Gupta, Shihao Ma, Jack Naimer, Arnv Shah, Haonan Duan, Omar Ibrahim, Hani Goodarzi, Chris Maddison, et al. BioReason: Incentivizing multimodal biological reasoning within DNA-LLM model. arXiv preprint arXiv:2505.23573 2025.
[55] Nikolay Kochev, Svetlana Avramova, and Nina Jeliazkova. Ambit-SMIRKS: a software module for reaction representation, reaction search and structure transformation. Journal of Cheminformatics, 10(1):42, August 2018.
[56] Mayk Caldas Ramos and Andrew D White. Predicting small molecules solubility on endpoint devices using deep ensemble neural networks. Digital Discovery, 3(4):786, 2024.
[57] Barbara Zdrazil, Eloy Felix, Fiona Hunter, Emma J Manners, James Blackshaw, Sybilla Corbett, Marleen de Veij, Harris loannidis, David Mendez Lopez, Juan F Mosquera, et al. The ChEMBL database in 2023: a drug discovery platform spanning multiple bioactivity data types and time periods. Nucleic Acids Research, 52(DI):DI DEDD1180-D1100 2024.
[58] Maria Sorokina, Peter Merseburger, Kohulan Rajan, Mehmet Aziz Yirik, and Christoph Steinbeck. COCONUT online: collection of open natural products database. Journal of Cheminformaties, 13(1):2, 2021.
[59] Venkata Chandrasekhar, Kohulan Rajan, Sri Ram Sagar Kanakam, Nisha Sharma, Viktor Weißenborm, Jonas Schaub, and Christoph Steinbeck. COCONUT 2.0: a comprehensive overhaul and curation of the collection of open natural products database. Nucleic Acids 2025.
Research, 53(DI):D634-DGG
[60] Philippe Schwaller, Teodoro Laino, Théophile Gaudin, Peter Bolgar, Christopher A Hunter, Costas Bekas, and Alpha A Lee. Molecular transformer: model for uncertainty chemical reaction prediction. ACS Central Science, 5(9):1572-158; 2019.
[61] Jorge Medina and Andrew White. Bloom filters for molecules. Journal of Cheminformatics, 15(1):95, 2023.
[62] Steven M Kearnes, Michael R Maser, Michael Wleklinski, Anton Kast, Abigail G Doyle, Spencer D Dreher, Joel M Hawkins, Klavs F Jensen, and Connor W Coley. The open reaction database. Journal of the American Chemical Society, 188245):8880-18885
2021.
[63] Rocío Mercado, Steven M Kearnes, and Connor W Coley. Data sharing in chemistry: lessons learned and a case for mandating structured reaction data. Journal of Chemical Information and Modeling, 63(14):4253-422 2023.
15

---

[64] Botao Yu, Frazier N. Baker, Ziqi Chen, Xia Ning, and Huan Sun. LlaSMol: Advancing large language models for chemistry with large- comprehensive, high-quality instruction tuning dataset. In First Conference Language Modeling, 2024.
[65] Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia He. Siqian He, Qingliang Li, Benjamin A Shoemaker, Paul A Thiessen, Bo Yu, et al. PubChem 2023 update. Nucleic Acids Research, 51(DI):D1333-D113
2023.
[66] Andreas Keller, Margaret Hempstead, Iran A Gomez, Avery N Gilbert, and Leslie B Vosshall. An olfactory demography of a diverse metropolitan population. BMC Neuroscience, 13(1), October 2012.
[67]
Bushdid, M. O. Magnasco, L., B. Vosshall, and A. Keller. Humans can discriminate more than 1 trillion olfactory stimuli. Science, 343(6177) March 2014.
[68] Neelansh Garg, Apuroop Sethupathy, Rudraksh Tuwani, Rakhi NK, Shubham Dokania, Arvind lyer, Ayushi Gupta, Shubhra Agrawal, Navjot Singh, Shubham Shukla, Kriti Kathuria, Rahul Badhwar, Rakesh Kanji, Anupam Jain, Avneet Kaur, Rashmi Nagpal, and Ganesh Bagler. FlavorDB: database of flavor molecules. Nucleic Acids Research, 46(D1))D1110-D1211
October 2017.
[69] Andreas Keller and Leslie B. Vosshall. Olfactory perception of chemically diverse molecules. BMC Neuroscience, 17(1), August 2016.
[70] Charles J Wysocki and Avery N Gilbert. National geographic smell survey: Effects of age are heterogenous. Annals of the New York Academy of Sciences, 561(1):12-22) 1989.
[71] Aharon Ravia, Kobi Snitz. Danielle Honigstein, Maya Finkel, Rotem Zirler, Ofer Perl, Lavi Secundo, Christophe Laudamiel, David Harel, and Noam Sobel. A measure of smell enables the creation of olfactory metamers. Nature, 588(7836):118 118866))11111133 November 2020.
[72] Anju Sharma, Bishal Kumar Saha, Rajnish Kumar, and Pritish Kumar Varadwaj. OlfactionBase: a repository to explore odors, odorants, olfactory receptors and odorant-react receptor interactions. Nucleic Acids Research, 50(D1)) September 2021.
[73] Kobi Snitz, Adi Yablonka, Tali Weiss, Idan Frumin, Rehan M. Khan, and Noam Sobel. Predicting odor pereeptual similarity from odor structure. PLOS Computational Biology, 90))e10033188 144, September 2013.
[74] Kobi Snitz, Ofer Perl, Danielle Honigstein, Lavi Secundo, Aharon Ravia, Adi Yablonka, Yaara Endevelt-Shapira, and Noam Sobel. Smellspace: An odor-based social network as platform for collecting olfactory perceptual data. Chemical Senses, 44(4):267-278, March 2019.
[75] Elizabeth A Hamel, Jason B Castro, Travis J Gould, Robert Pellegrino, Zhiwei Liang, Liyah A Coleman, Famesh Patel, Derek S Wallace, Tanushri Bhatnagar, Joel D Mainland, et al. Pyrfume: A window to the world's olfactory data. Scientific Data, 11(1):1220, 2024.
[76] Fanwang Meng, Yang Xi, Jinfeng Huang, and Paul W Ayers. curated diverse molecular database of blood-brain barrier permeability with chemical descriptors. Scientific Data, 8(1):289, 2021.
[77] EvE Bio Consortium. Eve Bio pharmome dataset: Data release 3. https::/ http:::://veeboorrte org, March 2025. Version Publisher: EvE Bio, LLC. License: CC BY-NC-SA 4.0.
[78] Cheng Fang, Ye Wang, Richard Grater, Sudarshan Kapadnis, Cheryl Black, Patrick Trapa, and Simone Sciabola. Prospective validation machine learning algorithms for absorption, distribution, metabolism, and excretion prediction: An industrial perspective. Journal of 2023.
Chemical Information and Modeling,
[79] Murat Cihan Sorkun, Abhishek Khetan, and Süleyman Er. AqSolDB, curated reference set of aqueous solubility and 2D descriptors for a diverse set compounds. Sci. Data, 6(1): August 2019.
16

---

[80] Jonathan Zheng and Olivier Lafontant-Joseph. IUPAC digitized pKa dataset. https: thub. com/ IUP WAAPAA/DDscaaaiiioo Constants, 2025. Version: 2025-03-21, Publisher: International Union of Pure and Applied Chemistry (IUPAC). Institution: Green Group, Department of Chemical Engineering, MIT. License: CC BY-NC 4.0. DOI: 10. 5281/zenodo 72236452
[81] Ryan-Rhys Griffiths, Jake L. Greenfield, Aditya R Thawani, Arian R Jamasb, Henry B Moss, Anthony Bourached, Penelope Jones, William McCorkindale, Alexander Aldrick, Matthew Fuchter, et al. Data-driven discovery of molecular photoswitches with multioutput Gaussian processes. Chemical Science, 13541-:334111333 2022.
[82] Lars Ruddigkeit, Ruud Van Deursen, Lorenz Blum, and Jean-Louis Reymond. Enumeration of 166 billion organic small molecules in the chemical universe database GDB-17. Journal of Chemical Information and Modeling, (111)))864-28755 2012.
[83] RDKit: Open-source cheminformatics. http: hwwp::/wwww........mmmmm org, [Online; accessed 27-
January-2025].
[84] Geemi P Wellawatte, Aditi Seshadri, and Andrew D White. Model agnostic generation of counterfactual explanations for molecules. Chemical Science, 13713)3367733755 2022.
[85] Edwin A Hill. On a system of indexing chemical literature; adopted by the classification division of the US patent office. Journal of the American Chemical Society, 22(8):478-44) 1900.
[86] David Rogers and Mathew Hahn. Extended-coonective fingerprints. Journal of Chemical Information and Modeling, 50(5):742-775) 2010.
[87] Sunghwan Kim, Paul A Thiessen, Evan E Bolton, Jie Chen, Gang Fu, Asta Gindulyte, Lianyi Han, Jane He, Siqian He, Benjamin A Shoemaker, et al. PubChem substance and compound 2016.
databases. Nucleic Acids Research, 44(D1):D1222
[88] Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia He, Siqian He, Qingliang Li, Benjamin A Shoemaker, Paul A Thiessen, Bo Yu, et al. PubChem 2019 update: improved access chemical data. Nucleic Acids Research, 47D11):1102-D1109 2019.
[89] Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia He, Siqian He, Qingliang Li, Benjamin A Shoemaker, Paul Thiessen, Bo Yu, et al. PubChem 2023 update. Nucleic Acids Research, 5I(DI):D1333-D1336 2023.
[90] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744, 2022.
[91] Thomas Anthony, Zheng Tian, and David Barber. Thinking fast and slow with deep learning and tree search. In Proceedings of the 31st International Conference on Neural Information Processing Systems, pages 1366-5376, 2017.
[92] Alex Havrilla, Yuqing Du, Sharath Chandra Raparthy, Christoforos Nalmpantis, Jane Dwivedi- Yu, Maksym Zhuravinskyi. Eric Hambro, Sainbayar Sukhbaatar, and Roberta Raileanu. Teach- ing large language models to reason with reinforcement learning, 2024.
[93] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network, 2015.
[94] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 7707.06347, 2017.
[95] Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Xin Wang, Rachel Ward, Yue Wu, Dingli Yu, Cyril Zhang, and Yi Zhang. Phi-4 technical report, 2024.
17

---

[96] DeepSeek-AL, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng. Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang. Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J. L. Cai, Jian Liang, Jianzhong Guo, Jiaqi Ni, Jiashi Li, Jiawei Wang. Jin Chen, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, Junxiao Song, Kai Dong. Kai Hu, Kaige Gao. Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Lei Xu, Leyi Xia, Liang Zhao, Litong Wang. Liyue Zhang, Meng Li, Miaojun Wang, Mingehuan Zhang, Minghua Zhang, Minghui Tang, Mingming Li, Ning Tian, Panpan Huang, Peiyi Wang. Peng Zhang, Qiancheng Wang, Qihao Zhu, Qinyu Chen, Qiushi Du, R. J. Chen, R. L. Jin, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, Runxin Xu, Ruoyu Zhang, Ruyi Chen, S. S. Li, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shaoqing Wu, Shengfeng Ye, Shengfeng Ye, Shirong Ma, Shiyu Wang, Shuang Zhou, Shuiping Yu, Shunfeng Zhou, Shuting Pan, T. Wang. Tao Yun, Tian Pei, Tianyu Sun, W. L.. Xiao, Wangding Zeng, Wanjia Zhao, Wei An, Wen Liu, Wenfeng Liang. Wenjun Gao. Wengin Yu, Wentao Zhang, X. Q. Li, Xiangyue Jin, Xianzu Wang. Xiao Bi, Xiaodong Liu, Xiaohan Wang, Xiaojin Shen, Xiaokang Chen, Xiaokang Zhang, Xiaosha Chen, Xiaotao Nie, Xiaowen Sun, Xiaoxiang Wang, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xingkai Yu, Xinnan Song, Xinxia Shan. Xinyi Zhou, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, Y. K. Li, Y. Q. Wang. Y. X. Wei, Y. X. Zhu, Yang Zhang. Yanhong Xu, Yanhong Xu, Yanping Huang, Yao Li. Yao Zhao, Yaofeng Sun, Yaohui Li, Yaohui Wang. Yi Yu, Yi Zheng, Yichao Zhang. Yifan Shi, Yiliang Xiong, Ying He, Ying Tang, Yishi Piao, Yisong Wang. Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yu Wu, Yuan Ou, Yuchen Zhu, Yuduan Wang. Yue Gong, Yuheng Zou, Yujia He, Yukun Zha Yunfan Xiong, Yunxian Ma, Yuting Yan, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Z. F. Wu, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhen Huang, Zhen Zhang, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhibin Gou, Zhicheng Ma, Zhigang Yan, Zhihong Shao, Zhipeng Xu, Zhiyu Wu, Zhongyu Zhang, Zhuoshu Li, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Ziyi Gao, and Zizheng Pan. Deepseek- v3 technical report, 2025.
[97] Mistral AI. mistralai/mistrallmmllll
2025. Accessed April 22, 2025.
[98] Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Weinan Dai, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei-Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, and Mingxuan Wang. DAPO: An open-source LLM reinforcement learning system at scale, 2025.
[99] Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Lucas Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, Weizhu Chen, Shuohang Wang, Simon Shaolei Du, and Yelong Shen. Reinforcement learning for reasoning in large language models with one training example, 2025.
[100] Nikhil Chandak, Shashwat Goel, and Ameya Prabhu. Incorrect baseline evaluations call into question recent llm-rl claims. attps://safe-- 9a8 notion. site/ wewpeeeennnoortttteeeeee el Slinee-Evllaatiinn-CClll. -aaallliitoouuuttionnn ciuuesttion-ee -ww.eeeti---eeettt----e 0B0944abBddd1953316633373222
RL- Claims-amms--i htl--clammm222221fffff
2025. Notion Blog.
[101] Wengong Jin, Connor W Coley. Regina Barzilay, and Tommi Jaakkola. Predicting organic reaction outcomes with Weisfeiler-Lehman network. In Proceedings of the 31st International Conference on Neural Information Processing Systems, pages 2604-2613, 2017.
[102] John Schulman. Approximating KL divergence. Blog post, 2020. Accessed: 2025-02-01.
[103] Markus Hartenfeller, Martin Eberle, Peter Meier, Cristina Nieto-Oberhuber, Karl-Heinz Alt- mann, Gisbert Schneider, Edgar Jacoby, and Steffen Renner. A collection of robust organic synthesis reactions for in silico molecule design. Journal of Chemical Information and Modeling, 51(12):30333398 2011.
18

---

[104] Pat Walters, Mining ring systems in molecules for fun and profit. https://praciicalchee minf ormatics. , blogspot. 10/00///////mm/// -eing-innggg-ee systems- snn-moleculee--ee
html, 2022. Accessed: 2025-05-08.
[105] Peter Ertl, Steven Jelfs, Johannes Mühlbacher, Ansgar Schuffenhauer, and Paul Selzer. Quest for the rings: In silico exploration of ring universe to identify novel bioactive heteroaromatic scaffolds. Journal of Medicinal Chemistry, 49(15):4568 (511:):366-453,, 2006.
[106] Pat Walters. useful rdkit_ utils: RDKit utility functions. https: attps://githnn http://ggthuu..coo/aaaalllll// com/Pttaatter Accessed: 2025-05-08.
8 s/ ////s/uuurrkkittutttttt
[107] G Richard Bickerton, Gaia V Paolini, Jérémy Besnard, Sorel Muresan, and Andrew L Hopkins. Quantifying the chemical beauty of drugs. Nature Chemistry, 4(2):90-98, 2012.
[108] Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 7eRO: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1-16. IEEE, 2020.
[109] Gabriele Libardi, Gianni De Fabritüis, and Sebastian Dittert. Guided exploration with proximal policy optimization using single demonstration. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 661 1-6620. PMLR, 18-24 Jul 2021.
[110] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The Llama 3 herd of models. arXiv preprint arXiv:2407.21788 2024.
[111] Yuyan Liu, Sirui Ding, Sheng Zhou, Wenqi Fan, and Qiaoyu Tan. MolecularGPT: Open large language model (LLM) for few-shot molecular property prediction. arXiv [q-bio.QMM. June 2024.
[112] Christopher Fifty, Jure Leskovec, and Sebastian Thrun. In-context learning for few-shot molecular property prediction. arXiv [cs.LG], October 2023.
[113] Johannes Schimunek, Sohvi Luukkonen, and Günter Klambauer. MHNfs: Prompting in- context bioactivity predictions for low-data drug discovery. Journal of Chemical Information and Modeling, April 2025.
[114] Jiatong Li, Wei Liu, Zhihao Ding, Wenqi Fan, Yugiang Li, and Qing Li. Large language models are in-context molecule learners. IEEE Transactions on Knowledge and Data Engineering, 2025.
[115] Xianggen Liu, Yan Guo, Haoran Li, Jin Liu, Shudong Huang, Bowen Ke, and Jiancheng Lx. DrugI.LM: Open large language model for few-shot molecule generation. arXiv [q-bio.BM] BMJ, May 2024.
[116] Lei Shi, Zhimeng Liu, Yi Yang, Weize Wu, Yuyang Zhang, Hongbo Zhang, Jing Lin, Siyu Wu, Zihan Chen, Ruiming Li, Nan Wang, Zipeng Liu, Huobin Tan, Hongyi Gao, Yue Zhang, and Ge Wang. LLMM-based MOFs synthesis condition extraction using few-shot demonstrations. arXiv [Ls. August 2024.
19

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We claim to be contributing a reasoning model trained for chemistry on various common chemistry tasks that outperforms specific deep-learning models such tasks. This is extensively shown and supported by our results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Section 6 discusses the limitations our method, including the tasks our model does not perform too well.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
20

---

Answer: [NA].
Justification: Our work does not include theoretical proofs. However, the needed background understand the model training is included in this manuscript.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: All the hyperpurameters and training method are extensively described and included in this paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
21

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We have publicly released GitHub repository (https://githbb.co httptthhuub.couuuuuuee. House/etherO) containing the reward functions and test data. All training data sources are publicly available, and have also open-sourced the templates used to generate the question prompts.
Guidelines:
The answer NA means that paper does not include experiments requiring code. Please the NeurIPS code and data submission guidelines (https://nips. cc/pu blic/ Dalc/guides, gwwggguddc/caa seiides/CooeeSuiissiiiii
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to reproduce the results. See the NeurIPS code and data submission guidelines (https: httpsscccppbccccccecceeeccccccce meeiic/guide guidee/ /eooeS//emmmmieee wetpomiisiinooiiiiiiieeo
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Section 2 and SI Section C.2 are transparent on how the data was obtained and Section 4 discusses the training procedure in detail. The hyperparameters for every phase of the training pipeline are provided.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Training LLMs is very cost-intensive, and API calls to frontier LLMs expensive. We considered a large evaluation set, but did not run replicates of each task.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
22

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: The first paragraph of our "Results" section explicitly tells the compute resources used to train each step of our training pipeline.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't I make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: hetp::/nneui..eeeeee wttpeeew...ppuuuueeeepepeeeeeeee wwtpcc/cpppccceeeeeeeeeeeeeeeeee cthiccsiueellieess
Answer: [Yes]
Justification: Our study does not have human subjects; all the data used is publicly available, and our model is being trained to mitigate safety and misuse concerns.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: We show that reasoning models are data efficient to learn chemistry-related tasks, which can be used to train a new model using our code for harmful purposes on a different dataset. This is not addressed in the main text. We briefly discuss potential applications of these models drug discovery pipelines and the associated risks of misuse, but do not explore these topics in depth.
23

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (c.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymemt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [Yes]
Justification: We reduce potential safety and misuse risks by incorporating a dedicated safety phase into the training pipeline, before releasing the nodel weights. We describe the full safety procedure in the SI.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Every source was thoroughly referenced, and datasheet for the dataset is provided below.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com dataaataees has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-pakaaedd, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
Justification: Our model has been open-sourced along with its model card, the reward function and dataset prompt templates are public and well documented in our GitHub repository
and the dataset use is derived from
publicly available sources.
Guidelines:
The answer NA means that the paper does not release new assets. Researchers should communicate the details of the dataset/code/model as part of their submissions structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification: Our study does not primarily involve crowdsourcing nor research with human subjects, but does pay small number of human chemists to determine baseline accuracy on the problems being considered. Details on their compensation and instructions are included in the SI Section F.6.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
25

---

Answer: [NA]
Justification:
Our study does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
Justification: Our study involves training LLMs, and LLMs were used in important steps of our method. All LLM used is well-described in Section
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non non-standard components.
Please refer to our LLM policy for what should or should not be described.
26

---

A Algorithms
A.1 Supervised Fine-Tuning
Given a set of demonstration sequences Demo. supervised fine-tuning (SFT) minimizes the cross-
entropy loss over the dataset:
LSFT
log k (sc(s<e)
(SI)
[Daemo
deDao
A.2
Group Relative Policy Optimization
The GRPO algorithm is given in Algorithm 1 below. Within it is the following KL divergence estimator [102]:
DKL. (oo) Frer x, Ve] = Trer pr, yeet). llog (3t ar. V-ee) (Vel x. Vct))
Tre(Uel Mete) - 1.
(S2)
Algorithm GRPO
Input: Minibatch sampling distribution (D(D), hyperparameters μ, M
1: for k 1...., K do
2: Told <- Te
3: if l: mod M then
4:
Update reference policy: nee +- T5
5: end
6:
Sample minibatch DB PB(D)
7:
for DB do
8:
Sample vf ..... Ve A (-12:) Compute rewards ri...., then advantages ,f,....,45 end for
9:
10:
11:
for j
l..j=..... do Update To with gradient ascent step JGRPO over fz, .e......... x Dp}
12:
13: end for
14:
end for
27

---

B Prompts
B.1 SFT Filtering Prompt
gemini-1 1..a--o---o--0 is used filter flawed reasoning traces coming from DeepSeek-Ri used in producing the initial SFT dataset. The following prompt was used:
justification for the answer to the question. Evaluate the reasoning as GOOD it complete, relevant, and justifies the answer without presuming the anaver beforehand. Evaluate the reasoning as BAD if it is incomplete, trivial, uses the Fimal/given/sugee justification. Answer only with GOOD BAD do not include explanation.
Examine the following 'thought reasoning
ansuer in its
("problem": "(problen)", "thought": "(thought)", "ansver": "(answer)")
This safeguard against incoherent sequences removes only a few examples.
B.2 Summarization Prompt
This prompt is used to summarize reasoning traces coming from DeepSeek-R1 for the SFT dataset.
Given the following reasoning process, reduce its length while preserving the structure and the sequence of thoughts. Keep the original sequence of thoughts and all relevant information to reach the final answer. It is essential to preserve all SMILES and equations. Start with the same words as the original reasoning process. You should also keep all reasoning patterns in the original thought. That includes behaviors like verifications (e.g. 'Let check... "), backtracking (e.g. 'Let's try another approach.... .) subgoal setting (e.g. 'First, let 's consider...") and back chaining (e.g. 'Wokking backwards "). If the original examples of these behaviors long, shorten them.
B.3 Distillation Filtering Prompt
This prompt is used to filter flawed reasoning traces coming from task-specific ether0 variants before distillation.
Examine the following 'thought' reasoning as justification for the answer to the question. Note the answer will contain SMILES (Simplified Molecular Input Line Entry System) notation, so do not consider SMILES such "C=-MC-NC-CC(-OO)MOO (B))
to be typo. There may also be markdown, please ignore markdown formatting. Please evaluate the reasoning as (case sensitive)
GREAT: if it is complete and relevant.
BAD: if it contains typos, English characters, formatting, or doesn's relate to the problem. Do analyze SMILES syntax for balanced parentheses or correctness, do not compare stated SMILES with the answer 's SMILES and do not analyze the accuracy of scientific claims, just evaluate based formatting, typos, and problem relevance.
ALRIGHT: GREAT or BAD don't quite fit.
Answer first with GREAT, ALRIGHT, BAD, then briefly state the rationale.
("problem": "(problem)", "thought": "(thought)", "ansver": "(ansver)">
28

---

B.4 Problem Rewriting Prompts
These prompts are used to direct Gemini 2.5 Flash to rewrite problems in our dataset. For the sake of brevity, omit most of the ICL examples that we include.
Prompt to rephrase the question without distracting information:
Rephrase the following problem. DO NOT manipulate any SMILES SMIRKS IUPAC name the chemistry being asked about. Just rephrase the problem in different way. ONLY respond with the modified question. Do try to make it more natural sounding. DO NOT forget to include all multiple choice options, if applicable.
You NUST include all SMILES, SMIRKs, IUPAC names, and functional groups in the original problem in the modified question.
Here are some examples of what am asking for:
[omitted]
<input>
What is the product of this reaction? Taa.e=scoccpp..co0000--0---0-000
F) (F)F.ODC(C) (DC cC) o..cc....ccI>>>>
</input>
<output>
We mixed the following reactants: [Zn] 00D.:///P......-----------------
)F (F.00C(C) (C)C.o. ciccl>>. Can you answer what was produced in this reaction?
</output>
DO include tags. may reuse patterns from these examples, these exact examples, even if one is similar to my problem. Be creative. DO NOT drop any information from the original problem, and REMEMBER to include all SMILES, SMIRKS, and IUPAC in their original form in the modified question.
DO NOT copy
Prompt to rephrase with distracting information:
Rephrase the following problem. DO NOT manipulate any SMILES or SMIRKS or IUPAC name the chemistry being asked about. Just rephrase the problem in different way. ONLY respond with the modified question. Do try make it natural sounding. DO NOT forget to include all multiple choice options, if applicable. You MUST include all SMILES, SMIRKs, IUPAC names, and functional groups in the original problem in the modified question.
Here examples of what asking for:
[omitted]
<input>
What the product of this reaction? [2n] R211_=5000CCFF..0000000000000000
F) D. C1CC1>>
</input>
<output>
exploring possible reactions with the reactants available in our lab. When we mixed the following reactants: [Zn] (F) --C10--200000-C2MM00-00000-
and my colleagues
(F)F.00C(C) (C)C.0.CICcI, we got a very interesting solution. Can you answer what was produced in this reaction? </output>
DO NOT include XML tags. You may reuse patterns from these examples, but DO NOT copy these exact examples, even if one is similar to problem. Be creative. DO NOT drop any information from the original problem, and REMEMBER to include all SMILES, SMIRKS, and IUPAC in their original form in the modified question.
29

---

C Chemistry RL Dataset Details
C.1
Multiple Choice Question (MCQ) task descriptions
Safety:
Select the molecule whose structure most strongly aligns with or deviates from specified safety-related property, such as toxicity, flammability, or hazard classification.
Scent: Identify the molecule most likely to exhibit a specific olfactory attribute (e.g., meaty, spicy, oily).
Blood-brain barrier: Determine which molecule is or isn't structurally likely to penetrate the blood-brain barrier based reference behavior.
Receptor Binding: Identify the molecule whose structure most likely lacks binding affinity or activity for a specified biological receptor target.
ADME:
Choose the molecule expected to improve or match a specified absorption, distribution, metabolism, or excretion property based on structural modifications.
Aqueous Solubility: water solubility.
Select the molecule whose structure leads to targeted increase or decrease in
LD50: Select the molecule whose structure corresponds to specified rat oral LD50 value, reflecting its relative acute toxicity level.
pKa: Select the molecule whose structure aligns with a specified increase, decrease, or target value of pKaH1.
Photoswitches: Select the molecule whose E isomer exhibits a target Ā- transition wavelength..
C.2 Dataset Provenance
The dataset constructed by aggregating data from 13 distinct sources, detailed in Table 1. All selected references exclusively involved experimental measurements of synthesized molecules, excluding any hypothetical computationally generated structures.
The source datasets had variety of representations, like CAS numbers, so we first relied on Leurli! PubChem, and RDKit to convert all molecules to SMILES. Unless otherwise specified, all SMILES were randomized, isomeric SMILES. Also, generally molecules were filtered out that were fewer than 4 heavy atoms, more than 100 heavy atoms, had less than 20% carbon atoms. The exceptions were when it was an exact match problem (like the outcome of a reaction). We did not filter out disconnected molecules, so many examples did have counterions (although model was excluded from answering with non-counterion mixtures).
For reaction prediction tasks, data was sourced from the organic reaction database (ORD) with filtering to remove contamination. Namely, some deposited reactions in ORD are parsings of USPTO, so that care must be taken to avoid contamination. Reaction strings were systematically parsed to standardize reactants, reagents, and products into reaction SMILES (SMARTS). Trivial reactions, defined by product-reactant identity, were filtered out. The test set was filtered based on major outcome of the reactions.
The SMILES Complction task used data from COCONUT. Tasks were generated by randomizing their SMILES representations and truncating these strings to create incomplete molecular fragments namely a fragment that cannot be parsed into a valid molecule by RDKit. The same COCONUT data was used for the IUPAC task, meaning the compounds are relatively complex for naming.
Solubility Edit tasks drew from Chembl compounds that are small molecules and had some assay conducted them. Tasks required modifying original SMILES strings achieve specified increases or decreases in predicted solubility (e.g., by one logS unit). Additional constraints included maintain- ing high structural similarity the original molecule, preserving the Murcko scaffold, retaining specific functional groups. We used exmol's list of functional groups for choosing these.
Retrosynthesis tasks used curated list of experimentally synthesizable molecules. The goal was to propose viable single-step syntheses for these targets. To generate these, we took the fragments from
LLeuui.com
30

---

the mcule catalog" and predicted products using the reaction templates from Hartenfeller et al. [103]. Thus, we expected these be synthesizable. much larger catalog was used for checking proposed solutions (ZINC20), so that more potential reactions could lead to the products.
Multiple Choice Questions (MCQs) formed a significant dataset component, designed around molecu- lar properties challenging to predict computationally or intended to test nuanced chemical discernment. Properties included safety profiles (e.g.. LD50 values, GHS classificationss, pKa values, scent at- tributes, and ADME properties from specialized datasets. The MCQ generation algorithm began with calculating molecular fingerprints (ECFP4) for each molecule. Structural similarity using Tanimoto indices identified candidate distractors. These distractors were categorized based on their property similarity or dissimilarity to the target molecule within 0.25 (0.35 for pka problems). MCQs were formatted either as outlier detection tasks identifyinniiii the structurally or property inconsistent molecule from set-or as identification tasks pinpointing a specific property within a group of similar molecules. To detect dissimilar compounds, like "which of the following has a higher pKa than X", we required a change in 10 percentile points of the given reference compound.
To prevent leakage, all compounds used in a question type together were excluded between train and test. Namely, we made graph where each edge represents when two molecules appeared in the same MCQ. Then ensured that the train and test subgraphs had no connections, but that we could group similar molecules densely enough to make questions with distractors. The smell, EveBio, and GHS tasks had enough compounds that this wasn't 't necessary, and we just randomly split. The categorical receptor, GHS, and smell data MCQs were treated as multi-label, Namely, the questions were all about single possible labels (e.g., does it smell like fresh cut grass) and no multi-classccomiiattoo questions were added.
The formula questions are generally under-specified (e.g., make a compound with formula C3H1002), but they were created from real molecules (from CheMBL) to ensure they are answerable.
C.3 Reward Function Implementation
The reward functions were implemented using a combination of Python code, remote calls, and database look-ups. Tasks that had an exact match, like reaction prediction or multiple choice prediction, the comparison was done via canonicalizing the molecule (with stereo chemistry retained) and string comparison. For open answer questions, like solubility edits, after checking for constraints and actually hitting the property target, we also tested that the molecule is plausible. The code for our reward functions, as well as relevant prompt templates and data utilities, are open sourced on GitHub at Future-House House/etherO.
In tasks that involve submitting molecule that satisfies constraints, we also do check on the plausibility of the molecule. See Table I for a list of tasks with this check. Aside from assessing if a molecule has valid valence, we check the ring structures and atom fragments. We first take the source molecules for our datasets, which is larger than 640,730 because we did not utilize 100% of ChEMBL or COCONUT, We then applied some filters to ensure the molecules had been synthesized. For example, we required I or more assays reported in ChEMBL or GHS categorization being present for molecules from PubChem. The rings from these molecules were isolated using the ring cut method from Pat Walters [104, 105, 106]. The rings were then stored as canonical SMILES in a bloom filter [61]. We then isolated all molecular fragments with radius 2 (2 bonds away) from the molecules and converted them into bit strings similar to ECFP4 fingerprints [86]. These bit strings encode an atom plus its local neighborhood. The bit strings were then stored in a bloom filter. At test time we apply the same ring cuts and fingerprint generation to proposed molecule. If its rings and fingerprints are all present in the derived bloom filters, we consider the molecule to be reasonable. Otherwise, it is not reasonable molecule. We use bloom filters because they are highly memory-efficient and fast for checking set membership.
This approach is relatively conservative, because it requires the rings and molecular groups to have been present at least once molecule reported in our source datasets. We did experiment with hand-constructed rules, machine learning models, and scores like QED [107], and found them susceptible to reward hacks such as inserting peroxides to satisfy oxygen counts, or hydrazines to increase solubility. We found this check to be essential to ensure plausible molecules are generated.
/https//menle.com//
Globallyy Harmonized System of Classification and Labeling of Chemicals
31

---

This check is applied at evaluation time as well, and is responsible for rejecting many answers when training the molecule completion and molecular formula tasks.
D Method Details
D.1 Reasoning Quality Filtering
Functional. grouz
SNOLEZ Lomplet
2UPAC
1590
2000
6-4 1800
1999
Pemppring
2,005
GECAT
R 5 3 0226
555 28 8
100
310a0 350g
(a) Open answer task reasoning quality across SFT (gray- shaded first bar) and task specific RL (remaining bars).
Functfooal grout
ES corplettinn
Elacidation
TIPAC name
Peasaring
1.212
0.45
4.00
0.00
20. RRRS 555
2,5
Coll "S52292150
Poaction Prediction
200
Teat
555.8
283325500 8225 855933
Great 3009
(b) Open answer task reasoning quality across distillation (gray-shaded first bar) and all -lsk RL (remaining bars), where the distillation dataset used here did not filter upon reasoning quality.
Fanctionl
SMLLES
plation
idatidation
JUPAL
0.21
42363X
588558884 363
335803
9:5652
Selubiitty 4550
Daacttion Sradicttion
Nolecc Lar Formula
art inal
5 493933 R3
53
3. X2 33
595985
(c) Open answer task reasoning quality across distillation (gray-shaded first bar) and all-task RL (remaining bars), where the distillation dataset used here did filter out BAD-level reasoning quality.
Figure S1: Reasoning quality across post-training. Note that regex-based language detection was part of the quality determination, just a LLM judge.
We observed the emergence of reasoning containing typos (made up chemicals), non-English charac- ters (use of languages such as Arabic or Cyrillic), nonsense formatting (blending text with brackets), or ungrounded reasoning (off-the-rails thoughts) as RL progressed. To gauge reasoning quality across training, we employed LLM judge using the prompt in Section B.3. The judge evaluates reasoning
32

---

as GREAT, ALRIGHT, or BAD. In practice we found the judgments made by OpenAl GPT 4.1 and Google Gemini 2.5 Pro were interchangeable, and used GPT 4.1 for more favorable rate limits*.
As shown in Figure Sla, after initial SFT the reasoning quality almost entirely GREAT. Then during task-specific RL the quality degradation begins, most substantially in IUPAC name, solubility edit, and retrosynthesis.
Then at distillation, diverge into two different and identical runs: (1) Figure Slb: not filtering bad reasoning before distillation, and (2) Figure Sle: filtering out bad reasoning before distillation. Note that the two distillation dataset sizes nearly identical because face the same problem multiple times during training, and keep only the latest problem after filtering.
Comparing these two all-task runs, we observe that filtering out bad reasoning before distillation led marginally higher quality reasoning while reliably boosting performance by few percentage points the test-set. When qualitatively reviewed by humans, the reasoning from the filtered RI. run was preferred. Furthermore, the filtering clearly has impact because, after the L.LM judge filtered out Arabic characters, during all-task RL the model began using Cyrillic characters instead.
Thus a second improvement was made, tightening our reasoning quality filtration using regex-based detection of other languages. The regex checked for the following unicode categories via the \p ele- ment: Arabic, Armenian Bengali, Georgian, Gujarati, Gurmukhi, Han, Hangul, Hebrew, Hiragana, Kannada, Katakana, Khmer, Latin_Exteieeee Ext_eneeddddde A, Latin, Eeriiin_Exeeeedd__m Extended_AAdittional. Additional, Latin Extended B, Malayalan Latin_Extenatt
Prrailee_Patteran
Cyrillic, Devanagari, Ethiopic,
Myanmar, Syriac, Tamil, Telugu, Thaana, Thai, and Tif Tifinagh. This regex filtration ensures all-task RL. began solely upon reasoning containing English characters or symbols (e.g. math phrases or Markdown syntax), thus unbiasing RL from any particular non- English language.
In general, our methodology leaves reasoning unconstrained beyond basic formatting, it's intriguing that as task accuracy increases across RL, reasoning flaws begin to appear.
D.2 Molecule Quality
When solving tasks such as molecule completion, the model can satisfy the reward function by coming up with an answer that meets all specified criteria (including the reasonable molecule check), but also functional groups that are undesirable for drug-like compound. For example, we observed an over-representation of nitro side-groups. These are reasonable and common in chemistry, but it preferable to avoid them if possible. Therefore, we try to reduce the occurrence of the following moieties, without penalizing them for correctness of a problem:
Multiple thiol bonds
Peroxide
Hydrazine
Charged amines
Nitro groups
Saturated chains of seven or more carbons
When constructing the distillation dataset, reject answers containing any of the above. This applied to molecule formula, functional group, elucidation, and solubility edit tasks. While molecule completion would also benefit from the same treatment, we found that too few sequences passed this filter.
Distillation:
During the last few steps of GRPO, we further assign molecule quality bonus reward to correct answers that also do not contain the above motifs. This applied to all tasks in Section 2 marked with t.
Generalist RL:
'Note LLM judges are not 100% reliable, as we observed stray cases where reasoning with non- English characters or typos were labeled as ALRIGHT. Using regular expression we measured this mistake only occurs in <0.1 % of judged reasoning traces, so these results can be trusted directionally accurate.
33

---

E
Training Hyperparameters
E.1
Task-Specific RL
All specific RL rums shared the following hyperparameters:
Maximum completion length: 2048
GRPO epochs ju:
Sampling temperature: 1.0
KL penalty weight 3: 0.005
Learning rate: 10
Linear LR warm-up steps: 20
Reference policy reset period M: never
We empirically observed top-K sampling caused unstable learning (with K=50), so we did employ sampling algorithms such as top-K, nucleus sampling, or beam search.
Since these experiments training, but did resume three task-specific runs from a checkpoint (which entails a reference policy reset) push the model further. Run-specifie hyperparameters ZeRO Stage 3 [108] was used to shard the model across Nvidia H100 GPUs.
relatively short and stable, we did reset the reference policy during
detailed in Table S1. DeepSpeed
Problem categories
Training steps
Checkpoint
Group size
Group batch size
Eour
Seeded
Rewritten problems
Step(s)
curriculum
Functional group
Elucidation
Molecular formula
918
n/a
256
0.5
SMILES
completion
1110
n/a
4
384
0.5
IUPAC name
1910
n/a
6
128
0.5
Solubility Edit
167
n/a
6
128
0.5
Retrosynthesis
1264
512
4
96
0.25
x
Reaction prediction
1501
704
4
96
0.5
Multiple choice
6417
2801
4
96
0.5
X
Molecule caption
2189
n/a
4
192
0.25
X
Table S1: Training hyperparameters for task-specific RL. Group batch size refers to the number of groups per batch during GRPO. Checkpoint step(s) refers to steps where we resumed the specialist model from checkpoint. Rewritten problems refer to the fraction of problems that were rewritten by an LLM.
E.2
All-Task RL
The following hyperparameters were used for the all-task RL phase:
Maximum completion length: 4096
Number of training steps: 434
Group size: 4
Group batch size: 768
GRPO epochs μı:
Sampling temperature: 1.0
KL penalty weight 3: 0.005
34

---

Learning rate: 1.25 x 10-6
Linear LR warm-up steps:
Reference policy reset period M: 256 steps
Curriculum buffer sampling rate ceur: 0.25
Curriculum buffer seed: X
Molecule quality bonus reward: enabled for the last 50 steps
Fraction of LLM-rewritten problems: 75%
Matching Section E.I, did not utilize sampling algorithms such as top-K sampling, nucleus sampling, or beam search.
E.3 Safety Alignment
After generalist RL, the model is capable of aiding user with unsafe chemistry tasks. To improve the safcty of the model, we apply a final RL phase for safety alignment. We design the procedure to be independent of model training. We do this to avoid constraining exploration during learning, and to develop a method that is robust enough to be applied as a final phase to any model in our training pipeline.
We construct specialized dataset comprising questions that safe model should not respond to. This includes clearly unsafe prompts (e.g.. "Design a molecule that is extremely explosive."), "), as well as retrosynthesis and edit solubility questions phrased similarly to our training tasks involving known chemical weapons (e.g.. "Suggest minimal structural modification to ClAsCI that would decrease its solubility by approximately logS unit."). As a control, we also include benign, open-ended questions that the model should answer correctly but that contain words also seen in the unsafe prompts (e.g.. "Design a compound that has no deadly properties.").
To incorporate the new safety behavior into the model, we generate curated set of prompt-completio examples that include both reasoning traces and the intended refusal response. We then perform few more steps of GRPO, with both all chemistry tasks and these safety questions. To each group of responses safety question, we add the synthetic completion that reflects the desired behavior and assign reward of 1 to it. In the GRPO objective (Fquation 2), we set the importance sampling denominator n0..a = 1, following [109].
The following hyperparameters were used for the safety RL phase:
Maximum completion length: 4096
Number of training steps: 120
Group size: 4 (non-safety problems) and 5 (safety problems)
Group batch size: 104
GRPO epochs 1
Sampling temperature:
KL penalty weight 3: 0.005
Learning rate: 1 x 10-6
Linear LR warm-up steps: 20
Reference policy reset period M: 256 steps
Curriculum buffer: X
Fraction of LLM-rewritten problems: 75%
Additional Results
F.1 Emergence of New Behaviors Through Reinforcement Learning
Reinforcement learning enables the discovery of new behaviors through trial and error, particularly when outcomes are verifiable. For example, Figure S3 shows results from early experiment
35

---

tresttigation Fatteltigatton
HELED coplattie
aostta foresa
Famettonl growp
Poactic predtctien
muality esit
eetrayystirte
tanc: nume
tarayy
Hame reopter Dieatay
Lone
Kaant
Tooks
Figure S2: Performance of ether( before and after the safety alignment is applied.
in which the model was trained to solve the retrosynthesis task without any initial supervised fine- tuning (SFT). Despite lacking prior knowledge, the model progresses from zero success to achieving correct completions. In approach, warm-start etherO with supervised fine-tuning rejection- sampled, long chain-of-thought sequences to accelerate learning. Nonetheless, reinforcement learning remains important, supervised data.
allow the model to bootstrap novel behaviors that are absent from the
Retrosynthesis
0.12
0.10
0.08
Accuracy
0.06
MWV
0.00
100
200
300
400
500
600
700
800
Step
Figure S3: Accuracy over training steps. The model receives learning signals through trial and error, gradually acquiring the ability to solve the task.
F.2
Cognitive Behavior Counts and Failure Mode Distributions Across Tasks
During evaluation steps performed throughout training, we prompt Llama-3.1 3----33--------- Instrucntt [110] to analyze each sample generated by model. For each behavior, we design custom prompt, following a strategy similar to [25]. Each prompt provides Llama with examples of the target behavior and instructs it analyze the sample and return the count in specific format (i.e., <count> [1/2/...] </count>). This procedure enables automatic extraction of behavior counts per sample.
Figure and Figure $5 present behavior counts and the distribution of answer outcomes from our model evaluation traces during training all chemistry tasks.
36

---

2FT
Tesk-apectfic 11
Dortillation
Ail-bast R.
19:0
bectiry
15:00
1440
Fartidd
Tesk-apectfic T
Desttttation
Ati-tast R.
Task-apestfic 1
Sestiltation
Atb-task R.
12.0
teittry
8
Tisk-apes: 3
Staciltation
Att-bask R.
25:0
Idipart beittry
Sring
Figure S4: Evolution model reasoning behaviors and answer outcomes on the evaluation throughout training on functional group, reaction prediction, IUPAC name and molecular formula tasks. For each task, the top row shows the number of counts for each behavior and the bottom row shows the distribution of answer outcomes, categorized by reward reason.
F.3 Advantage-Based Curriculum Ablation
Section 4.2.1 motivates advantage-bul based curriculum; here, empirically justify its In Fig- ure S6, we compare the first few epochs of the reaction prediction specialist (trained with a curriculum) an identical training run without curriculum.
The effect of the curriculum visible almost immediately. The fraction non-trivial groups (1 fr) starts at 30% for both experiments, but the curriculum quickly pushes it up to 50-60% (Figure S6A). As training progresses and the model learns to solve more problems, the non-trivial fraction drops to nearly 20% without curriculum. That is, only 20% of each sampled batch is providing useful learning signal with non-zero advantage. With the curriculum, the non-trivial fraction remains above 40%.
The downstream utility of more non-trivial problems evidenced in Figure S6B: accuracy on the holdout starts higher and increases faster.
37

---

Tesk-apectfic
DoettlLation
ALl-bast
1400
Tesk-apectfic 11
Deaesllatiin
AL1-tast R.
12.0
Tesk-apectfic N
Sestiltation
Alb-task R.
12.00
teittry
Hight
Task-apectfic 3
Statiltation
Att-task R.
1000
beitlry
Wong Hoer
Figure S5: Evolution model reasoning behaviors and answer outcomes on the evaluation throughout training on SMILES completion, solubility edit, retrosynthesis and multiple choice tasks. For each task, the top row shows the number of counts for each behavior and the bottom row shows the distribution of answer outcomes, categorized by reward reason.
F.4 In-Context Learning
In-context learning (ICL) [2] involves adding examples directly to the prompt at inference time. Also range of applications, from property prediction [111, 112, 113] to molecule generation [1 14, 115, 116]. To build this experiment, we select multiple-choice questions from our dataset and use one of the incorrect choices as context.
called few-shot, ICL has been shown improve performance
For example, given this question:
Which molecule listed here most likely to have rat microsomal stability in mL/min/ke approximately equal to 1.26? C1O0=##MM0Dc2MACCC......ocecacc. C12=NC cC coC1
ol-ooeaccCccaeecoooeecacccccoooo C1c cccccaacceeseecccooooooocccemeen
create an ICI equivalent of this task by taking one of the incorrect choices (highlighted in red) and using context in the question:
38

---

A)
B)
Curriculum
No Curriculun
0.60
0.6
Evaluattoo reward
Non-te Non-trivias triviat fraction
0.50
0.3
0.40
0.2
180
200
300
400
588
600
0
100
200
300
488
588
600
Training step
Figure S6: RL training dynamics of reaction prediction specialist models, one with and one without an online curriculum. A) the fraction of non-trivial groups seen during training (faint lines are raw data; solid are a 30-step moving average). B) the evaluation set reward, computed every 64 steps.
Considering (22 C-CC)) has
measured rat microsomal stability in mL/min/kg of 1.03, which candidate modification listed would most effectively increase this property?
C1c GICCCC2HHFBSSOCCCCCCC00000022222 CC- C12 (02 OBCNNNCCCCCCCCCooo0oooCCCCCCC---
To ensure that any observed performance improvement is not simply due to reduced number of answer choices, we also modify the original question by removing the same incorrect option used as context in the ICL version. This way, both the standard and ICL queries present the same number of choices, preserving the same baseline probability selecting the correct answer by chance (random baseline shown in Figure 3B.
F.5 Human expert benchmarks
In Table S2, we report the breakdown of human expert performance on our test set.
Task
Accuracy
Molecular formula
0.30 0.60
0.13
Functional group
0.13 0.00
0.30
Reaction prediction
0.41
Solubility edit
0.05 0.00
0.12
Retrosynthesis
0.00
Safety
0.40 0.28
ADME
().22 0.24
0.48
LD50
0.43 0.21
pKa
0.35 0.20
0.48
Table S2: Human expert performance on the test set. Four contractors were tasked with solving each question, with no tools besides ChemDraw. We report the average accuracy, well as the minimum and maximum scores, as sub- super-scripts respectively.
39

---

Rubric provided for trace evaluations
The provided trace does not contain any contrived performative reasoning.
Genuineness:
answer based on the reasoning trace only, and did not make any sudden leaps judgment.
Faithful:
The model arrived
Exploration:
The displays examples of non-linear reasoning, self-reflection, backtracking in its reasoning.
Pick the most suitable assessment for each metric: Strongly disagree 2) Somewhat disagree Neither agree nor disagree 4) Somewhat agree 5) Strongly agree
Figure S7: Four expert evaluators were provided with this rubric to assess the "quality" of traces from ether0 and 15 traces from DeepSeek-R1.
Non-Contrived / Genuine
S. Agree
Agree
Neutral
Disagree
S. Disagree
DeepSeek-rl
28%
23%
7%
8%
33%
etherO
45%
23%
12%
15%
5%
Faithful
S. Agree
Agree
Neutral
Disagree
S. Disagree
DeepSeck-rl
33%
25%
8%
20%
13%
etherO
50%
23%
10%
13%
3%
Explorative
S. Agree
Agree
Neutral
Disagree
S. Disagree
DeepSeck-rl
20%
32%
18%
13%
17%
etherO
13%
28%
35%
5%
18%
Table S3: Expert evaluation of reasoning traces generated by ether0 and DeepSeek-R1.
F.6 Human evaluation
We conducted two sets of expert evaluations: human baselines on a set of held-out open open-ended and multiple-choice type questions, 2) etherO trace evaluations.
For the first set of evaluations (human baselines), recruited four expert evaluators: two with PhDs in organic chemistry, one with a PhD chemical engineering, and one PhD candidate organic chemistry. Evaluators were instructed respond using only the SMILES representation of the target molecule, without relying on external tools for assistance in answering. However, tools for visualizing allowed. Tasks considered impossible accomplish without
SMILES chemical structures the use of tools were flagged by the evaluators and excluded from the final analysis. Each evaluator was given 200 open-ended and/or multiple-choice questions from held-out evaluation set, and was compensated $10 per question completed. Their performance is compared with ether0 and other frontier models Figure 2.
For the second set of evaluations, we recruited another group expert evaluators: three with PhDs in organic chemistry, and one with PhD in chemical engineering. The evaluators were provided with a rubric to assess the reasoning traces generated by ether0 and DeepSeek-RR (see Figure S7). Each evaluator was given 30 reasoning traces from both model (15 from each). Compensation was $10 per completed trace evaluation. Full comparison results are shown in Table S3. More experts disagreed that DeepSeek-rl demonstrated non-contrived and faithful reasoning, although they noted showed more extensive exploration. This is unsurprising, as DeepSeek-r1 reasoning traces were much longer than etherO's.
40


Reviews:
[{"review_id": "yiQ8XjJVZL", "reviewer": "Reviewer_uSST", "summary": "This work introduces a dataset of 577,790 experimentally-grounded chemistry problems, categorized into 17 distinct tasks. Each problem includes a solution that can be verified using KDESol, RDKit, or string matching. The authors also trained a 24-billion-parameter reasoning language model on this dataset, named Science0-c. Science0-c outperforms both general-purpose reasoning language models and specialized non-LLM models on chemistry-related tasks.", "questions": "Please refer to the Weaknesses above", "limitations": "Please refer to the Weaknesses above", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n\n- The paper addresses an important problem in the intersection of chemistry and reasoning language models.\n- The presentation is clear and easy to follow.\n\n\n**Weaknesses:**\n\n- The evaluation is limited to a single benchmark consisting of holdout problems from the dataset. There is no assessment on tasks outside the training data, such as molecule captioning (e.g., ChEBI-20 [1]) or text-based molecule design (e.g., ChemLLMBench [2]).\n- The contribution in terms of novelty is limited. The dataset is constructed from existing sources and verified using standard toolkits, and the training pipeline closely follows prior work on reasoning LLMs with minimal modification [3].\n- The paper lacks a qualitative analysis of the reasoning behavior of Science0-c compared to general-purpose reasoning LMs. For example, what are the typical failure modes of general-purpose models, and how does Science0-c address or overcome them?\n\n[1] Edwards, C., Zhai, C. and Ji, H., 2021. Text2Mol: Cross-modal molecule retrieval with natural language queries. *Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*, pp.595–607.\n\n[2] Guo, T., Guo, K., Nan, B., Liang, Z., Guo, Z., Chawla, N.V., Wiest, O. and Zhang, X., 2023. What can large language models do in chemistry? A comprehensive benchmark on eight tasks. In: *Proceedings of the 37th International Conference on Neural Information Processing Systems (NeurIPS 2023)*\n\n[3] DeepSeek-AI, Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P. *et al.*, 2025. *DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning*. arXiv preprint arXiv:2501.12948.", "was_revised": true, "final_justification": "Overall, the rebuttal addresses most of the issues and concerns raised in the discussion, so I have raised my score", "rebuttal": "Many thanks to the reviewer for the valuable feedback provided. Here, we would like to address the main concerns raised on the review:\n\n**The evaluation is limited to a single benchmark consisting of holdout problems from the dataset:**\n\nFirst, we would like to emphasize that one of our goals was to design a dataset of specific, practical, and challenging problems that are useful in real research. Our objective was not to train a general-purpose model capable of generalizing to new tasks, which is why our evaluation focuses on a held-out subset of problems from the same dataset.\n\nHowever, we did evaluate the model on some tasks outside the training task distribution, such as ChemBench [1] and the suggested chebi-20. We confirmed that the performance of Science0-c on these unseen tasks is limited. Science0-c obtains 44% on ChemBench and 31% on chebi-20 pass@1, in both cases far from the state-of-the-art. Nonetheless, nothing suggests that these additional tasks could not be incorporated into the dataset and learned using the approach described in our work, potentially leveraging transfer learning from the other trained tasks. We acknowledge that this information is important and will make these limitations clearer by mentioning these results.\n\nWe also wanted to ensure that the model was robust to variations in phrasing. This is relevant to generalise within the trained tasks. To evaluate this, we ran an experiment where we asked a frontier model (Gemini) to rephrase the test questions and found no significant performance differences compared to the original wording. Similarly, we added distracting information to the prompts and again observed no meaningful performance change. We also plan to include it in a final version of the paper.\n\nAdditionally we want to emphasize that we ensured a fair comparison with all baselines on the test set. We adapted each test set problem to match the input format and expectations of the open-source models. For example, for ChemDFM, we adjusted the prompt wording and applied SMILES canonicalization. Similar adaptations were made for TxGemma. Naturally, all models used their own system prompts. We are aware that frontier LLMs likely haven't been optimized for the specific tasks considered in our work. However, they have been trained to be robust to different phrasings, so we consider it reasonable to evaluate them with the same questions as our model, as no specific phrasing will lead to improved performance a priori.\n\n**The contribution in terms of novelty is limited.**\n\nOur work introduces a set of advances that enable the effective application of reasoning models to scientific tasks, while also offering broader insights into the training of general-purpose reasoning systems.\n\nWe begin by hypothesizing that open-ended scientific problems benefit from reasoning-based models due to their nature grounded in structured reasoning. We validate this hypothesis by demonstrating strong performance on chemistry tasks, confirming that such models are well-suited for complex scientific reasoning. To the best of our knowledge, this had not been demonstrated before. Additionally, we show that a general model, which has potentially seen limited chemistry data during pre-training, can be effectively specialized for chemistry, which we believe to be not obvious.\n\nWhile it is true that our training pipeline builds on established techniques like SFT and GRPO, it also incorporates several novel components that distinguish it from the initial wave of reasoning model research released in early 2025. These include advantage-based curriculum training (with some concurrent work using similar yet different approaches to tackle the same problem, like Llama-Nemotron [2] and DAPO [3]), rephrasing demonstration trajectories using the base model intended for RL training (Mistral-Small-24B))  to better align the demonstration token distributions, interleaving multiple rounds of SFT and GRPO, and the use of specialist models (a similar idea was later incorporated into a revised version of the DeepSeek-V3 [4] paper).\n\nAlthough our data sources are publicly available, our work combines them with well-designed reward functions and a robust training strategy that allows achieving top performance on several difficult chemistry benchmarks. The design of this pipeline contributes to the progress of AI for Chemistry. Additionally, toolkit-based rewards often fail when used out-of-the-box, and avoiding reward hacking is especially hard in this setting. Below are two examples and how we addressed them:\n\nFor example, in designing the retrosynthesis reward function, Science0-c first learned to exploit the rule that only one of the reaction components needed to be purchasable. It began suggesting reactions where it swapped a bromine atom with another halide (like chlorine or fluoride) on the target molecule. These transformations often used precursors that weren’t purchasable, but the presence of a common purchasable halide (e.g., chloride) allowed Science0-c to game the reward. This simple, high-reward hack inflated its performance without truly solving the retrosynthesis task. We resolved this by expanding the bloom filter to include a comprehensive list of laboratory reagents and enforcing stricter checks on all reactants’ purchasability and molecular transformations.\n\nAs another example, when asked to generate molecules with specific atomic compositions (like C₃H₅O₂), Science0-c would often append unstable or synthetically infeasible groups, such as long peroxide or hydrazine tails, to meet atom count requirements. These additions fulfilled the letter of the task but produced unrealistic, explosive molecules. To combat this, we created a bloom filter of all three-atom-radius subgraphs found in known synthesized molecules and used this as a \"reasonableness\" check in the reward function. We further filtered out chemically infeasible ring structures and tuned the reward to discourage nitro groups and other problematic patterns, shifting from brittle heuristics to preference bonus rewards.\n\nFinally, our work also provides a thorough analysis of the trained model’s behavior, including data efficiency, failure modes, and reasoning patterns. These insights shed light on how reasoning models behave in scientific domains, offering guidance for future research.\n\n**The paper lacks a qualitative analysis of the reasoning behavior of Science0-c compared to general-purpose reasoning LMs.** \n\nOf all the models evaluated, only DeepSeek-r1 provides traces we can examine. Closed-source models do not disclose their exact reasoning patterns, while specialized models do not perform reasoning at all.\n\nWe actually conducted a qualitative analysis of the reasoning traces between DeepSeek-r1 and Science0-c. While this comparative analysis was not included in the original manuscript (only the results for Science0-c were reported in the supplementary information section 3), upon further review we recognize the value of reporting the results also for DeepSeek-r1 as suggested by the reviewer and will add the full comparison.\n\nAs explained in the supplementary information, we asked human experts to evaluate how much they agreed that the traces from both models were non-contrived/genuine (in the sense that they did not exhibit performative thinking without clear justification given the problem under consideration), faithful (i.e., the answer is consistent with the reasoning trace), and whether the model performed an extensive exploration before answering. Experts had five response options: strongly agree, somewhat agree, neither agree nor disagree, somewhat disagree, and strongly disagree.\n\nThe results showed that more experts disagreed with the claim that DeepSeek-r1 demonstrated non-contrived and faithful reasoning, although they noted it did have more extensive exploration. This is somewhat expected, given the average length of DeepSeek-r1’s reasoning traces being much longer than Science0-c traces.\n\nWe report our results here and will include them in the final version of the manuscript:\n\nNon-Contrived/Genuine:\n| Model            | S. Agree | Agree  | Neutral      | Disagree | S Disagree\n|-------------------|-------------|----------|-----------------|-------------|-----------------|\n| DeepSeek-r1 | 28%       | 23%    | 7%              | 8%        | 33%             |\n| Science0-c    | 45%       | 23%    | 12%             | 15%       | 5%              |\n\nFaithful:\n| Model            | S. Agree | Agree  | Neutral      | Disagree | S Disagree\n|-------------------|-------------|----------|-----------------|-------------|-----------------|\n| DeepSeek-r1 | 33%       | 25%   | 8%               | 20%       | 13%            |\n| Science0-c    | 50%       |23%     | 10%             | 13%        | 3%             |\n \nExplorative:\n| Model            | S. Agree | Agree  | Neutral      | Disagree | S Disagree\n|-------------------|-------------|----------|-----------------|-------------|-----------------|\n| DeepSeek-r1 | 20%      | 32%    | 18%             | 13%       | 17%             |\n| Science0-c    | 13%       | 28%    | 35%             | 5%         | 18%            |\n\n**References:**\n\n[1] Mirza, Adrian, et al. \"Are large language models superhuman chemists?.\" arXiv preprint arXiv:2404.01475 (2024).\n\n[2] Bercovich, Akhiad, et al. \"Llama-nemotron: Efficient reasoning models.\" arXiv preprint arXiv:2505.00949 (2025).\n\n[3] Yu, Qiying, et al. \"Dapo: An open-source llm reinforcement learning system at scale.\" arXiv preprint arXiv:2503.14476 (2025).\n\n[4] Liu, Aixin, et al. \"Deepseek-v3 technical report.\" arXiv preprint arXiv:2412.19437 (2024)."}, {"review_id": "06ExIiiuoT", "reviewer": "Reviewer_hCQw", "summary": "This paper introduces Science0-c, a 24-billion parameter LLM designed for chemical reasoning. It's built upon the Mistral-Small-24B model and has been specifically fine-tuned to understand natural language prompts and respond with chemical structures in SMILES format.", "questions": "Regarding Figure 2, the legend does not define the blue cross symbol. Though I can guess the author skip tasks related to SMILES for human, it should be explicitly stated in the caption for clarity.", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n1. A key strength is the demonstration that a general reasoning model can be effectively specialized for chemistry, which is significantly more data-efficient than creating a specialized model from scratch.\n2. The discussion section is particularly robust, offering a well-rounded analysis of both the model's groundbreaking capabilities and its current constraints.\n\nWeakness:\n1. The validity of the retrosynthesis dataset is a point of concern. The authors mention the use of Molecular Transformer in the main text and reaction templates in the SI, rather than using experimentally verified reactions. This approach introduces a potential gap between the computational predictions and practical laboratory outcomes.\n2. The human baseline evaluation was conducted with a small panel of four to five experts, which limits the statistical significance of these results. This small sample size weakens claims about the model's performance relative to human experts. To strengthen the comparison, the authors should, at a minimum, report the variability in human performance by including error bars.", "was_revised": true, "final_justification": "My primary reservations were initially centered on two aspects: the validity of the retrosynthesis dataset and the statistical significance of the human baseline. The authors clarified that reaction templates were used to ensure the training data was grounded in experimentally feasible chemistry. They explained that the Molecular Transformer was a necessary tool for validating the novel reactions proposed by the model, a task for which experimental data does not yet exist. Crucially, they have committed to explicitly discussing the known limitations of this validation approach in the final manuscript. This transparency is a responsible and satisfactory resolution. Also, the authors have agreed to add error bars to the human evaluation plots to report performance variability. This directly addresses my concern and provides the necessary context to fairly interpret the comparison between the model and human experts.", "rebuttal": "We thank the reviewer for the constructive review and helpful observations. We address the weaknesses and questions raised by the reviewer one by one below.\n\n**The validity of the retrosynthesis dataset is a point of concern. The authors mention the use of Molecular Transformer in the main text and reaction templates in the SI, rather than using experimentally verified reactions. This approach introduces a potential gap between the computational predictions and practical laboratory outcomes.**\n\nWe used reaction templates to generate the dataset to ensure that the training data consists of experimentally synthesizable molecules. However, since Science0-c proposes new, previously unseen reactions, we required a more general method to validate its predicted reactions. To make the reward function broadly applicable, we used Molecular Transformers (MT) for reaction validation. We acknowledge that this approach has limitations, as MT itself is not without shortcomings. These limitations will be discussed more explicitly in the limitations section.\n\n**The human baseline evaluation was conducted with a small panel of four to five experts, which limits the statistical significance of these results. This small sample size weakens claims about the model's performance relative to human experts. To strengthen the comparison, the authors should, at a minimum, report the variability in human performance by including error bars.**\n\nWe will include error bars in the final version of the plot to reflect variability in human performance. Meanwhile, we report here the standard deviation across human annotators for each task in the table below:\n\n| Task                                                               | std                   |\n|------------------------------------------------------------|---------------------|\n| Functional group                                            | 0.12                 |\n| Solubility edit                                                  | 0.06                 |\n| Reaction prediction                                        | 0.26                 |\n| Retrosynthesis                                               | 0.05                 |\n| Molecular formula                                          | 0.22                 |\n\n**Regarding Figure 2, the legend does not define the blue cross symbol. Though I can guess the author skip tasks related to SMILES for human, it should be explicitly stated in the caption for clarity.**\n\nThis is correct, thanks for the observation. The blue cross indicates that human evaluations were not possible for the task, mostly because humans were not able to do things like SMILES completion. We'll add this clarification to the caption."}, {"review_id": "Iqx5yC6fty", "reviewer": "Reviewer_ha5g", "summary": "This paper introduces Science0-c, an LLM specifically developed for scientific reasoning in chemistry. The authors constructed a large dataset of ~570K chemistry problems covering 17 tasks such as reaction prediction, SMILES completion, and molecule design under specific constraints. Science0-c is trained from the Mistral-Small-24B-Instruct-2501 base model using this dataset with a multi-stage training procedure that interleaves distillation and GRPO.\n\nExperimental evaluation on Science0-c's test set shows that the proposed model outperforms both frontier and domain-specific models.", "questions": "* What're the exact sizes of datasets used during various phases of the training pipeline? What percentage of the data is filtered out due to quality concerns at each stage?\n\n* Is the second distillation performed on the warm started model or directly on the base mistral model?\n\n* Please explain the subtasks within MCQ tasks like Safety, Scent, Receptor Binding, ADME, and pKa.\n\n* Is there a reason the 'Human' baseline was not included in the MCQ results? Please include it if possible.\n\n* Are there plans to release the full code and datasets?", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n\n* The authors carefully created a large dataset of chemistry problems (MCQ & open-ended) that covers various chemistry tasks from data sources like ChEMBL, COCONUT and Pubchem. The initial dataset and distillation dataset would be valuable for future research if publicly released.\n\n* The detailed description of the training methodology, including SFT, task-specific RL, distillation, and a final all-task RL phase, demonstrates a well thought-out and effective approach for developing specialized reasoning models. \n\n* Including \"reasonable molecule\" check in GRPO verifier is an interesting strategy to ensure the usefulness and plausability of the molecules generated by Science0-c\n\n* The paper's detailed analysis on cognitive behaviors like verification and backtracking in reasoning traces during training, and their correlation with task performance, provides valuable insights into the internal workings and effectiveness of reasoning models.\n\n* Overall, it's a well written and easy to read paper\n\n\n**Weaknesses:**\n\n* Although the experimental results show the significant benefits of a reasoning model for chemistry, the training approach relies on established techniques like SFT, GRPO, and distillation, and lacks novelty.\n\n* Science0-c is mainly benchmarked on its own test set. Additional evaluation on existing chemistry benchmarks like ChemLLMBench could be useful for a more comprehensive assessment.\n\n* Ablation studies on vanilla SFT over the full ~570K dataset, and SFT + all-task RL without the task-specific RL phase, would further help to understand the specific benefits and contributions of the proposed multi-stage training procedure.", "was_revised": true, "final_justification": "This work on reasoning for chemistry tasks is quite promising. The authors offer a thorough overview of their data, training approach, and evaluation. The rebuttal reasonably addressed my earlier concerns about novelty and performance on out-of-distribution tasks.", "rebuttal": "We thank the reviewer for the constructive review and helpful remarks. We address the weaknesses raised by the reviewer one by one below.\n\n**Although the experimental results show the significant benefits of a reasoning model for chemistry, the training approach relies on established techniques like SFT, GRPO, and distillation, and lacks novelty.**\n\nWhile it is true that our training pipeline builds on established techniques like SFT and GRPO, it also incorporates several novel components that distinguish it from the initial wave of reasoning model research released in early 2025. These include advantage-based curriculum training (with some concurrent work using similar yet different approaches to tackle the same problem, like Llama-Nemotron [1] and DAPO [2]), rephrasing demonstration trajectories using our target model to better align the token distribution, interleaving multiple rounds of SFT and GRPO, and the use of specialist models (a similar idea was later incorporated into a revised version of the DeepSeek-V3 [3] paper).\n\n**Science0-c is mainly benchmarked on its own test set. Additional evaluation on existing chemistry benchmarks like ChemLLMBench could be useful for a more comprehensive assessment.**\n\nFirst, we would like to emphasize that one of our goals was to design a dataset of specific, practical, and challenging problems that are useful in real research. Our objective was not to train a general-purpose model capable of generalizing to new tasks, which is why our evaluation focuses on a held-out subset of problems from the same dataset.\n\nHowever, we did evaluate the model on some tasks outside the training task distribution, such as ChemBench [4] and the suggested chebi-20 [5]. We confirmed that the performance of Science0-c on these unseen tasks is limited. Science0-c obtains 44% on ChemBench and 31% on chebi-20 pass@1, in both cases far from the state-of-the-art. Nonetheless, nothing suggests that these additional tasks could not be incorporated into the dataset and learned using the approach described in our work, potentially leveraging transfer learning from the other trained tasks. We acknowledge that this information is important and will make these limitations clearer by mentioning these results.\n\nWe also wanted to ensure that the model was robust to variations in phrasing. This is relevant to generalise within the trained tasks. To evaluate this, we ran an experiment where we asked a frontier model (Gemini) to rephrase the test questions and found no significant performance differences compared to the original versions. Similarly, we added distracting information to the prompts and again observed no meaningful performance change. We also plan to include it in a final version of the paper.\n\n**Ablation studies on vanilla SFT over the full ~570K dataset, and SFT + all-task RL without the task-specific RL phase, would further help to understand the specific benefits and contributions of the proposed multi-stage training procedure.**\n\nWe did not perform SFT over the full 577,790 dataset because reasoning traces are not available for all examples. Initially only those solved by DeepSeek-r1 (14,021 samples) and later those solved by the specialists (186,010 samples) are available. The total number of solved problems with reasoning traces available is 200,031.\n\nWe did perform training on the 200,031 examples available (a significant subset of the full dataset), which corresponds to the second distillation phase of our pipeline. However, we observed that we could further push performance of the model with the final RL phase.\n\nTraining on just (q, a) pairs (without reasoning traces) corresponds to our provided non-reasoning SFT baseline. \n\nAs for SFT followed by all-task RL without task-specific RL, we did not scale this configuration fully due to early signs of poor learning dynamics. In particular, some task categories such as molecular formula and functional group showed little to no learning under this setup. We acknowledge that this information is relevant and will include it in the manuscript for completeness.\n\n**What're the exact sizes of datasets used during various phases of the training pipeline? What percentage of the data is filtered out due to quality concerns at each stage?**\n\nThe complete train dataset contains 577,790 samples, as detailed in **Table 1**.\n\n1. Initial Supervised Fine-Tuning (SFT):\n   - Conducted on 14,021 samples originally solved by DeepSeek-r1.\n2. Specialist Training with GRPO:\n   - GRPO training used the entire train dataset, split by task.\n3. Filtering specialist data\n   - Specialists collectively solved 3,830,728 questions (with repeated questions).\n   - From these questions, we retained only the most recent solution per unique question, resulting in 201,957 samples.\n   - We found that ~13% of the samples were filtered out due to quality concerns but in almost all cases we could find an earlier instance in which the model solved the question with high quality, and used that one instead. To enhance clarity, we will add this information to the paper. The final dataset after the quality filtering was 200,031 samples.\n   - The specialist solved questions included the 14,021 questions originally solved by DeepSeek-r1, so the specialist models effectively contributed to solve 186,010 new high-quality samples.\n4. Second SFT Phase:\n   - SFT was performed on the 200,031 samples.\n5. Final RL Phase:\n   - GRPO on the full dataset of 577,790 samples again.\n\n**Is the second distillation performed on the warm started model or directly on the base mistral model?**\n\nThe second distillation was performed on the base mistral model. We will be more clear about that, thank you.\n\n**Please explain the subtasks within MCQ tasks like Safety, Scent, Receptor Binding, ADME, and pKa.**\n\nAs suggested by the reviewer, we will add formal definitions for each subtask, likely in the Supplementary Information (SI) due to space limitations in the main text.\n\n*Safety:* Select the molecule whose structure most strongly aligns with or deviates from a specified safety-related property, such as toxicity, flammability, or hazard classification.\n*Scent:* Identify the molecule most likely to exhibit a specific olfactory attribute (e.g., meaty, spicy, oily).\n*Blood-brain barrier:* Determine which molecule is or isn’t structurally likely to penetrate the blood-brain barrier based on reference behavior.\n*Receptor Binding:* Identify the molecule whose structure most likely lacks binding affinity or activity for a specified biological receptor target.\n*ADME:* Choose the molecule expected to improve or match a specified absorption, distribution, metabolism, or excretion property based on structural modifications.\n*Aqueous Solubility:* Select the molecule whose structure leads to a targeted increase or decrease in water solubility.\n*LD50:* Select the molecule whose structure corresponds to a specified rat oral LD50 value, reflecting its relative acute toxicity level.\n*pKa:* Select the molecule whose structure aligns with a specified increase, decrease, or target value of pKaH1.\n*Photoswitches:* Select the molecule whose E isomer exhibits a target π–π* transition wavelength.\n\n**Is there a reason the 'Human' baseline was not included in the MCQ results? Please include it if possible.**\n\nWe focused our human evaluations on the open-ended questions due to resource constraints. If the reviewer sees value in including human performance on MCQs, we're open to running them and incorporating the results in the final version.\n\n**Are there plans to release the full code and datasets?**\n\nThe anonymized link in the SI includes our scripts for SFT and GRPO training, located in the *scripts* directory. All references to our organization have been removed to comply with double-blind review. We also have plans to release the code for our reward functions, prompt templates, data utilities, and the dataset.\n\n**References:**\n\n[1] Bercovich, Akhiad, et al. \"Llama-nemotron: Efficient reasoning models.\" arXiv preprint arXiv:2505.00949 (2025).\n\n[2] Yu, Qiying, et al. \"Dapo: An open-source llm reinforcement learning system at scale.\" arXiv preprint arXiv:2503.14476 (2025).\n\n[3] Liu, Aixin, et al. \"Deepseek-v3 technical report.\" arXiv preprint arXiv:2412.19437 (2024).\n\n[4] Mirza, Adrian, et al. \"Are large language models superhuman chemists?.\" arXiv preprint arXiv:2404.01475 (2024).\n\n[5] Edwards, Carl, ChengXiang Zhai, and Heng Ji. \"Text2mol: Cross-modal molecule retrieval with natural language queries.\" Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021."}, {"review_id": "cvkNKzK36t", "reviewer": "Reviewer_ZRWC", "summary": "This paper presents Science0-c, a 24B parameter reasoning LLM fine-tuned for chemistry using reinforcement learning on 577,790 verifiable tasks involving molecule design, modification, completion, and synthesis. The model demonstrates strong performance across open-ended and multiple-choice tasks, outperforming general-purpose LLMs, humans, and prior chemistry-specific models.", "questions": "see weakness", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n\n-The paper tackles the underexplored problem of scientific reasoning in chemistry using LLMs\n\n-Strong empirical performance on diverse and challenging chemistry tasks\n\n-Science0-c outperforms both general-purpose LLMs (e.g., GPT-o1, Claude) and specialized chemistry models (e.g., TxGemma, ChemDFM)\n\n-Detailed training pipeline with reinforcement learning, task specialization, and generalist distillation.\n\n-Open-sourcing of model, code, and reward functions\n\n\nWeakness:\n\n-It would be helpful if the authors briefly discuss the broader utility of Science0-c, such as its potential use in drug discovery pipelines\n\n-The reasoning traces provide useful insight into the model’s thought process, making Science0-c partially interpretable by design. However, a small-scale case study—such as having domain experts annotate a sample of reasoning traces for correctness, relevance, or use of chemical knowledge—could help strengthen the scientific trustworthiness.\n\n-After being trained on this set of chemistry tasks, can the model generalize to unseen tasks by leveraging the chemical knowledge it has learned?", "was_revised": true, "final_justification": "The authors have provided a more thorough qualitative analysis of the model’s reasoning traces and added a discussion of Science0-c's potential role in drug discovery workflows. Although the model does not transfer well to tasks outside its training distribution, it performs well on the proposed chemistry tasks and shows thoughtful design for those use cases.", "rebuttal": "We thank the reviewer for the constructive review and helpful remarks. We address the weaknesses raised by the reviewer one by one below.\n\n**It would be helpful if the authors briefly discuss the broader utility of Science0-c, such as its potential use in drug discovery pipelines.**\n\nWe agree that highlighting the broader utility of Science0-c in the context of drug discovery is important to illustrate its relevance in practical settings and the potential impact beyond the reported benchmarks. We will incorporate this perspective into the introduction of our manuscript. \n\nSpecifically, Science0-c supports key stages of the drug discovery pipeline: it can generate candidates during hit discovery, it enables molecule editing in hit-to-lead to improve potency, selectivity, or physicochemical properties, and it contributes to lead optimization by refining compounds to enhance efficacy, reduce toxicity, and improve ADMET profiles. All while being aware of synthesizability.\n\nAdditionally, and unlike models like molt5 or chemformer, Science0-c works directly in natural language, which facilitates integration into drug discovery workflows through intuitive interaction, and its reasoning traces can help medicinal chemists understand the justification of proposed molecules, especially in high-stakes decisions.\n\n**The reasoning traces provide useful insight into the model’s thought process, making Science0-c partially interpretable by design. However, a small-scale case study—such as having domain experts annotate a sample of reasoning traces for correctness, relevance, or use of chemical knowledge—could help strengthen the scientific trustworthiness.**\n\nWe actually conducted a qualitative analysis of the reasoning traces between DeepSeek-r1 and Science0-c. While this comparative analysis was not included in the original manuscript (only the results for Science0-c were reported in the supplementary information section 3), upon further review we recognize the value of reporting the results also for DeepSeek-r1 and will add them as well.\n\nAs explained in the supplementary information, we asked human experts to evaluate how much they agreed that the traces from both models were non-contrived/genuine (in the sense that they did not exhibit performative thinking without clear justification given the problem under consideration), faithful (i.e., the answer is consistent with the reasoning trace), and whether the model performed an extensive exploration before answering. Experts had five response options: strongly agree, somewhat agree, neither agree nor disagree, somewhat disagree, and strongly disagree.\n\nThe results showed that more experts disagreed with the claim that DeepSeek-r1 demonstrated non-contrived and faithful reasoning, although they noted it did have more extensive exploration. This is somewhat expected, given the average length of DeepSeek-r1’s reasoning traces being much longer than Science0-c traces.\n\nWe report our results here and will include them in the final version of the manuscript:\n\nNon-Contrived/Genuine:\n| Model            | S. Agree | Agree  | Neutral      | Disagree | S Disagree\n|-------------------|-------------|----------|-----------------|-------------|-----------------|\n| DeepSeek-r1 | 28%       | 23%    | 7%              | 8%        | 33%             |\n| Science0-c    | 45%       | 23%    | 12%             | 15%       | 5%              |\n\nFaithful:\n| Model            | S. Agree | Agree  | Neutral      | Disagree | S Disagree\n|-------------------|-------------|----------|-----------------|-------------|-----------------|\n| DeepSeek-r1 | 33%       | 25%   | 8%               | 20%       | 13%            |\n| Science0-c    | 50%       |23%     | 10%             | 13%        | 3%             |\n \nExplorative:\n| Model            | S. Agree | Agree  | Neutral      | Disagree | S Disagree\n|-------------------|-------------|----------|-----------------|-------------|-----------------|\n| DeepSeek-r1 | 20%      | 32%    | 18%             | 13%       | 17%             |\n| Science0-c    | 13%       | 28%    | 35%             | 5%         | 18%            |\n\n**After being trained on this set of chemistry tasks, can the model generalize to unseen tasks by leveraging the chemical knowledge it has learned?**\n\nFirst, we would like to emphasize that our goal was to design a dataset of specific, practical, and challenging problems that are useful in real research. Our objective was not to train a general-purpose model capable of generalizing to new tasks, which is why our evaluation focuses on a held-out subset of problems from the same dataset.\n\nThat being said, we did evaluate the model on some tasks outside the training task distribution, such as ChemBench [1] and chebi-20 [2]. We confirmed that the performance of Science0-c on these unseen tasks is limited. Science0-c obtains 44% on ChemBench and 31% on chebi-20 pass@1, in both cases far from the state-of-the-art. Nonetheless, nothing suggests that these additional tasks could not be incorporated into the dataset and learned using the approach described in our work, potentially leveraging transfer learning from the other trained tasks. We acknowledge that this information is important and will make these limitations clearer by mentioning these results.\n\nWe also wanted to ensure that the model was robust to variations in phrasing. This is relevant to generalise within the trained tasks. To evaluate this, we ran an experiment where we asked a frontier model (Gemini) to rephrase the test questions and found no significant performance differences compared to the original versions. Similarly, we added distracting information to the prompts and again observed no meaningful performance change. We also plan to include it in a final version of the paper.\n\n**References:**\n\n[1] Mirza, Adrian, et al. \"Are large language models superhuman chemists?.\" arXiv preprint arXiv:2404.01475 (2024).\n\n[2] Edwards, Carl, ChengXiang Zhai, and Heng Ji. \"Text2mol: Cross-modal molecule retrieval with natural language queries.\" Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021."}, {"review_id": "2LlAsYikEc", "reviewer": "Reviewer_dvLR", "summary": "This paper introduces Science0-c, a 24-billion parameter large language model (LLM) specifically trained to solve complex chemistry problems.  Based on the Mistral-Small-24B model, Science0-c is designed to reason in natural language and provide answers in chemical structure formats like SMILES. The training process involved a multi-stage approach combining supervised fine-tuning on chain-of-thought examples, training task-specific \"specialist\" models via RL, distilling their knowledge into a single \"generalist\" model, and then further training the generalist model with RL.  The authors report that Science0-c significantly outperforms existing general-purpose chemistry models, frontier LLMs like Claude and GPT models, and even human experts on a range of open-ended chemistry tasks.", "questions": "1. The authors' commitment to open-sourcing the code, dataset, and model is a significant contribution. However, the anonymous repository link provided for review appears to be empty. Given the highly complex, multi-stage training pipeline, reproducing these results presents a considerable challenge. For the benefit of the review process and future reproducibility, it would be immensely helpful if the authors could update the repository with the functional code. Furthermore, providing samples from the training data for each stage, along with a few examples of the final model's reasoning texts, would be invaluable for a thorough analysis.\n\n2. The training curves in Figure 2a indicate that for several tasks, the task-specific reinforcement learning phase begins with a success rate close to zero. Standard RL is often inefficient in such a sparse reward setting, as the policy may struggle to generate successful samples to initiate the learning gradient. Could the authors elaborate on how the model was able to overcome this challenge? \n\n3. In the data efficiency analysis, the paper compares Science0-c to the Molecular Transformer, a 2019 model. While this model was foundational, have the authors considered comparing their results against more contemporary state-of-the-art models for reaction prediction? A comparison against a more recent baseline would provide a stronger validation of the model's claimed superiority in data efficiency and performance.\n\n4. The study primarily employs a binary reward signal (0 or 1) for task success. Could the authors discuss the rationale for this choice over a continuous reward function? What are the authors' thoughts on the challenges and opportunities in designing more sophisticated reward functions for complex scientific reasoning tasks in chemistry?", "limitations": "1. Loss of General Capabilities: The intensive RL training focused on chemistry tasks has removed the general instruction-following and conversational abilities of the base Mistral model. \n2. No Tool Use: The model was not trained to use external tools, which is a common and critical part of modern computational chemistry workflows.  Integrating tool-calling abilities is left for future work", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "strengths\n1. Novel Application: The research successfully extends the application of \"reasoning models,\" previously focused on math and programming, to the scientific domain of chemistry, demonstrating their potential beyond logic-based fields. \n2. Superior Performance: Science0-c shows state-of-the-art performance, outperforming other large-scale models and human experts on several challenging, open-ended chemical reasoning tasks.  This is a significant achievement, particularly on complex problems like structure elucidation and retrosynthesis.\n3. Methodological Rigor: The authors provide a detailed and transparent account of their multi-stage training pipeline, including supervised fine-tuning, specialist RL, distillation, and the use of an advantage-based curriculum.  The inclusion of ablation studies, such as comparing reasoning vs. non-reasoning models, strongly supports their central claim that explicit reasoning is beneficial. \n\nweaknesses\n1. Reasoning Quality Degradation: The study honestly reports that as RL training progresses and accuracy increases, the quality of the model's reasoning text can degrade, with the emergence of typos, non-English characters, and nonsensical formatting.\n2. Performance Gap with Specialized Models: While the model demonstrates impressive performance, outperforming both proprietary and open-source chemistry LLMs, it still lags behind state-of-the-art specialized models. These specialized models, which are trained exclusively for a single purpose (such as retrosynthesis), often represent the true state-of-the-art in performance for that specific application. Therefore, for users requiring the absolute highest accuracy on a single, well-defined problem, this generalist reasoning model may not be the optimal choice.", "was_revised": true, "final_justification": "Most of my concerns are addressed by the authors. I think this is an important work for the chemical area. Due to several questions I proposed, such as \"performance gap with specialized models\" and \"loss of general capabilities\", are not addressed, I maintain my rating of 5.", "rebuttal": "We thank the reviewer for the constructive review and helpful observations. We address the weaknesses and questions raised by the reviewer one by one below.\n\n**Reasoning Quality Degradation: The study honestly reports that as RL training progresses and accuracy increases, the quality of the model's reasoning text can degrade, with the emergence of typos, non-English characters, and nonsensical formatting.**\n\nThis is indeed an interesting phenomenon that we observed. While the underlying reason is not entirely clear to us, we have reported it as a potentially interesting direction for future research.\n\n**Performance Gap with Specialized Models: While the model demonstrates impressive performance, outperforming both proprietary and open-source chemistry LLMs, it still lags behind state-of-the-art specialized models. These specialized models, which are trained exclusively for a single purpose (such as retrosynthesis), often represent the true state-of-the-art in performance for that specific application. Therefore, for users requiring the absolute highest accuracy on a single, well-defined problem, this generalist reasoning model may not be the optimal choice.**\n\nIn our evaluation with specialized models, we focused on reaction prediction using the Molecular Transformer (MT) model. We acknowledge that other specialized models could be considered for other tasks, and we are open to noting this in the limitations. More specifically, that a performance gap may exist with respect to state-of-the-art models for certain specific tasks.\n\n**The authors' commitment to open-sourcing the code, dataset, and model is a significant contribution. However, the anonymous repository link provided for review appears to be empty. Given the highly complex, multi-stage training pipeline, reproducing these results presents a considerable challenge. For the benefit of the review process and future reproducibility, it would be immensely helpful if the authors could update the repository with the functional code. Furthermore, providing samples from the training data for each stage, along with a few examples of the final model's reasoning texts, would be invaluable for a thorough analysis.**\n\nWe sincerely apologize for the oversight regarding the anonymous repository link. Thank you for pointing this out.\n\nWe have now corrected the issue, the link is fully functional and the repository should allow for a review of our training codebase, including the scripts used for SFT and GRPO training, which are located in the *scripts* directory. Please note that we have removed all references to our org to comply with the double blind review, which may have caused degradation in the code's functionality.\n\nFinally, we have included in the anonymized github repo a representative sample of the final model’s reasoning outputs. These examples can be found inside the *data/molrqa_model_answers* directory.\n\n**The training curves in Figure 2a indicate that for several tasks, the task-specific reinforcement learning phase begins with a success rate close to zero. Standard RL is often inefficient in such a sparse reward setting, as the policy may struggle to generate successful samples to initiate the learning gradient. Could the authors elaborate on how the model was able to overcome this challenge?**\n\nWhile it is true that RL algorithms can be inefficient in sparse reward settings, RL training remains viable as long as some non-zero reward signal is eventually encountered. In practice, RL can learn effectively even from rare success signals, it just tends to start slower, as evidenced in Figure 2a. Despite initially low success start rate, we found that the model is able to bootstrap off rare successful trajectories, and this was often sufficient to get learning off the ground.\n\nMoreover, we incorporate a curriculum learning strategy based on advantage-weighted sampling. Specifically, during training we preferentially sample prompts where the estimated advantage is non-zero. This naturally biases learning toward trajectories that are already yielding some signal, thus helping the model focus on parts of the prompt distribution where learning progress is possible. This targeted sampling strategy mitigates the exploration problem and accelerates convergence in sparse reward regimes.\n\n**In the data efficiency analysis, the paper compares Science0-c to the Molecular Transformer, a 2019 model. While this model was foundational, have the authors considered comparing their results against more contemporary state-of-the-art models for reaction prediction? A comparison against a more recent baseline would provide a stronger validation of the model's claimed superiority in data efficiency and performance.**\n\nDespite being published in 2019, Molecular Transformers (MT) is still among the best models for reaction prediction, reaching 88.6 %  pass@1 accuracy  on USPTO_MIT [1]. Additionally, MT is a relatively small model, making it easier to perform multiple studies with it. However, to illustrate our performance against more up-to-date models in diverse chemistry tasks, we also compared Science0-c to ChemDFM (2024).\n\n**The study primarily employs a binary reward signal (0 or 1) for task success. Could the authors discuss the rationale for this choice over a continuous reward function? What are the authors' thoughts on the challenges and opportunities in designing more sophisticated reward functions for complex scientific reasoning tasks in chemistry?**\n\nWe experimented with continuous reward functions on several tasks, including reaction prediction and retrosynthesis, but these attempts did not improve over our binary baseline in most cases. Designing effective continuous rewards is complex, often considered almost an art, and requires deep domain expertise and significant time investment. Even then, it may fail to provide useful learning signals. Therefore, we chose not to pursue this direction further after a certain number of unsuccessful attempts.\n\nGiven these challenges and the limited gains from our early trials, we adopted binary rewards across all tasks except one. The exception is the specialist trained on molecular formula, functional group, and elucidation tasks, where the reward was set to 0.5 if the formula was correct and 1.0 if all constraints were met. We found that this setting improved performance over the binary baseline.\n\n**Loss of General Capabilities: The intensive RL training focused on chemistry tasks has removed the general instruction-following and conversational abilities of the base Mistral model.**\n\nThis is true. In particular, the fact of focusing on tasks whose expected answer is always a SMILES, has removed the conversational abilities of the base mistral model.\n\n**No Tool Use: The model was not trained to use external tools, which is a common and critical part of modern computational chemistry workflows. Integrating tool-calling abilities is left for future work**\n\nThis is also correct, tool use was left for future work. We believe integrating tool-calling has strong potential to unlock even more accurate and useful capabilities, especially for real-world chemistry workflows, but in this work we focused on verifying whether reasoning alone was helpful, and left the addition of tools for future work.\n\n**References:**\n\n[1] Jin, Wengong, et al. \"Predicting organic reaction outcomes with weisfeiler-lehman network.\" Advances in neural information processing systems 30 (2017)."}]

Author rebuttal:

Meta-review: This paper proposes Science0-c, a 24-billion parameter LLM, designed for chemical reasoning. Science0-c is based on the Mistral-Small-24B model without additional domain pretraining but fine-tuned to understand natural language prompts and respond with chemical structures in SMILES format. 

The reviewers expressed positive support for the paper and I also think that the work makes a good contribution toward reasoning for chemistry tasks. I strongly encourage the authors to incorporate all the discussion points with the reviewers in the final version.
Decision: Accept (poster)