SCI-Verifier: Scientific Verifier with Thinking
Authors: Shenghe Zheng
Chenyu Huang
Fangchen Yu
Junchi Yao
Jingqi Ye
Tao Chen
Yun Luo
Ning Ding
LEI BAI
Ganqu Cui
Peng Ye
Venue: ICLR 2026 Poster (2026)
Keywords: LLM-as-a-judge; Large Language Model
Primary area: foundation or frontier models, including LLMs
TL;DR: We introduce SCI-VerifyBench, a cross-disciplinary benchmark, and SCI-Verifier, a reasoning-augmented verifier, to provide systematic evaluation and reliable solutions for answer verification in scientific domains.

Abstract: As large language models (LLMs) are increasingly applied to scientific reasoning, the complexity of answer formats and the diversity of equivalent expressions make answer verification a critical yet challenging task. Existing verification studies in scientific domains suffer from two major limitations: (a) the absence of systematic evaluation standards and insufficient disciplinary coverage, which hinders their comprehensive assessment; and (b) heavy reliance on cumbersome rule design or prompt engineering, which reduces their effectiveness in complex reasoning scenarios or limits their cross-disciplinary generalization. To address these challenges, we propose solutions at both the data and model levels. On the data side, we construct **SCI-VerifyBench**, a cross-disciplinary benchmark covering mathematics, physics, biology, chemistry, and general scientific QA. The benchmark is built from real LLM responses and enhanced with domain-specific equivalence transformations that generate challenging and realistic data. Model-based and expert annotations ensure both quality and diversity, enabling rigorous evaluation of verification ability. On the model side, we emphasize the importance of reasoning for verification and introduce **SCI-Verifier**, a unified reasoning-augmented verifier for scientific domains. Through post-training, SCI-Verifier demonstrates strong logical reasoning and equivalence judgment capabilities while maintaining concise and stable outputs. Together, SCI-VerifyBench and SCI-Verifier provide a principled framework for scientific verification, offering both systematic evaluation and practical pathways to enhance the reliability and applicability of LLMs in scientific domains.

Published as a conference paper at ICLR 2026
SCI- VERIFIER: SCIENTIFIC VERIFIER WITH THINKING
Shenghe Zheng .2,, Chenyu Hu Fanghhen Yu Junchi Yao!.7, Jingqi Ye!.3,
Tao Chen', Yun Luo', Ning Dingl. 1,5 Lei Bai', Gannuu Cui Peng Yel.
Shanghai Artificial Intelligence Laboratory
Harbin Institute of Technology
3 Fudan University 4 The Chinese University of Hong Kong 5 Tsinghua University
6 CUHK- Shenzhen 7 University of Electronic Science and Technology of China
8 University of Science and Technology of China
ABSTRACT
As large language models (LLMs) are increasingly applied to scientific reasoning, the complexity of answer formats and the diversity of equivalent expressions make answer verification critical yet challenging task. Existing verification studies in scientific domains suffer from two major limitations: (a) the absence of systematic evaluation standards and insufficient disciplinary coverage. which hinders their comprehensive assessment; and (b) heavy reliance on cumbersome rule design or prompt engineering, which reduces their effectiveness in complex reasoning scenar- ios or limits their cross- pross-disciplinary generalization. To address these challenges, we propose solutions at both the data and model levels. On the data side, we construct SCI-Verify Bench, a cross-disciplinayy benchmark covering mathematics, physics, biology, chemistry, and general scientific QA. The benchmark is built from real LLM responses and enhanced with domain-specine equivalence transformations that generate challenging and realistic data. Model-based and expert annotations ensure both quality and diversity, enabling rigorous evaluation of verification ability. On the model side, we emphasize the importance of reasoning for verification and introduce SCI-Verifier, unified reasoning-augmentt mains. Through post-training, SCI- Verifier demonstrates strong logical reasoning and equivalence judgment capabilities while maintaining concise and stable outputs. Together, SCI- Verifyrrrryeee and SCI- Verifirr provide a principled framework for scientific verification, offering both systematic evaluation and practical pathways to enhance the reliability and applicability of LLMs in scientific domains.
verifier for scientific do-
INTRODUCTION
As large language models (LLMs) become increasingly prevalent in scientific reasoning (Yang et al., 22255] Ren et al., 2025 Liu et al. 2024a Bai al.) (025), ensuring the reliability of their outputs has emerged as a critical challenge (Chang et al. 2024 Liu et al., 2025). Scientific reasoning often involves intricate multi-step processes and wide range of equivalent answer formulations (Chen ct al. 2025b), posing substantial difficulties for answer verification. The essence of verification lies in accurately determining whether an output of LLM is equivalent to the reference answer, task that serves both as the foundation for evaluating capabilities of LLMS and as a key bottleneck to further advancement (Zhang et al., 2025a Chen al., 2025a Zhang et al.) (0256))
Despite recent progress, verification research in scientific domains still faces two major challenges. First, high-quality and systematic benchmarks are lacking. Existing benchmarks cover only a narrow range of scientific disciplines and fail to account for discipline-specific equivalence forms (Liu et al.) 2024b; Li et al., 2025 Yan et al. 2025 Chen et al. 2025a), making it difficult to comprchensively evaluate verification capabilities. Second, current methods are limited in complex reasoning scenarios and lack cross-disciplinary applicability. Rule-based approaches rely on manually crafted templates and heuristics, which are labor-intensive and insufficient for complexity (Hynek Kydlfček 2024 Gao
jet al. 2021). More recently, general LLMs or specialized verifiers leverage LLM generalization to
achieve promising results (Li et al. 2023 Liu al. 2025; Zhang al. 2025b Luo et al. 2023
"Equal Contribution. Work done during the internship at Shanghai AI Lab.
Project: SCI-Verifier
1 "oorrsponndig Author

---

Published as a conference paper at ICLR 2026
Quaed 1is becuut deeet sp A bitte
Quied l leteu Queet boaae
22
Question:
Fedi pos ine sadethar costo kullt i di the Aufef, eel Aurfecrfec are befl look right or book clrose.
oD) (000
Gold Ans:
X Verifier (thinking]:
Vaiiday Cleck The carild ais seee for complte naternatiaal iri vilil vissor
Compartion
(newinng ug
LLM Ans:
rono one mambova excpe as mmad be poettie insger
Cascldate sanver a (-m. 53 (S totach mahoralcait
C U) (S.m)
Equiearaie Aealysk The prtbles witu fhe poolier longem sarisfingg Recraad for The soriad
a S-epp escladio and prodtis
longer,
Verifier (no thinking):
postre aagges pucpeard) a velhou sdding or croenting any valack... so ogan dene mtaffed.
Dovasttan Ths caskuac's
cetaso adiiiin ectoal
Figure 1: (a) and (b) show the performance gains with reasoning enabled on VerifierBench and VerifyBench-hare hard, respectively: (c) presents a case where reasoning leads to the correct judgment.
Liu et al. 2023ap, but they require extensive prompt engineering, produce unstable outputs, and still struggle with complex reasoning and cross-disciplinary tasks. To overcome these limitations, we propose a solution from both the data and model perspectives, with the dual goals of cstablishing a systematic evaluation framework and designing robust verifier tailored to scientific reasoning.
On the data side, we introduce SCI- OCI-VerifyBench, a cross-disciplinrry and highly targeted scientific verification benchmark. We first collected over 100K responses from LLMs across mathematics, physics, biology, chemistry, and general scientific of domains, question types, and answer formats. To increase both difficulty and specificity, apply domain-specific equivalence transformation to some samples, enriching the test set with challenging cases. These transformations include, but are not limited to, formula rewrites, and logical equivalence substitutions, simulating error patterns that verifiers are likely to encounter in real- world scientific scenarios. Finally, by combining godel-generrted labels with expert human annotations, we ensured the quality and diversity of the data, making SCI-VerifyBenr cross- cooss-disciplinaryy and rigorous for evaluating verification capabilities. In benchmark evaluations, we are surprised to find that reasoning abilities, which are often overlooked by many current specific verifiers, can significantly enhance the model's performance on scientific verification tasks.
tasks, ensuring broad coverage
a benchmark that is both
At the model level. we propose SCI-Verifier domains. In Fig. D we highlight the critical role of reasoning in scientific verification. Enabling ofain-of-Thougee (CoT) across models consistently boosts judgment accuracy. Motivated by this observation, SCI- Verifier adopts a two-stage post-training pipeline that combines supervised fine- tuning and reinforcement learning incorporate logical reasoning into the verification process. This design enables it to handle complex equivalence judgments and multi-step reasoning while maintaining concise and stable outputs for deployment. It assesses equivalence from multiple perspectives while maintaining concise and stable outputs for practical deployment. Experimental results show that SCI- Verifier substantially improves accuracy on challenging and easily confusable samples compared to current verifiers and exhibits stronger drossddisciplinary generalization. Notably, the 8B version of SCr-Verifier achieves verification performance on par with the current state-of-the- art closed-souree model GPT-5 (OpenAl 2025b).
gaasoning-augmentt verifier tailored for scientific
In summary, this work makes three key contributions as following:
We propose a cross-disciplinary. high-challenge benchmark for scientific verification, SCI- VerifyBench, which covers mathematics, physics, biology, chemistry and general question-answe fields. Using real LLM responses and domain-specific equivalence transformations, it evaluates verification performance in complex scenarios and sets unified standard for LLM assessment.
We design a reasoning-enhaneee high nigh-performance scientific verifier, SCI-Veritier. By integrat- ing logical reasoning via supervised post-training. SCI-Verifier gains the capability to perform complex equivalence judgments and conduct multi-step scientific reasoning, thereby significantly outperforming existing verification models across multiple domains.
Extensive experiments show that SCI-Verify Bench and SCI- Verifier together provide precise eval- uation framework and practical guidance for improving LLM capabilities, reliability, and reasoning in scientific domain, setting new standard for cross-disciplnaary verification research.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORKS
The unstructured LLM outputs makes the verification of the answers challenging, motivating efforts to construct benchmarks for evaluating the verifiers. VAR (Chen et al. 2025a) evaluates 19 LLMs on 24 datasets train and assess x Verrffy VerifyBench (Li et al. 2025) covers general, logical, mathematical reasoning. and VerifierBench (Liu et al. 2025) has 4,000 expert- level questions across STEM domains. VerifyBench (Yan et al. (025) aggregates model outputs with manual meta-error analysis across math, science, knowledge, and general reasoning tasks. Existing works face two main issues: (1) pointless samples, such as multiple-choice questions that require no specially designed verifiers, and (2) limited disciplinary coverage that restricts generalization assessment of scientific domain. To address these problems, we introduce SCI- SeIl-erifyBert spanning mathematics, physics, chemistry, biology, and general QA, with filtered tasks and expert annotations to enable rigorous evaluation of scientific verification.
Verification Benchmark.
Verification Models. The verifier compensates for gaps in rule-based answer evaluation. x Xeer- ify (Chen et al. 2025a) is efficient but lacks reasoning. limiting performance; General- Verifier (Ma) al. (2025 has partial reasoning capabilities for cross-domain equivalence assessment. Compass Veri- fier (Liu ct al. 2025) aims to provide efficient, high--peffrmmance, and robust answer verification using carefully designed error templates. Existing verifiers, constrained by limited reasoning capabilities, are inadequate for complex scientific reasoning, while using general models as verifiers requires careful prompt design with unstable outputs. Then, we propose SCI- Verifier, reasoning-augmentte scientific verifier offering strong reasoning with concise, stable outputs.
Reward Models. Reward models differ from verifiers in that they rank response quality, while verifiers assess correctness. Prior work primarily follows discriminative paradigm, outputting scalar score directly (Ouyang et al. 2022; Snell et al. 2025). More recent approaches leverage reasoning capabilities to enhance reward model performance. For example, J1 (Whitehouse et al. 2025) proposes an RL framework for training Thinking -iaiing-LLM----
models; Think-J (Huang et al. 2025) introduces offline and online RL-based methods for judgment thinking optimization; Conpass-Judger2 (Zhang et al.) 2025b) uses verifiable rewards and rejection sampling to guide critical reasoning, improving robustness and generalization. Despite these advances, reward models aim to rank response quality rather than verify correctness, and this difference in objective introduces new challenges for data construction and training strategies.
SCI-VERIFYBENCE
Current research on scientific verification faces a major bottleneck in the lack of com- prehensive and rigorous benchmarks, which creates blind spots in evaluating LLMs' scien- tific reasoning capabilities and guiding their training. To address this problem, we con- struct SCI- Verify systematic cross- disciplinary benchmark covering mathematics, physics, chemistry, biology, and general scientific QA, designed to comprehensively evaluate the verification abilities of verifiers. We first present the characteristics of SCI- SCI-Verify VerifyBench in Sec. 3.] followed by the construction pipeline in Sec. 3.2
Table 1: Comparison of verification benchmarks.
Espiivleence Transformation
Difneulty
Scale
Domains
Control
VerifyBench Verify BericyBenchha hard
2000
X
x
1000
3
X
/
VerifierBench
2817
X
x
SCI- Seriiterity Beraf Beach
2500
5
/
/
3.1 DATA OVERVIEW
Table 2: Benchmark Statistic.
We construct SCI- Verifyyench to assess verifiers' scien- tific verification capabilities. this part, we compare it with existing benchmarks and present static data analyses. Tab. [ higghlights the key differences, showing that SCI- VerifyBench spans wider range of domains, incorporates more challenging yet commonly encountered equivalence transformations, and applies difficulty control mechanisms to better reflect realistic scientific reasoning while increasing overall task complexity. Tab. presents the static analysis results, with further details provided in Appendix A.
Statistic
Number
Total Data (each domain)
500
Real QA (each domain)
350
Synthetic QA (cach domain)
150
Average Answer Tokens
24.98
Average Response Tokens
2980.43
3

---

Published as a conference paper at ICLR 2026
SCI-VerifyBench
Equivalent Representation
Q: What solvents are used in the reaction.... ...?
Chemistry
Question Collection
Data Generation
Data Annolation
Data Filter
Gold Ans: The salvents for this reaction are CO. Evaluative
Name
Conversion
Equivalent Ans:
monoxide.
Question & Generato
Anurdaaii Anower Label
Fiber by Difficale
Q: Compute the red number salislying.
Answer
Anowe
LLM Answer
X X
Filtered Data
Evaluative Ans: V3
Gold V
Numerical Equivalence
Math
Physies
VVVV
vilter by Cormettens
Equivalent Ans: 34,
LLMs
Annolator
Belance Answer Label
Q:
does the lens need moved...?
Physies Biology
Math
Gold Ans:
Unit Conversion
SCI-
Evaluative Ans: -0.04
VerifyBench
Equivalent Ans: -0.000000000000 m.
Chemistry
Q: Please generate the protein sequenec... ...?
Babanan Answer Label
Gold Ans: GMD...
Equivalent Representation
Biology
Evaluative Ans: GMD...
Label
Equivalent Ans: Clty Met--ppt.......
Equivalent Answer
Diliinilyy
Filtered Data
Q: What natural phecomenoon Equivalent
rainbows?
QA
Ganuzate Anawe
Annocate AESPNCE
Fiher by Carmetman
Evaluative Ans: Light refraction.. Representation
Gold Ans: Light refraction.
QA
Equivalent Anst Bending sunnlgght.
Figure 2: SCI- SCI-VerifyBench construction pipeline (left) and cases of Equivalent Representation (right).
3.2 DATA COLLECTION
Question-Answer Data Generation. Comprehensive scientific verification requires coverage of diverse question areas, answer types, and response formats. To achieve this, we collect over 15k question-answer pairs across mathematics (Gao et al., (025), physics (Zheng et al.] (025). chemistry (Alampara et al., (024), biology (Jiang et al.j 2025), and general QA (Wang et al. 2024 Wei et al. (024), and generate 100K+ responses using eight models of varying scales, while controlling response length. This design allows the verifier to adapt to different question and answer styles. The prompts used for data generation are provided in the Appendix A.
Synthetic Data Generation. Current verifiers perform well on responses identical to reference answers. However, even carefully trained specialized verifiers can fail when confronted with complex, domain-specific equivalence transformations, which are common in real-world scientific reasoning. Fig. Dillustrates typical equivalence transformations across mathematics, physics, chemistry, biology, and general QA. To address this, we select 500 representative questions from each domain that allow for answer equivalence transformations and generate five equivalent answers for each using the methodology described in Appendix A. During this process, five LLMs assist in assessing the quality of generated equivalences. If the equivalence is clearly invalid and multiple models agree, the sample is discarded and regenerated. This approach increases the challenge for verifiers and closely simulates the diverse answer forms encountered in realistic scientific reasoning scenarios.
Data Annotation. The previous two pathways produced both real and synthetic question-ane answer data. To ensure annotation quality, we adopt a hybrid approach combining LLMs and human experts. First, five LLMS evaluate the accuracy of generated or synthetic answers against the reference answers as shown in Appendix [A.2] To reduce human effort and maintain dataset difficulty, we retain only samples where the five LLMs disagree. From the data in the two methods described above, we select 2,500 samples with the highest model disagreement (500 per domain) for human annotation. Each sample is assessed by least two experts with bachelor's degree or higher, and answers are considered equivalent if they can be transformed into each other. In cases of disagreement, third expert consulted, ensuring labels reflect true equivalence while maintaining diversity.
Data Filter. Using the above procedure. we obtained a dataset comprising 5,000 annaanttd samples and large collection of L.nM-anottaeed samples. We partitioned the data into a training set and test set, with the latter corresponding to SCI- CCI--VrifyBBnch For the test set, we sampled 350 real LLM responses and 150 equivalence-bant synthetic responses per domain. The selection criterion required full agreement among human experts, while samples with disagreement between human experts and LLMs are preferentially included to increase difficulty. This process results in a test set of 2,500 samples in total. The remaining non-overlapping data formed the training set, where only portion was human-annan annotated and the majority relied on LLM annotations. Samples with substantial disagreement among L.MMs were filtered out ensure label reliability, yielding a training set of 14K
4

---

Published as a conference paper at ICLR 2026
samples. Both the training and test sets can be expressed in the following format:
D {(q,, ds, rs; 4))==
(1)
where zi denotes the question, o, denotes the reference answer, rs denotes the response whose correctness needs to be evaluated, and 4 denotes the label, which can be either true or false.
4 SCI-VERIFIER
In this section, we develop a sassoning-aummenteen verifier for scientific verification. First, Sec. 4.1 presents the motivation and necessity for incorporating reasoning capabilities into scientific verifica- tion. Then, Sec. 4.2 describes the approaches for integrating reasoning through supervised fine-tuning (SFT) and reinforcement learning (RL). This design emulates human step-by-step reasoning and improves verification reliability and robustness.
4.1 MOTIVATION
We begin by motivating the introduction of reasoning capabilities into scientific verification. While Chain-of-Thought (CoT) has been widely recognized for enhancing model performance across various domains, most existing verifier studies have overlooked this aspect. As shown in Fig. ! we evaluate models of different scales under two conditions: outputting only the final answer versus producing intermediate reasoning before the answer. The results demonstrate that reasoning brings substantial gains in scientific verification, largely because responses of scientific questions inherently complex and often involve multiple equivalent forms. Reasoning is thus essential for assessing equivalence from different perspectives. Building this insight, argue that optimizing reasoning for scientific verification can significantly boost verifier performance. Next, we introduce SCI-Verifier, a unified verifier designed to deliver concise yet powerful reasoning capabilities.
4.2 BAST-TRAINING
In this section, we present our approach to enhancing the reasoning ability of models for scientific verification. While reasoning essential, the nature of verification requires reasoning paths to as concise as possible to minimize resource consumption. Accordingly, with short and stable outputs. Based on these characteristics, we utilize a two-stage post-training paradigm that combines supervised fine-tuning (SFT) with reinforcement learning (RL).
aim for lightweight verifier
Supervised Fine- Tane-Tunngg (SFT). this stage, we employ large models with rejection sampling to generate diverse of reasoning paths in structured format shown in Appendix A.2 We then perform strict filtering to retain only valuable and concise traces. For reasoning models, we keep only the conclusive summary, while for non-reasoning models, discard overly long unstructured responses. The filtered reasoning paths are used to fine-tune a smaller model, effectively injecting the essential reasoning ability with minimal overhead. The training objective defined as follows:
LsrT (0)
=-Rqg.u)>Daw log mo xx)
where DSFT denotes the curated training dataset of high-quality reasoning traces. Unlike SFT in other domains like mathematics physics where the focus is mostly output formatting, verification SFT centers on transferring domain-specific verification knowledge to small models, which is essential for developing concise and useful reasoning for verification.
Reinforcement Learning (RL). After SFT, the model has basic verification ability and can follow the required output format, but prone to overfitting,
DAPO (Yu al.) (025). refined GRPO (Shao et al. 2024) variant that filters both overly easy and overly hard samples and adds length penalty encourage concise reasoning. The training objective is to maximize:
lo:1
(8) = E(g.a)_,, (0:)9 m (+l0)
Σ
Hor
min (rr,(0) Are, clip(r..(f), Clow + Chigh) An.))
5

---

Published as a conference paper at ICLR 2026
Table 3: Performance of different verifiers on SCI-VerifyBench. Specialized verifiers use default prompts, while all other models are allowed to use reasoning.
Models
Physies
Chemistry
Biology
QA
Total
Avg. Token
Closed-souree Models
GPT-5 (OpenAl AL
2025b
90.0
89.0
85.4
84.8
95.4
88.92 87.56
384.59
Gemini 2.: Flahh Comanici et
2025
90.2
81.8
86.0
88.2
86.6
478.48
04-mini OpenAl
88.8
84.0
84.6
86.6
90.8
86.96
437.27
Open-source Instruct models
Qwen2.5 5-7stutntt (Qwen et al
2025
90.2
80.6
77.8
80.6
82.0
82.24
400.40
Qwen3-30H-- --3em3--0--------- Ins.Inssttl--a 2507 Wwen
2025
79.6
88.4
85.6
684.58
MaMMa-33--70B-nstttee Grattafiori
76.6
67.4
78.8
84.6
85.8
78.64
364.36
pen-source Reasoning Models
Qwen3-4B Yane et al.
88.2
77.2
81.4
83.8
83,4
82.80
1466.92
Qwen3-8b Yang
2025
90.0
76.2
81.8
83.0
81.4
82.48
1033.07
GPT-oss-200 OpenA
2025
85.0
70.0
81.0
84.6
72.4
78.60
522.55
Qwen3-30B -308-A38-10
1113-30B-33355
Thiiknn 2507 Yang et al. 2025
82.2
70.4
86.0
88.4
84.8
82.36
1714.66
GPT-oss- 120B OpenAA
2025
79.8
69.8
86.2
87.0
90.0
82.66
110.21
Wwen3-235B-A72B Yang et 2025
82.4
67.0
87.8
91.2
79.8
81.64
4601.16
Specific
Verifiers
XVenff-888 (Chn
77.8
CompassVer Vermer--BB
Liu 21.
2025
86.2
79.4
80.2
86.6
87.0
83.88
192.00
Verifir-e
Liu ct 0 2025
87.4
82.0
84.0
84.6
87.2
85.04
162.34
Compass Verifier-32B
Liu al
2025
90.0
84.0
85.4
89.8
86.24
212.04
Ours
SCI- 1BII-eeiiier-41 SCI- SCI-Verifier-8B
92.4
84.6
86.4
94.2
93.4
90.20
485.13
93.8
90.4
87.8
96.4
95.2
92.72
490.66
where (0)) mo or, 18,00 ke)
Here, Ty and To,y denote the updated and previous policies.
The advantage function As, is calculated from the final reward R₁;
A Rs mean(
(t
The final reward R, sum of the alignment reward Ranggn and overlong penalty Povertong.
R = Ralign, + Povertong.i
(5)
where Ralign, is if the final prediction for example i matches the ground-truth answer, and otherwise, the overlong penalty is defined as:
if 0g S Lmax
P P
loi| loil-Lae Lonz
Apenally
if Lmax < |ool I Lmax + Lbaffer
(6)
Louiter
Apenally
if 0 > Lmnxx + Loatter
Here, lo:] is the length the response, Laux is the maximum allowed length, Lbailer is the overlong buffer length, and Xpenalty is the penalty weight. Since verification is binary classification task, imbalanced data may lead the model to rely on label priors instead of reasoning. To address this, we rebalance the dataset during RL training ensure equal positive and negative examples.
Through this two-stage post-training paradigm, we obtain a verification model with concise reasoning ability, applicable across domains for scientific answer validation. This approach not only improves the reliability of model capability evaluation, but also provides reward function with clear semantics, thereby facilitating the training of stronger reasoning-oriented language models.
EXPERIMENTS
5.1 BASELINES AND SETUP
We conduct systematic evaluation of SCI- VerifyBench on SCI-Verifier-4B and 8B, which are trained
from Qwen3-4B, 4B-Baas (Yang et al. 2025) and Qwen3-8B-Ban (Yang et al.) 2025), respectively. In addition, we benchmark on two established datasets: VerifierBench (Liu et al. 2025) and VerifyBench- hard (Yan al. 2025). The baselines cover four categories: (1) closed-source models, (2) open-source
instruct models, , (3) open-source reasoning models, and (4) specialized verifiers. Details are provided since positive and negative samples are balanced by construction. On VerifierBench and Vericy hard, we additionally report F1 score alongside Accuracy. In all cases, higher values indicate stronger verification performance.
in Appendix A.2 For evaluation, report Accuracy

---

Published as a conference paper at ICLR 2026
Table 4: Performance of different verifiers on VerifierBench and VerifyBench-Harte Specialized verifiers use default prompts, while all other models are allowed to use reasoning.
Models
VerifierBench
Verify Bench-Hard
Acc.
F1
Avg. Token
Acc.
F1
Avg. Token
Clored-source Models
GPT-5 OpenAA
2025b Comanici et al.
91.80
90.48
203.45
90.40
85.34
245.64
Gemini 2.5 Flash
2025
87.63
87.56
265.49
87.70
83.65
302.65
Open-souree Instruct models
Qwen2.5 72B-Instruct @wwn et al
2025
82.61
81.67
550.73
85.20
81.31
381.27
1.13-300-AA3-Instmmme 2507 Owen of
2025
88.78
88.88
972.30
88.70
85.03
810.24
-. Grattafion et al.
2024
79.84
79.00
398.99
85.20
81.10
382.04
Upen-Soure
Reasoning Models
Qwen3-4B Yang et al.
2025
84.42
84.54
2119.87
83.40
78.80
1755.61
Qwen3-8b Yang al. 2025
85.55
85.56
1857.95
84.40
79.58
1588.45
GPT-oss-201 (OpenAl
2025
83.36
83.73
523.10
85,90
80.36
328.50
Qwen3 11133-3B-A3111 hinking- 2507
(ang al. 2025
90.42
90.05
2438.52
88.60
84.92
2226.46
Swen3-235B-A11 Yang et al.
2025
88.36
88.01
5044.43
86.80
82.26
4690.13
Specific Verifiers
Verify-SB (Cen et al. 2025a
78.03
75.53
1.00
83,20
79.60
1.00
CompassVeriteers
Al.
2025
82.39
83.37
1.00
86.60
84.16
CorpassVeriffer--P Al.
85,56
84.83
1.00
87.50
84.13
Compass Verifee-erifier32222
Ltu al.
2025
39.88
88.91
1.00
88.30
85.86
Ours
SCI-Verifier-4B
92.37
92.01
703.47
88,90
85.98
470.26
SCI-Verifier-80
93.01
93.06
636.53
90.30
87.45
393.61
5.2 EVALUATION AND ANALYSIS OF OCI-VERIFYBELE BENCH
In this part, present and analyze the evaluation results SCI- SCI-VerifyBench. Tab. reports the performance of both closed-source and open-source models on SCI-VerifyBench This comprehensive evaluation enables to compare the verification capabilities of LLMs of different types and scales under the same settings. We then provide a detailed analysis of the experimental results.
Open-source models are gradually closing the gap with proprictary models, yet noticeable performance gap remains. On the verification task, many open-source models have approached the performance of closed-source models, including specialized verifiers, but proprietary models still maintain an edge. For instance, GPT-5 outperforms current open-source models by more than 5%. Notably, our proposed SCI-Verifier achieves performance comparable to GPT-5 on the scientific verification task, which confirms the effectiveness of the proposed verifier.
Reasoning models and chat models do not exhibit significant differences on this task. On this task, reasoning models show no clear advantage over chat models. We attribute this to the fact that, unlike chal- lenging problems such as IMO-level mathematics, scientific verification tasks are straightforward, quiring domain-specific knowledge and only brief reasoning. Since both model types share similar pri- ors and lack reasoning-specific optimization, perfor- mance gains limited. This observation under- scores the need for reasoning tailored to the unique characteristics of verification tasks.
QA
Physies
Blology 100
Chemistay
GPT-5
Gerrini-5..Rech
Equivalence-based answers poses significant chal- lenges for current LLMSs. As shown Fig. 3 on our sewgivalence-uugmente VerifyBenche even state-of-the-art GPT-5 models per- form poorly, with scores dropping below 60% in mathematics and physics. This highlights clear deficiency in handling complex equivalence transformations. Remarkably, our SCI- Verifier, both its 4B and 8B configurations, achieves substantially higher performance the same tests, owing to targeted optimization for this challenge. These results provide strong evidence for the effectiveness of integrating reasoning capabilities specifically tailored for equivalence verification.
GPT-cs 120B
CoupessVeillees
SCI-Verifier-11
SCI-Verifiee-B
test set derived from SCI-
Figure 3: Evaluation on Equivalent Answer.
7

---

Published as a conference paper at ICLR 2026
SCI-Narly-
10
Plyder
SCE-Nwaly-nn
Crequs
Vortaas32
Vorlae-n
Oreakdil
Urodarta
065-120n 1280
Chosiity
Oweet
QA
sVerty-it
-Taake
GF1400031 Jastratt
76
Model Billiii parameten))
SCI-VvttyBBeeh
Yerilooteekk
Yesdly Brads-are
(a)
(b)
(c)
Figure 4: (a) Performance on SCI- Benyeerffyeecch versus model size. (b) Difficulty comparison across domains in SCI-VerifyBnnch. (c) Ablation study of training methods.
Model scale does not have decisive impact on results. Experiments across model scales show that increasing model size does not consistently improve performance (Fig. a)). We attribute this to the fact that verification mainly relies on prior knowledge for answer equivalence, and current models are not specifically optimized for this objective, so larger capacity does not yield better performance.
Task characteristics across domains lead to domain-dependent performance differences. As shown in Tab. Lah.Baa and Fig. hb), performance varies across disciplines but follows consistent trends across models. Mathematics and physics achieve lower scores due complex transformations such as Taylor ex pansioon, whereas other subjects involve more straightforward judgments once prerequisite knowledge is available. These findings suggest the need for discipline-specifie verifiers.
5.3 EVALUATION AND ANALYSIS OF CEI-VERFEEER
Generalization of SCI- We conduct experiments on our SCI-VerifyBench and two existing verification benchmarks. VerifierBench and VerifyBench-Hard shown in Tab. 4 The results demonstrate that, both sizes of SCI- Verifier achieve strong performance even at small sizes, reaching levels comparable to the state-of-ther ofettheart closed-source model GPT-5. Mcanwhile, Fig. demonstrates the strong capability of SCI- Verifier in judging equivalence transformations. The consistent advantage of SCI-Vent ECI-Verifier across all three benchmarks indicates its strong verification ability and generalization capability across tasks. Notably, on XCI-VerifyBe Benih, SCI- Verifier outperforms current open-source models in all disciplines, further validating its coos-disciplinary generalization in verification.
Prompt Robustness of SCI- Verifier. We investigate the robustness of SCI- Verifier to property
Table 5: Comparison of model robustness across different prompts. our: default prompt; other: modified prompt. Models
prompt variations, that is crucial for real-world ap- plications where prompts must often be adapted to user re- quirements (Liu et al.) (202bb) We evaluate multiple models three benchmarks using both our proposed CoT prompt and the x Verify prompt (modified to al- low reasoning for alignment pur- poses). Details are provided in Appendix A.2, and the results are summarized in Tab. 5 From these results, we draw two key conclusions: (I) SCI-Verifier exhibits strong robustness to prompt modifications, maintaining competitive performance even when the prompt differs from those seen during training: and (2) general models are considerably more sensitive to prompt variations in verification tasks, largely because they lack an intrinsic notion of answer equivalence and must instead rely on contextual cues. Notably, this sensitivity tends to diminish as model size increases.
ACI-VerifyBende our other
Verify Bench-Hard
other
Qwen3-30B-A33
84.40
82.92
88.70
75.40
GPT-oss-20b
78.60
79.08
85,90
79.50
Qwen3-235B-A22B
81.64
79.40
86.80
81.00
CompassVerifice Verificr-BB
83.88
81.72
86.60
79.30
Compuss Verifier-7B
85.04
84.84
87.50
80.30
Compuss Verifier-3BB
85.04
86.00
88.30
84.70
SCI- Verifier-BB SCI-Verifie--8B
90.20
89.80
88.90
88.30
92.72
91.90
90.30
89.70
5.4 ABLATION STUDY
Training Methods. In this section, we analyze the contribution of each component in our two- stage training framework. Experiments both 4B and 8B models (Fig. 5(a)) show that SFT alone
8

---

Published as a conference paper at ICLR 2026
RM-FFT
RAGALL
OureSFT
COT
via COT
COT
Auaraay
Verifylleec--lted
SCI-Voity/Becch
Vediiyyeoch-llar
SCT-NweifyBboch
(a)
(b)
(c)
Figure 5: (a) Ablation study of training data impact. (b) Ablation study of SFT distillation methods.
(c) Ablation study of training with CoT in scientific verification.
already yields strong verification performance, indicating that supervised adaptation effectively equips the model with basic task-specific reasoning ability. Starting RL from reasoning model also achieves competitive results, whereas directly applying RL to the Base model performs poorly, likely because the absence of SFT warm-up limits the acquisition of targeted reasoning skills. In contrast, combining SFT and RL consistently delivers the best overall performance, particularly in cross-dataset generalization, demonstrating that the two stages play complementary roles and are both crucial to the final effectiveness.
Training Data. this part, we evaluate the quality of our constructed training dataset by comparing it with a commonly used dataset (RM) (Zhao et al.) 2025) in the Reward Model domain. Using Qwen3-4B-Ban as the initial model, we conduct experiments with both datasets under SFT and SFT+RL settings, and the detailed results are presented in Fig. b). The results show that our dataset consistently enables the model to achieve strong performance across three benchmarks, whether used for SFT RL. This demonstrates the high quality of our data, from which the model can learn richer distributional information about the verification task. The RM dataset also yields reasonable performance under SFT, mainly because of its large scale with more than 180K samples. However, its effectiveness under RL is limited since the heterogeneous quality within such a large dataset slows down model improvement, which makes data filtering necessary in practice. These findings confirm that our constructed training dataset, like our test data, is of high quality and reliability.
Distillation Data. We investigate the effectiveness of our proposed short CoT distillation. Specifically, we compare the outcomes of distilling complete CoT versus short CoT, with results presented in Fig.5(b) The findings reveal that distilling complete CoT only fails to improve performance but also substantially increases output length, rendering it impractical. We attribute this to the nature of the verification task, which is relatively simple and does not require long reasoning chains. Instead, concise reasoning from fixed perspectives is sufficient to achieve strong performance. Therefore, distilling short reasoning traces during the SFT stage is both reasonable and efficient choice.
Inference Mode. In this part, we investigate the impact of incorporating reasoning capabilities on model performance. We compare models trained with and without reasoning modes using the same training data, with results shown in Fig. S(c). We find that omitting chain-of-thought leads to more efficient inference but results in a substantial performance drop. This clearly demonstrates the importance of incorporating reasoning abilities for verification tasks in scientific domains.
CONCLUSION
We highlight verification as a critical step toward advancing the scientific reasoning capabilities of LLMs. To this end, we introduce SCI-VerifyBench, a high-quality and diverse benchmark spanning mathematies. physics, chemistry, biology, and commonsense scientific QA tasks, designed to rigorously and systematically assess models' cross-disciplinary scientific verification capabilities. Our study further demonstrates that chain-of-thought reasoning is essential for scientific verification, particularly when answers are complex or admit multiple equivalent forms. Building this insight, we develop SCI-Verifier a verifier endowed with concise reasoning abilities specifically tailored for verification tasks. Together, SCI- VerifyBench and SCI-V Verifiri provide both comprehensive evaluation framework and a practical solution for scientific verification, offering strong potential to guide the continued advancement and reliability of L.MSs in scientific reasoning.

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the Shanghai Artificial Intelligence Laboratory and a locally commis- sioned task from the Shanghai Municipal Government.
REFERENCES
Nawaf Alampara, Mara Schilling- Wilhelmi, Martiño Ríos-García, Indrajeet Mandal, Pranav Khetarpal, Hargun Singh Grover, N. M. Anoop Krishnan, and Kevin Maik Jablonka. Prob- ing the limitations of multimodal language models for chemistry and materials research. arXiv preprint arviv: 2411 1. 16955, 2024.
Lei Bai, Zhongrui Cai, Maosong Cao, Weihan Cao, Chiyu Chen, Haojiong Chen, Kai Chen, Pengcheng Chen, Ying Chen, Yongkang Chen, et al. Intern-sl: A scientific multimodal foundation model. arXiv preprint wxXiv:250.11760 2025.
Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang. Yidong Wang, et al. A survey on evaluation of large language models. ACM transactions on intelligent systems and technology, 15(3):1-45, 2024.
Ding Chen, Qingchen Yu, Pengyuan Wang, Wentao Zhang, Bo Tang, Feiyu Xiong, Xinchi Li, Minchuan Yang, and Zhiyu Li. xverify: Efficient answer verifier for reasoning model evaluations. arXiv preprint arXiv: 10481, 2025a.
Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang. Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards reasoning era: A survey of long chain-of- thought for reasoning large language models, 2025b. https:////// http:p/s///aaxvi.rg//abbbba wrt.://///aa//////oo
09567
Gheorghe Comanici, Eric Bieber, Mike Schackermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06266 2025.
Bofci Gao, Feifan Song, Zhe Yang. Zefan Cai, Yibo Miao, Qingxiu Dong, Lei Li, Cheng- hao Ma, Liang Chen, runxin xu, Zhengyang Tang, Wang Benyou, Daoguang Zan, Shang- haoran Quan, Ge Zhang, Lei Sha, Yichang Zhang, Xuancheng Ren, Tianyu Liu, and Baobao Chang. Omni-math: A universal olympiad level mathematic benchmark for large language models. In Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (eds.), In- ternational Conference on Representation Learning, volume 2025, pp. 100540-100569, htt.aiccr...ccrrrreeiiee http://r.....pppopeeeeeeeee////a
2025. httpt//s/pppceeeee.........pppps
Leo Gao, Jonathan Tow, Stella Biderman, Shawn Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jasmine Hsu, Kyle McDonell, Niklas Muennighoff, et al. A framework for few-shot language model evaluation. Version v0. 0. Sept, 10:8-9, 2021.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, and Kadian. The laama herd of models, November 2024.
Hui Huang, Yancheng He, Hongli Zhou, Rui Zhang, Wei Liu, Weixun Wang, Wenbo Su, Bo Zheng, and Jiaheng Liu. Think-j; Learning to think for generative llm-as-a-judge, 2025. URL https
//arxiv.org/ abs /200s./5555..0 14268
Greg Gandenberger Hynek Kydlíček. GitHuub huggingfacce/atth A robust mathematical ex- pression evaluation system designed for assessing Large Language Model outputs in mathematical tasks., 2024. URL httpthttt//// / thub. cow www./ouggii..ooooooomoooooooo.oo ing facee Mat -
Jiyue Jiang, Pengan Chen, Jiuming Wang. Dongchen He, Ziqin Wei, Liang Hong. Licheng Zong, Sheng Wang, Qinze Yu, Zixian Ma, et al. Benchmarking large language models on multiple tasks in bioinformatics nlp with prompting. arXiv preprint 20l::55030041..3 2025.
10

---

Published as a conference paper at ICLR 2026
Xuzhao Li, Xuchen Li, Shiyu Hu, Yongzhen Guo, and Wentao Zhang. Verifybench: A systematic benchmark for evaluating reasoning verifiers across domains, 2025. URL httpp:://axx:t
org/abs/ 2507. 09884
Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen. Making language models better reasoners with step-aware verifier. In Proceedings of the 61st Annual Meeting the Association for Computational Linguistics ( (Volume I: Long Papers), pp.
Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10. 10.1633311/2023 acl-long.29) URL htt.hhttss:.. / a heteeaantthllooooo logy wwt. acl ong 291/
Hanmeng Liu, Ruoxi Ning, Zhiyang Teng, Jian Liu, Qiji Zhou, and Yue Zhang. Evaluating the logical reasoning ability of chatgpt and gpt-4. arXiv preprint WxXi:2234.033399 2023a.
Junnan Liu, Hongwei I Liu, Linchen Xiao, Ziyi Wang. Kuikun Liu, Songyang Gao, Wenwei Zhang, Songyang Zhang, and Kai Chen. Are your ilms capable of stable reasoning? arXiv preprint arXiv:2412.13147. 2024a.
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM computing surveys, 55(9):1-35, 2023b.
Shudong Liu, Hongwei Liu, Junnan Liu, Linchen Xiao, Songyang Gao, Chengqi Lyu, Yuzhe Gu, Wenwei Zhang, Derek F Wong. Songyang Zhang, et al. Compassverifier: A unified and robust verifier for Ilms evaluation and outcome reward. arXiv preprint arXiv:2508.0300 2025.
Yantao Liu, Zijun Yao, Rui Min, Yixin Cao, Lei Hou, and Juanzi Li. Rm-bench: Benchmarking reward models of language models with subtlety and style. arXiv preprint arXiv:2410.16184, 2024b.
Zheheng Luo, Qianqian Xie, and Sophia Ananiadou. Chatgpt factual inconsistency evaluator for text summarization. arXiv preprint arXiv:2303. 15621, 2023.
Xueguang Ma, Qian Liu, Dongfu Jiang, Ge Zhang, Zejun Ma, and Wenhu Chen. General-reasoner: Advancing Ilm reasoning across all domains, 2025. WRRLhttps::0 http://wwwwwvv.vaa////o//
14652
OpenAl. Introducing OpenAl o3 and 04-mini, April 2025a. URL https://openai.com
hndd://nttroicccnn--o--aaaa
OpenAI. Introducing gpt-5, August 2025b. Lntroducing-gpt-
URI.
OpenAI, :, Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K. Arora, Yu Bai, Bowen Baker, Haiming Bao, Boaz Barak, Ally Bennett, Tyler Bertao, Nivedita Brett, Eugene Brevdo, Greg Brockman, Sebastien Bubeck, Che Chang, Kai Chen, Mark Chen, Enoch Cheung, Aidan Clark, Dan Cook, Marat Dukhan, Casey Dvorak, Kevin Fives, Vlad Fomenko, Timur Garipov, Kristian Georgiev, Mia Glaese, Tarun Gogineni, Adam Goucher, Lukas Gross, Katia Gil Guzman, John Hallman, Jackie Hehir, Johannes Heidecke, Alec Helyar, Haitang Hu, Romain Huet, Jacob Huh, Saachi Jain, Zach Johnson, Chris Koch, Irina Kofman, Dominik Kundel, Jason Kwon, Volodymyr Kyrylov, Elaine Ya Le, Guillaume Leclere, James Park Lennon, Scott Lessans, Mario Lezcano-Casado, Yuanzhi Li, Zhuohan Li, Ji Lin, Jordan Liss, Lily, Liu, Jiancheng Liu, Kevin Lu, Chris Lu, Zoran Martinovic, Lindsay McCallum, Josh McGrath, Scott McKinney, Aidan McLaughlin, Song Mei, Steve Mostovoy, Tong Mu, Gideon Myles, Alexander Neitz, Alex Nichol, Jakub Pachocki, Alex Paino, Dana Palmie. Ashley Pantuliano, Giambattista Parascandolo, Jongsoo Park. Leher Pathak, Carolina Paz, Ludovic Peran, Dmitry Pimenov, Michelle Pokrass, Elizabeth Proehl, Huida Qiu, Gaby Raila, Filippo Raso, Hongyu Ren, Kimmy Richardson, David Robinson, Bob Rotsted, Hadi Salman, Suvansh Sanjeev, Max Schwarzer, D. Sculley, Harshit Sikchi, Kendal Simon, Karan Singhal, Yang Song, Dane Stuckey, Zhiqing Sun, Philippe Tillet, Sam Toizer, Foivos Tsimpourlas, Nikhil Vyas, Eric Wallace, Xin Wang, Miles Wang, Olivia Watkins, Kevin Weil, Amy Wendling, Kevin Whinnery, Cedric Whitney, Hannah Wong, Lin Yang, Yu Yang. Michihiro Yasunaga, Kristen Ying, Wojeiech Zaremba, Wenting Zhan, Cyril Zhang, Brian Zhang, Eddie Zhang, and Shengjia Zhao. gpt-oss- opt-oss-120b & gpt-oss-20b model card 2025. URL https::::: //arx/ax//// www.rrwwwv..rrogoooommmmo wbwgg//bb///// /508. 2tt.//1000880221.51

---

Published as a conference paper at ICLR 2026
Long Ouyang, Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang. Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730- 27744, 2022.
Qwen, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang. Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang. Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, January 2025.
Shuo Ren, Pu Jian, Zhenjiang Ren, Chunlin Leng, Can Xie, and Jiajun Zhang, Towards scientific intelligence: A survey of llm-based scientific agents, 2025. URL
abs/
2503.24047
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits mathemat- ical reasoning in open language models, 2024.
Charlie Victor Snell, Jachoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute opti- mally can be more effective than scaling parameters for reasoning. In The Thirteenth International Conference on Learning Representations, 2025.
Yubo Wang, Xueguang Ma. Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj. Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi- task language understanding benchmark, Advances in Neural Information Processing Systems, 37: 25266,95290, 2024.
Jason Wei, Nguyen Karina, Hyung Won Chung, Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, and William Fedus. Measuring short-form factuality in large language models. arXiv preprini arXiv:2411.00 000e:2111..444 1.04668, 2024.
Chenxi Whitehouse, Tianlu Wang, Ping Yu, Xian Li, Jason Weston, Ilia Kulikov, and Swarnadeep Saha. J1: Incentivizing thinking in judgge via reinforcement learning, 2025. URL
org/ abs 2505.10320 10320
Yuchen Yan, Jin Jiang, Zhenbang Ren, Yijun Li, Xudong Cai, Yang Liu, Xin Xu, Mengdi Zhang, Jian Shao, Yongliang Shen, Jun Xiao, and Yueting Zhuang. Verifybench: Benchmarking reference- based reward systems for large language models, 2025. URL https: www.aaaiivvoooo org/ /bs// 25C5.15801
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint 0093:8850, 2025.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint arXiv:2503. 14476, 2025.
Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Aurojit Panda, Jinyang Li, and He He. Reasoning models know when they'r re right: Probing hidden states for self-verification. arXiv preprint 00w:25044.5419, 2025a.
Taolin Zhang, Maosong Cao, Alexander Lam, Songyang Zhang, and Kai Chen. Compassjudger--: Towards generalist judge model via verifiable rewards, 2025b. URL htt.tttss//ariiiv.oorr
abs/2507.090 09104
Yulai Zhao, Haolin Liu, Dian Yu, SY Kung, Haitao Mi, and Dong Yu. One token to fool llm-as-a- judge. arXiv preprint arXiv:2507.087.. 2025.
Shenghe Zheng, Qianjia Cheng, Junchi Yao, Mengsong Wu, Haonan He, Ning Ding, Yu Cheng, Shuyue Hu, Lei Bai, Dongzhan Zhou, et al. Scaling physical reasoning with the physics dataset. arXiv preprint arXiv:2506. 2025.
12

---

Published as a conference paper at ICLR 2026
Appendix for SCI-Verifier
A DETAILS OF SCI- VERIFYBENCH
In this section, we introduce details of the process of constructing SCI-VerifyBench, including the prompts and models used for data generation described in Sec. A.1 the model annotation process and the prompts and parameters designed for practical use described in Sec. A.2 as well as the data details and sample cases in SCI- SCI-VerifyBench described Sec. A.3
A.1 DATA GENERATION
The data generation involves two parts. The first part uses LLMs to generate answers for existing questions, and correctness is determined by comparing the generated answers with the reference answers. The second part generates equivalent answers based on the characteristics of different subjects, testing whether the model can correctly recognize these equivalent forms. For both parts, multiple LLMs are used to generate candidate answers, including Qwen3-32B, Thinking-2507, Wwwem3--0B--A3--nsttrut--------- 32B-Instruct, Gemma-3-27b-it, and Qwen3-8B. The prompt used in the first part is shown in Box. A. For the second part, different prompts are used for each subject according to the corresponding task. The Math prompts refer to Box. A.2 to Box. A.4 The Physics prompts refer to Box. A.5 to Box. A.7
Wwen3-30B-A3B Inntttttttt GPT-oss-20B, Qwen2.5
The Chemistry prompts refer to Box. A.8 to Box. A.16 The Biology prompts refer Box. A.17 to
Box. A.2T
The QA prompts refer to Box. A.22
Box A.1: Prompt for generating LLM response
Please answer the problem adhering to the following rules: 1. Please LaTeX format to represent the variables and formulas used in the solution process and results. 2. Please put the final answer(s) in boxed ]. note that the unit of the answer should be included in boxed ). 3. If the problem requires multiple answers, list them in order, each in a separate (answer) (answer)
boxed }. Problem:
Box A.2: Prompt for generating equivalent answers to mathematical interval problems
You are given mathematical interval answer. Generate 10 different equivalent forms of this interval, using transformations such as:
Rewriting as incqualities: |] <<
Rewriting as set operations: (0, U 00.3]
Open/closed interval limit definitions: b). = lim,-0t [a c,b c]
Converting numeric sets: 2] f0, 1,2] (for integer endpoints)
Format: Output exactly 10 forms. Each form must be wrapped in LaTeX \boxed(...).
Separate each answer with a newline (In).
Example:
Input: [0, 1]
Output:
0 0<x51
(0, 11
0.0001, +
x R 1
0, 0.25, 0.5, 0.75,
Only output the converted results, do not output the conversion process! Now, process the following answer: (answer).
13

---

Published as a conference paper at ICLR 2026
Box A.3: Prompt for generating equivalent answers to mathematical expression prob- lems
You are given mathematical expression or numeric answer. Generate 10 different equiva- lent forms of this expression, using transformations such as:
Factoring or expansion: x² + + - -x (x+
Leaving fraction unsimplified: (2) (1) -
Fraction simplification:
(unchanged)
Partial fraction decomposition: 1/(x(x - + 1/z
Fraction to decimal conversion: 1/2 0.5
Trigonometric identities: sin x: + cos x =1
Trigonometric transformations: sin 2x cosr
Taylor expansion: sin 2 x² x²//3
Exponential/logarihnn rules: In(ab) Ina Inb
Substitution; let y=x+1, then x² + 2x + 1= y²
Approximating special constants: π = 3.14159, = 2.718
Angle-radian conversion: T/6 30
Format: Output exactly 10 forms. Each form must be wrapped in LaTeX \boxed(...). Separate each answer with newline (In). Example:
Input: (x + 1)2
Output:
(xr + () + 1)
x² + 2x +
(x + 1)s
x + 1
Let y x + 1.
(x + 1)2 + 2x; for small x
Only output the converted results, do not output the conversion process! Now, process the following answer: (answer).
14

---

Published as a conference paper at ICLR 2026
Box A,4: Prompt for generating equivalent answers to mathematical equation problems
You are given a mathematical equation. Generate 10 different equivalent forms of this equation, using transformations such as: Moving terms: x + + x
Mulliplying/dividing both sides: 2r 4 + -
Factoring: x² = 0 + (x - 1) +1 =0
Completing the square: x² 6r + 5
(x 333 -4=0
Root extraction: x² - =
Substitution in equations: +1 leta theni + 1 =
Trigonometric transformations: sin x
cos x
Taylor expansion for approximate solution: sin æ r x ~2 0 Domain restrictions: Vz - = requires æ 2 1
Format: Output exactly 10 forms. Each form must be wrapped in LaTeX \boxed(...). Separate each answer with newline (In). Example:
Input:
x² 1 = 0
Output:
x²
()
x = +1
(r 0)²
cos² f sin 2 (substitution cos
Only output the converted results, do not output the conversion process! Now, process the following answer: (answer).
Box A.5: Prompt for generating equivalent answers physies numerical problems
Given the following question and an answer in expression form, generate 10 different equiva- lent forms of this interval, using transformations such as:
Arithmetic operations or evaluation (e.g.. + +)))
Substitution of variable values (e.g., 2x + 3,z = -
Scientific notation, fractions, or decimals (e.g., 0.00012 - 1.2 x 10-1)
Unit conversion or dimensional adjustment (e.g., 1km + 1000m)
Dimensional conversion (e.g., 1N + 1kg - m/s²)
Format: Output exactly 10 forms. Each form must be wrapped in LaTeX \boxed(...). Separate each answer with a newline (In). Example:
Input: [0.1]
Output:
0 x $
00, 1)
0.0001, 1 + 0.0001)
R <
0, 0.255 0.5, 0.75,
Only output the converted results, do not output the conversion process!
Now, process the following answer: Question: (question). Answer (Numeric): (answer),
15

---

Published as a conference paper at ICLR 2026
Box A.6: Prompt for generating equivalent answers to physies expression problems
Given the following question and an answer expression form, generate 10 different equiva-
lent forms of this interval, using transformations such as:
Factoring or expansion: x² + 2x
(x 1)
Fraction simplification: (I² -22=1)/xx+)) +
---
Leaving fraction unsimplified: (2- (2 + (unchanged)
Partial fraction decomposition: 1/(x(x 1)) - 1/x 1/(x 1)
Fraction to decimal conversion:1/2-1 0.5
Trigonometric identities: sin x + cos x =
Trigonometric transformations: sin 2x c
Taylor expansion: sinz = 2 x /3!
Exponeniial/ogarrihn rules: In In(ab)
Ina Inb
Substitution; let + 1, then x²² + 2xr = y²²
Approximating special constants: π Rs 3.14159, e RJ 2.718
Angle-radian conversion: 7/6 = 30
Dimensional conversion (e.g., 1NV 1kg m/s²)
Format: Output exactly 10 forms. Each form must be wrapped in LaTeX \boxed(...). Separate each answer with newline (In). Example:
Input: (z + 1)2
Output:
(x + (x + 1)
x² + 2x +
(x + 1)3
xr + 1
Let y x + 1
(x + 1) 1 1+ 2z; for small
Only output the converted results, do not output the conversion process! Now, process the following answer: Question: (question). Answer (expression): [answer),
16

---

Published as a conference paper at ICLR 2026
Box A.7:
Prompt for generating equivalent answers to physies equation problems
Given the following question and an answer in expression form, generate 10 different equiva- lent forms of this interval, using transformations such as:
Moving terms: x +
+ x
Multiplying/divdding both sides: 2r 4
Factoring: x² = 0 + (x - 1) + =0
Completing the square: x² 6r + 5
(x+ - 4
Root extraction: x²²
+ +2
Substitution in equations: _ +1=0 leta theni 1 = 0
Trigonometric transformations: sin x = cos x
Taylor expansion for approximate solution: sin x r x R 0
requires I > 1 Dimensional conversion (e.g., F = ma, 1000g, 2m/s² F= 2N) Format: Output exactly 10 forms. Each form must be wrapped in LaTeX \boxed(...).
Domain restrictions: / (a - 1)
Separate each answer with newline (ln). Example:
Input: x²
1
Output:
(r (2 + 0
+1
(x 0)2
cos 2 0 - sin
(substitution x cos 0)
Only output the converted results, do not output the conversion process!
Now, process the following answer: Question: {question). Answer (equation): (answer)
17

---

Published as a conference paper at ICLR 2026
Box A.8: Prompt for generating equivalent answers to chemical solvent prediction problems
Given chemical reaction and a predicted solvent, generate 10 equivalent forms of the solvent answer, considering:
Different chemical names (IUPAC, common, trivial)
Abbreviations (e.g., EIOH)
Molecular formulas (e.g., C2H50H)
SMILES representations
Solvent class equivalence (e.g., polar protic, polar aprotic)
Mixture equivalences (e.g., FiOH:H2O 1:1 = H20:FIOH 1:1)
Format: Output exactly 10 forms, each wrapped in LaTeX \boxed[...) and separated by
newline.
Example:
Input: Ethanol
Output:
Ethanol
FiOH
C2H5OH
CCO
alcohol
polar protic solvent
EIOH:H2O 1:1
H2O:E:OH 1:1
ethyl alcohol
C-C-O
Only output the converted results, do not output the conversion process!
Now process: Reaction: [question] Predicted solvent: (answer).
18

---

Published as a conference paper at ICLR 2026
Prompt for generating equivalent answers to chemical property prediction problems
Box A.9:
Given molecule and a predicted property value, generate 10 equivalent forms, using:
Unit conversions (e.g., Celsius ++ Kelvin ++ Fahrenheit)
Ranges vs single values
Approximate vs exact
Different notations (e.g., logP, Kow)
Descriptive labels (e.g., high solubility)
Format: 10 forms wrapped in LaTeX \boxed(...).
Example:
Input: Water boiling point 100 C
Output:
100 C
373 K
212
100 101 C
high boiling point
approx. 100
373. 15K
0.1 x 10 OC
100 Celsius
water boils at 100
Only oulput the converted results, do not output the conversion process! Now process: Molecule:
(question) Property: (answer),
19

---

Published as a conference paper at ICLR 2026
Box A.10: problems
Prompt for generating equivalent answers to chemical yield prediction
Given chemical reaction and a predicted yield, generate 10 equivalent forms:
Percentage ++ decimal
Ranges ++ single value
Descriptive labels (e.g., high yield, trace)
Approximate expressions
Format: 10 forms, LaTeX \boxed(...). newline separated.
Example:
Input: 85%
Output: 85%
0.85
high yield
80 90%
approx. 85%
y 0.85
yield 80%
quantitative yield
major product
trace product
Only output the converted results, do not output the conversion process!
Now process: Reaction: qquestion Predicted yield: [answer),
20

---

Published as a conference paper at ICLR 2026
Box A.11: lems
Prompt for generating equivalent answers to chemical retrosynthesis prob-
Given target molecule, generate 10 equivalent retrosynthesis answers:
Different valid disconnections
Different reagents or reducing oxidizing agents
Alternative synthetic routes (single-step or multi-step)
Protecting group alternatives
Functional group stereochemical equivalence
Format:
forms wrapped in LaTeX \b.xed(...).
Example:
Input: Target= Benzyl alcohol
Output:
Benzaldehyde + NaBH4 + Benzyl alcohol
Benzyl chloride + NaOH - Benzyl alcohol
Benzaldehyde reduced by LiAIH4
Benzyl bromide KOH Benzyl alcohol
Benzaldchyde H2 Pd/C - Benzyl alcohol
C6H5CH2CI NaOH -> C6H5CH2OH
Alternative protecting group: Boc route
Multi-step oxidation-reduction route
Reductive amination route to same alcohol
Electrochemical reduction route
Only output the converted results, do not output the conversion process!
Now process: Target molecule: (question] Retrosynthesis answer: (answer),
21

---

Published as a conference paper at ICLR 2026
Box A.12: Prompt for generating equivalent answers to chemical temperature predic- tion problems
Given reaction and predicted temperature, generate 10 equivalent forms:
Different units (?C, K, *F)
Ranges Vs single value
Descriptive labels (low, room temp, high)
Common lab descriptions (ice bath, reflux)
Format: LaTeX \boxed(...] 10 forms.
Example:
Input: 80
Output:
80 C
353 K
176 SF
70 90 "C
approx. 80
reflux
room temperature
ice bath
moderate heating
high temperature
Only output the converted results, do not output the conversion process!
Now process: Reaction: {question] Predicted temperature: (answer).
22

---

Published as a conference paper at ICLR 2026
Box A.13: Prompt for generating equivalent answers chemical product prediction problems
Given reactants and reaction conditions, generate 10 equivalent forms of the predicted product:
Different structure representations: SMILES, InChl, molecular formula
IUPAC name, common name, trivial name
Stereoisomers (R/S)
Tautomers, salts, hydrates
Alternative valid representations (chair/boat conformers)
Format: 10 LaTeX \boxed(...] outputs.
Example:
Input: Nitrobenzene
Output:
C6H5NO2
clcco(cc)]N+[[[O]]]]
Nitrobenzene
Benzene nitro compound
C6H5-NO2
Nitrobenzol
C6H5NO2 H2O
aromatic nitro compound
benzene derivative
Racemic mixture if applicable
Only output the converted results, do not output the conversion process!
Now process: Reactants: {question] Predicted product: [answer),
23

---

Published as a conference paper at ICLR 2026
Box A.14: Prompt for generating equivalent answers chemical mol2caption problems
Given molecular structure, generate 10 equivalent textual descriptions:
IUPAC name
Common/trivial names
Chinese name
Functional description (solvent, reagent, drug)
Usage property description
Format:
LaTeX \boxed(...I. 10 forms.
Example:
Input: C2H5OH
Output:
Ethanol
EiOH
alcohol
ethyl alcohol
common solvent
disinfectant
flammable liquid
reagent in reactions
soluble in water
Only output the converted results, do not output the conversion process!
Now process: Molecule: (question) Mol2caption answer: (answer).
Box A.15: Prompt for generating equivalent answers to chemical caption2mol problems
Given a textual description of a molecule, generate 10 equivalent molecular representations:
Different SMILES strings (canonical and non-canonical)
InChI
Molecular formula
Graphical structure representation (if feasible)
Different naming conventions resolved to the same structure LaTeX \boxed(...), 10 forms.
Format:
Example:
Input: "alcohol used for disinfection"
Output:
C2H50H
CCO
OCC
IIChI-ISCCC660001235H22222..
ethanol
ErOH
ethyl alcohol
common lab alcohol
solvent for reactions
Only output the converted results, do not output the conversion process!
Now process: Caption: question Predicted molecule: answer,
24

---

Published as a conference paper at ICLR 2026
Box A.16: problems
Prompt for generating equivalent answers chemical name conversion
Given molecule name in one format, generate 10 equivalent forms:
IUPAC <> trivial/common name
English ++ Chinese name
SMILES
InChI
Hydrate/salt forms if applicable
Synonyms and international spelling variants
Format: LaTeX \boxed(-..). 10 outputs.
Example:
Input: Acetic acid
Output:
Acetic acid
ethanoic acid
CH3COOH
C2H402
Vinegar acid
Acetic acid, glacial
Acetic acid aqueous solution
Sodium acetate form
Acid CH3COOH
Only output the converted results, do not output the conversion process!
Now process: Name: (question) Predicted conversion: (answer)
25

---

Published as a conference paper at ICLR 2026
Box A.17: Prompt for generating equivalent answers to biological protein inverse folding problems
Given the following question and an answer in protein sequence form, generate 10 different equivalent forms of this sequence, using transformations such as:
Synonymous sequences folding into the same structure
Conservative amino acid substitutions (e.g., Lys +> Arg, Asp ++ Glu)
FASTA format plain sequence string
One-letter code ++ three-letter code
Sequence with small tolerated mutations that preserve structure
Adding/removing header lines in FASTA
Using lowercase uppercase amino acid letters
Introducing gap symbols to indicate alignment but preserving structure Representing sequence JSON array form
Grouping amino acids by domains/regions but preserving the overall sequence
Format: Output exactly 10 forms. Each form must be wrapped in LaTeX \boxed(.....
Example:
Input: MKAILVVLLYTAA
Output:
MKAILYYLLYTAA
>sq(InMKAIVVVLLYTAA
mkailvvll ytaa
MKAILVVI-YTAA
['M','K' "'"'''''''""""""" 'L' 'V, YY,
MKAII.WYLLYY(STTAAA
MKAILVVILY RAA (Arg substitution)
[Domainl: MKAILV, Domain2: VLLYTAA]
MKAILVVLLYTAA (unchanged)
Only output the converted results in LaTeX
boxed.... do not output the conversion process!
Now, process the following answer: Question: (question) Answer: (answer).
26

---

Published as a conference paper at ICLR 2026
Box A.18: Prompt for generating equivalent answers biological protein structure prediction problems
Given the following question and an answer in protein structure form, generate 10 different equivalent forms of this structure, using transformations such as:
PDB format ++ mmCIF format
3D atomic coordinates ++ Ca-only coordinates
3D structure contact map <+ distance matrix
Cartesian coordinates ++ internal torsion angles (ø, 3/. X)
Secondary structure representation (helix/sheet/loop) instead of full 3D
A Attenaatve but equivalent chain numbering or atom ordering
Superimposed structures with RMSD < threshold
JSON or graph-based adjacency representation of the structure Searse-grained models (e.g., backbone only)
Visual encodings (e.g., dot-bracket for secondary structure, schematic diagrams)
Format: Output exactly 10 forms. Each form must be wrapped in LaTeX \boxed(.....
Example:
Input: PDB file with Ca coordinates helix
Output:
ATOM CA ALA 0.000 0.000 0.000
loop helix helix sheet
contact map: (3.6).
distance matrix: [[O, [[0.1.2...11112220.......
torsion angles o -00, v 45
mmCIF equivalent entry
JSON: "
graph: nodes=7, ((gp6sc77
heaabbonn-oolyyyppeeeetttttt
cartoon schematic:
helix symbol
Only output the converted results in LaTeX \boxed(...). do not output the conversion process! Now, process the following answer: Question: (question) Answer: (answer),
27

---

Published as a conference paper at ICLR 2026
Box A.19: Prompt for generating equivalent answers to biological transformed agent- clinic problems
Given the following question and answer in agent action/trajectory form, generate 10 different equivalent forms of this solution, using transformations such as:
Different but equivalent action sequences leading same outcome
Merging multiple actions into macro-actions
Splitting macro-actions into finer- aneerggriiedd steps
JSON ++ tabular + natural language action logs
Reordering independent actions without changing outcome
Adding no-op (null) actions that do not affect outcome
Representing state transitions instead of actions
Graph or tree representation of the policy trace
Summarizing actions high-level clinical outcome description
Encoding with symbolic tokens instead of natural language
Format: Output exactly 10 forms. Each form must be wrapped in LaTeX \boxedf.....
Example:
Input: Give drug -> Measure blood pressure >
Stop treatment
Output:
Give drug A Measure BP + Stop treatment
Action1, Action2, Action3
[Give drug A, Measure blood pressure, Stop treatment]
MacroAction: Treat+Monitor
State transitions: SO S1 + S2
Add no-op: Give drug A + Wait + Measure BP + Stop
JSON: "actions": Give drug A", "Measure BP", "Stop")
Table: 1. Give drug A 2. Measure BP 3. Stop
Graph representation: nodes=states, edges=actions
Outcome: Patient stabilized after drug A
Only output the converted results in LaTeX (b.xed(...). do not output the conversion process! Now, process the following answer: Question: question Answer: answer""".
28

---

Published as a conference paper at ICLR 2026
Box A.20: Prompt for generating equivalent answers to biological rna structure predic- tion problems
Given the following question and an answer in RNA structure form, generate 10 different equivalent forms of this structure, using transformations such as:
Dot-bracket notation ++ CT format <+ BPseq format
Base-pairing list ++ adjacency matrix representation
2D secondary structure ++ 3D atomic coordinates
Minimal free energy structure + near-optimal structure
Adding base-pair probabilities annotation
Grouping stems/loops/bulges into modular blocks
JSON array representation of base pairs
ASCII art of RNA fold representation
Graph representation of paired/unpaired nodes
Annotated sequence with paired regions in brackets
Format: Output exactly 10 forms. Each form must be wrapped in LaTeX \boxedf.....
Example:
Input: GCGCUUAGC, Structure: (((..)))
Output:
((...)))
dot-bracket: (((..)))
CT: 9,, 2,288 8, 3 7
BPseq: 2 8,
(19)))()))
matrix: [[0,0.1.........]]]
near-optimal structure (((..)))
ASCII: stom-loop diagram
JSON:
annotated sequence: GCG(CUU)AGC
Only output the converted results LaTeX (boxed(...), do not output the conversion process! Now, process the following answer: Question: (question) Answer: (answer)
29

---

Published as a conference paper at ICLR 2026
Box A.21: Prompt for generating equivalent answers to biological rna inverse folding problems
Given the following question and an answer in RNA sequence form, generate 10 different equivalent forms of this sequence, using transformations such as:
Different sequences folding into the same target structure
++ T replacement (RNA vs DNA notation)
FASTA format plain string sequence
Lowercase vs uppercase bases
Adding alignment gaps without changing fold
JSON or array encoding of sequence
Annotating sequence with regions (stem, loop, bulge)
Replacing synonymous positions with alternative nucleotides that preserve folding Annotated sequence with secondary structure alignment marks
Splitting long sequence into chunks and recombining
Format: Output exactly 10 forms. Each form must be wrapped in LaTeX \boxedf.....
Example:
Input: AUGCGAU
Output:
AUGCGAU
augegau
ATGCGAT
SeqInAUGGGAUU
AU-GCGAU
TATU;GCCCGG.A.UU
stem(AUG) loop(CGAU)
AUGCGAU (unchanged)
JSON: "sequence": "aareeecec""AAACCCCC"" "AUGCGAU"
chunks: AU GCIGGAAA IU
Only output the converted results in LaTeX \boxed(...), do not output the conversion process!
Now, process the following answer: Question: (question) Answer: (answer)
30

---

Published as a conference paper at ICLR 2026
Box A.22: Prompt for generating equivalent answers to QA problems
You are given an answer general question. Generate different equivalent forms of this answer, using transformations such as:
Paraphrasing the text while keeping the meaning the same.
Changing the sentence structure word order.
Using synonyms or alternative expressions.
Expressing the answer a list, table, bullet points if applicable.
Rewriting as formal statements, equations, or logical expressions relevant.
Approximating special constants: R 3.14159, R 2.718
Angle-radian conversion: 7/6 30
Dimensional conversion (e.g.. 1NV 1kg m/s²)
Format:
Output exactly 10 forms. Each form must be wrapped in LaTeX \boxed(.....
Separate each answer with newline (In).
Example:
Input:
"Water boils at 100 degrees Celsius sea level." "
Output:
Water reaches its boiling point at 100CC level.
At sea level, the boiling temperature of water is 100 degrees Celsius.
100"C is the temperature at which water boils at sea level.
Only output the converted results, do not output the conversion process!
Now, process the following answer: Question: (question) Answer (expression): [answer]
A.2 DATA ANNOTATION AND EVALUATION
Next, we describe the configurations and prompts used during data annotation and the actual evalua- tion. In this process, the inputs are the question, the reference answer, and the answer to be evaluated, and the output the correctness of the answer being evaluated. To ensure stable outputs during the experiments, temperature of 0 is used. The prompts with CoT are shown in Box. A.23 the prompts without CoT are shown Box. A.24 and the prompts used in the main experiments to measure prompt stability shown in Box. A.25 The LLMs used during the data annotation process were Qwen3-30B-AM 100-A3---nnttutt255555 GPT- 20BB Qwen2.5-P7 5-BBB In-struct, LLaMa33.3Instruu Instruct, and CompassVeriffer-3333
31

---

Published as a conference paper at ICLR 2026
Box A.23: Prompt of Inference with Thinking
As grading reward model, your task evaluate whether the candidate's final answer matches the provided standard answer. You must first output a detailed step-by-step analysis. then give a final structured judgment. Do not regenerate or improve answers, only compare.
Evaluation Protocol:
1. Reference Standard:
The standard (gold) answer is definitive and always correct.
The question is always valid never challenge
Do not regenerate answers; only compare candidate's 's answer with the gold answer.
2. Comparison Method:
Analyze the question's requirements and the gold answer's structure.
Determine the question requires exact matching or allows equivalence.
Compare ONLY the candidate's 's final answer. Ignore reasoning errors.
Ignore differences in formatting style.
For math expressions: check algebraic equivalence step by step; if uncertain, test numerically at multiple points.
For multiple-choice: only compare the final choice and content.
3. Multi-part Answers:
All parts must match the gold answer exactly.
Partial matches incorrect.
not specified, order may vary. For example,
7: are equivalent.
4. Validity Check:
If incomplete (cut off, unfinished sentence) + Label as INCOMPLETE. If repetitive (looping words/phrases) - Label REPETITIVE explicit refusal (e.g., "I cannot answer... ") Label REFUSAL.
Gives an answer hut then negates at end + Label as REFUSAL.
Any of the above + classify as with the correct error type.
Grading Scale:
A
CORRECT:
Matches gold exactly equivalent (including algebraic/numeric equivalence). numerical values: equivalent if equal under rounding tolerance.
Semantic equivalence allowed.
B
INCORRECT:
Any deviation from gold.
Partial matches for multi-part answers.
C
Invalid answers (must specify error type).
Execution Steps and Output:
Analysis step by step:
1. First check validity (INCOMPLETE, REPETITIVE, REFUSAL).
2. Compare candidate's 's final answer vs. gold answer detail.
Identify strict requirements (e.g., exact match, order, completeness).
Allow tolerances (format differences, equivalent math forms, unsimplified fractions, provide the full answer for completion-lype questions).
Check for equivalences, e.g., [=+1)=-2] and x-2 are equivalent.
2x-7
Consider:
Factoring/cxpansion: + + 1)
Fraction simplification: 2²_1
2=1 +
Partial fraction decomposition, decimals, trig identities, substitutions, etc.
For multiple-choice, answer must exactly match gold or be fully equivalent.
3. Provide thorough reasoning chain, highlighting subtle equivalences deviations.
Final Judgment: A/B/C Here is your task:
<Original Question
<Original Question End>
<Standard Answer Begin> [gold answer] <Standard Answer End>
<Candidate's 's Answer Begin> [llm response) <Candidate's Answer End>
Analysis step by step (not to try solving the problem yourself) and Final Judgment:
32

---

Published as a conference paper at ICLR 2026
Box A.24: Prompt of Inference without Thinking
As grading reward model, your task evaluate whether the candidate's final answer matches the provided standard answer. You must only give final structured judgment. Do not regenerate or improve answers, only compare.
Evaluation Protocol:
1. Reference Standard:
The standard (gold) answer is definitive and always correct.
The question is always valid never challenge
Do not regenerate answers; only compare candidate's 's answer with the gold answer.
2. Comparison Method:
Analyze the question's requirements and the gold answer's structure.
Determine the question requires exact matching or allows equivalence.
Compare ONLY the candidate's 's final answer. Ignore reasoning errors.
Ignore differences in formatting style.
For math expressions: check algebraic equivalence step by step; if uncertain, test numerically at multiple points.
For multiple-choice: only compare the final choice and content.
3. Multi-part mullt--part&o Answers:
All parts must match the gold answer exactly.
Partial matches incorrect.
not specified, order may vary. For example,
7: are equivalent.
4. Validity Check:
If incomplete (cut off, unfinished sentence) + Label as INCOMPLETE. repetitive (looping words/phrases) - Label REPETITIVE explicit refusal (e.g., "I cannot answer...
Label REFUSAL.
Gives an answer hut then negates at end - Label as REFUSAL.
Any of the above - classify as with the correct error type.
Grading Scale:
A CORRECT:
Matches gold exactly equivalent (including algebraic/numeric equivalence). numerical values: equivalent if equal under rounding tolerance.
Semantic equivalence allowed.
INCORRECT:
Any deviation from gold.
Partial matches for multi-part answers.
C
WEWPPLETT//EEEEEIVEEREEE Invalid answers (must specify error type).
Annotation Criteria:
1. First check validity (INCOMPLETE, REPETITIVE, REFUSAL).
2. Compare candidate's 's final answer vs. gold answer in detail.
Identify strict requirements (c.g., exact match, order, completeness).
Allow tolerances (format differences, equivalent math forms, unsimplified
fractions, provide the full answer for completion questions).
Check for equivalences, c.g., [=+1)==-]
25-7 and m are equivalent.
Consider:
Factoring/cxpansion: x² +z +1 1)²
Fraction simplification: 2²-1
2-1
Partial fraction decomposition, decimals, trig identities, substitutions,
For multiple-choice. answer must exactly match gold or be fully equivalent.
3. Only give the final judgment.
Final Judgment: A/B/C Here is your task:
<Original Question Begin> questtion| <Original Question End>
<Standard Answer Begin> (gold answer) <Standard Answer End>
<Candidate's 's Answer Begin> [llm response) <Candidate's Answer End>
Final Judgment:
33

---

Published as a conference paper at ICLR 2026
Box A.25: Prompt of Inference for the evaluation robustness
You diligent and precise assistant tasked with evaluating the correctness of responses. You will receive a question, oulput sentence, and the correct answer. Your task determine the output sentence accurately answers the question based on the provided correct answer. Respond with either [Correct] or [Incorrect]. Special considerations:
1. Multiple Answers: If the output contains multiple answers, evaluate whether later answers modify or correct earlier ones. In such cases, compare the final answer with the correct answer. If the final answer is unclear or incorrect, respond with [Incorrect].
2. Mathematical Problems: If the formats differ but the answers are mathematically equivalent, respond with [Correct].
3. Explicit Options: If the question provides explicit candidate answers, the output will be considered correct if it clearly indicates the correct option's code the correct option's content.
4. No Explicit Options: If the question does not provide explicit options, the output must align with the correct in content and meaning to considered [Correct].
Question: {question)
Output sentence: [llm response) Correct answer: (gold answer) Judgement:
A.3 DATA DETAILS
In this section, present several data cases, focusing primarily examples of the equivalent forms we generated. Box. A.26 shows an equivalent example in mathematics, where the answers in the Outputs will undergo both LLM annotation and human annotation. Box. A.27 presents equivalent example in physics, Box, A.28 in chemistry, Box, A.29 in biology, and Box. A.30 provides
an example for QA questions.
Box A.26: A sample for generating equivalent answers to mathematical problems
Question: Find all functions : Q+ + Q such that:
f(x) + (w) + 2ryl(wy) -(x+)))
f(wy)
Gold Ans:
Outputs: 2
1
for x 3
x
1
a:2
34

---

Published as a conference paper at ICLR 2026
Box A.27: A sample for generating equivalent answers to physical problems
Question: Set a crystal prism with an apex angle of 60", and the optical axis is perpendicular to the main cross-section of the prism. The sodium yellow light is refracted in the prism in the direction of the minimum deviation angle and focused using lens with a focal length of 1 m. What is the separation between the o-ray and e-ray spectral lines?
Gold Ans: 7.16 mm
Outputs:
7.16 mm
7.16 mm
0.00716
7160 pm
7.16 x 10 m
Box A.28: A sample for generating equivalent answers to chemical problems
Question:
I'm looking for the SMILEES version of this IUPAC chemical name.
hwwp://lww...oooooooooooooooo/oo Gold Ans: CCC-AATCCCCCIINCCCCCCCCCCCCCCCCC
Outputs:
CINNGCCNIJCC-CCCNNNCCCCCCCCCCCCC
35

---

Published as a conference paper at ICLR 2026
Box A.29: A sample for generating equivalent answers to biological problems
Question: Please predict the dot-bracket notation of the secondary structure directly from the RNA sequence:
Gold Ans: ((((
Outputs:
)))),
CT: 30, 3 29, 4 28, 5 27, 26, 25, 24, 9 23, 10 22, 11 21, 12 20, 13 19, 14 18, 15 17
Bescq
30, 29, 4 28, 27, 6 26, 7 25, 8 24, 23, 10 22, 11 21, 12 20, 13 19, 14 18, 15 17
pairs=
02220000222011222002220004444414
matrix:
[0000000000000000000000000000000
|0000000000000000000000000000000
....
[0. 10.00000000000000000000000000000
Box A.30: A sample for generating equivalent answers to QA problems
Question: What's the name of the active president of the United States in Season of Motherland: Fort Salem?
Gold Ans: President Kelly Wade Outputs:
The active president of the United States in Season
of Motherland: Fort Salem is named Kelly Wade.
Kelly Wade is U.S. President in Season of Motherland: Fort Salem.
In the first season of Motherland: Fort Salem, the president is Kelly Wade.
Kelly Wade is the incumbent U.S. leader Season of Motherland: Fort Salem. In Season of Motherland: Fort Salem, Kelly Wade serves as the U.S. president.
B DETAILS OF TRAINING
In this section, we present the parameter configurations used during the training process. The training mainly involves two parts: SFT and RL. For SFT, adopt full fine-tuning, and the detailed parameter configurations are shown in Tab. 6 For RL, a modified version of GRPO is used, with detailed parameters also provided in Tab.
36

---

Published as a conference paper at ICLR 2026
C LIMITATIONS AND FUTURE WORK
In this section, we discuss the limitations of our work and outline directions for future research. We propose verifier for scientific verification tasks that demonstrates strong reasoning capabilities, achieving high performance with concise and interpretable reasoning outputs. However, some scenarios demand both high accuracy and extreme efficiency. future work, we plan to leverage the model's explicit reasoning abilities to further enhance its implicit reasoning, allowing it to maintain strong performance even without explicitly generating detailed reasoning steps. This approach could provide significant efficiency gains while preserving the model's reliability and robustness across wider range of scientific verification tasks.
Table SFT Configuations for SCI- Verifier.
Parameter
Value
BF16
True False
Gradient Checkpointing
Learning Rate
5 x 10-5
LR Scheduler Type
Soine_withh
Minimum LR Rate
0.1
Packing
False
Maximum Sequence Length
1024
Maximum Steps
Number of Training Epochs
2
Per Device Train Batch Size
2
Per Device Eval Batch Size
16
GPUs Per Node
4
Number of Nodes
1
Seed
42
Use Liger Kernel
True
Warmup Ratio
0.02
Table RL Configuations for Verifiei..
Parameter
Value
BF16
True
Temperature
1.0
Top
1.0
Clip Ratio Low
0.2
Clip Ratio High
0.28
Max Response Length
2048
Overlong Buffer Len
1024
Learning Rate
1 10
Number of Training Epochs
20
GPUs Per Node
4
Number of Nodes
Seed
42
D THE USE OF LARGE LANGUAGE MODELS
In paper, LLMs are first used to polish the writing, improving the clarity and readability of the manuscript. At the same time, as mentioned multiple times the main text and Appendix, we employ LLMs to generate and annotate training and test data. Since LLM outputs sometimes be unreliable, as noted in the text, all selected data are subsequently manually re-am a--annotaeed by human experts and carefully filtered.
37

---

Published as a conference paper at ICLR 2026
E DETAILS OF SCI- VERIFIER
E.1 CASES OF SCI- VERIFIER
In this part, we present representative cases showing how SCI- SCI-Verifier evaluates model outputs when Chain-of-Thought Thought (CoT) is enabled, making its reasoning trajectory more transparent. We select one example from each domain, and the detailed step-by-step results are provided in Box |.. Box E.5] In this setting, a verifier output of A indicates that it judges the answer to be correct, while an output of B indicates that judges the answer be incorrect.
Box E.I: sample for SCI- VCr-Verieeer generating answers to Math problems
Question: Let ABCD
quadrilateral with A (3, B) B = (9, (9,-
(-5, 1-5, (-7,24). Let P be point in the plane (not necessarily inside the quadrilat- eral). Find the minimum possible value AP + BP + CP DP.
Gold Ans: 16v 8v5
LLM Ans (extracted): 66 + 8y5
Gold Judgment: False
Verifier Judgment: To evaluate whether the candidate's final answer matches the provided standard answer, we will follow the evaluation protocol step by step.
Analysis Step by Step
1. Check Validity: The candidate's 's answer is complete and does not show any signs of repetition refusal answer. Therefore, we can proceed to compare the final answer directly.
Compare Candidate's Final Answer vs. Gold Answer: The candidate's final answer is (66 8V5). The gold answer (16v17 5v/)).
We need to determine if these two expressions are equivalent or if there is a significant difference. Let's break down the components: Both answers contain the term (8v5), which common.
The remaining terms are (66) and (16.
To check for equivalence, we can compare the numerical values of these terms: (66) is constant. (T) is approximately (16 4.123 65.968).
Clearly. (66) and (65.968) are very close in value, but they are not exactly the same. The difference minimal, but is still present.
Provide Thorough Reasoning Chain: The candidate's answer (66 8v5) very close the gold answer (16v/77 + 8v5), but exactly the same. The difference between (66) and (16v17) is approximately (0.032), which negligible in most practical contexts but
zerO.
Final Judgment:
Given that the candidate's final answer (66 + 8v5) very close the gold answer (16/17+
8V5) but not exactly the same, we can conclude that the candidate's answer is incorrect. Therefore, the final judgment is: boxed J.
38

---

1-imine is Published as a conference paper at ICLR 2026
Box F.2: sample for SCI- Verifier generating answers to Physies problems
Question: There are two identical objects, with internal energy given by = NCT, where N is the number of particles in the object and C' is constant. Initially, the temperatures of the two objects are ii and 72, respectively. Now, the two objects act as high and low temperature heat sources drive Carnot engine, and finally, the two objects reach a common temperature Tr. Find the work W done by the Carnot engine.
Gold Ans: NC (T1 + T2 2vT]T2)
LLM Ans (extracted): NC (VTi VTb)
Gold Judgment: True
Verifier Judgment: To evaluate whether the candidate's 's final answer matches the provided standard answer, we will follow the evaluation protocol step by step.
Analysis Step by Step
Check Validity: The candidate's answer complete and does not show any signs of repetition refusal to answer. Therefore, can proceed compare the final answer with the gold answer.
Compare Candidate's 's Final Answer vs. Gold Answer:
The candidate's ' final answer NC (VTi -T?)"
The gold answer is NC (T1 + T- - 2/TTT2
We need to determine these two expressions are equivalent. Let's expand the candidate's
answer:
NC -VT--VV NC (Ti 2 + T2) This can be rewritten as:
NC (Ti +7 -2VTTT5
This is exactly the same the gold answer.
3. Provide Thorough Reasoning Chain:
The candidate's 's answer NC (VTi- v vT²)² simplifies to NC (T) T2 2VTTT2). which
identical the gold answer NC (Ti T2 2vT172).
Final Judgment
Since the candidate's s final answer matches the gold answer exactly, the final judgment is:
[boxed[A]]
Box E.3: sample for SCI- Verifier generating answers to Chemistry problems
Question: How would you express this IUPAC name in SMILES format?
Gold Ans:
The SMILES representation for 2-dizaanylidenn---meeeey
CCIC)CCENNNC=NN
LLM Ans (extracted): CCIC)CC/-N)CCN
Gold Judgment: True
Verifier Judgment: To evaluate whether the candidate's final answer matches the provided standard answer, we will follow the evaluation protocol step by step.
First check validity: The candidate's answer is complete and does not show any signs of repetition or refusal. Therefore, we can proceed to compare the candidate's 's final answer with the gold answer.
2. Compare candidate's 's final answer vs. gold answer:
-The candidate's 's final answer is 'CCIC)CC(=NNJC-N:
The gold answer is also CCIC)CC/NNNCC-N: Since the candidate's 's final answer exactly matches the gold answer, there are no deviations or equivalences that need to be considered. The structure and content of the SMILES notation are identical.
3. Provide thorough reasoning chain: The candidate's 's answer and the gold answer identical, indicating that the candidate has correctly expressed the IUPAC name in SMILES format.
Final Judgment:
boxed[A) CORRECT: The candidate's final answer matches the gold answer exactly.
39

---

Published as a conference paper at ICLR 2026
Box E.4: sample for SCI- Veriferre generating answers to Biology problems
Question:
Please predict the dot-bracket directly
notation
the secondary
LCGUCCCUCCCGGAGAACCCAAAAAAACAACC ALCCUUAAAACAAUGU
structure
from the RNA sequence:
GABUCCCCACCAAUCCCCUUUUUUACCCCCCC Gold Ans:
AUCCACCCCAGGACCUUCCCCCCCC
(C...((((((((........ M)D)..))))
LLM Ans (extracted): dot-bracket:
Gold Judgment: False
Verifier Judgment: To evaluate the candidate's final answer against the standard answer, we
will follow the evaluation protocol step by step.
1. First check validity (INCOMPLETE, REPETITIVE, REFUSAL). The candidate's answer complete and not repetitive or refusing to answer.
2. Compare candidate's 's final answer vs. gold answer in detail:
The candidate's answer is: (i(......(((..(
)..... )..
................................ ( (.....
The gold answer is:
We can see that the candidate's answer has an additional part at the end: ((.....)))).. ()))....)))..................... .............. This part is not present in the gold answer. 3. Provide thorough reasoning chain, highlighting subtle equivalences or deviations. The candidate's 's answer includes an extra section that is not part of the gold answer. This deviation means that the candidate's 's answer does not match the gold answer exactly.
Final Judgment: boxed (B) INCORRECT, because the candidate's final answer includes an additional part that is not present in the gold answer.
Box E.5: A sample for SCI- Verifier generating answers to Scientific QA problems
Question: A packet filter firewall filters at the.
Gold Ans: Network or transport layer
LLM Ans (extracted): network layer
Gold Judgment: True
Verifier Judgment: To evaluate whether the candidate's final answer matches the provided
standard answer, we will follow the evaluation protocol outlined:
First check validity: The candidate's 's answer incomplete, repetitive, refusal. is a clear and concise response.
Compare candidate's 's final answer vs. gold answer detail:
The candidate's s answer is network layer.
The gold answer is Network or transport layer.
The candidate's answer is very close to the gold answer. The only difference is the use of backslash before layer the candidate's 's answer, which is likely a typographical error. The intended meaning is the same the gold answer, which refers to the network layer the transport layer.
3. Provide thorough reasoning chain, highlighting subtle equivalences deviations:
The candidate's answer network layer can be interpreted as referring to the network layer, which is one of the layers in the OSI model. The gold answer Network or transport layer indicates that either the network layer or the transport layer could be correct. Since the candidate's s answer is essentially referring the network layer, it aligns with the gold answer.
Final Judgment:
Given that the candidate's answer essentially correct, with only a minor typographical error, and considering the context of the question, the final judgment is: boxed(A) CORRECT.".
40

---

Published as a conference paper at ICLR 2026
E.2 FAILURE CASES OF SCI-VERIFIER
In this section, we analyze several failure cases of SCI-Verifier to better understand its current limitations and guide future improvements. We find that most errors arise during the equivalence- checking phase, where SCI-Verifier makes incorrect reasoning steps when deciding whether two expressions are equivalent. While such mistakes do not always lead an incorrect final verdict, they reveal clear weakness of the current
In Box E.6 and Box E.7 we present several representative failure cases from SCI-Verifier-BBD with the erroneous reasoning steps highlighted in red. In this setting, a verifier output of A indicates that judges the answer be correct, while an output of indicates that it judges the answer to be incorrect. These errors fall into two categories: incorrect reasoning with correct final answer, and incorrect reasoning that leads to an incorrect final answer. These examples show that, despite our efforts to enhance SCI- Verifier's own reasoning capability. it remains imperfect and still produce flawed intermediate reasoning. This may be related to the capability of the base model. In future versions, plan to explicitly target and mitigate these issues to further improve the robustness and reliability of SCI-Verifier
Box F.6: case where SCI-Verifier reasons wrongly but judges correctly.
How many different collections of letters are there? A letter can appear multiple times in collection. Two collections are equal if each letter appears the same number of times both collections.
Question:
Gold Ans: (54)
LLM Ans (extracted): 2517660288
Gold Judgment: False
Verifier Judgment: To evaluate whether the candidate's final answer matches the provided standard answer, will follow the evaluation protocol step by step.
Step 1: Check Validity
The candidate's 's answer is 2517660288.
There are signs of incompleteness, repetition, refusal in the candidate's response. Therefore, we proceed to compare the candidate's final answer with the gold answer.
Step 2: Compare Candidate's ' Final Answer vs. Gold Answer
The gold answer is ((+)).
The candidate's answer is 25 2517660288,
To determine if these two values are equivalent, we need to calculate ((tt) and compare it to
251 7660288.
(3))
34'
91(34-9)I 05
Calculating this directly can be complex, but we can use a calculator or software to find the
value: (34) 5, 311, 735
However, the candidate's answer is 2517660288, which is significantly larger than 5,31
This indicates that the candidate's s answer does not match the gold answer.
Step 3: Provide Thorough Reasoning Chain
The candidate's answer 25 76602888 is not equal to ($2). which is 5,311,735.
There are no other forms of equivalence (such as different representations of the same number) that would make the candidate's answer correct.
Final Judgment
Since the candidate's final answer does not match the gold answer, and there other forms of equivalence, the final judgment is: (boxed(B).
41

---

Published as a conference paper at ICLR 2026
Box E..: case where SCI-Verifier reasons wrongly and judges wrongly.
Find all pairs of positive integers (I, y) with the following property: a, are relative prime and positive divisors of x² + y² then + - 1 is divisor of x. y (Cyprus) Gold Ans: (2*, 2*), (2 -3 ,,)3),, ). (3*; 2 3*) for non-negative integers k.
Question:
LLM Ans (extracted): (2.2)
Gold Judgment: False
Verifier Judgment:
To evaluate the candidate's final answer against the standard answer, will follow the evaluation protocol step by step.
Check Validity: The candidate's answer complete and does not show any signs of repetition or refusal. Therefore, we can proceed to the comparison.
Comparison Method:
The question asks for all pairs of positive integers(x, y) such that if (a, b) are relative prime and positive divisors of (x + y²), then (a + 1) is divisor of (xx +
The candidate's i final answer is (2, 2). We need to check if this pair satisfies the condition given the question.
3. Detailed Comparison:
For the pair (2, 2), we have (x = 2) and 2). Therefore, (x +y² 23 +23 = 8+8 16).
The divisors of 16 arc(1, 2, 16). Since (a) and (b) are relative primes, the only pairs of divisors that are relative primes are((1, 2)), ((I, ()), 8)), ((1, 16)), (2, 4), (2, 8), ((2, 16)), ((4, 8)). ((4, 16)), and ((8, 16)). However, since (a) and (b) must be relative primes, we only consider ((1, 2)), ((1, 4)), ((1, 8)), and ((1, 16)).
For each pair, we check if (a b-
divisor of 16:
For ((1, 2)), (a + b- 1
2). and 2 is divisor of 16.
For ((1, 4)), (a + 1-=bb1=11+
4), and 4 a divisor of 16.
For ((1, 8)), 1=
8), and 8 is divisor of 16.
For ((1, 16)), (a+b-1=1+ 16 = 16), and 16 is a divisor of 16.
All these conditions are satisfied, so the pair ((2, 2)) meets the criteria. Final Judgment:
The candidate's 's final answer ((2, 2)) is correct according to the standard answer, which includes ((2 2 for non-negative integers (k). Since (k = 0) gives us ((20, 22) = (1, 1)) and (k = gives us ((2) 2 (2, 2)). the candidate's 's answer indeed part of the standard answer.
Therefore, the final judgment is: [oxed(A)
42


Reviews:
[{"review_id": "vFAwQXZZ7f", "reviewer": "Reviewer_w2ZX", "summary": "The paper introduces SCI‑VerifyBench, a cross‑disciplinary benchmark for verifying the correctness of LLM answers in scientific domains (math, physics, chemistry, biology, and general QA), and SCI‑Verifier, a reasoning‑augmented verifier trained via SFT followed by RL (DAPO/GRPO‑style). The benchmark mixes real LLM responses and synthetic, domain‑specific equivalence transformations. The model generates short chain‑of‑thought before issuing a Correct/Incorrect judgment. Experiments show SCI‑Verifier‑8B performs comparably to or better than large closed models on SCI‑VerifyBench and two external benchmarks (VerifierBench, VerifyBench‑Hard), and is robust to prompt variants.\n\nThe paper focuses on an important gap: reliable verification of scientific answers. The dataset construction is well‑documented, and the training recipe is clearly summarized. However, the definitions of equivalence transformations for several tasks appear loose and subjective, potentially giving label ambiguity.", "questions": "See Weakness.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "+ The motivation for reasoning in verification is clear. The authors show consistent gains when enabling CoT on two benchmarks, directly motivating a reasoning‑augmented verifier (Fig. 1 and 5).\n\n+ The cross‑disciplinary benchmark is valuable, and the idea of using targeted equivalence transformations is interesting. The benchmark fills the breadth gap compared to previous works, covering five domains with 500 samples per domain. Concrete equivalence transformation templates are provided (Fig. 2 and Appendix A). \n\n+ The experimental evaluations are comprehensive. The results on SCI-VerifyBench include both closed and open models with specific verifiers. Generalization to VerifierBench and VerifyBench‑Hard is also reported (Table 4). SCI‑Verifier surpasses or matches strong baselines across domains (Table 3), and excels on equivalence‑augmented tests where others degrade sharply (Fig. 3). This directly targets the paper’s core problem.", "weaknesses": "- There may be selection bias in benchmark construction. The test set intentionally prioritizes model‑disagreement samples for human annotation (Sec. 3.2: “retain only samples where five LLMs disagree”), which may skew distribution toward hard/atypical cases. This complicates external validity. Furthermore, “samples with disagreement between human experts and LLMs are preferentially included” in the test set (Sec. 3.2), possibly amplifying adversarial difficulty beyond natural settings. Another potential issue is the extremely long average response length (2,980 tokens; Table 2), which suggests contexts unlike many real deployments, potentially overemphasizing long‑context quirks. \n\n- The definitions of equivalence transformation are loose or subjective in several domains. Chemistry prompts treat descriptors (high boiling point, room temperature, solvent class) as equivalent to numeric and speciﬁc answers, which is semantic similarity rather than strict equivalence. This risks label ambiguity. Biology prompts allow “conservative substitutions/synonymous sequences”, which need not preserve exact answers without explicit structural/functional constraints. QA prompts accept generic paraphrases, where scope/entailment nuances could flip correctness without careful normalization rules. \n\n- The evaluation may lack statistical rigor. Table 3 and 4 report single accuracies without confidence intervals or multi‑seed variance, which is especially important for RL‑trained models. Prompt‑robustness Table 5 presents point estimates only; no variability across runs or bootstrap CIs. Fig. 3 (p. 7) shows sharp drops on equivalence stress tests but omits per‑domain n/CIs, hindering significance assessment. \n\n- Some concerns:\n\nCaptions of Tables 3-4 specify: \"Specialized verifiers use default prompts, while all other models are allowed to use reasoning\". This design may disadvantage specialized verifiers that could also benefit from tailored prompts or concise CoT.\n\nAverage token counts differ drastically across models (e.g., many thousands for some reasoning LLMs vs 1.00 for specialized verifiers; Table 4), implying heterogeneous inference modes and costs that complicate fairness of accuracy‑only comparisons. \n\nWhile Sec. 5.1 references Appendix A.2 for prompt details, it is unclear that each baseline was evaluated under its best prompt setting, especially for specialized verifiers. No direct evidence found in the manuscript. \n\nThe alignment reward (R_{align,i}) (Eq. 5) is not defined operationally. \n\nThe policy ratio (r_{i,t}(\\theta)) in Eq. 3 is never explicitly defined.\n\nThe over‑long penalty uses (-\\infty) for responses exceeding (L_{\\max}+L_{buffer}) (Eq. 6), which is not implementable as‑is; practical handling (e.g., very large negative constants) is unspecified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kpkkJuoQ5z", "reviewer": "Reviewer_9b7g", "summary": "This paper presents SCI-VerifyBench, a cross-disciplinary benchmark covering math, physics, biology, chemistry, and general scientific QA, and SCI-Verifier, a reasoning-augmented verifier. It addresses scientific verification challenges via data (high-quality benchmark) and model (two-stage post-training) solutions, outperforming baselines and matching GPT-5.", "questions": "Please refer to the Weaknesses.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. SCI-VerifyBench uses real LLM responses and domain-specific equivalence transformations, ensuring diverse, high-quality data across 5 disciplines, filling gaps in existing narrow-coverage benchmarks.\n2. SCI-Verifier integrates logical reasoning via SFT and RL, enabling accurate complex equivalence judgments and stable outputs, outperforming rule-based or prompt-dependent methods.\n3. It shows strong cross-disciplinary performance, matches closed-source GPT-5 with 8B parameters, and maintains accuracy under prompt variations, aiding reliable LLM use in science.", "weaknesses": "1. I have doubts about the \"w/ thinking\" setup. RLVR enables large-scale RL training through a rule-based verifier, and specialized verifiers also need to minimize inference costs to prevent training from getting stuck during reward calculation. However, the paper does not quantify the inference efficiency of SCI-Verifier. It remains unclear whether the verifier in thinking mode can meet practical requirements during large-scale RL training.\n\n2. Continuing the previous question, in the process of calculating rewards in RL, what is the role of steps other than Final Judgment, such as Validity Check and Comparison of Final Answers? Are they directly ignored? I suggest the authors conduct small-scale RL experiments to demonstrate the advantages of SCI-Verifier during the training process (refer to [1]).\n\n3. It is recommended that the authors add more verifier cases under the thinking mode to facilitate a better understanding of the verification process of SCI-Verifier.\n\n4. There seems to be an inversion in the citations here: \"VerifyBench (Li et al., 2025) covers general, logical, mathematical reasoning, and VerifyBench (Yan et al., 2025) has 4,000 expert-level questions across STEM domains\". The correct citation should be: VerifyBench (Li et al., 2025) has 4,000 expert-level questions across STEM domains, and VerifyBench (Yan et al., 2025) covers general, logical, mathematical reasoning.\n\n[1] Huang Y, Zeng W, Zeng X, et al. Pitfalls of Rule-and Model-based Verifiers--A Case Study on Mathematical Reasoning[J]. arXiv preprint arXiv:2505.22203, 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Cbzzl3Yx58", "reviewer": "Reviewer_x8GA", "summary": "This paper introduces (i) SCI‑VerifyBench, a cross‑disciplinary verification benchmark built from real LLM responses and domain‑specific equivalence transformations across Math, Physics, Chemistry, Biology, and general scientific QA, and (ii) SCI‑Verifier, a reasoning‑augmented verifier trained with SFT+RL that aims for concise, stable judgments. The test set comprises 2500 human‑verified items (500 per domain) drawn from 15k QA pairs and 100k+ LLM responses; the construction explicitly includes equivalence‑transformed answers to stress verifiers. On SCI‑VerifyBench, an 8B SCI‑Verifier matches or exceeds closed‑source models (e.g., near GPT‑5), and it also performs strongly on VerifierBench and VerifyBench‑Hard.", "questions": "- Human annotation quality. As I mentioned above, please report IAA and adjudication rates for the 2,500 test items; also, clarify annotator backgrounds and per‑domain expertise distributions in detail...\n\n- How do you formally distinguish strict equivalence from approximate/semantic equivalence? Would you consider splitting SCI‑VerifyBench into strict vs approximate tracks to avoid conflating them? \n\n- Could specialized verifiers be run with a CoT‑enabled prompt (when feasible) to check whether the gap persists?\n\n- Maybe I missed something. Do you have results where entire transformation families seen in training are held out in testing? \n\n- Please report wall‑clock latency and cost/token comparisons vs GPT‑5 and CompassVerifier on SCI‑VerifyBench.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The motivation is good. The paper articulates why scientific verification is challenging (multi‑step reasoning, many equivalent forms...) and positions verification as foundational to evaluation and reliability. The narrative is easy to follow. \n\n- The scope is reasonable. SCI‑VerifyBench spans 5 disciplines, filters tasks, and injects equivalence transformations (e.g., unit/name conversions, algebraic rewrites, symbolic identities) to create realistic, hard cases, with a 2500‑item test set that requires unanimous human agreement (but I do have some concerns on this benchmark). \n\n- The authors add CoT to boost accuracy across models; the paper highlights this with Fig.1 (and ablations in Fig.5), and builds SCI‑Verifier around short CoT distilled via SFT and then refined with DAPO/GRPO‑style RL. \n\n- The performance seems promising. SCI‑Verifier‑8B achieves good performance on SCI‑VerifyBench compared with other models like GPT5. On equivalence‑only subsets, many LLMs—including GPT‑5—drop sharply (below 50% in Math/Physics), while SCI‑Verifier remains substantially higher. The authors also show that SCI‑Verifier is less sensitive to prompt changes than general models. Pls report statistical significance or CIs around headline numbers.", "weaknesses": "(also questions for authors)\n\n- The pipeline generates 100k+ responses using eight models, and five LLMs assist in judging equivalences; only 2500 high‑disagreement samples (500/domain) are escalated to human annotation for the test set, while much of the training set relies on LLM labels (≈14k after filtering). Although test items require full human agreement, the selection by model disagreement may be biased toward tricky/ambiguous patterns, and the training labels inherit LLM biases.  Besides, regarding the annotation, please provide inter‑annotator agreement (IAA) or adjudication statistics are reported for the human‑labeled set (beyond the “full agreement” requirement) to gauge label consistency. The benchmark & data generation part is the core contribution in this paper, so this part is very important and must be carefully evaluated and justified.\n\n- What counts as “equivalent” is sometimes approximate, not strictly equal. Several transformation templates explicitly include approximations (e.g., Taylor expansions or “approx. 80°C”, descriptive lab phrases like “reflux”) and semantic paraphrases in QA. These are context‑dependent rather than strictly equivalent and could blur the task definition or inflate ambiguity.\n\n- Method novelty is limited. SCI‑Verifier uses a fairly standard two‑stage SFT + RL recipe with short‑CoT distillation and a DAPO/GRPO‑style objective plus a length penalty—sensible engineering, but incremental. The paper’s main novelty is task focus + dataset rather than a new learning principle.\n\n- Because the equivalence categories used to synthesize data appear in both *training and test pipelines*, there is a risk that models overfit to those templates rather than general principles of equivalence (?). The authors partly mitigate this with human curation for the test set, but a strict “held‑out transformation family” evaluation would strengthen the generalization claim.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RLHPTKF37y", "reviewer": "Reviewer_Hp78", "summary": "This paper introduces a challenging benchmark for scientific verification that spans five distinct subjects and domains. In addition, the authors present a reasoning-based scientific verifier trained on carefully curated datasets, which substantially outperforms existing verification models. Interestingly, they highlight the inherent difficulty of equivalence judgment—an essential aspect of scientific evaluation. This work is well-written and solid.", "questions": "1. Could you conduct an error analysis on the problems that the current verifier fails to handle well? I’d like to understand what major issues still exist in the current verification process.\n\n2. Could you sample some data and conduct a comparative analysis between human evaluations and the verifier’s results?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The writing is clear and easy to follow.\n\n2. The paper highlights the importance of equivalence transformations in answers, identifying them as a crucial and challenging aspect of verification. Once this issue is addressed, the results show remarkable improvement.\n\n3. This work is comprehensive and well-executed, covering data creation, benchmark construction, and verifier training—each of which involves substantial effort—and it achieves outstanding performance overall.", "weaknesses": "1. There is a lack of error analysis for Sci-Verifier, making it difficult to understand what problems still remain.\n\n2. There is a lack of human evaluation. Although Sci-Verifier performs well on the benchmark, it is unclear how consistent its results are with human judgments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)