TaTToo: Tool-Grounded Thinking PRM for Test-Time Scaling in Tabular Reasoning
Authors: Jiaru Zou
Soumya Roy
Vinay Kumar Verma
Ziyi Wang
David Wipf
Pan Lu
Sumit Negi
James Zou
Jingrui He
Venue: ICLR 2026 Poster (2026)
Keywords: Process Reward Model
Tabular Reasoning
Tool Integration
Test-time Scaling
Primary area: foundation or frontier models, including LLMs
TL;DR: A tool-augmented process reward model that improves tabular reasoning at test time.

Abstract: Process Reward Models (PRMs) have recently emerged as a powerful framework for enhancing the reasoning capabilities of large reasoning models (LRMs), particularly in the context of test-time scaling (TTS). However, their potential for supervising LRMs on tabular reasoning domains remains underexplored. 
Through detailed empirical analyses, we identify that existing PRMs, though widely adopted for supervising text-only reasoning steps, struggle with table-specific operations such as sub-table retrieval and schema interaction, leading to critical performance bottlenecks. To address this limitation, we propose TaTToo, a novel table-grounded PRM framework that (i) reasons explicitly over tabular reasoning steps and (ii) integrates tool-based verification to provide precise reward supervision. Concretely, we first design a scalable data curation pipeline that constructs over 60k high-quality step-level annotations by integrating table verification rationales with tool-based executions. Building on the collected data, we train TaTToo with a dual-stage paradigm: cold-start supervised fine-tuning to capture tool-use reasoning patterns, followed by reinforcement learning with tool-grounded reward shaping to align our model with table-based verification.
We provide a comprehensive evaluation of the policy improvement induced by our newly designed PRM.
Across 5 challenging tabular reasoning benchmarks covering numerical reasoning, fact-checking, and data analysis, TaTToo improves downstream policy LRMs by 30.9\% at inference, surpasses strong PRM baselines such as Qwen-2.5-Math-PRM-72B with only 8B parameters, and demonstrates strong generalizability across diverse TTS strategies.

Published as a conference paper at ICLR 2026
TATTOO: TOOL-GROUNDED THINKING PRM FOR BEST-TIME SCALING IN TABULAR REASONING
Jiaru - Soumya Roy?, Vinay Kumar Verma,, Ziyi Wang"; David Wipf
Pan Lu Sumit Negi². James Zou Jingrui He UIUC, AAmaoon, *Purdue University, 'The University of Hong Kong, SStanford University
ABSTRACT
Process Reward Models (PRMs) have recently emerged as a powerful framework for enhancing the reasoning capabilities of large reasoning models (LRMs), particu- larly in the context of test-time scaling (TTS). However, their potential for supervis- ing LRMs on tabular reasoning domains remains underexplored. Through detailed empirical analyses, we identify that existing PRMss, though widely adopted for supervising text-only reasoning steps, struggle with table-specific operations such sub-table retrieval and schema interaction, leading to critical performance bot- tlenecks. To address this limitation, we propose TATToO, a novel table- grool-gd PRM framework that (i) reasons explicitly over tabular reasoning steps and (ii) inte- grates tool-based verification to provide precise reward supervision. Concretely, we first design scalable data euration pipeline that constructs over 60k high-quality step-level annotations by integrating table verification rationales with tool-based executions. Building on the collected data, train TATTOO with dual-stage paradigm: cold-start supervised fine-tuning to capture tool-use reasoning patterns, followed by reinforcement learning with tool-grounded reward shaping align our model with table-based verification. We provide comprehensive evaluation of the policy improvement induced by our newly designed PRM. Across 5 challenging tabular reasoning benchmarks covering numerical reasoning, fact-checking, and data analysis, TATTOO improves downstream policy LRMs by 30.9% at inference, surpasses strong PRM baselines such as Qwen- 2wen-225-Maal Pttth-PRR-222 with only 8B parameters, and demonstrates strong generalizability across diverse TTS strategies.
TATTOO DATA QURATION
Ressoring Staps
Collected Verification
Original Irput Table
LRM'ss
Reasoning Trojectory Generation
Verification Synthesis
Assignment
Tool Use Synthesis
Irput: Tabie Quary
Ovenol Velifietion Beriander froe
ssurced from Tableinstret, HybnidQA ToTTo, stc..
thining Step
Retriand
labeled date
Stop
Sleg
Ste
Yiurs
Entegration &
Stap
Scha na
prefie
X
reeerd
Toble lookup tools be Computation toels
Seeepretatoon Seeadia Schems Interaction Stop.
labeled data
Teel-augmantedd Verification
Table Retrievel Step
Diman-thikingg
Stop
Expert Regponses yev
TATTOO DUAL-STAGE TRAINING
Supervised fine tuning
Al-based Policy Optimization
O
Identify Evaluate sub-
Automatically tuble-prefix
Confidance Gellaboration Teol -So-unddn
1itt
Learn. Teel-Use Pattern
OSFT
ByrTOoO
Figure 1: Overview of TATTOO framework. We first curate 60k high high-quality instances by collecting expert verification rationales with tool integration (Section 4.2). We then train our PRM through dual-stage training paradigm to achieve nol-grrunded step-by-step reward supervision (Section 4.3).
"Work done while at Amazon
1 Corresponse to ajiar j iinrri 10111 woniiiiia a..is.e. edu, jamesz@stanforde

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Tabular reasoning has become fundamental capability for emerging large reasoning models (LRMs) across various real-world applications, including numerical analysis (Akhtar et al., 2023), fact- checking (Chen et al., 2019; Parikh et al., 2020), and question answering (Vakulenko and Savenkov, 2017; Li et al., 2023a). Unlike free-form text, tables encode information in rows and columns with an implicit relational semi-structure. Effective reasoning over tables therefore requires both accurate interpretation of tabular content and step logical inference to produce precise answers (Wang et al., 2024c: Zhang et al., 2025a). To support such multi-step reasoning, recent studies such as Table-R1 series (Wu et al., 2025b; Yang et al., 2025b; Jín et al., 2025) have incorporated reinforcement learning (RL) techniques (Schulman et al., 2017; Shao et al., 2024) to better align LRMs with the demands of complex table understanding and reasoning.
On the other hand, process reward models (PRMs) (Setlur et al., 2024; Wang et al., 2024b; Yang et al., 2024) have been developed to provide step-level supervision over model reasoning trajectories during test-time scaling (TTS), offering fine greiigraned verification that enhances LRMs' performance at inference. However, despite growing computational budgets and increasing emphasis advancing LRMs' tabular reasoning abilities (Ye et al., 2025; Muennighoff et al., 2025), a corresponding step- level PRM to supervise the reasoning quality of these models in table domains is equally important but remains notably absent. This gap motivates our study of a fundamental question:
How can we provide reliable step-level supervision to advanced LRMs in tabular reasoning?
To investigate this question, we first revisit several general-domain advanced PRMs and evaluate their effectiveness in supervising table-involved reasoning steps generated by LRMs. Our analysis reveals that existing PRMs struggle to reliably verify two critical types of tabular CoT steps: D Table Retrieval, where PRMs fail to supervise whether LRMs extract the correct sub-region of the input table relevant to the query; and @ Schema Interaction, where PRMs cannot detect attention collapse (Dong et al., 2021), as LRMs often overlook long-range table dependencies due to inherent locality bias. Beyond challenges arising from the tabular input modality, we also observe that current PRMs frequently introduce supervision errors within their own evaluation process, stemming from inaccurate table lookups or failed operations tables. These shortcomings amplify bias and noise during TTS, ultimately creating persistent performance bottlenecks.
Motivated by our preliminary analyses, we propose TATTOO, new Table Thinking PRM with Tool integration abilities to provide more reliable and precise supervision for tabular reasoning. Distinct from prior PRMs that provide weak supervision over table-specific operations, TATTOO provides step- level supervision tailored to different input steps, applying both table-grounded rewards for tabular operation steps and inner-reasoning rewards for text-based reasoning steps. In addition, TATTOo can leverage several external tools to interact with table contents, execute code-based operations, and incorporate the results back into the step-by-step verification process. To build TATTOO, we first design scalable data curation pipeline that yields over 60k high-quality supervision instances by integrating expert verification rationales with tool-based executions. We then train our PRM under dual-stage paradigm: supervised fine-tuning to capture step-level tool-use reasoning patterns, followed by reinforcement learning with a newly designed reward shaping scheme to encourage effective tool manipulation and faithful reasoning for accurate verification. Finally, provide theoretical intuition on the policy improvement induced by incorporating TATTOO during inference.
To demonstrate the effectiveness of TATTOO, conduct extensive experiments on five challenging tabular reasoning benchmarks, covering table-based question answering, numerical reasoning, fact- checking, and data analysis. Across all benchmarks, incorporating 8B-size TATToo improves downstream policy models by 30.9%. In addition, TATTOO consistently outperforms strong PRM baselines such 2025) with up to 9x parameter efficiency. In-depth analyses further demonstrate that incorporating our dual-stage training paradigm yields 10.2% improvement over standard PRM training, and TATToo exhibits strong generalizability across diverse TTS strategies, including Beam Search and DVTS.
(Zhang et al., 2025b) and GenPRM-32B (Zhao et al.,
2 PRELIMINARY
Table Understanding with LRMs. We denote T = (H, R) as a semi-structured table, where H is the set of column headers defining the schema-level semantics, and R is the set of rows, with each row
2

---

Published as a conference paper at ICLR 2026
Fact Checking
Numerical Reasoning
Data Analyses
8,8%
Accuracy (90)
67.7%
78-
70-
Table Retrieval
Inner thinking
Sohema Interaction
23
Number Solutions (N)
24
22
22 Solutions
2'
27
27
27
Solutions (N)
2
Figure 3: Error Distribu-
tion over step categories across 500 incorrect cases after Best-of-N selection.
Figure 2: Best-of-N performance of -eeySeek-R---isil----ee- 1owen-14B
across 3 table tasks TableBench with different types of step verifiers.
composed of cell entries aligned with H. Given a table T and an associated natural language query q. define reasoning model conditional generation policy .(r (), where y= {a....., a-)- Here, r denotes the reasoning model's generated reasoning trajectory, including both intermediate reasoning steps {a, { V--1 and the final answer ag. In our problem setup. the intermediate reasoning steps consist of both model inner-thinking reasoning traces and tool-integrated programs that operate directly on the table to retrieve or compute intermediate results. The final answer can take different formats depending the query type, including textual numerical values, boolean outputs (e.g.. True/False), or executable programs (e.g., Python, SQL).
Reward Modeling for Tabular Reasoning. Given table T, a query q, and a candidate response generated by policy LRM, a standard step-level verifier (i.e., PRM) parameterized by computes a scoring function (-)-) that assigns step-level rewards r; evaluating the correctness of each step a, E т. The trajectory-level reward r, for each response T is then obtained by aggregating these step-level rewards. Formally, we have:
Ro(a: TTq, Tei), with A(r1, (2.
(1)
where A(-) denotes an aggregation function such as MEAN and SUM (Liu et al., 2025). The rewards provided by the PRM Ro can be further leveraged by test-time compute strategy & (e.g.. Best-of-N (Brown et al., 2024), Beam Search (Snell et al., 2024)) to guide resampling, refinement, and candidate selection among the responses generated by the policy model.
WHY TABLE REASONING REQUIRES VERIFIERS BEYOND CURRENT PRMs?
We begin by revisiting existing general-domain PRM methods to assess their effectiveness in super- vising LRMs on tabular reasoning tasks and to identify potential performance bottlenecks. To this end, we conduct a pilot study guided by two key questions:
RQI- Beyond free-form text inputs, can common general-domain PRMs combined with TTS strategies also enhance the performance of LRMs on tabular reasoning tasks?
RO2 When step-level reward supervision is crucial for tabular reasoning performance, how can PRMs effectively supervise and guide the quality of each reasoning step generated by LRMs?
For brevity, we defer detailed experimental setups Appendix F. To investigate RQI, we evaluate various step-level verification methods, including two advanced PRMs 5-Mee22.5-MMth-PRMMM (Zhang et al., 2025b) and Skywork-PRM--1
(He al., 2024a)), majority voting (Liu al., 2025). and LLM-as-a-judge (Zheng et al., 2023) with the Best-of-N TTS strategy. We choose DeepSeek-RL Distill-Qwen- 14B (Guo et al., 2025) as the common LRM and evaluate on TableBench (Wu et al., 2024), which includes three fundamental table tasks (Fact Checking, Numerical Reasoning, and Data Analysis). As shown in Figure 2, we observe that for small values of N, incorporating step-level verifiers into Best-of-N generally improves LRM's performance over single-shot generation, with PRMs providing the largest gains. However, once the number surpasses a threshold (N 2 8). accuracy across all three table tasks converges to bottleneck. For example, the performance of Qwen2.5-Maan 1...55-MMtt--MM----- PRM-72B on fact-checking is 79. 19%, 79.82%, and 79.84% for N = {8, 16, 32}. indicating that further increases in N yield negligible gains, even though with PRM incorporation.
3

---

Published as a conference paper at ICLR 2026
Table Representative error cases in 3 different reasoning step categories. Each example highlights the erroneous model step in red, the corresponding error description, and the PRM's (mis)judgment reward, illustrating where existing PRMs fail detect mistakes.
Category
Wrong Model Answer
Error Description & PRM output
<think> Step I:
need to figure out the average prominence of
The model inirially retrieves the wrong part of the table (columns 'Country' and 'Elevation (m) which leads errors all subsequent reasoning steps.
mountain peaks the Democratic Republic of the Cangu that have an elevation at least 3000 meters.
Table Retrieval
Step 2: First, I'll review the provided table. My main focus is "country" 'elevation (m) columns. (Wrong Step)
</think>>
Final Answer Vboxed: (a0ee::22066666 65;
PRM [correct) Fail detect the
<think> Step Let me start by looking at the table provided
The model identifies the conrect coloma for the answer, bat fails to extract all elements, omitting the last incorect reasoning all
The retrieved columns are
Schemn Interaction
Step I'll go through
row and
episodes:
cell. This emission subsequent steps.
Episode Rating- 5.1, Episode 2: Rating - 5.2 (Wrong Step!
Step S: Adding the rating logether is
<ithink>
Final Answer is: \boxed (11.
PRM + (currect) x Fail to detect the error
<think> Step 1: Based on the table contents, need to count
The model incorrectly does the calculation by averaging season-lovel mcans, giving instead of avergiing across all team-sensoos.
podiums per scason, 2008: (1+++ 13/3 2009: 6, 2010:
equal weight,
Iunes-thinking
0, 201 I: (2 0)/2
Step
will averuge across seasons, average (1
6- 0 1)/4 = (Wrong Step)
<tthink>
Final Answeris is \boxed (2)
(incorrect] Detect the eroor
% instancees out of
X0-
Bool Sob-tatie
Random Sub Cable
Average attention mass
Layer
Layert
With Tablo Praffx
oo-
Lagerr
Accuracy (96
73%
68%
70
65
QwenZ Meth-PRM PRM 728
Stop
Skywort-PAM.77
60
correct
Incorrect
2
PRM Predicted Outputs
Relative step distance
Number of Solutions (N)
Figure 4: Left: PRM's rewards on 500 reasoning steps with the real-retivve//radoollyeeeepee
sub-table. Middle: Layer-wise average attention mass vs. relative step distance in tabular reasoning. Attention concentrates nearby steps, with sharp decay distance increases. Right: Best-of-N
results on DeepSeek-R1-DD R1-DDstill- Qwllll-een-eee 14B for numerical reasoning with/without the table prefix.
Observation 1 (Limitation on TTS): Existing PRMs yield modest improvements on tabular reasoning, but their efficacy quickly saturates, failing fully exploit additional test-time compute.
Error Analysis. Building the observation, further investigate the underlying causes of the performance bottleneck by conducting an error analysis on LRM's generation and PRM's supervision processes. Specifically, we sample 500 erroneous Best-of-N responses (N= 32) selected by the PRM from LRM outputs, and ask human experts to classify them into 13 well-defined tabular error types (see Appendix C). We then connect these errors with 4 reasoning-step categories reflecting the typical flow of LRMs' reasoning process: (i) Table Retrieval Steps, locating relevant rows/columns regarding the input query; (ii) Schema Interaction Steps, reasoning over the retrieved table contents, (iii) Inner-thinking Steps, models' inner reasoning independent of table contents, and (iv) Others, initial setup or final output steps that are irrelevant to core reasoning process. Figure 3 presents the error distribution across 4 reasoning step categories. We find that most errors arise in Table Retrieval (47.7%) and Schema Interaction (34.3%), implying that PRMs perform reasonably well on independent reasoning but fall short when reasoning steps involve table-specific operations. For better demonstration, we provide representative examples for each category in Table I.
Why do PRMs fail on table-involved reasoning steps? Next, we take closer look at why PRMs lose their supervisory effectiveness when reasoning steps involve table operations. For Table Retrieval Steps, conduct contrastive experiment focusing particularly on the table contents retrieved by LRMs within their responses. We randomly sampled 500 responses and constructed two variants by (i) retaining the original -eM-reerieeed sub-table, and (ii) replacing it with randomly selected sub-table
4

---

Published as a conference paper at ICLR 2026
region from the original input table. Figure 4 (left) shows the output rewards of Qwen2.5-Math- MRM-- 72B on both variants. The nearly identical distributions between real and random sub-tables indicate that current PRMs fail to distinguish retrieval correctness, suggesting that they are unable to assess whether the LRMss retrieved portion of the table corresponds to the query.
Takeaway 1 (Table Retrieval): Existing PRMs are insensitive to table retrieval correctness in the reasoning steps and fail to recognize whether the retrieved content corresponds to the query.
For Schema Interaction Steps, we found in prior experiments that in the logic flow of LRMs' trajectories, table retrieval steps typically occur at the beginning, as the model must first extract relevant information from the table to answer the query. In contrast, schema interaction steps frequently occur far sentences from the beginning table retrieval steps, since LRMs tend to perform intermediate reasoning before revisiting their retrieved contents when needed. Figure 4 (middle) illustrates the attention distribution of the LRM between the schema interaction step (step 8) and the table retrieval step (step 0). Due to the regr-ssgees nature of LRMs. the schema interaction step attends primarily to nearby steps while assigning little attention to the earlier retrieval step. This inherent locality bias causes the model to frequently misinterpret discard previously retrieved contents, even when the retrieval step has already extracted the correct information. Moreover, current PRMs fail supervise such misinterpretations,, as their evaluations are highly localized to the current step rather than capturing dependencies on distant prior steps (Zou et al., 2025b; Feng et al., 2025b).
Takeaway 2 (Schema Interaction): Schema interaction steps under-attend to distant table retrieval contents due to locality bias. PRMs miss these failures since they can't look ahead and capture long-range dependencies among distant steps.
Table Prefix the Key. To explore potential solutions to the limitation above, we begin with a simple input modification for PRMs: prepending the retrieved table contents prefix to each schema interaction step. This grants PRMs direct access to the retrieval context, alleviating the need for long-range dependencies. We evaluate this modification and report the results in Figure 4 (right). Incorporating the table prefix indeed improves PRM supervision and leads to stronger downstream LRM performance. However, directly applying the prefix remains challenging, current PRMs cannot automatically identify schema interaction steps, and the table prefixes obtained from LRMs are not guaranteed be correct without proper supervision.
Motivation for TATTOO. Our analyses above highlight the need for a principled step-level verifier capable of providing robust supervision over both table- broundgr operations and models' inner reasoning. Motivated by this, we propose new process reward model specifically designed to support LRMs in tabular reasoning.
BUILDING A TABLE-GROUNDER STEP VERIFIER
We introduce TATTOO, generative PRM that provides reward supervision over both table operations and model inner thinking steps. Our method builds on two key components: (i) a large-scale data curation pipeline that synthesizes reasoning and tool usage for PRM training, and (ii) dual-stage training paradigm that learns step-level verification with tool use optimization.
4.1 TABLE-AWAL AND TOOL-INTEGRATED SUPERVISION
Table-Aware Reward. To align with the LRM's reasoning process on table tasks, separate the supervision of table operations from model's 's inner reasoning part and decompose TATToO's step-level reward (Eq. 1) into two components:
Tirrea ag E inner-thinking.
r;
and Ty
T/
ri,ub, if a; E table retrieval or schema interaction,
where ri,rea captures the correctness of the model neasonne process, Ti,ub reflects the accuracy of table-grounded operations, and r, denotes the rujectory-leve reward.
Tool Integration in Verification. A major limitation of current PRMs is their inability to supervise table-involved reasoning steps (as shown in Section 3). Meanwhile, recent studies (Feng al.,
5

---

Published as a conference paper at ICLR 2026
2025a; Qian et al., 2025) have shown that LLM agents can autonomously tools to interact with external environments and iteratively refine their reasoning. In a similar spirit to address current tools into TATToo's verification
PRM's limitation, we incorporate several external process to enable more reliable step supervision. We next describe how we curate a training set with tool-augmented, table-aware rewards and use it to train TATTOO.
4.2 TATToO DATA CURATION PIPLINE
We design large-scale data curation pipeline that simulates real-world scenarios of PRM tool use and step verification scale. As illustrated in Figure 1, there are three main stages:
Reasoning Trajectory Generation. We begin by collecting trajectory responses from expert LRMs (e.g.. DeepSeek-R (Guo et al., 2025) and Claude-Opus-4. (Anthropic, 2025)) on table-based questions drawn from diverse benchmarks, including TableInstruct (Wu et al., 2024). HybridQA (Chen et al., 2020), ToTTo (Parikh et al., 2020), and WikiTQ (Pasupat and Liang, 2015b). We generate multiple responses per query and apply dual verification with human annotators and expert LLMs to filter out low-quality data, yielding a high-quality trajectory pool Tpoos for subsequent labeling.
Verification Synthesis & Reward Assignment. We next provide step-level verification rationales and reward labels for each candidate response in Tpaot- (i) For table retrieval steps, we extract the sub-table in each step and use LLM-as-a-judge to assess its relevance to the query, assigning table reward ri,tah E {-1,1) based on retrieval correctness. (ii) For schema interuction steps, we prepend the accurate sub-table as a table prefix to each collected verification rationale (according to our table-prefix analysis in Section 3) and assign T',sab E [-11, 13 based on the correctness of the specific table-based operations or reasoning. (iii) For inner-thinking steps, which involve no table contents, we apply judee and follow established labeling strategies (Zhao et al., 2025; Khalifa et al., 2025) to assign rirr E {-1,1) 1} based on reasoning quality.
Tool Use Synthesis. To train TATTOO to leverage tools for more accurate verification, we further augment the collected verification rationales with tool invocations, execution results, and feedback at the step level. Specifically, inside the rationale contents, we replace manual reasoning for table lookups or calculations with the corresponding tool call and its execution output. We primarily employ two types of table tools: (i) Computation tools: code snippets (e.g., Python, SQL) for arithmetic and aggregation over table inputs; (ii) Table Lookup tools: DataFrame APIs (e.g., Polars) or Lookup Utilities (e.g., CSV/Excel readers) for retrieving specific rows, columns, cells during verification.
Finally, we construct over 60k high- training instances with complete verification rationales and step-level rewards. This dataset is then used to train TATToO integrate tool use with reasoning for robust step supervision. We leave additional data curation details in Appendix D.
4.3 TOOL-GROUNDED SUAL-STAGE TRAINING
With the training data recipe in place, we train TATTOO via dual-stage paradigm: supervised fine-tuning to capture tool-integrated verification patterns, followed by RL-based policy optimization with newly designed reward shaping scheme further refine our PRM's step-level rationales and ensure accurate verification.
Table-Aware Verification with Tools via SFT. We first finetune our PRM R4 on the curated dataset (Section 4.2). Specifically, given a training instance (T, g, T) consisting of table T, a query and an LRM-generated trajectory T (o.,..., az), we train the PRM to output, for each step a; T, verification rationale together with its corresponding step-level reward rú. By formulating PRM training as language modeling, Rg is optimized retr-segresy to (i) identify accurate sub-table regions, (ii) learn to dynamically incorporate the retrieved table prefix into each schema interaction step, and (iii) generate verification rationales with tool-integration patterns.
Toou-Gro Reward Shaping in RL. Prior generative PRM approaches (Liu et al., 2025; Khalifa et al., 2025; Zhao et al., 2025) typically conclude PRM training after the SFT stage. In contrast, we draw inspiration from recent advances in agentic RL (Jaech et al., 2024; Guo et al., 2025; Zou et al., 2025c) and further apply policy optimization to more tightly align the PRM's verification process with effective tool utilization. Specifically, optimize R2 with modified GRPO (Shao et al., 2024) by providing dense, trol-grounded supervision signals during policy optimization. During RL rollouts of each training instance (T, q, q), we replace the original rule-based GRPO supervision

---

Published as a conference paper at ICLR 2026
signal with denser per-step reward signal 8;. defined as:
8; 1i = m
Aeal
log (r (T4,))) +
Annol (
label-matching
tool-grounding
confidence calibration
where f is the PRM's predicted step-reward and r; the ground-trutt step-reward for the input step a, E 0g denotes the verification rationale generated by the PRM at step i, and support {0. 1 measures whether the rationale correctly incorporates tool outputs; and Acal, Atool are tunable coefficients. Besides enforcing correctness with the label-mafching Ierm, the confidence calibration term stabilizes training by encouraging higher probability on the ground-trume truth label, and the tool- grounding term encourages rationales that effectively incorporate tool outputs. We then aggregate the per-step signals s; into a trajectory-leve training reward, normalize it within each sampled group to compute group- group-relative advantages, and update the PRM Ry under the GRPO objective.
4.4 INFERENCE-TIME POLICY IMPROVEMENT AN INTUITIVE VIEW
To intuitively elucidate the role of TATTOO and its table-aware rewards on LRM's tabular reasoning process (Eq. 2), we provide theoretical analysis on the policy improvement induced by TATToo.
Recall that the goal of our PRM is to improve the generated trajectory T sampled from policy LRM æ, i.e., T TT- T. q). By combining the input table and query. we represent (T, Q1,..... a;-1) as the current state 8;. At step i, the policy LRM samples an action ~ TT. 8:). We define the Q-value of policy æ as the expected future success, measured by the final answer az correctness, i.e.,
Q a.) Q ((T, di,01,..,, 01- a)) Be.1i...a2~<T((()) [Lazis
The value of policy at state s; is defined as the expectation of Q-values under x's next action distribution: V" (s. = Ea.- (s.) 1Q" (s., a- We now analyze the policy improvement afforded by TATToO's table-aware reward supervision under one step of natural policy gradient updating.
Theorem 4.1 (Policy Improvement (Lower Bound)). Given the current policy æ, after one natural policy gradient update step guided by the PRM reward Ti defined in Eq.2. we obtain the revised policy N' r'a s,) x exp(Q" (s,, a,) + rr(ss,, a.)). The resulting expected policy improvement over the state distribution p then satisfies:
Bacep ( (s:) - 130 (s:) 2 Exerp Vara Vara [ri a:)] +
a (sx)) [ri, a:)
distinguishobbliiyy from toble reward rs.m
distnguissabbiity from inner-reatuming reward a.) (8, a.)):
Ba, Bee (-lre)) [Femm (St; a.)AA (st, a:)]
( [Te,een (ort(()
alignment nab and
alignment between res A
(5)
where AT a.) Q7 a.) (s:)s) denotes the advantage of policy
Theorem 4. 1 (proof in Appendix E) explains that our decomposable reward design r; enables each component additively contribute to policy improvement, provided that the reward components each individually aligned with the policy advantage function. In this way, the table-aware rewards provided by TATTOO help ensure targeted supervision on both inner reasoning and table-involved operations generated by LRMs. Below, we further empirically evaluate the effectiveness of TATTOO across various downstream tabular reasoning tasks.
5 EMPIRICAL EVALUATIONS
Baselines and Models. We compare TATTOO against various types of step-level verification methods, including advanced ORMs/PRMs, majority voting (Liu et al., 2025), and LLadaa--ajudgee (Zheng et al., 2023). The setups for these baselines are aligned with our preliminary analyses in Section 3. For PRM approaches, we include both discriminative (Qwen-PRM series (Zhang et al., 2025b), Math- Baah--heeperd-PPe -PRM (Wang et al., 2024b), and Skywork (ThinkPRM (Khalifa et al., 2025) and GenPRM (Zhao et al., 2025)). Regarding the policy reasoning models, we evaluate our proposed method on DeepSeck- -iieeck-RR-DD -Distill Qwen- -eeen-14BBBB (Guo et al., 2025). Further details the baselines and policy models setups provided in Appendix F.I.
(He et al., 2024a)) and generative
7

---

Published as a conference paper at ICLR 2026
Table 2: Main results of TATTOO on 5 different tabular reasoning tasks. We report the best-of-N -1--s-tiil--s--- 14B as the policy model
(with N = {4, 8, 16, 32)) performance using and compare against various step verifiers. The best and second-bes results are highlighted. TATToo consistently achieves state-of-the-art TTS performance with significantly fewer parameters.
Veriferr (Best-af-N)
Params
TB-NR
TE-FC
TE-DA
WTQ
MMQA
16
32
Majority Vote
65.5
65.9
66.8
66.5
76.2
77.3
77.3
77.4
23.5
24.5
26,0
26
64.7
65.3
67.3
67.0
18.4
19.4
20.4
20.1
LLMM-as--juuge
66.7
66.9
67.1
6699
71.2
78.3
78.4
78.6
23.5
27.4
28.0
28.4
65.2
66.4
68.1
68.1
19.6
21.3
22.5
22.7
Skywork PRM-7
66.1
69.5
70.3
70.1
76.8
78.4
78.6
78.3
24.1
27.5
28.9
29.1
65.9
67.5
68.4
68.6
21.4
24.6
25.3
Math Seih-SShphen--PPP PRM-7B
7B
67.2
70.6
71.5
718
76.2
76.9
76.8
77.1
22.7
24,8
26.4
25.9
66.8
68.7
69.6
69.
22.0
25.2
25.9
26.1
Qwen25 Mathh PRM-7B
66.9
70.1
71.7
72.5
75.4
77.2
77.9
77.4
23.2
25.4
26,3
26.6
65.2
68.5
69.6
69.7
23.5
25.2
27.1
27.3
ThinkPRM
148
69.2
70.7
73.5
73.8
75.8
75.4
76.3
76.9
21.6
22.7
23.1
22.8
64.3
66.1
65.7
65.9
22.4
22.7
23.6
23.0
GenPRM
Owenn.5-Mathh Math-PRMM72BB
72B
70.4
73.8
24.9
25.3
77.8
79.2
79.8
79.8
25.5
31.5
3200
32.4
69.2
71.8
73.0
72.6
24
26
28.7
28.6
TATTOO
Datasets. We evaluate on four representative and challenging benchmarks spanning diverse tabular reasoning tasks, including (i) TableBench (TB) (Wu et al., 2024), a complex tabular reasoning benchmark with 886 questions covering tasks of numerical reasoning (NR), fact checking (FC), and data analysis (DA). (ii) WTQ (Pasupat and Liang, 2015b), benchmark for complex question answering over Wikipedia tables. (iii) MMQA (Wu et al., 2025a), a multi-table understanding benchmark covering table retrieval, multi-hhop& & multi-table QA and text-to-SQL generation. We leave the additional dataset descriptions in Appendix F.2.
Implementation Details. We train TATTOO the off-the-shelf Qwen-3-8B model (Yang et al., 2025a) using our 60k curated training instances (Section 4.2). All training and inference experiments conducted 8xA 100-80G GPUs. To evaluate TATToO under different TTS strategies, adopt three representative methods, including Best-of-N (Brown et al., 2024), Beam Search (Snell et al., 2024), and Diverse Verifier Tree Search (DVTS) (Beeching et al., 2024). Additional implementation details on training setup and configurations of TATTOO are provided in Appendix F.3.
5.1
MAIN RESULTS
Table 2 reports the Best-of-N performance of incorporating TATTOO on the DeepSeek- Ri-eck--R--iitii Qwen-14B model across five tabular reasoning tasks. Notably, TATTOO consistently outperforms strong baselines such as GenPRM-32B and Owwen2.5MMath-PRM-222 despite using only 8B pa- rameters. On TB-DA, TATTOO achieves the largest accruacy performance across each level of N, rising from 27.7% at N=4 34.3% N=32. Moreover, while existing PRMs often suffer from performance bottlenecks beyond certain response threshold (as observed in Section 3), TATTOO continues to scale effectively, yielding consistent gains as the response group size increases. For example, on TB-NR, -wen2.5-Matt-PRMM-
saturates after N-16 (74.9% - 75.3%), whereas TATTOO continues to improve from 74.2% at N to 78. 1% at N-32. These results demonstrate that TATTOO provides stronger reward supervision on LRMs' reasoning trajectories, therefore yielding better performance improvement compared with other step-verification baselines.
55
60
Accuracy(%)
Accuracy(%)
45
Owen2 Moth FRM4-22
40-
Caeez.5 Mii.s.Sttt---22a
Skywort
Skywork
To
35
20
2
2a
24
2
20
2
2
2
2
Beam Search
DVTS
Figure 5: Performance of TATTOO on two additional TTS strategies, Beam Search and Diverse Verifier Tree Search (DVTS). We report the average accuracy across all 5 tabular reasoning tasks.
Generalizability Other TTS Strategies. Beyond best-of-N, we also evaluate TATToo under two additional TTS strategies (Beam Search and DVTS) and compare with the strongest PRM baselines. Figure 5 reports the average performance across the five tabular reasoning tasks. Under each TTS
8

---

Published as a conference paper at ICLR 2026
Table In-depth analysis of TATToO on three table datasets. We evaluate the contributions of SFT and RL training stages, and assess the impact of reward shaping components during RL optimization.
TB-NR
TB-FC
TB-DA
Training Variants
8
16
32
8
16
32
8
16
32
TATTOO (SFT only)
67.9
69.1
72.0
73.7
71.5
73.0
74.6
75.2
23.3
25.6
26.2
26.4
TATTOO
71.2
74.2
76.4
78.1
77.4
79.6
81.2
82.0
27.7
31.9
33.6
34.3
w/o tool- groundinn
68.5
71.1
72.7
74.6
73.2
75.6
75.5
76.3
26.2
28.1
28.7
30.3
wlo confidence caliboration
71.1
73.7
74.3
76.2
76.4
76.7
78.4
80.5
27.4
29.5
31.3
33.2
rule-based (GRPO)
67.0
68.4
70.4
73.1
71.6
74.0
74.9
75.8
25.5
27.4
28.0
28.6
strategy, TATTOO consistently yields steady improvements as the number of responses N increases. whereas other baseline PRMs often plateau. For example, in beam search, TATTOO improves from 45.0% to 54.8% while GenPRM saturates around 51% % and Skywork- remains below 46%. These results highlight the strong generalizability of TaTToo across diverse TTS strategies.
5.2 IN-DEPTH ANALYSES ON TATTOO
Mastery RL with Bootstrapping from SFT. examine the respective roles of SFT and RL in TATToo's dtal-stage training paradigm, compare against variant TATToo (SFT only), which is trained solely on the first SFT stage. As shown in Table 3, under the Best-of-N evaluation, the second-stage RL policy optimization consistently improves performance over the SFT-only initialization. Specifically, we observe that the average accuracy across all three tasks improves from 72.3% (SFT only) 78.5% after RL. training, yielding total gain of 10.2%. This demonstrates that bootstrapping from SFT provides a solid initialization, while RL optimization further enhances our PRM's reasoning and tool-use effectiveness during the verification process.
Reward Shaping during RL Training. Next, we ana- lyze the effectiveness of each supervised component our per-step reward signal 8; design (Eq. 3), with the ablation results reported in Table Removing the tool- grounding term yields the largest drop (e.g., J4.0% on TB-DA at N-32), highlighting its critical role encour- aging effective tool use during RL training. In addition, excluding confidence calibration reduces performance by 1.6% average, showing its complementary effect in stabilizing reward signals. We also compare TATTOO with the original rule-based group-relative reward from GRPO, which yields only marginal improvement over SFT. Finally, Figure 6 visualizes the training dynamics of TATTOO and other variants during RL optimization.
Training reward
644
wio tool aroundnnn
wil confidence callboration
(GHPO)
Step
239
Figure 6: Training dynamies of TATTOO and ablated variants. We report the train- ing reward across 280 training steps.
Additional Experiments. Additional experiments, including ablations on the training coefficients and case studies on TATTOO's 's effective tool usage, are provided in Appendix G.
RELATED WORKS
LLMs on Tabular Reasoning, Reasoning over tables poses a unique challenge for LLMs, requiring them to bridge natural language understanding with structured reasoning over rows, columns, and cell values (Jin et al., 2022; Zhang et al., 2025a). Recent works (Tang al., 2020; lida al., 2021; Deng et al., 2022; Zou et al., 2025a) have investigated tabular reasoning several downstream tasks, including table QA (Chen et al., 2020), table fact verification (Chen et al., 2019; Parikh et al., 2020), text-to-SQL (Mohammadjafari et al.. 2024), etc. Early-stage tabular reasoning methods, such as TAPAS (Herzig et al., 2020) and TaBERT (Yin et al., 2020), encode table data into transformer based encoder representations. Later methods leverage the capabilities of LLMs to apply either prompt engineering (Sui et al., 2023; Wang et al., 2024c) or supervised fine-tuning techniques (Su et al., 2024; Zhang al., 2023) for enhanced tabular reasoning. More recent works, including the Table-R1

---

Published as a conference paper at ICLR 2026
series (Wu et al., 2025b; Yang et al., 2025b; Jin et al., 2025) and Reasoning- -easoning-Tabbee Table (Lei et al., 2025), leverage RL to acquire higher- quallty reasoning paths during reasoning over tables.
Table Question Answering. The evolution of Table Question Answering (Table QA) research (Jin et al., 2022) has been propelled by the creation of sophisticated evaluation resources that facilitate semantic parsing capabilities (Yang et al., 2020; Li et al., 2023b; 2024). Foundational works, including WTQ (Pasupat and Liang, 2015a) and TabFact (Chen et al., 2019). established initial evaluation paradigms through Wikipedia-derived HTML table QA pairs. Structured supervision has also been explored in alternative benchmarks such as WikiSQL (Zhong et al., 2017) and Spider (Yu al., 2018), where logical expressions serve explicit annotations to encourage systematic reasoning. More recent studies such as MultiTableQQA (Wu al., 2024), MT-RAIG (Seo al., 2025), and MMQA (Wu et al., 2025a) has shifted towards multi-hop reasoning.
PRMs for Test-time Scaling. Process Reward Models (PRMs) (Lightman et al., 2024; Uesato et al.,
2022; Zhang et al., 2024) deliver fine ganiiid, step-level feedback guide model reasoning, assigning intermediate rewards to individual reasoning steps rather than only judging final answers (Guan et al., 2025; Chen et al., 2025). Prominent PRMs, including Math-Shepherd (Wang al., 2024b), Skywork- PRM (He et al., 2024a), and the -aatnn..--Mat-PPPR family (Zhang et al., 2025b), are trained using a mix of human annotations and synthesized supervision to score model-generated solution steps across domains such as math (Maxwell-Jia, 2024; Wang et al., 2026), scientific reasoning (Rein et al., 2023), and programming (He et al., 2024b); more recently, Think-PRMM proposes a generative verifier produce long-chain CoT evaluations (Khaliff et al., 2025). PRMs have been incorporated into training-time -timi optimization as reward signals via step-verified online RL and verifier verifier-giided self-training (Li and Li, 2024; Guan et al., 2025; Cui al., 2025), and into inference-time scaling by coupling step-level scoring with search/decoding strategies (Zhao et al., 2025; Khalifa et al., 2025), including beam search, reward-guidee tree search, and Best-of-N sampling. We leave additional related works on Process Reward Models and Tool Integration with RL in Appendix B.
CONCLUSION
We introduced TATTOO, novel tool-augmented thinking PRM tailored for tabular reasoning. By diagnosing why existing verifiers fail on table retrieval and schema interaction, we built scalable pipeline with expert rationales, table prefixes, and tool-angmented verification, and trained our model via SFT followed by RL with reward shaping. TATToO achieves comparable performance across five table benchmarks, surpassing strong PRMs with up to 9x parameter efficiency and generalizing across multiple TTS strategies. Our results underscore the importance of aable-grounde reward supervision and point toward future directions in reward modeling for structured reasoning tasks.
ACKNOWLEDGMENT
This work supported by National Science Foundation under Award No. 118-2117902. The views and conclusions are those of the authors and should not be interpreted as representing the official policies of the funding agencies or the government.
ETHICS STATEMENT
This work does not involve human subjects, sensitive personal information, or proprietary datasets. All datasets used in our experiments are publicly available table reasoning benchmarks, such as TabFact, FeTaQA, and WikiTableeuessionn.. We provide detailed descriptions of data processing steps in Section 5 and the Appendix D. The goal of our method is to improve process reward modeling for table reasoning, which we believe contributes positively to advancing trustworthy and interpretable reasoning with structured data. Nevertheless, acknowledge that stronger table reasoning capabilities could be misused for generating misleading or biased tabular summaries applied irresponsibly. We encourage responsible use and adherence the ICLR Code of Ethics.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
We have made careful efforts to ensure reproducibility. The main text describes our model architecture, training procedure, and evaluation protocols in detail (Sections 4,5). Additional hyperparameters, implementation details, and ablation study configurations are included in the Appendix F. All datasets are publicly available, and preprocessing steps are documented in Section 5. Theoretical analyses, including formal definitions of preference-based reward modeling and proofs of consistency guarantees, are provided in Appendix E.
REFERENCES
Mubashara Akhtar, Abhilash Shankarampeta, Vivek Gupta, Arpit Patil, Oana Cocarascu, and Elena Simperl. Exploring the numerical reasoning capabilities of language models: A comprehensive analysis tabular data. arXiv preprint arXiv:231 231.22221.02166, 2023.
Anthropic. Claude opus 4 and claude sonnet 4 system card. Technical report, Anthropic, 2025.
Edward Beeching, Lewis Tunstall, and Sasha Rush. Scaling test-time compute with open models, 2024. URL comss--tmmmmmmpute....ee
co/ coosspaces/Hiee /ttp//ppee//uggggggggettt/t/
blogpost - www.sssaaannniiiiaeeeeeeeeeaaeea testg-test-ao - time
Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoe V. Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling, 2024. URL https:tt///// / www.xivooo www.v.oor//aaa. aaborg/ab///aa /2407. 21787.
Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. Tabfact: A large-scale dataset for table-based fact verification. arXiv preprint arXiv: 1909.0029002444 02460,1114444 2019.
Wenhu Chen, Hanwen Zha, Zhiyu Chen, Wenhan Xiong, Hong Wang, and William Wang. Hy- bridqa: A dataset of multi-hop question answering over tabular and textual data, arXiv preprint arXiv:2004.07347, 2020.
Xiusi Chen, Gaotang Li, Ziqi Wang, Bowen Jin, Cheng Qian, Yu Wang. Hongru Wang, Yu Zhang, Denghui Zhang. Tong Zhang, Hanghang Tong, and Heng Ji. Rm-rl: Reward modeling as reasoning, 2025. URL www.tttpps..ooo //tpxvvaaivvvv///////i org/ abs 2505. 02387.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110. 14168, 2021.
Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, et al. Process reinforcement through implicit rewards. arXiv preprint MxXi:22002.01556. 2025.
Xiang Deng. Huan Sun, Alyssa Lees, You Wu, and Cong Yu. Turl: Table understanding through representation learning. ACM SIGMOD Record, 51(1):33-40, 2022.
Yihe Dong, Jean- San-ttatiste Cordonnier, and Andreas Loukas. Attention not all you need: Pure attention loses rank doubly exponentially with depth. In International conference machine learning, pages 2793-2803. PMLR, 2021.
Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. Retool: Reinforcement learning for strategic tool use in llms. arXiv preprint wwXiv:2544.15555 2025a.
Zhangying Feng Qianglong Chen, Ning Lu, Yongqian Li, Siqi Cheng, Shuangmu Peng, Duyu Tang, Shengcai Liu, and Zhirui Zhang. Is prm necessary? problem-solving rl implicitly induces prm capability in Ilm. arXiv preprint 2.Ki::2505.12222 2025b.
Xinyu Guan, Li Lyna Zhang, Yifei Liu, Ning Shang, Youran Sun, Yi Zhu, Fan Yang, and Mao Yang. rstar-math: Small Iims can master math reasoning with self-evolved deep thinking. arXiv preprint arXiv:2501.045// 2025.
11

---

Published as a conference paper at ICLR 2026
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning, arXiv preprint arXiv:2501. 2025.
Jujie He, Tianwen Wei, Rui Yan, Jiacai Liu, Chaojie Wang, Yimeng Gan, Shiwen Tu, Chris Yuhao Liu, Liang Zeng, Xiaokun Wang, Boyang Wang, Yongcong Li, Fuxiang Zhang, Jiacheng Xu, Bo An, Yang Liu, and Yahui Zhou. Skywork-ol open series. https : htt.g/gggggggacagaeo/ so/kkyorrk November 2024a. URL https: htt.g/gggggggggooeeo wigiiinffacao co/ skyookk
Xinyi He, Jiaru Zou, Yun Lin, Mengyu Zhou, Shi Han, Zejian Yuan, and Dongmei Zhang. CoCoST:
Automatic complex code generation with online searching and correctness testing. In Yaser Al-Onaizam, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1433-19451, Miami, Florida, USA, November 2024b. Association for Computational Linguistics. doi: 10. 121131866110//// 1/te53/1112222..lmmmmaaa 1082. URL https:tt//// http://aaaaaattthoooooooo org /t/gg/20/ 2024. emn malpppmminn... 1082/.
Jonathan Herzig, Pawet Krzysztof Nowak, Thomas Müller, Francesco Piccinno, and Julian Mar- tin Eisenschlos. Tapas: Weakly supervised table parsing via pre-training. arXiv preprint arXiv:2004.02349, 2020.
Arian Hosscini, Xingdi Yuan, Nikolay Malkin, Aaron Courville, Alessandro Sordoni, and Rishabh Agarwal. V-star: Training verifiers for telf-t reasoners. arXiv preprint arXiv:240.006457 2024.
Hiroshi lida, Dung Thai, Varun Manjunatha, and Mohit lyyer. Tabbie: Pretrained representations of tabular data. arXiv preprint arXiv:2105.02554 2021.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Camney, et al. Openai ol system card. arXiv preprint arXiv:2412.16720, 2024.
Nengzheng Jin, Joanna Siebert, Dongfang Li, and Qingcai Chen. A survey on table question answering: recent advances. In China Conference on Knowledge Graph and Semantic Computing. pages 174-186. Springer, 2022.
Rihui Jin, Zheyu Xin, Xing Xie, Zuoyi Li, Guilin Qi, Yongrui Chen, Xinbang Dai, Tongtong Wu, and Gholamreza Haffari. Table-rl: Self-supervised and reinforcement learning for program- table reasoning in small language models. arXiv preprint arXiv:2506.066 06177, 2025.
Sham Kakade and John Langford. Approximately optimal approximate reinforcement learning, In Proceedings of the nineteenth international conference on machine learning, pages 267-274, 2002.
Muhammad Khalifa, Rishabh Agarwal, Lajanugen Logeswaran, Jaekyeom Kim, Hao Peng. Moontae Lee, Honglak Lee, and Lu Wang. Process reward models that think, 2025. URL https: // //arxiv. org/ abs/ 2504. 1 6828.
Fangyu Lei, Jinxiang Meng, Yiming Huang. Tinghong Chen, Yun Zhang, Shizhu He, Jun Zhao, and Kang Liu. Reasoning-table: Exploring reinforcement learning for table reasoning. arXiv preprint 20Xi:25506.01700. 2025.
Peng Li, Yeye He, Dror Yashar, Weiwei Cui, Song Ge, Haidong Zhang, Danielle Rifinski Fainman, Dongmei Zhang, and Surajit Chaudhuri. Table-gpt: Table-tuned gpt for diverse table tasks. arXiv 2023a.
preprint arXiv:2310.099
Wendi Li and Yixuan Li. arXiv:2410.11288 2024.
Process reward model with q valuee rankings. arXiv preprint
Yinghui Li, Zishan Xu, Shaoshen Chen, Haojing Huang, Yangning Li, Yong Jiang, Zhongli Li, Qingyu Zhou, Hai-Tao Zheng, and Ying Shen. Towards real-world writing assistance; A chinese character checking benchmark with faked and misspelled characters. CoRR, abs/2311.11266 2023b. doi: 10.48550/ARXIV.2311 1226. URL https: /doiii www..oooo. org/10........ 181.1..8555555 2311. 11268.
12

---

Published as a conference paper at ICLR 2026
Yinghui Li, Qingyu Zhou, Yuanzhen Luo, Shirong Ma, Yangning Li, Hai-Tao Zheng, Xuming Hu, and Philip S. Yu. When Ilms meet cunning questions: A fallacy understanding benchmark for large language models. CoRR, abs/2402.11100) 2024. doi: 1044550/ARXIV.200.11100 URL
https:: //oii .rgg/.r.g..... 4 arxii..iv..... 2400.11100.
Hunter Lightman. Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. arXiv preprint arXiv:2305.200000 2023.
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference Learning Representations, 2024. URL https: httpp:://ppeeeeeeeeeeeee /pwpn/evee..e.e.eeeeeee
net hest/Eorum?iice (itorrmmiivvvLLNNGG22
Runze Liu, Junqi Gao, Jian Zhao, Kaiyan Zhang, Xiu Li, Biqing Qi, Wanli Ouyang, and Bowen Zhou. Can 1b I1m surpass 405b Ilm? rethinking compute-optimal test-time scaling. arXiv preprint arXiv:2502.06703, 2025.
Dakota Mahan, Duy Van Phung, Rafael Rafailov, Chase Blagden, Nathan Lile, Louis Castricato, Jan-Philipp Fränken, Chelsea Finn, and Alon Albalak, Generative reward models. arXiv preprint arXiv:2410. 12832, 2024.
Maxwell-Jia. AIME 2024 dataset. http:://bugggngf.......aaatsssss AIME_2024, 2024. Accessed: 2025-05-15.
Ali Mohammadjafari, Anthony S Maida, and Raju Gottumukkala. From natural language to sql: Review of llm-based text-to-sql systems. arXiv preprint arXiv:2410.01066, 2024.
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling, 2025. URL https/////........//aaabsseeea
19393.
Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. Totto: A controlled table-to-text generation dataset. In EMNLP 2020, pages 1173-1186, 2020.
Panupong Pasupat and Percy Liang. Compositional semantic parsing on semi-structured tables. In Chengqing Zong and Michael Strube, editors, Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1470-1480, Beijing. China, July 2015a. Association for Computational Linguistics. doi: 10.3115/v1/P1555 1142. URL https: //
Panupong Pasupat and Percy Liang. Compositional semantic parsing on semi-structured tables. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers, 2015b.
Cheng Qian, Emre Can Acikgoz, Qi Hc, Hongru Wang, Xiusi Chen, Dilek Hakkani-Tür. Gokhan Tur, and Heng Ji. Toolrl: Reward is all tool learning needs, 2025. URL https: / aaapaaxxvvvvoo...//.a org/ abbs/2504 13958.
David Rein, Betty L. Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. Gpqa: A graduate-level google-proof q&a benchmark, 2023. URL https: / /awpiv.iiiv./////////aaa/.aaaav. org/ wtgg/bb/////eoooooooo/oooo//aae/ /2//2.11..20 12022.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:/707.000 1707.06377, 2017.
Kwangwook Seo, Donguk Kwon, and Dongha Lee. Mt-raig: Novel benchmark and evaluation framework for retrieval- -uggeette insight generation over multiple tables. arXiv preprint arXiv:2502. /1735, 2025.
13

---

Published as a conference paper at ICLR 2026
Amrith Setlur, Chirag Nagpal, Adam Fisch, Xinyang Geng, Jacob Eisenstein, Rishabh Agarwal, Alekh Agarwal, Jonathan Berant, and Aviral Kumar. Rewarding progress: Scaling automated process verifiers for Ilm reasoning. arXiv preprint arXiv:2410.08146, 2024.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL https://arxiv. arg/abs/2402.033000 2024.
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409. 19256, 2024.
Charlic Snell, Jachoon Lee, Kelvin Xu, and Aviral Kumar. Scaling IIm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXivv2408.033114 2024.
Aofeng Su, Aowen Wang, Chao Ye, Chen Zhou, Ga Zhang, Guangcheng Zhu, Haobo Wang, Haokai Xu, Hao Chen, Haoze Li, et al. Tablegpi2: A large multimodal model with tabular data integration. arXiv preprint arXiv:241 1.02059, 2024.
Yuan Sui, Jiaru Zou, Mengyu Zhou, Xinyi He, Lun Du, Shi Han, and Dongmei Zhang. Tap4Ilm: Table provider on sampling, augmenting, and packing semi-structured data for large language model reasoning. arXiv preprint arXiv:2312.0000. 2023.
Nan Tang, Ju Fan, Fangyi Li, Jianhong Tu, Xiaoyong Du, Guoliang Li, Sam Madden, and Mourad Ouzzani. Rpt: relational pre-trained transformer is almost all you need towards democratizing data preparation. arXiv preprint arXiv:20/2.02469 2020.
Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. Solving math word problems with process- and outcome-based feedback, 2022. URL https: /tass////rvvv..///a/ org/ abs 222222211111 14275.
Svitlana Vakulenko and Vadim Savenkov. Tableqa: Question answering on tabular data. arXiv preprint a
2017.
Jun Wang, Meng Fang, Ziyu Wan, Muning Wen, Jiachen Zhu, Anjic Liu, Ziqin Gong, Yan Song, Lei Chen, Lionel M Ni, et al. Openr: An open source framework for advanced reasoning with large language models. arXiv preprint arXiv:2410.09671 2024a.
Peiyi Wang. Lei Li, Zhihong Shao, R. X. Xu, Damai Dai, Yifei Li, Deli Chen, Y. Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce Ilms step-by-step without human annotations, 2024b. URL
org/ abs/ /2322222222222 2. 08935.
Zilong Wang, Hao Zhang, Chun-Liang Li, Julian Martin Eisenschlos, Vincent Perot, Zifeng Wang, Lesly Miculicich, Yasuhisa Fujii, Jingbo Shang, Chen Yu Lee, al. -haile:f-table: Evolving tables in the reasoning chain for table understanding. arXiv preprint arXiv:2401.043300 2024c.
Ziyi Wang, Siva Rajesh Kasa, Ankith M S. Santhosh Kumar Kasa, Jiaru Zou, Nan Jiang, Sumit Negi, Ruqi Zhang, and Qifan Song. DIVERSED: Relaxed speculative decoding via dynamic ensemble verification. In The 29th International Conference Artificial Intelligence and Statistics, 2026.
URL wtwpttsss/pppeeeeeeeette //wpnreeeiiieeeeeeeeeeeeeee net wwwp:/ent/ett/tieeoeoomo (eedddyyNtoooyyy.
Jian Wu, Linyi Yang, Dongyuan Li, Yuliang Ji, Manabu Okumura, and Yue Zhang. Mmqa: Evaluating Ilms with multi-table multi-hop hop complex questions. In The Thirteenth International Conference on Learning Representations, 2025a.
Xianjie Wu, Jian Yang, Linzheng Chai, Ge Zhang, Jiaheng Liu, Xinrun Du, Di Liang, Daixin Shu, Xianfu Cheng, Tianzhen Sun, al. Tablebench: A comprehensive and complex benchmark for table question answering. arXiv preprint arXiv:2408.09) 0774, 2024.
Zhenhe WI, Jian Yang, Jiaheng Liu, Xianjie Wu, Changzai Pan, Jie Zhang, Yu Zhao, Shuangyong Song, Yongxiang Li, and Zhoujun Li. Table-rl: Region-based reinforcement learning for table understanding. arXiv preprint arXiv:2505. /2415, 2025b.
14

---

Published as a conference paper at ICLR 2026
An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jian- hong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2.5 -wah2.5-math technical report: Toward mathematical expert model via self eelloomprovement. arXiv preprint (rXiv2249..12122 2024.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report, 2025a. URL https: pecttp:://axiis org/abs 2505. 09388.
Jian Yang, Shuming Ma, Dongdong Zhang, Zhoujun Li, and Ming Zhou. Improving neural machine translation with soft template prediction. In Dan Jurafsky, Joyce Chai, Natalic Schluter, and Joel R. Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 5979-5989. Association for Computational Linguistics, 2020. doi: 10.
URL https: //oii//diii/ii org/10. 100
18653/v1/202/ poll-miir. 531.
Zheyuan Yang, Lyuhao Chen, Arman Cohan, and Yilun Zhao. Table-rl: Inference-time scaling for table reasoning, 2025b. URL https:
23621.
Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning. arXiv preprint arXiv:2502.03387 2025.
Pengcheng Yin, Graham Neubig. Wen-tau Yih, and Sebastian Riedel. Tabert: Pretraining for joint understanding of textual and tabular data, arXiv preprint arXiv:2005.08314 2020.
Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang. Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, et al. Spider: A large-scale -abanel dataset for complex and cross-domain semantic parsing and text-to-sql task. In EMNLP 2018, pages 3911-3921, 2018.
Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, and Jie Tang. Rest-mets*; Llm self-training via process reward guided tree search. arXiv preprint 1eXiv:2406.03812 2024.
Tianshu Zhang, Xiang Yue, Yifei Li, and Huan Sun. Tablellama: Towards open large generalist models for tables. arXiv preprint arXiv:2311.00 1.092069 2023.
Xuanliang Zhang, Dingzirui Wang, Longxu Dou, Qingfu Zhu, and Wanxiang Che. A survey of table reasoning with large language models. Frontiers of Computer Science, 19(9):99348,, 2025a.
Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. The lessons of developing process reward models in mathematical reasoning. arXiv preprini arXiv:2501.07300 2025b.
Jian Zhao, Runze Liu, Kaiyan Zhang, Zhimu Zhou, Junqi Gao, Dong Li, Jiafei Lyu, Zhouyi Qian, Biqing Qi, Xiu Li, and Bowen Zhou. Genprm: Scaling test-time compute of process reward models via generative reasoning, 2025. URL https: arxiv. org/ alos 2504. 00891.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595-46611 2023.
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Pro- ceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), Bangkok, Thailand, 2024. Association for Computational Linguistics. URL http::tt:///aaa http:////axxvv./o org/ abs abs/2403... 13372.
Jialun Zhong, Wei Shen, Yanzeng Li, Songyang Gao, Hua Lu, Yicheng Chen, Yang Zhang, Wei Zhou, Jinjie Gu, and Lei Zou. comprehensive survey reward models: Taxonomy, applications, challenges, and future. arXiv preprini 2504 2025.
Victor Zhong, Caiming Xiong, and Richard Socher. Seq2sql: Generating structured queries from natural language using reinforcement learning. CoRR, 2017.
15

---

Published as a conference paper at ICLR 2026
Jiaru Zou, Dongqi Fu, Sirui Chen, Xinrui He, Zihao Li, Yada Zhu, Jiawei Han, and Jingrui He. Rag over tables: Hierarchical memory index, multi-stage retrieval, and benchmarking. arXiv preprint arXiv:2504.01346, 2025a.
Jiaru Zou, Ling Yang, Jingwen Gu, Jiahao Qiu, Ke Shen, Jingrui He, and Mengdi Wang. Reasonflux- PRM: Trajectory-aware PRMs for long chain-of-thought reasoning in LLMs. In The Thirty- ninth Annual Conference on Neural Information Processing Systems, 2025b. URL https: / //tppe/weeww..nee/eeeeeeeeeeeeee (tt.:/tttooommmddd33333333331
Jiaru Zou, Ling Yang. Yunzhe Qi, Sirui Chen, Mengting Ai, Ke Shen, Jingrui He, and Mengdi Wang. Autotool: Dynamic tool selection and integration for agentic reasoning. arXiv preprint 137733,512.13278 2025c.
16

---

Published as a conference paper at ICLR 2026
TABLE OF CONTENTS
A The Use of LLMss
18
Additional Related Work
18
Detailed Error Analysis
18
D TATTOO Data Curation Pipline
20
Proof of Theorem 4.1
21
Experimental Setups
23
F.1
Policy Model Configurations
23
F22
Evaluation Dataset Details
23
F3
Training Details
24
Additional Experiments
24
G.1
Comparison with ORM Baselines
24
G.2 TaTToo on RL-Trained Table-RI
25
G.3 Ablation Study on TATTOO
25
G.4 Efficiency Analyses on TATTOO
25
G.5 Performance Gain of TATTOO with Increasing Number of Responses
26
H Case Study on TATTOO
26
I Limitations and Broader Impacts
27
17

---

Published as a conference paper at ICLR 2026
Appendix
A
THE USE OF LLMS
LLMs were used in this work in two main capacities. First, they served as the base models whose outputs were ranked and evaluated by our proposed TablePRM framework. We tested across variety of publicly available pre-trained models, including Qwen and LLaMA families, ensure robustness and generality, as described in Section and Appendix F. Second, during the construction of preference pairs for supervision, we used LLM -_M-ggeeertted responses as candidates, which were then compared and ranked according to factual consistency with gold tables. In addition, LLMs were used limited capacity for writing assistance, specifically to improve phrasing and readability of manuscript. They did not contribute to research design, methodological innovations, experimental results; all scientific contributions are the responsibility of the authors.
B ADDITIONAL RELATED WORK
Discriminative vs. Generative PRM. In general, PRMs can be categorized discriminative and generative evaluators (Zhong et al., 2025). A discriminative PRM treats verification as classification, directly predicting the correctness of each reasoning step with a scalar score. It is typically trained on step-level labels using cross-entropy loss, making it heavily reliant on step-level reward annotations. A generative PRM instead frames verification as conditional generation. is trained with the standard language modeling objective to first generate rationales and then verify each step's correctness via judgment token (e.g., [correct, incorrect)).
C DETAILED ERROR ANALYSIS
Error Distribution by Type
logical error others
(0.0%)
contradictory steps
(2,85))
misinterpretation
(9.89)
double counting
incorrect grouping context missing output format partial aggregation column mis-seles sis-seeletiin miscalculation row mis-selection unit mismatch
(7.35)
38 (7.6%)
(8((745)
52 (10.4%)
(13.25))
75 (15.0%
10
20
30
40 Count
50
60
70
Figure Error distribution over 500 incorrect LRM responses after Best-of-N, The errors are grouped into 13 predefined types, with the majority arising from table retrieval and schema interaction.
In Section 3, perform fine fine-grained error analysis 500 erroneous responses sampled after Best-of-N selection with Siven2.5-MtthPRRM-77 to better understand the limitations of LRMs and PRMs. Each response is inspected and categorized by human experts into 13 predefined error types, covering both reasoning and table-specific mistakes. Figure illustrates the overall error distribution.
Error Type Distribution. The most frequent errors are unit mismatch (15.0%). row mis- mis-selection (14.8%), and miscalculation (11.2%). Other common issues include column mis mis-selection (10.4%), partial aggregation (9.6%), and missing or incomplete context (7.6%). Less frequent but still notable categories include output format errors, incorrect grouping, double counting, misinterpretation. and
18

---

Published as a conference paper at ICLR 2026
contradictory steps. small portion of errors is grouped under others and logical errors. This diverse distribution highlights that model failures are not restricted to arithmetic slips but extend to schema understanding and structural reasoning.
Mapping to Reasoning-Stes Categories. To reveal deeper patterns, we align the 13 error types with four reasoning masoning-steep step categories reflecting the typical flow LRMs:
Table Retrieval Step: Includes row/column mis selectio,,,,, unit mismatch, and partial aggregation. These account for 47.7% total errors, indicating difficulty locating and extracting the correct
table region.
Schema Interaction Step: Covers miscalculation, grouping mistakes, double counting, and misinterpretation of table semantics. This represents 34.3% errors, reflecting challenges in reasoning over structured contents once retrieved.
Tmmr--hinniin Step: Logical errors or contradictory reasoning steps independent of table contents. These contribute 12.0% of total errors, suggesting LRMs remain relatively competent pure logical chains compared to table-centric operations.
Others:
Errors arising from context omission or improper output formatting.
table-related operations, including table retrieval and schema interaction, rather than general logical reasoning. PRMs, when supervising such steps, face greater challenges since they must not only validate the correctness of reasoning but also verify alignment between the retrieved sub-table and the query.
Key Findings. The analysis confirms that most model weaknesses
19

---

Published as a conference paper at ICLR 2026
D TATTOO DATA CURATION PIPLINE
We design large-scale data curation pipeline that simulates real-world scenarios of PRM tool use and step verification scale. As illustrated in Figure there are three main stages:
Reasoning Trajectory Generation. We begin by collecting trajectory responses from expert LRMs (e.g.. DeepSeek-RI (Guo et al., 2025) and Claude-Opus-4.1 (Anthropic, 2025)) table-based
questions drawn from diverse benchmarks, including TableInstruct (Wu et al., 2024), HybridQA (Chen al., 2020), ToTTo (Parikh al., 2020), and WikiTQ (Pasupat and Liang. 2015b).
We generate multiple model responses per query, capturing both correct and incorrect reasoning patterns. We then adopt dual-verification procedure (Feng al., 2025a), where both human annotators and expert LLMs are employed to examine and filter out low-quality incomplete CoT data. Through this, we receive high-quality set of LRMs' output responses Tpoal for subsequent data labeling.
Verification Synthesis Reward Assignment. Our next step is to provide step-level verification rationales and assign PRM step-reward labels for each candidate response in Tooo) To this end, we first identify the table retrieval and schema interaction steps within each response in Tpoot:
Table retrieval steps We first extract the retrieved sub-table from each step. Then we apply LLM-as- a-judge to evaluate whether retrieved contents are accurate and provide complete rationales for the judgment. We assign step-level table reward Tr.tb E {-1,1} (in Eq. 1) based on the correctness of the retrieval, while setting rs, to O. This reward supervision explicitly trains PRMs to recognize the retrieved sub-table aligns with the input query, addressing the limitation shown in Takeaway 1.
Schema interaction steps and use it as table prefix. If the retrieval is incorrect, manually replace it with the correct sub-table corresponding to the query. We then prepend this table prefix to the verification rationale generated by LLM-as-a-judge. Finally, we assign the PRM's step-level table reward ri E {- based on the correctness of the schemaa interaction, and ria to 0. By explicitly attaching the retrieved sub-table each schema interaction step, we mitigate the dependencies issue noted in Takeaway 2.
We collect the sub-table retrieved from the preceding table retrieval step
Other steps without table operations involved - We directly query expert LLM (DeepSeck-Ri) to generate verification rationales. We assign the PRM's step-level reasoning reward rirea E {-1, based on the correctness of the reasoning, while setting the table reward Ti,ub to 0.
Tool Use Synthesis. To help PRMs learn to leverage tools for more accurate verification, we augment the collected verification rationales by incorporating tool invocation, execution, and feedback into the verification steps. Specifically, whenever the model's inner reasoning involves a calculation or table lookup operation, we replace it with the corresponding tool call and its execution result. We primarily employ two types of tools:
Computation tools Applying Python SQL code snippets for arithmetic or aggregation operations. E.g., a step verifies the sum of a table column, we replace the model's manual calculation with code snippet that executes the summation and returns the result.
Table lookup tools Locating and extracting specific rows, columns, or cells from the table. E.g.. if a step requires referencing sub-table cell value during the verification, we replace the model's self-extraction with an explicit lookup tool call that retrieves the corresponding entry.
By integrating verification processes with code snippets and real-time interpreter feedback, construct roughly 60k data for ToTToO's verification reasoning and tool usage.
20

---

Published as a conference paper at ICLR 2026
E PROOF OF THEOREM 4.
Notational conventions. We use s; for state, a; for action, for the current policy, and x' for the updated policy. The advantage is defined as
(6) The PRM signal at step is the overall process reward, defined in Eq.2. For a fixed st, we
(s a.) (s, (s.)..
write E+ = Barst(s) Var = (ss and Co a (s;, a.))
Cov Cova,nt(/ss (S,, a;), (i,eea a;)) Expectations over states use the subscript explicitly, e.g.,
Bxpsp() We use dy for the state distribution induced by the policy x' starting from the initial distribution p. Finally, X 2 Y means there exists a universal constant > 0, independent of (T,, T (')), such that X cYY
We start the proof by introducing two standard lemmas that will be used repeatedly; both are well-known results in the RI. literature, and we omit their proofs here for brevity.
Lemma E.1 (Performance Difference Lemma (PDL)). For any pair of policies a and n' defined over the same Markov decision process with initial state distribution p. the following identity holds:
Eximp VT (s.) - (s.)) = Ba Bui (118; (1 (st,
See proof of Lemma 6.1 in (Kakade and Langford, 2002). Lemma E.2 (Natural policy gradient (NPG) update form). Fix a step size > O. If the NPG update is guided by the signal A (Si, 0s) + (S;, 0z), then
T' (a; 8:)
T(a s:) exp (A (s;, + +s)ss))))))
ndd (s.)
A Σ (a, | st) exp (7 (17 a:) + (s, (8))
exp 7 az) tr (s ai))
r'a (s)
so that
T(a s:)
(s;ss)
See proof of Lemma F.2 in (Setlur et al., 2024). Next, we restate Theorem 4.1 in the following proposition.
Proposition E.3 (Full-strength policy improvement lower bound). Let n² be the NPG update in Lemma E.2. We can have:
Ex-op (s.)) - ((())
2 Ex/=p Far=-[iiron (S a:)
+ Var=[Titath (s;, a:)]
distinguishabbiiity (reasoning reward))
distinguishability (table reward)
+2 Cov (Tisre az), Fi Fi,aabl (Si, a;))
+ E [r (S;, a:) AT a:)]
+ Em [r (s; 0;) A (s; a:)]
alignment between mo and (,)
alienment "i,nb with A
alignment Piove with A
Proof of Proposition E.3. We now combine the performance difference lemma with the NPG update to derive variance-aligmment lower bound, while first retaining the covariance term between the reward components. By Lemma E. have
E Es V (s.) V* (s:) -E Ba Ba, (st
Exponential tilting and a log- eag--tatition bound. Let us define the log- sege-patiiion at state s, by
log (s:) log E, (s)) (A + re(ss), ai)).
From Lemma E.2, we have
1 a'(a; st)
1 log ndd (s:).
A (s,, a.)
= log
(s as) +
7
T(a; s:)
7
21

---

Published as a conference paper at ICLR 2026
Averaging over a, x'(- (+ 8;), using E, (log I'1 2 0, Jensen's inequality log Z* and En [A" (s,, 0:)] = gives
Baror' ((-aa (A" (s;, at)]
(1a)) a:)] + E. (a,
(10)
Plugging this into equation 9 yields the basic inner-product lower bound
Ea V (s.) - (*))) 2 B Ba (5 s.) - T(. s:), rs(s
(11)
Using first-order expansion of the exponential tilt implies
(T' [() x- st), (sa. +) 2 (Varx[ri(s,, a:)] + E. A as),
(12)
Combining equation 11 and equation 12, and weakening dy to (componentwise monotonicity) gives
Es (V VT(s) 2 Ear [ars(ri(ss, a:)] + Ee(ri(s,, as) (s)) (13)
Variance decomposition with covariance. Next, using r3 Fi,ra + Ti,tb, have
/arg/ri(s;, [s)) Vareerireeffe a:)] Varz/Ti,na a:)] (ra a:), Fi,ta aa (Si, a:)). (14)
Substituting into equation 13 complete proof of Proposition E.3 (equation 8).
Covariance elimination under our reward design. By construction in our setup (see Section 4.2),
for each state-action pair (st, az), the two components of the PRM signal, i.c., table reward and reasoning reward, are mutually exclusive. Formally, we have
Ti,ab (S; a.) E f-1,0,1): (s; a:) E 1-1, 0, 1}, and Ti,ub as) (st, a,) = 0.
Dolicy-gradient updates are invariant adding any per-state baseline, so we may center each component without loss, i.e.,
(Si, a.) = ri (S;; Ex) az)];
(Si ai) = Fi,ubb (8i, (.)-Errl Eriat as)]
Mutual exclusivity yields [Fr (s,, az) Tu a:)] hence Cova(Fi,nen Fr,ta)) 0 and
Varz/i:(s;; a:) Var=/P rea (s; a;) + Var, (r, a;)) Fg & Pizeen + fi,ub.
Plugging these centered quantities into the bounds of Proposition E.3 (which is NPG-invariant under per-state centering) gives exactly Theorem 4.1's inequality:
E Exixp V (s:) (((i)) 2 Expop
a:)] Var a:)]
(15)
+ +EE a;) A (s; a:)] Er [rirreee ai) A (si, ai]]]
which completes the proof of Theorem 4.1.
Remarks. (i) Proposition E.3 strictly more general; Theorem 4.1 follows as a corollary under mutual exclusivity plus per-state centering (baseline invariance). (ii) Mutual exclusivity alone yields Ex, [Farea ri. = 0, but per-state centering is what ensures Cov, (Fiiees Titta)) = O. (iii) The alignment term necessarily uses the composite signal r; because the NPG step is guided by A + rz-
22

---

Published as a conference paper at ICLR 2026
F EXPERIMENTAL SETUPS
F.1 POLICY MODEL CONFIGURATIONS
In our experiments, we adopt LRM SeepSeek-R--Disill---vvee downstream policy model. During inference, we configure the model with temperature of 0.7, a maximum generation length of 16,384 tokens, and top-p sampling with p = 0.95. We evaluate the LRM on several inference-time scaling strategies:
(Guo et al., 2025) as the
Best-of-N (BoN). The policy model generates N candidate responses independently. A verifier (PRM) scores each response, and the final output is selected based on voting or scoring method.
Beam Search. Given beam width N and branching factor M, the model generates N initial steps. The verifier then selects the top N/M continuations, and the model expands each with M new candidates. This process repeats until termination, enabling guided exploration of high-quality reasoning paths.
Diverse Verifier Tree Search (DVTS). DVTS is a variant of beam search where the search process divided into multiple subtrees. Each subtree is explored independently using verifier-guided expansions, with candidates selected at every step based on PRM scores.
Majority Voting. After generating multiple responses, the final answer is determined by simple majority over identical outputs, regardless of intermediate step scores. This method provides baseline aggregation mechanism.
LLM-as-a-Judge. Instead of relying solely on PRMs, a separate LLM is prompted to compare and evaluate candidate responses directly, selecting the most plausible or logically consistent output.
F.2 EVALUATION DATASET DETAILS
TableBench (Wu et al., 2024). TableBench is comprehensive benchmark specifically designed to evaluate the reasoning abilities of LLMSs over tabular data. It consists of 3,681 unique tables drawn from diverse domains such as finance, sports, politics, and science, with each table containing on average 16.7 rows and 6.7 columns. The dataset emphasizes numerical reasoning, with over 65% of table cells containing numerical values. TableBench questions are organized into four major categories: fact-checking, numerical reasoning, data analysis, further divided into 18 subcategories, yielding a total of 886 carefully annotated samples. Each question typically requires 6.3 reasoning steps, making the dataset significantly more complex than prior TableQA corpora.
WiliiTableQQestionns (WTQ) (Pasupat and Liang, 2015b). The WikiTableQuestion dataset introduces question answering over semi-structured HTML tables, aiming to test both compositional reasoning and domain generalization. It comprises 22,033 natural language questions paired with 2,108 Wikipedia tables, where the training and test tables are disjoint to ensure generalization to unseen schemas. The tables are stri-structured and heterogeneous, often containing multi-part cell values (c.g.. "Beijing, China") that require normalization into multiple semantic types such as numbers or dates. Questions range from simple lookups highly compositional queries involving comparison, aggregation, arithmetic, and superlatives. Each table contains at least 8 rows and 5 columns, and the question collection was conducted with quality control through multiple annotators.
MMQA (Wu al., 2025a) MMQA a large-scale benchmark for evaluating LLMSs on multi-table and multi-hop question answering. The benchmark includes total of 3,312 relational tables across 138 domains, where each instance consists of two or three interlinked tables. The dataset features 5,000 multi-table samples, annotated with natural language questions, SQL queries, gold answers, and explicit primary/foreign key relations. To ensure annotation quality, foreign and primary keys were labeled by human experts with inter-annotator agreement exceeding 80%. MMQA questions span four main categories, including numerical, list, count, and select, with an average length of 77-85 tokens, reflecting their compositional complexity.
23

---

Published as a conference paper at ICLR 2026
Table 4: Comparison of TATTOO with Output-Reward-Monn (ORM) baselines.
Method (Best-of-16)
TB-NR TB-FC
TB-DA
WTQ
MMQA
Discriminative ORM
66.4
72.0
26.8
68.1
25.3
Generative ORM
70.6
75.9
28.5
69.2
26.6
TaTToo
76.4
81.2
33.6
73.5
29.1
F.3 TRAINING DETAILS
We train TATTOO using the off-the-shelf Qwen-3-8B model (Yang et al., 2025a) on our curated 60k dataset. For supervised fine-tuning, we adopt the LLaMA-Factory framework (Zheng et al., 2024). The training setup uses learning rate of 1 x 10-5, 5, a weight decay of x 10-4, maximum sequence length of 20,000, and is run for 3 epochs. For the RL training stage, we adopt the VeRL framework (Sheng et al., 2024) further optimize the SFT checkpoint via policy optimization. The model is trained with batch size of 32, generating 8 samples per question as the group size, and is run for 3 epochs. During inference, we use the OpenR framework (Wang et al., 2024a) deploy our trained TATToO-8D which serves verifier to guide the downstream LRM under different test-time scaling strategies.
G ADDITIONAL EXPERIMENTS
G.1 COMPARISON WITH ORM BASELINES
To evaluate the efficacy of our approach, we compare TaTToo against distinct Outcome Reward Model (ORM) baselines, both implemented on the same Qwen3-8B backbone. The comparison is reported Table 4. We detail the training and implementation settings for these ORM baselines below. For the policy model and all additional configurations, we follow exactly the same setup as outlined in our main experiments. First, following (Mahan et al., 2024) and our established experimental settings, we employ Generative ORM trained via dual-stage process (SFT followed by RL) using our curated dataset. In this configuration, the model is optimized to generate a rationale chain before producing final output-level reward token ("correct" or "incorrect") for each complete instance. Second, adhering to the methodologies of (Hosseini et al., 2024; Cobbe al., 2021), I), we implement Discriminative ORM by training a classification head directly on the backbone model. This baseline is designed to classify whether a candidate answer's final solution correct or incorrect without generating any intermediate rationales.
In Table 4, we show that TaTToo provides significantly stronger supervision than both ORM base- lines. This performance advantage indicates that TaTToo's 'ooos process-level supervision delivers denser and richer reward modeling signals, which in turn contribute positively to the performance of the downstream policy model. Furthermore, when comparing the two baselines, the Generative ORM consistently outperforms the Discriminative ORM. This gap suggests that the inclusion of generative rationales offers more informative supervision than binary correctness labels alone. As the Generative ORM effectively leverages the reasoning paths within our training data, this result further highlights the rich verification rationales and high-quality supervision signals provided by our curated dataset.
Table 5:
Performance of Table R1 1-eeero with and without TaTToo under the Best-of-N setting. TaTToo consistently improves the RL-trained table reasoner across all tasks.
Method
TB-NR TB-FC
TB-DA
WTQ
MMQA
Table-RI-Zero
34.8
61.6
16.4
77.3
24.2
Table-R1 -Zero + TaTToo (BoN-4)
39.6
64.7
18.3
80.9
26.8
Table-RI ---ero + TaTToo (BoN-8)
45.1
69.0
20.1
82.6
28.4
Table-R1-Zero + TaTToo (BoN-16)
48.2
72.3
23.5
84.5
30.3
24

---

Published as a conference paper at ICLR 2026
G.2 TATTOO ON RL-TRAINED RIBLE-RI
To demonstrate that TaTToo is complementary to RL-based table-reasoning methods, we further evaluate its performance when paired with Sable-R1-Zent (Yang et al., 2025b), a 7B table-specialist policy model trained with program-based reinforcement learning. Under the Best-of-N test-time sampling strategy (NV E {4, 8, 16)). we apply TaTToo as the verifier following the same evaluation protocol described in Section 5 and compare Table-RI-Zeru alone against Table-RI-Zer augmented with ToT across all five tasks. As shown in Table 5, incorporating ToTToo consistently improves accuracy and yields larger gains as N increases, indicating that ToTToo reliably identifies higher- quality trajectories during test-time scaling. These results further confirm that ToTToo is architecture- agnostic and serves as flexible verifier that enhances diverse table-reasoning systems, including RL-trained approaches such as Table-R1.
G.3 ABLATION STUDY ON TATTOO
Table 6: Ablation on confidence calibration Aeal-
Table 7: Ablation on tool-grounding Atool-
N=32
TB-NR
TB-FC
TB-DA
N=32
TB-NR TB-FC
TB-DA
0.3
76.8
80.9
33.1
0.1
75.2
76.3
30.8
0.5
77.3
81.3
33.6
0.5
75.9
76.9
32.2
0.8
78.1
82.0
34.3
1.0
78.1
82.0
34.3
1.0
78.5
81.4
33.8
1.3
77.5
81.2
34.6
Ablations Xeal and Auaa- In Eq. 3, the contributions the corresponding reward terms in GRPO. To examine their influence, we separately train our verifier model (initialized from the same SFT checkpoint) by varying Aaal {0.3, 0.5, 0.8, 1.0} and Avool E {0.1, 0.5, 1.0, 1.5} during RL, and then evaluate on TableBench with 3 As shown in Table and 7, performance improves Aeal increases, peaking 0.8-1.0. For Auat accuracy rises steadily and is strongest around 1.0-1.3. These results empirically confirm the effectiveness of confidence calibration and tool-grot mol-grounding in enhancing TTS.
Aeal and Aosu tunable coefficients to balance
G.4 EFFICIENCY ANALYSES ON TATTOO
To ensure computational efficiency, design prioritizes concise data curation. As detailed in Section 4.2, we implement a progressive filtering strategy that yields high-quality training set of approximately 60,000 examples. This dataset size is significantly more compact than standard Process Reward Model (PRM) corpora; for comparison, the baseline Qwen-PRM models utilized in our experiments required roughly 800,000 training samples (Lightman al., 2023; Zhang et al., 2025b). Leveraging this compact dataset alongside lightweight backhone allows the complete TaTToo training process conclude in fewer than 8 GPU hours. Beyond efficiency, the resulting PRM demonstrates robust generalization capabilities. Once trained, TaTToo provides effective step- level supervision across diverse spectrum of tabular reasoning tasks, including question answering, fact-checking, and data analysis, while still remaining compatible with various Test- Time Selection (TTS) strategies such as Best-of-N and Beam Search. This versatility underscores TaTToo's broad applicability across the tabular reasoning domain.
Table 8: Computational cost breakdown of ToTToo training stages.
ToTToo Training Stage
Training Data Size # of Training Steps GPU Setup Total Hours
Total Cost
SFT RL
50K
~400
8x A100
5,4
$25.9
10K
~80
8xA100
8.3
$39.8
To quantify the accessibility of our method, we analyze the training overhead of ToTToo. We present detailed cost breakdown for the 8B model, encompassing both the Supervised Fine- Tine-Tuning (SFT) and Reinforcement Learning (RL) stages. To provide a tangible economic metric, we evaluate the total GPU computational cost based standardized pricing rate of $4.8 per 8-GPU hours.
25

---

Published as a conference paper at ICLR 2026
TB-NR
80
Accuracy
75
70
65
GenPRMM 328
Skywonk PRM178
ToTToo
60
2
2
2
24
23
Best-of-N
Figure 8: Performance of TATTOO and baseline PRMs on TB-NR under Best-of-N test time scaling. While baseline models plateau as N increases, TATToO continues to scale effectively, yielding consistent accuracy gains.
While the RL stage utilizes only approximately one-fifth of the data and one-ninth of the training steps relative the SFT phase, it incurs higher computational cost due the overhead associated with rollout sampling and tool executions. However, the absolute duration of the RL stage remains limited. As demonstrated in Table 8. given the substantial performance improvements observed over the SFT-only baseline, this additional computational expenditure represents favorable efficiency- effectiveness tradeoff. Overall, the complete TaTToo training pipeline requires approximately 14 GPU hours, translating to cost of roughly $65. This resource requirement is considerably lower than that of training larger PRM baselines, such as those with 32B or 72B parameters.
G.S PERFORMANCE GAIN OF TATTOO WITH INCREASING NUMBER OF RESPONSES
Figure 8 presents the best-of-N performance TB-NR. We observe that baseline PRMs such as Qwen2.5-Math-PRRM- and GenPRM-32B quickly saturate beyond N=16, achieving only marginal improvements larger N. Skywork-PRM-77B shows even weaker scalability, plateauing below 71%. In contrast, TATTOO continues to improve steadily as N increases, reaching 78.3% N=32, the highest among all models despite having significantly fewer parameters (8B). These results highlight the scalability advantage of TATTOO, demonstrating its ability to leverage larger response pools more effectively than existing PRMs.
H CASE STUDY ON TATTOO
As shown in Figure 9, we present a case study on TATTOO illustrating the difference between the verification processes at the two training stages specific instance in Figure 9 (Appendix H). When facing the same step (Step 3), the SFT-stage relies on inner text reasoning to verify the calculation, but introduces numerical errors that lead to incorrect justification of the step's correctness. In contrast, the RL-stage learns to leverage the computation tool with concise Python code, ensuring accurate calculations and thereby providing In addition, we randomly sample 500 trajectories from both stages of ToTToO on the same set of inputs and observe 26.3% improvement in the tool-integration ratio after RI. training, indicating our model learns to utilize tools better for seep- verification during RL rollouts.
reliable reward supervision the policy model's d's responses.
26

---

Published as a conference paper at ICLR 2026
I LIMITATIONS AND BROADER IMPACTS
While reinforcement learning with reward shaping enhances our PRM's ability to capture fine- grained tabular reasoning signals. introduces more computational overhead. Compared SFT-only training, the RL stage requires additional rollouts, reward evaluations, and optimization steps, which can increase training cost and resouree demands. This overhead may hinder reproducibility and accessibility in low-resource environments, motivating future work on more efficient reward objectives and lightweight reward modeling strategies. addition, our current framework limited to text-table reasoning, and extending it to multimodal settings (e.g., integrating charts or image-based tables) remains an important direction for future work.
From a broader perspective, this work highlights the potential for process reward models to enhance structured reasoning in domains such as fact-checking, scientific analysis, and decision support. At the same time, reliance on automated verification carries risks: if tools or training data contain errors, these may be amplified rather than corrected. We encourage future research to explore mechanisms for auditing verifier reliability, reducing the energy footprint of RL training, and ensuring equitable performance across diverse application domains.
27

---

Published as a conference paper at ICLR 2026
Case Study TATTOO
(i) Before RL Training
Okay, need verify step by step for the provided answer determine each step correct or not.
[Step Evaluation]
For step 0, let me first check whether it retrieves the correct information from the input table [Step Evaluation]
Looking at step and table data provided, I need to verify whether the summarization calculation on column "capacity in persons hour" is correct. Let me add each part together:
350 500 970 10000 2000 2800 650 650 1000 1000 720 820 1000 16,920 (wrong PRM calculation)
Since the provided step 3 gives a final calculation result of 22,460, which is not equal to 16,920. Step 3 is incorrect due to the wrong summarization the overall carrying capacity.
The step is \boxed (incorrect) Wrong Step Reward)
(ii) After RL Training
Alright, I given complete table, question, and corresponding answer. I need check the correctness of each step the answer. Let me verify step by step:
[Step Evaluation]
start checking if Step 0 pulls the proper information from the table
[Step Evaluation]
To verify Step 3, need to check the summarization (sum) over the "capacity in persons hour" column. will write Python code to verify this using the input table.
Code Sandbox
table
"columns". ""name or route", "capacity in persons / hour"),
"data"
(betten betten village", 3501, [bbetten village bettmeralp", 500],
"ttrainerift", 820),
"lager 1", 1000],
...
1,
capacity_ index table("col ("columsss ". index ("capacity in persons / hour") total_capacity srm(row [capacitty for row in table("data"))
print ("Total capacity in persons/hour: ", total_ capacity
Code Interpreter
Total capacity persons/hour: 22460
Since computed total 22,460 aligns with the result in step 3, this step correct.
The step is \boxed (1 (correct) (V Correct Step Reward)
Figure 9: Case study TATToo before and after RL training. Applying RL training enhances PRM's dynamic utilization of tool calls, which in turn provides more reliable supervision over the input reasoning trajectories of LRMs.
28


Reviews:
[{"review_id": "EJiuWXMIKC", "reviewer": "Reviewer_sq6W", "summary": "This paper proposes a process reward model (PRM) named TATTOO to address the common failure modes in current RMs, like subtable retrieval, schema/column interaction, and so on. The authors build a dataset of 60k+ step-level annotations and design a two-stage training pipeline (SFT+RL) for the better PRM. Specifically, they propose a GRPO variant with label matching, confidence calibration, tool grounding as shaping signals to optimize stepwise validation and scoring. Experiments spanning five tasks show that TATTOO yields monotonic, non-saturating gains as N increases under Best-of-N, beam search, and DVTS, and surpasses strong baselines like Qwen2.5-Math-PRM-72B and GenPRM-32B.", "questions": "Please see the weaknesses above", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Re-diagnoses PRM failure from table-specific steps, clearly separating error sources into retrieval and schema interaction. The “table prefix” study shows existing PRMs’ insensitivity to distant retrieval context, motivating table-aware rewards + tool execution for stepwise verification.\n2. End-to-end data pipeline: from expert LRM trajectory pools and dual validation filters to replacing subjective heuristics with executable tools for verification.\n3. Introduces tool-supported reward shaping (label matching, confidence calibration, tool grounding) that explicitly rewards using tool outputs inside verification rationales—distinct from prior PRMs that rely on purely textual discrimination/generation.", "weaknesses": "The main weaknesses are in experiment settings. Addressing these would substantially strengthen the paper.\n1. The competitive baseline about outcome reward model is absent. And I wonder how an outcome reward model trained on your training samples (which are surely a solid, data-aspect contribution) performs.\n2. The policy model in experiments is Distilled-Qwen-14B while the reward model is trained from Qwen3-8B (weak policy model + strong reward model). And I wonder if TATTOO still works on strong policy models such as Qwen3-series (like Qwen3-32B, Qwen3-30B-A3B) or gpt-oss.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RliJflFCWd", "reviewer": "Reviewer_xVtW", "summary": "This paper addresses a critical issue that existing Process Reward Models (PRMs) struggle with table-specific operations and result in performance bottelnecks in tabular reasoning. To this end, this paper proposes TaTToo, a novel table-grounded PRM framework that provides precise reward supervision in reasoning steps. Valuable contributions contain the large-scale high-quality step-level annotations and tool-grounded dual-stage training paradigm involving table-aware supervised fine-tuning and RL training with step-level reward shaping. Experimental analysis demonstrates the effectiveness and generalizability of TaTToo.", "questions": "See the weaknesses part.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. This paper is very well organized and presented, integrating clear and logical descriptions of motivations, pre-analysis, theoretical analysis and experimental validation. I am very impressed to read this manuscript and learn much from it. \n\n2. The motivation of addressing step-level supervision in tabular reasoning is critical and impressive as a very novel contribution, as the limitations of previous LRMs have been demonstrated in the pre-analysis section. \n\n3. The design of the RL reward integrates multiple reasonable components, and the policy improvement has been validated by robust theoretical analysis.\n\n4. Experimental results clearly show the effectiveness and generalizability of TaTToo, revealing the contributions of the key modules of TaTToo.", "weaknesses": "1. TaTToo is trained based on Qwen3-8B and outperforms previous verifers with larger model sizes. I am curious about the result of the setting that utilizes the raw Qwen3-8B-model without further training, yet this setting is missing.\n\n2. As TaTToo integrates large-scale dual-stage training within TTS scenarios, the computational overhead may be significant. Is there some possible solutions to further improve the efficiency?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "R18OGpFxVQ", "reviewer": "Reviewer_ysau", "summary": "This paper introduces TaTToo (Tool-Grounded Thinking PRM), a Process Reward Model (PRM) framework specifically designed for Test-Time Scaling (TTS) in the domain of tabular reasoning. The authors empirically demonstrate that existing PRMs, which are widely adopted for supervising text-only reasoning steps, struggle with table-specific operations such as sub-table retrieval and schema interaction. To address this, TaTToo proposes a novel Tool-Grounded thinking process to finely supervise Large Reasoning Models (LRMs) during inference on structured data. The work leverages Reinforcement Learning (RL) and Reward Shaping to enhance the PRM's ability to capture fine-grained tabular reasoning signals, highlighting the significant potential of PRMs in structured reasoning domains like fact-checking, scientific analysis, and decision support.", "questions": "Q1:How do tool execution errors (e.g., code runtime exceptions) impact PRM reward supervision? The paper integrates tool outputs into rationales but lacks error handling mechanisms.\n​​Q2:The paper notes schema interaction steps fail due to locality bias, and TATTOO uses table prefixes as a fix. Does this scale to very long reasoning chains (e.g., >10 steps)? Are there plans for dynamic prefix mechanisms?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1: The paper is highly original, successfully extending the advanced PRM paradigm to the complex, structured domain of tabular reasoning. It precisely identifies the core limitation—the inability of current PRMs to handle table-specific operations—and provides a critical solution with the TaTToo framework. This is a vital step toward improving the reliability of LLMs on structured data.\nS2: The methodology is sophisticated, employing Reinforcement Learning and Reward Shaping to effectively capture and optimize fine-grained signals of the reasoning process, demonstrating high-quality research execution.\nS3: The paper is clear in articulating the problem, the proposed method, and its potential limitations. Method details (e.g., data pipeline) are well-described.", "weaknesses": "W1: The paper acknowledges the significant computational overhead from the RL stage, which may \"hinder reproducibility and accessibility in low-resource environments.\" The authors should provide a deeper discussion and specific efficiency analysis in the main body (e.g., training time/cost comparison to SFT-Only models). \nW2: Shallow comparison with recent tabular reasoning RL methods (e.g., Table-R1 series), only mentioned as related work.\nW3: A critical risk, as highlighted in the ethics section, is that reliance on automated verification could amplify errors if the underlying tools or training data are flawed. The authors should include experiments to measure TaTToo's robustness against tool errors (e.g., errors in SQL compilers/executors) or noisy training data, and propose concrete, preliminary mechanisms for auditing verifier reliability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)