High Accuracy, Less Talk (HALT): Reliable LLMs through Capability-Aligned Finetuning
Authors: Tim Franzmeyer
Archie Sravankumar
Lijuan Liu
Yuning Mao
Rui Hou
Sinong Wang
Jakob Nicolaus Foerster
Luke Zettlemoyer
Madian Khabsa
Venue: ICLR 2026 Poster (2026)
Keywords: LLMs
hallucination
abstention
Primary area: foundation or frontier models, including LLMs

Abstract: Large Language Models (LLMs) currently respond to every prompt. However, they can produce incorrect answers when they lack knowledge or capability -- a problem known as hallucination. We instead propose post-training an LLM to generate content only when confident in its correctness and to otherwise (partially) abstain. Specifically, our method, HALT, produces capability-aligned post-training data that encodes what the model can and cannot reliably generate. We generate this data by splitting responses of the pretrained LLM into factual fragments (atomic statements or reasoning steps), and use ground truth information to identify incorrect fragments. We achieve capability-aligned finetuning responses by either removing incorrect fragments or replacing them with "Unsure from Here" -- according to a tunable threshold that allows practitioners to trade off response completeness and mean correctness of the response's fragments. We finetune four open-source models for biography writing, mathematics, coding, and medicine with HALT for three different trade-off thresholds. HALT effectively trades off response completeness for correctness, increasing the mean correctness of response fragments by 15% on average, while resulting in a 4% improvement in the F1 score (mean of completeness and correctness of the response) compared to the relevant baselines. By tuning HALT for highest correctness, we train a single reliable Llama3-70B model with correctness increased from 51% to 87% across all four domains while maintaining 53% of the response completeness achieved with standard finetuning.

Published as a conference paper at ICLR 2026
HIGH ACCURACY, LESS TALK (HALT): RELIABLE LLMS VIA CAPABILITY-ALIGNED FINETUNING
Tim Franzmeyer"
Archie Sravankumar
Lijuan Liu
Yuning Mao
Rui Hou?
Sinong Wang
Jakob Foerster
Luke Zettlemoye
Madian Khabsa
University Oxford
Anthropic
3 Meta
University of Washington
Work done while at Meta.
ABSTRACT
Large Language Models (LLMs) currently respond every prompt. However, they can produce incorrect answers when they lack knowledge or capability problem known as hallucination. We instead propose post-training an LLM to generate content only when confident in its correctness and to otherwise (partially) abstain. Specifically, our method, HALT, produces capability-aligned post-training data that encodes what the model can and cannot reliably generate. We generate this data by splitting responses of the pretrained LLM into factual fragments (atomic state- ments reasoning steps), and use ground truth information to identify incorrect fragments. We achieve capability-aligna incorrect fragments replacing them with "Unsure from Here" according a tunable threshold that allows practitioners to trade off response completeness and mean correctness of the response's 's fragments. We finetune four open-source models for biography writing, mathematics, coding, and medicine with HALT for three different trade-off thresholds. HALT effectively trades off response com- pleteness for correctness, increasing the mean correctness of response fragments by 15% on average. By tuning HALT for highest correctness, we train single reliable Llama3-70B model with correctness increased from 51 % to 87% across all four domains while maintaining 53% of the response completeness achieved with standard finetuning.
finetuning responses by either removing
INTRODUCTION
Most current language models attempt to respond to the majority of prompts, regardless of how complex they are or how much domain knowledge is required to answer them. While this behavior is desirable for creative tasks (e.g.. poem writing), it undesired when factual correctness is crucial. This phenomenon, often referred to hallucination, poses risks in high-stakes applications such as medicine or law. We propose HALT, which finetunes an LLM to only respond with information is confident about, and (partially) abstain otherwise. For example, when asked write a person's biography, the LLM would list only facts it is confident in and would omit others. Similarly, when presented with complex math problem, would show only the reasoning steps in which it has high confidence, terminating its response with "Unsure from here" if it cannot proceed potentially forgoing a final answer. Naturally, restricting an LLM to provide only high-confidence information reduces the total number of correct statements, as it withholds any information it deems uncertain evvn if some of that information might ultimately be correct. To balance this trade-off, HALT allows practitioners to adjust a confidence threshold. This threshold controls whether the LLM should respond more eagerly (and risk occasional errors) or more conservatively (and risk omitting correct information).
We remark that no model will ever achieve perfect performance across all possible tasks, requiring ceaability-aligneee responses regardless of model strength (OpenAl 2025). Additionally, HALT is not about distilling from stronger model; rather, it directly addresses the problem ensuring that an LLM's outputs aligned with its uncertainty.

---

Published as a conference paper at ICLR 2026
Training Prompts
Test-Time User Prompts
Salve 4y 22--7-00
How can contact Mete?
Remander 125438 dnviddllty (1?
Tell about Obarr''s dog.
Standard Finetuning
Example Responses for Standard Finetuning
Example Resporses after Standard Fineturing
Compute 2-22-bab2--05
Format bx2-bxxec0 [..]
The email
support@mmta.com and the
Use the divasbill by rule digits numbers. Result 22
Their dog was called Bo and German Shophordd
Solutions: and
07767124521.
Beyond Copabiittiss Pretrained
Incorrect Responses from Firctuned LLM
HALT Finetuning
Exomple Responses used for HALT Fineturing
Example Responses after HALT Finetuning
Format mt2ebsenceO [.]
Compute x=2a-bhhb---a Aac..
Unsure from here.
The email is supportgmeta.com
Use the divisbiiity rule for 11.
Unsure from
Thair dag was called
Adjusted to Capabittoes of Pretrained LLM
Responses from HALT Finetured LLM
Figure I: Comparison between Standard and HALT Finetuning for LLMs:
Standard finetuning trains the LLM on responses that potentially exceed the pretrained LLM's capabilities, which results in incorrect outputs at test time. HALT finetuning trains the model only on content within the pretrained LLM's capability limits, omitting unknowns or replacing them with "Unsure from here". HALT finetuning improves response correctness as the I.I. is trained to generate responses according to its capabilities, which may result in partially incomplete responses when the pretrained L.M's capabilities are insufficient.
HALT's key insight is to finetune the pretrained LLM only on correct content it is capable of generating, i.e. content within the bounds of the knowledge and reasoning capabilities obtained during pretraining, we illustrate in Figure 1. Further, HALT finetunes the L.MM to output "Unsure from here" when uncertain about the remainder of the generation.
HALT is motivated by the observation that LLMs possess internal calibration (Tian et al. 2023b
Kadavath et al. 2022). suggesting they can produce responses aligned with their internal confidence in the relevant facts or reasoning steps. HALT's approach of generating capability-aligned finetuning samples according to the pretrained LLM's capabilities is based on recent work (Lin et al. 2023
Zhou et al.] (024) which shows that LLMs do not acquire novel capabilities during finetuning but only learn to effectively utilize knowledge and reasoning capabilities obtained during pretraining. Moreover, HALT's approach of finetuning only on content that the LLM is capable of generating is supported by recent work (Gekhman et al. 2024 Kang et al., 2024) showing that finetuning LLMS on concepts unknown to them increases hallucination. While HALT requires additional model generations and post-processing for each pretrained LLM it is applied to, we believe this is reasonable, as recent work (Zhou et al., 2024) found that merely 1000 finetuning examples can be sufficient to finetune generally capable models. Unlike previous methods ( (Yadkori et al.) [2224] Farquhar et al. 2024 Cheng et al.] [024] |Brahman et al. 2024 Feng et al. [202]] /Zhang et al. 2024a, Kang et al. 2024). HALT requires no additional computations at test time, it does not rely on post-processing
or sampling. Instead, it produces finetuned model that attempts the best possible answer within its capability. Also, unlike earlier work, HALT extends to prompts involving not only knowledge retrieval but also complex reasoning.
Given a pretrained LLM and a finetuning dataset, for each prompt, HALT generates a response aligned to the capabilities of the LLM. For given prompt, the HALT pipeline (1) generates an initial response via few-shot prompting of the pretrained LLM, (2) splits it into factual fragments, (3) assesses correctness of individual fragments via an Evaluator LLM with access to ground truth information (e.g. the ground truth response), and (4) post-processe the fragments to arrive at the final HALT finetuning response- which only contains fragments that the pretrained LLM is capable of generating. Figure 2 provides an example of the pipeline.
For the second step of decomposing the response into fragments, we make the assumption that responses either consist of independent fragments or causally dependent fragments, i.e., fragments that build on top of each other in logical sequence. This distinction is made for each prompt based on its domain, e.g., math samples are assumed to consist of causally dependent fragments, while biographies are assumed to consist of independent ones.
2

---

Published as a conference paper at ICLR 2026
For responses composed of independent fragments, rely prior work (Song al. 2024 and use a finetuned LLM to extract atomic statements, i.e. independently verifiable statements. For example, "Barack Obama was born in Hawaii" is atomic statement that could be extracted from his biography. In step 3, the correctness of each fragment is assessed independently using the ground- truth-conditioned evaluator LLM (e.g. conditioned on the respective Wikipedia article). The HALT finetuning response (4) is then composed of all correct fragments, or set to "Unsure from here" if no fragments were correct.
For responses composed of causally dependent fragments, such as responses to math questions, HALT instead splits responses natural boundaries like new lines or equation statements. HALT then determines the fragment at which the first error occurs using an evaluator LLM with access to the ground truth response. The HALT response then composed by replacing the incorrect fragment and all subsequent fragments with a single "Unsure from here" statement, arriving at a capability- aliinedd finetuning response.
Importantly, HALT enables practitioners to choose the desired trade-off between response pleteness (analogous to Recall) and response correctness (analogous to Precision) according to the deployment scenario. This is effected by estimating model capabilities based on multiple responses of the pretrained LLM. For example, composing the HALT response from a response with thsn-than- average correct fragments yields conservative estimate of the model's capability, resulting in an L.M that responds more carefully.
We evaluate HALT wide variety of language models, including LLama3-8B. LLama3-70B, Gemma2-9B, and Mistral-7B, and on a series of tasks, including writing Wikipedia-style biographies, solving competition mathematics problems, answering medical questions, and solving coding prob- lems posed in natural language. We compare HALT to standard finetuning, prior work on factuality improvement (FactTune, (Tian et al. 2023a)). prior work on training models to abstain (IDK. (Cheng et al. 2024)), and a heuristic baseline that trims responses to match the response length of HALT. We split generated responses from prompts the eval dataset into factual fragments and assess the completeness and average correctness of fragments for each response. Tuning HALT based on our threshold allows increase the correctness by 17% for Llama3-70B on average. Across all settings, HALT improves the Fl score, i.e. the harmonic mean of completeness and correctness, on average by 4%, as compared to the baselines. Tuning HALT for correctness, we combine data from all domains and finetune single reliable ALama3-70BB model that achieves accuracy of 87% across the four domains - which is in contrast to 51% accuracy achieved with standard finetuning while maintaining 51% completeness of responses. Last, demonstrate that slightly modifying HALT to annotate fragments as "Unsure" instead of omitting them - informs users about uncertainty while retaining complete responses.
METHODS
Overview. A pretrained LLM M maps from input sequences to output sequences, M X' + D. Additionally, we have finetuning dataset D = (arj- Uj y j=1 which consists of m pairs of prompts zj and their corresponding ground truth responses yy- The goal of our method, which we refer to as HALT, is to create modified finetuning dataset = (I ""'""" where each target response
vff is aligned with the capabilities of the pretrained model M. We assume access to an evaluator & that takes as input a statement f and information J and determines whether f is correct or incorrect according to the given information, i.c., & (f + {0, 1}. This evaluation could be performed by a human annotator, an additional LLM, or any other means.
Figure 2 provides an overview of how HALT processes each prompt First, preliminary response is generated using the pretrained model M, then split into factual fragments. Next, the evaluator & assesses the correctness of individual fragments, using the given ground truth response y/, potentially supplemented by other relevant context. Finally, incorrect fragments are removed or replaced by "Unsure from here" arrive at the target response yf.
3

---

Published as a conference paper at ICLR 2026
Prompt
Few-Shot prompted response Pre-Trained
Response split
Fragments
Fragment Correctness Assessment with Evaluator
HALT Finetuning Response
Causally Dependent Response Fragments
Solve
4xx2-3xe7 -0
Format ax2-bx+c=0, then compute 2-2a-bbb2-2ac and find the solutions as x=3 and -10
(F2) Compute
(F1) Format ax2-bx-cc00
x-22-brbb-2aac
[F1] Format ax2+bx+c=0
(F2) Compute
c-22-bee2--2 2ac (F3) Sol: and x=-10
Unssuee from here.
Format as ax2+bxep=0.
(F3) Sol: x-3 andan-d- x=-10
Independent Response Fragments
IF') Benack Obemm is the 45th
(F1) Burask Obamaia the 458h
me about Barack Obama's dog!
Barack Obama, the 45th President of the USA, had deceased Garman Shepherd called Sunny.
prasidemt
(F2)
Surry:
Surny
Barack Obarna had dog called Suny.
(F3) Berock Oburm''s dog Surry deceused)
(F3) Burack Obama dog Surny
Opamn''s
(F4) Bareck Obama's dog Surry was Gorman Shaphard
Berman Sheplherd.
Figure 2: The HALT pipeline includes generating a preliminary response via few-shot prompting of the pretrained LLM, splitting into factual fragments, and assessing each fragment's 's correctness before compiling the HALT finetuning response. For causally dependent response, as displayed the top, an error results in an "Unsure from here" marker to indicate uncertainty. For response consisting of independent fragments, as shown at the bottom, incorrect fragments are removed.
2.1 CREATING CAPABILITY-ALIGNE FINETUNING DATASET WITH HALT
Generating the Preliminary Response. each prompt Zij, we first generate response that aligns with the format of the ground truth responses in the finetuning dataset 1) through few-shot prompting of the pretrained model. We sample a set of four
k= uniformly at random from D \ { Bj}} as in-context learning examples. The context C { 31)) )1 is concatenated with the target prompt z using
pairs {( }x)
quesstionansser format. The pretrained model M is then prompted to create a preliminary response vf"' for the prompt æj. i.. vt. = M (concat(C); x,)). where concat denotes the concatenation of the context Cj and the target prompt æ; into a single input sequence.
Splitting the Preliminary Response into Factual Fragments. Next, we split the preliminary responses v into individual factual fragments, denoted as v/" = (e), where each fragment represents verifiable unit. The method used for fragmentation generally depends on the type of response. Simple, inherently structured responses may allow fragmentation along natural boundaries, such as equal signs in math equations, bullet points or newline characters. Long and complex sentences may require
sophisticated methods for fragmentation, such as machine learning methods specifically trained to extract verifiable statements from complex sentences (Song et al. 2024 Min al.] 2023) Wei al. 2024). We empirically find that grouping prompts by type of question (e.g., math, coding, knowledge) allows us to determine the required fragmentation method. We describe fragmentation in Section 3.1
Assessing Correctness of Fragments. We distinguish between two types of response structures when determining the correctness the individual response fragments f1...., . fx: Responses con- sisting of either independent fragments, or of causally dependent fragments that build on each other in logical sequence. Figure 2 shows the HALT processing pipeline for example prompts of both categories. We note that not every response will perfectly fit one of the two categories but leave more complex dependency structures, such as dependency graphs, for future work.
For responses consisting of independent fragments, the correctness of each fragment is independent of the correctness of other fragments. Examples are Wikipedia-style information overviews, lists of independent recommendations, and results for independent tasks. For responses consisting of causally dependent fragments, the correctness of a fragment depends on the correctness of prior fragments. Examples step-by-step solutions, code implementations, sequential reasoning, where each part builds upon previous ones.
For responses consisting of independent fragments, assess the correctness of each fragment individually using the evaluator ε, with information J given by the ground truth response y, and potentially additional information. For responses consisting of causally dependent fragments,
4

---

Published as a conference paper at ICLR 2026
prompt the evaluator & conditioned on the ground truth response to identify the first incorrect fragment of the response, and consider all subsequent fragments as incorrect.
Creating the HALT Finetuning Response. For responses with independent fragments, we merely remove any incorrect fragments. For responses with causal dependency, we remove all statements starting from the first incorrect statement, and, in case any incorrect statements were present, place "Unsure from here" the end of the response, as illustrated in Figure 2
2.2 TRADING OFF BETWEEN RESPONSE CORRECTNESS AND COMPLETENESS
The trade-off between average correctness of fragments and response completeness can be tuned by choosing the preliminary response from set of sampled responses the pretrained LLM. Specifically, instead of relying single greedily decoded response to estimate the pretrained model's capabilities, we sample multiple preliminary responses. Selecting the "worst" response among the sampled preliminary responses, i.e., the response in which the lowest number of fragments is correct, provides a conservative estimate of the model's ' capability. Composing the HALT response based on the worst response results in finetuned model with higher correctness but lower completeness. The opposite holds true when selecting the best response instead.
Specifically, for each prompt F,, we sample N preliminary responses {v/ pL " with different Ju-1
random seeds and compute the average fragment correctness for each response outlined in Section
3.1 We sort the preliminary responses in ascending order of average correctness and select the a-percentile response as n' = =[aN] and v = ui Here, a determines the trade-off, where lower a favors correctness over completeness. The selected response v/5" is then processed with the HALT pipeline shown in Figure
EXPERIMENTAL VALIDATION
Overview. As HALT responses are derived from few-shot prompted responses of the pre-trained model, we first validate that finetuning on these yields performance similar to finetuning on the ground truth responses. We then assess the accuracy of our evaluator ε, which we implement using Llama3-405B. We observe in Table [ that the evaluator has an average per-response absolute error ranging from 0.27 to 1.14 fragments. We then evaluate HALT on four different datasets and for four open-source LLMs, namely Llama3-8B (Dubey et al. (0244) Llama3-70B (Dubey et al. 2024). Gemma2-9B (Team al. 2024), and Mistral-7B (Jiang al. 2023). We demonstrate that HALT allows to trade off between response completeness and correctness while simultaneously achieving the best arithmetic mean of completeness and correctness. Utilizing HALT. we train a single reliable model on all four datasets that achieves 87% correctness, an increase of 36% compared to the correctness achieved by standard finetuning. Last, we demonstrate that HALT can be modified to annotate response fragments that the model is unsure about as "Unsure" - instead of omitting them. This allows provide users with complete responses, while simultaneously informing users about likely incorrect parts of the response.
3.1 IMPLEMENTATION AND VALIDATION OF RELEVANT COMPONENTS
Prior Work and Baselines. We implement FactTune (Tian al. 2023a), IDK (Cheng et al. 2024 and supervised finetuning, which we refer to as "Unchanged". We remark that none of these baselined allow for adjusting the trade-off between completeness and correctness. FactTune, applicable to knowledge tasks, i.c. the Wikipedia dataset, first finetunes the pretrained model, c.g. on Wikipedia biographies. It then generates responses from the finetuned model, evaluates atomic statements using FactScore (Min et al. 2023), and creates a preference dataset to finetune the model again using Direct Preference Optimisation (Rafailov et al., 2024). IDK evaluates which prompts a model can answer, then finetunes the model to either entirely abstain or completely answer to given prompts. We additionally implement a RandomTrim baseline, which, instead removing fragments according to the capability of the pretrained model done with HALT, removes the last fragments of a given response. n is sampled from a Poisson distribution, which is chosen such that the average response length matches that of HALT responses.
5

---

Published as a conference paper at ICLR 2026
Wikip. Blographiss (Wikiboos)
Math (MATH)
Med. Questions (MadEXQA)
Coding (APPS)
075
0.05
0.30
2.00
Comeenese
0.45
0.76
0.52
0.60
0.00
0.2 Complatonoss
0.45
5:45 Comploeeeesa
0.86
0.66
000
0.23 Completeness
Complstanuss
Unchanged
HALT
RandomTrim
Abstain (IDK)
FactTune
Wkip. Biographies (Wikibios)
Math (MATH)
Med. Questions (MedExQA)
Coding (APPS)
Concerees
0.35
0.45
0.75
0.77
005
0.25
0.55
0:90
0.00
0.25
0.55
2.05
520 Completenees
5:86
0.05
0:40
0.75
D/02
0.17
0:36
Comgletenees
Completenees
Completeraes
Unchangad
RandomTrim
FacfTLnna
Figure 3: Response correctness (y-axis) and completeness (x-axis) for LLama3-70B (top) and Llama3-8B (bottom) when finetuned with different methods (desired number of fragments nau). Fl Score is constant along curved grey lines, and highest in the top right corner. For HALT and Randomtrim, results are shown with different trade-offs between completeness and correctness, where lighter colors indicate tuning for higher correctness. We omit results with less than 5% completeness, e.g., RandomTrin results for Wikipedia Biographies and Math. We observe that HALT allows for strongly influencing the trade-off between correctness and completeness across all four datasets while achieving higher Fl scores (closer to the top right corner) than baseline methods. We show results for Gemma2-9E and Mistral7B in Figure in the Appendíx.
Datasets. Wikibios (Lebret et al. 2016) contains the summary paragraph of individuals listed on Wikipedia, sampled uniformly randomly. Hence, this dataset contains many individuals who are only little known. Since the original dataset was assembled 8 years ago, we updated all information using the official Wikipedia API, and excluded all individuals their name on the Wikipedia API. We added each person's 's full Wikipedia article to the dataset, which used as additional information for the evaluator E. In the reasoning domain, we only consider datasets that provide step-by-step solutions, as we found that these increase the accuracy of the evaluator & significantly. MATH (Hendrycks et al. (022b) contains mathematical problems across various fields. MedExQA (Kim et 20.24 is medical mnestion-answering dataset covering five distinct specialties. APPS (Hendryeks al. 2021a) evaluates code generation, containing wide range of programming challenges; we consider those marked as "introductory"".
uniquely identified when searching for
Splitting Responses into Fragments. We decompose a response into factual fragments as follows. Wikipedia answers are often complex and nested, which prohibits heuristic- based fragmentation. We instead use state-of tfrthe fact-extraction LLM, made available by [Sng et al. (2024). Although responses well structured for MATH, found that heuristic-based fragmentation yields unsat- isfactory results and instead prompted III Lama3-405B to fragment responses at natural boundaries. For MedExQA found satisfactory to split answers full stop signs. Similarly, code responses in APPS are split line-for-line.
Evaluator Implementation. For Wikibios, we prompt Llama3-405F with the fragment assess, as well as with the entire Wikipedia article of the relevant person, and ask it to assess whether the given fragment is correct or incorrect in the given context (see example prompt App. B.T). Hence, the evaluator is called multiple times to assess a single response. For MATH, MedExQA, and APPS, we prompt Llama3-405B with the numbered fragments that make up the response and the ground-truth step-by-step solution, and prompt it to identify the first incorrect fragment.
F1 Score to Measure the Mean of Completeness and Correctness. We follow Wei et al. (0244) and use the Fl score combined measure of response correctness and completeness, analogous to

---

Published as a conference paper at ICLR 2026
Table I: Avg. number of fragments per response and avg. number of fragments incorrectly la- belled by the Llama3-405B evaluator. Analysis run on 300 manually- Llama3-708 re- sponses per dataset.
(Fragment 1) Format ax2+bxtce0 [..]
Completeness: 100%
(Fregment Compute 2=2s bub2-4ap
Correctness: 100%
(Pragment Solutions:
(Fragment Unsure from
(Fragment Format ax2+bx+c=01 [..]
Completeness: 30%
Correctness: 100%
F1:0.46
Wikipedia
MATH
Medical Q. Coding
(Fragment 1) Format ax2=bx+c=0 [..] (Fragment 2) Compute s=2a-b-b2-6an (Fragment Solutions: and
Completenes:: 30%
Avg fragm. 9.58
5.54
5.26
7.85
Correctness: 30%
Avg incorrect
0.27
0.63
0.41
1.14
F1:0.30
Completeness: 0%
Table 2: Finetuning models responses gen- erated via best-of-5 few-shot prompting results in only marginally lower performance (average correctness of statements) than finetuning on the Unchanged (ground truth) responses.
(Fragment 1) Unsure from here.
Correctness: 0%
F1:0
Figure 4: Example responses (ground truth re- sponse in the top left comee), with completeness, correctness, and F1 scores annotated for each. The number of required fragments here is three. We observe that the highest F1 score is achieved when the LLM answers with the correct first frag- ments and abstains after.
Pretrained Model
Responses
MATH
Medical Q.
Coding
Llama-3-SB
Row-Shot
Unchanged
0.25
0.30
0.68
0.69
0.30
0.27
Llama-3-70B
Few-Shot
Unchanged
0.81
0,80
0.35
0.38
Unchanged
0.30
0.69
0.30
Mistral-7B
Row-Shot
0.25
0.69
0.24
Gemma-2-9B
Fow-Shot
Unchanged
0.33
0.39
0.69
0.26
0.27
binary classification. Here, response completeness (Recall) measures the ratio of correct fragments Response correctness (Precision) is the relative
relative to the desired number fragments, i.e., Rsanna
Miduired
correctness of all given response fragments, i.c.,
Raumas
We consider two definitions for the desired
Rigies
number of fragments. The first defines the desired number of fragments as the number required to answer the question fully (see App.[A A.. for details), which we refer to as Rall (which is independent of the model's capability). Hence, any method can only achieve full recall under this definition if the pretrained I.LM's s capabilities are sufficient to answer all questions completely and correctly. The second option defines the number of desired fragments according to the pretrained LLM's capability, which we refer to as Reapable- Specifically, Reapable is the number of correct fragments the few-shot prompted response of the pretrained LLM. We remark that under the latter definition, method could attain full recall (and an score of 1) if the finetuned LLM answers exactly with those response fragments that within its capability limits. Figure 4 for different example responses, with the number of desired fragments defined as the number required answer the question fully (nai)).
shows response completeness and correctness
Wkkp. Blographies (Wikibios)
Math (MATH)
Med. Questions (MadEXQA)
Coding (APPS)
0.s
2.00
0.36
0.86
Correneeee
0.50
0.78
0.56
0.62
0.05
0.25
0.45
0.05
5:45 Compodeereea
0.50
0.70
0.90
0.00
0.17
0.35
Complatinnss
Complstaniss
Compliienea
Unichanged
HALT (Combined Model)
HALT
Figure 5: We show results for training a single reliable LLama3 70B model with HALT tuned for increased correctness, trained equal shares of all four datasets, referred to HALT (Combined Model). We additionally plot results for HALT trained on each dataset individually, referred to HALT, and results when finetuning on the Unchanged dataset. We observe that HALT allows for increasing average accuracy across all datasets by 36% to 87% while maintaining an average completeness of 25%.
Informatton-Conndttooned Llama-3-405B Yields Strong Correctness Evaluator. We evaluate the performance of our evaluator in assessing the correctness of individual fragments (denoted as fi,..., f& for response split into & fragments, defined in Section 4) of responses of all four
7

---

Published as a conference paper at ICLR 2026
datasets. For each dataset, we randomly sampled 300 responses generated by Llama3-70B and manually annotated the correctness of all fragments. We observe in Table I that the Llama3-4055 evaluator's 's error ranges from 0.27 incorrectly assessed atomic statements per Wikipedia response, composed of 9.58 atomic statements on average, to a discrepancy of 1.14 lines on Coding, with an average number of 7.85 lines of code per response. In MATH, the evaluator misjudges the location of the first incorrect fragment by only 0.6 reasoning steps, at an average response length of 5.54, We conclude that Llama3-405B prompted with relevant information well suited assess the correctness of individual logical fragments.
Finetuning on Few-Shot Prompted Responses is comparable to Finetuning on Ground Truth Responses, HALT relies on responses derived from few-shot prompting of the pretrained LLM. For the MATH, Medical QA, and Coding datasets, we compare the performance of the finetuned model when either finetuned on the dataset's 's unchanged ground truth response or the best-of-5 few-shot prompted response of the pretrained LLM. Table 2 shows that the average gap ranges from 1.7% to 3.7% only across the models, and could be further closed by sampling more responses. As a result, the expected correctness for finetuning methods that rely the ground truth response is 1.7% to 3.7% higher than that of HALT responses; however, we empirically find that HALT can still outperform such methods. We remark that for Mistral-7B, the gap is 5% and 6% in Math and Coding, respectively, suggesting that this model has worse in-context learning capabilities than the other examined models. As discussed later, we found that this performance gap influences the finetuned Mistral-7B HALT models. These findings support the assumption that LLMs do not acquire novel knowledge or capabilities during finetuning (Lin et al. 2023; Zhou et 2024).
3.2 HALT ENABLES TRADING OFF COMPLETENESS AND CORRECTNESS
Next, we finetune all four models on all four datasets, comparing HALT to the baselines. For HALT, we sample five few-shot prompted preliminary responses, sort them by relative correctness, choose a response according to a E [40%, 60%, 80%]. That is, we either choose the most, second most, or third most accurate response, and then process it with the HALT pipeline. We provide examples of We discard the response with
finetuning responses for different tradeoff parameters in App. B.2 the lowest and second lowest relative correctness, as we empirically found that finetuning on these can result in close to zero completeness and correctness. Figure 3 shows that choosing different for HALT allows to trade off correctness and completeness effectively, altering correctness by + 17% and + 12% for Llama3-70B and Llama3-8B, respectively. Table in the Appendix shows that HALT further achieves the highest harmonic mean of completeness and correctness in most cases, as compared to baseline methods, both for fldesired fall and to Neaapable.
HALT Does Not Harm General Instruction-Followte Capabilities. To assess whether HALT maintains coherence and general we conducted an AlpacaEval-Se
capabilities beyond domain-speciife accuracy, evaluation (Li et al. (223) comparing Llama3-70B HALT (trained with a = 0.6) against the base Llama3-70B model across 578 prompts spanning all four evaluation datasets, using GPT-5 as judge. HALT achieves near near-parity with the base model overall (50.8% win rate), demonstrating that selective abstention does not significantly harm general instruction- following capabilities. Notably, HALT outperforms the base model on wikibios v2 0%6 win rate), where factual accuracy is pritical-ppeeiselyy the use case HALT is designed for. On mathematical reasoning and general-domain tasks (MATH, appsintro, medex), HALT shows modest degradation (41.8-45.4%% win rates), expected since judges may penalize abstentions when completeness is valued over guaranteed correctness (see Table 5 in the Appendix for full results).
3.3 TRAINING A MULTI-DOMAIN LLM ACHIEVING 87% CORRECTNESS
We now evaluate the real-world use case of training a single model that achieves high correctness across multiple domains. Specifically, we train a single reliable Llama3-70B model on an equal-parts mix of data from all four domains with HALT tuned for increased accuracy, i.e., with = 40%. Figure 5 shows that this model achieves an average accuracy of 87% across all four domains, as compared to 51% achieved by training on a mix of the ground truth datasets. Meanwhile, the response completeness remains at 25% on average. This demonstrates that HALT allows training a generally capable model that users can trust significantly more than models trained with standard finetuning.
8

---

Published as a conference paper at ICLR 2026
3.4 ADD-ON HALT: MARKING FRAGMENTS AS "UNCERTAIN" INSTEAD OF OMITTING THEM.
Users might desire to observe the model's full response while being informed which parts might be incorrect, enabling them to edit uncertain code or verify statements externally. We evaluate this use case by modifying HALT to annotate uncertain response fragments with "Uncertain", instead of omitting them. Specifically, we modify the finetuning dataset for Math, Medical Q. and Coding by marking the first incorrect response fragment and all subsequent fragments as "Uncertain". Note that these fragments are replaced by "Unsure from here" in the default HALT response. We do not investigate the open-ended Wikipedia biography writing task as it allows for generating an unbounded number of "Uncertain" statements. Figure 6 in the Appendix shows that training Llama3-70B at
-0% with Add-On HALT results in significant correctness improvements when fragments marked as "Uncertain" are excluded from the correctness evaluation, achieving improvements similar to those of HALT's default implementation. When the "Unsure" markers are ignored Add-On HALT achieves response completeness similar to that achieved when finetuning on the Unchanged responses. In conclusion, slight modification of HALT enables the retention of response completeness while reliably informing the user about likely incorrect fragments.
4 RELATED WORK
A large body of prior work studied detecting LLM generations that likely contain hallucinations, i.e., likely contain incorrect statements. Approaches include training probes (Su et al. 2024). directly inspecting hidden states (Chen et al. 2024), or evaluating semantic entropy of generations (Farquhar al. 2024). These works are complemented by approaches to mitigate hallucinations via weight- space editing (Zhang et al.] (024b), modified decoding techniques (Chuang et al. 2023), preference training on samples that contain fewer or more hallucinations TTin et al. 2023a), or post-processing of generated responses (Mohri & Hashimoto, 20244] Gui et al. 2024 Wang et al. 2024). In contrast to these works, HALT addresses the trade off between response completeness and correctness, does not require any post-processing of generations at test-time, and applies to reasoning problems.
Recent works have discovered that finetuning LLMs on unknown examples increases hallucinations (Kang et al. 2024 /Gekhman et al., 2024 Tian et al. 2023b), which motivates HALT's approach of only finetuning on samples that are within the LLM capabilities.
Another line of prior work has found that LLMs are well-calibrated. and found that confidence scores can, for example, be obtained via inspection of logits (Kadavath et al. 2022) or via prompting LLMs to state their confidence (Lin et al., 2022). This provides basis for recent works that train models
to abstain when uncertain (Zhang et al. 2024a Chen et al.) 2003] Yadkori et al. 2024 Wen et al..
2024| BBahhman et al., 20244 Feng al. [224]] Cheng et al. [2024] Tuan et al. 2024). Recent work has explored training models to express verbalized uncertainty through confidence scores (Band et al., 2024) and calibrated hedging (Stengel-Eskin al., 2024). Unlike prior works that either fully abstain, fully answer, or preserve complete responses with uncertainty qualifications, HALT trains LLMs to selectively compose partial responses according their capabilities, enabling modification of the desired trade off between completeness and correctness.
5 CONCLUSION
We present HALT, novel finetuning paradigm that trains models to generate responses according to their internal confidence by omitting response fragments they are uncertain about. HALT training yields models that are up to 37% more accurate than answers while maintaining high response completeness.
those trained using unchanged ground-truth
Limitations. Our method assumes independent causally dependent fragments; dependency graphs could improve complex tasks. HALT uses binary capability assessment; fine-grained scaling may help. We train separate models per trade-off;; single adaptive model would improve flexibility. Extending to multi-modal domains is promising.

---

Published as a conference paper at ICLR 2026
REFERENCES
Neil Band, Yueqi Anand, Xander Bertran, Rafael Rafailov, Ellie Peng, Shangqian Zou, Chelsea Finn, and Stefano Ermon. Linguistic calibration of long-form generations. In International Conference on Machine Learning, 2024.
Faeze Brahman, Sachin Kumar, Vidhisha Balachandran, Pradeep Dasigi, Valentina Pyatkin, Abhilasha Ravichan- der, Sarah Wiegreffe, Nouha Dziri, Khyathi Chandu, Jack Hessel, al. The of saying no: Contextual noncompliance in language models. arXiv preprint arXiv:2407. 12043, 2024.
Chao Chen, Kai Liu, 7e Chen, Yi Gu, Yue Wu, Mingyuan Tao, Zhihang Fu, and Jieping Ye. Inside: Llms' internal states retain the power of hallucination detection. arXiv preprint arXiv:2402.03744, 2024.
Jiefeng Chen, Jinsung Yoon, Sayna Ebrahimi, Serean O Arik, Tomas Pfister, and Somesh Jha. Adaptation with self-evaluation to improve selective prediction in lims. arXiv preprint arXiv:2310. /6699, 2023.
Qinyuan Cheng, Tianxiang Sun, Xiangyang Liu, Wenwei Zhang, Zhangyue Yin, Shimin Li, Linyang Li, Kai Chen, and Xipeng Qiu. Can ai assistants know what they don't know? arXiv preprint arXiv:2401.13275. 2024.
Yung-Sung Chuang. Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprint arXiv:2309.03885 2023.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman. Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The Ilama 3 herd of models. arXiv preprint arXiv:2407.217800 2024.
Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy. Nature, 630(8017):625- 630, 2024.
Shanghin Feng, Weijia Shi, Yike Wang, Wenxuan Ding, Vidhisha Balachandran, and Yulia Tsvetkov. Don't hallu- cinate, abstain: Identifying I1m knowledge gaps via mulli-llim collaboration. arXiv preprint arXiv:2402.00300 2024.
Zorik Gekhman, Yona, Race Aharoni, Matan Fyal, Amir Feder, Roi Reichart, and Jonathan Herzig. Does line-tuning ilms on new knowledge encourage hallucinations? arXiv preprint wvXiv:240500590 2024.
Yu Gui, Ying Jin, and Zhimei Ren. Conformal alignment: Knowing when to trust foundation models with guarantees. arXiv preprint arXiv:2405.1033 2024.
Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Jackson Jiang, Andy Zou, Dawn Song, and Jacob Steinhardt. Measuring coding challenge competence with apps. arXiv preprint 1rXiv:2105.099 009889 2021a. URL http::tttss/tt /tt:://axxxv/// org. /tceg//bbb.. abs 2105. 09938
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03877 2021b.
103 33874
Edward Hu, Yclong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, arXiv preprint arXiv:2106.00 09685, 2021.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint 11..022310.06625. 2023.
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer. Zac Hatfield-Dodds Nova DasSarma, Eli Tran-Johnson, et al. Language models (mostly) know what they know. arXiv preprint arxiv2200..0522), 2022.
Katie Kang, Eric Wallace, Claire Tomlin, Aviral Kumar, and Sergey Levine. Unfamiliar finetuning examples control how language models hallucinate. arXiv preprint 055/6:2-00.005612 2024.
Yunsoo Kim, Jinge Wu, Yusuf Abdulle, and Honghan Wu. Medexqa: Medical question answering benchmark with multiple explanations. arXiv preprint arXiv:2406.06331 2024.
Diederik P Kingma. Adam: method for stochastic optimization. arXiv preprint 14X.::1412.6998. 2014.
10

---

Published as a conference paper at ICLR 2026
Rémi Lebret, David Grangier, and Michael Auli. Neural text generation from structured data with the
Wikipedia Infoboxes. In Proceedings of the 2016 Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP), pp. 1061-1071 Association for Computational Linguistics, 2016. URL
https://aclantiin
Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpacacval: An automatic evaluator of instruction-oolooinng models. https
//github. wataa/aasuuuuaa waw/aaaaaaaaa alpaca eval 2023.
Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chandra Bhagavatula, and Yejin Choi. The unlocking spell on base Ilms: Rethinking alignment via in-context learning. In The Twelfth International Conference Learning Representations, 2023.
Stephanie Lin, Jacob Hilton, and Owain Evans. Teaching models to express their uncertainty in words. arXiv preprint arXiv:2205. 14334, 2022.
Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit lyyer, Luke Zettlemoyer, and Hannanch Hajishirzi. Factscore: Fine-grained atomic evaluation of factual precision in long form text generation. arXiv preprint arXiv:2305.1422 2023.
Christopher Mohri and Tatsunori Hashimoto. Language models with conformal factuality guarantees. arXiv preprint arXiv:2402. 2024.
OpenAl. Why language models hallucinate, 2025.
URL hctps://openal ccm/ index/ why- wwwguaaaa--mdeessaallllllo huc.iiintee Accessed: 2025-09-26
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning. Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. Advances in Neural Information Processing Systems, 36, 2024.
Yixiao Song, Yekyung Kim, and Mohit Tyyer. Veriscore: Evaluating the factuality of verifiable claims in long-form text generation. arXiv preprint arXiv:2406. 19276, 2024.
Elias Stengel-Eskin, Peter Rawls, Arnaud Roncone, and Benjamin Van Durme. LACIE: Listener-aware finetuning for confidence calibration in large language models. In Advances Neural Information Processing Systems, 2024.
Weihang Su, Changyue Wang, Qingyao Ai. Yiran Hu, Zhijing Wu, Yujia Zhou, and Yiqun Liu. Unsupervised real-time hallucination detection based on the internal states of large language models. arXiv preprint arXiv:2403.0644 2024.
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma Improving open language models at practical size. arXiv preprint arXiv:2408.00) 18, 2024.
Katherine Tian, Eric Mitchell, Huaxiu Yao, Christopher Manning, and Chelsea Finn. Fine-tuning language models for factuality. arXiv preprint arXiv:231 1,08401, 2023a.
Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D Manning. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. arXiv preprint arXiv:2305. 14975, 2023b.
Yi-Lin Tuan, Xilun Chen, Eric Michael Smith, Louis Martin, Soumya Batra, Asli Celikyilmaz, William Yang Wang, and Daniel M Bikel. Towards safety and helpfulness balanced responses via controllable large language models. arXiv preprint arXiv:2404.01295 2024.
Zhiyuan Wang, Jinhao Duan, Lu Cheng, Yue Zhang, Qingni Wang, Xiaoshuang Shi, Kaidi Xu, Hengtao Shen, and Xiaofeng 7hu. Conu: Conformal uncertainty in large language models with correctness coverage guarantees. arXiv preprint arXiv:2407.00499, 2024.
Jerry Wei, Chengrun Yang, Xinying Song. Yifeng Lu, Nathan Hu, Dustin Tran, Daiyi Peng, Ruibo Liu, Da Huang, Cosmo Du, et al. Long-form factuality in large language models. arXiv preprint arXiv:2403. 240..8802. 2024.
Bingbing Wen, Jihan Yao. Shangbin Feng, Chenjun Xu, Yulia Tsvetkov, Bill Howe, and Lucy Lu Wang. The art refusal: A survey of abstention in large language models. arXiv e-prints, pp. arXiv-2407, 2024.
Yasin Abbasi Yadkori. II Kuzborskij, David Stutz, András György, Adam Fisch, Arnaud Doucet, Iuliya Beloshapka, Wei-Hung Weng, Yua-Yuan Yang, Csaba Szepesvári, et al. Mitigating Ilm hallucinations via conformal abstention. arXiv preprint 20550000. 2024.

---

Published as a conference paper at ICLR 2026
Hanning Zhang, Shizhe Diao, Yong Lin, Yi Fung. Qing Lian, Xingyao Wang, Yangyi Chen, Heng Ji, and Tong Zhang. R-tuning: Instructing large language models to say ' don't know'. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies (Volune 1: Long Papers), pp. 7106-7132, 2024a.
Shaolei Zhang, Tian Yu, and Yang Feng. Truthx: Alleviating hallucinations by editing large language models in truthful space. arXiv preprint arXiv:2402.178/ 1, 2024b.
Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36, 2024.
12

---

Published as a conference paper at ICLR 2026
A SUPPLEMENTARY EXPERIMENTAL DETAILS AND RESULTS
A.1 DEFINING THE NUMBER OF FRAGMENTS REQUIRED TO FULLY ANSWER PROMPT.
We remark that the number of fragments necessary to answer question cannot be unambiguously determined and, in practice, slightly varies for different pre-trained L.Mss. However, when, e-g.. an LLM responds with two correct fragments, we must define nall to be able to compute response completeness. As different pretrained LLMs will answer the same few-shot prompt with slightly different numbers of fragments, as shown in Table 4 choose according to the pre- rreitriied model's s response statistics the MATH and Medical Question datasets. For biography writing and coding, we found that the number of fragments of the ground truth response defines aal well.
A.2 ADDITIONAL IMPLEMENTATION DETAILS
LLM implementations. We implement all LLMs using their open-source implementations given at http:///hhuub.cooo/uggggggg.oeo/ transsterrrraeeeeoeeeeeeees formers
Finetuning Details. We use supervised finetuning with LoRA (Hu et al. 2021) for compute-efficient training. with a rank 256 and alpha 512. Training is performed using the AdamW (Kingma 2014) optimizer with betas set to (0.9, 0.95), weight decay of 0.01, and an initial learning rate of le 5. The learning rate schedule follows a warmup fraction of 5%, with a decay to over epochs. The effective batch size is set to 128.
Computational Resources. Finetuning of the 7-9B LLMS took around 5-6 hours single Nvidia A100 GPU. Finetuning of the Llama3-70B model took around 3-4 hours on a node Nvidia A100 (80GB) GPUs.
Datasets. We downloaded the datasets from the following domains. The Wikibios dataset was downloaded from the MATH dataset from htt.a//ppp//g.tto..ccaaaaaaaaa/a htt.://g.....g.goooo/./aaaaaaaas
aataaabiiggggaaaa the MedexQA dataset from and the APPS dataset from
https://github. wwt.:/..occceeeeeeeeeeeeeeeeeeee htt.:///ww..ccoo//eenrrr///////p
Baseline Implementations. We implemented FactTune (Tia et al. 2023a) using the authors'
implementation given at httpp::/ggtti:t cwm/ com/kttian/lim httpallllaaaaaaaaaaaaaaaaaaaaaaa factuality tuning) For the
IDK baseline (Cheng et al. 2024) we follow the authors' implementation given athhttps://itthe
com/ http///eeMMsS/aaaaaa Dont Know
Math (MATH)
Med. Questions (MedExQA)
Coding (APPS)
0.80
0.00
0e5
Correshwees
0.05
0.36
0.50
0.55
0.65
0.78
0.30
0:15
0.22
0.30
Completenese
Complexonese
Completeness
Unchanged (Few-Shool
HALT
Add-On HALT (remove Unnuree)
Add-On HALT (eveluxie 'Unsure')
Figure 6: We evaluate an Add-On version of HALT for LLama3-70B. Add-On HALT annotates response fragments as 'Unsure' instead of omitting them. Removing fragments marked by Add-On HALT as 'Unsure' results in significant increase correctness.
the same time, evaluating 'Unsure' fragments yields results similar to training on the Unchanged (Few-Shot generated) re- sponses. This demonstrates that Add-On HALT allows for increased correctness while preserving the completeness of unchanged finetuning.
13

---

Published as a conference paper at ICLR 2026
Table 3: Fl score (arithmetic mean of response correctness and completeness) for all models and finetuning methods. We show results when setting the desired number of fragments as those required fully answer the question (left side of Table) and when setting the desired number as those that the pre-trained model is capable of generating (right side). Note that a perfect Fl score is only attainable when setting Vedesired = Neapable- We find that HALT largely outperforms prior work and baselines, by larger margins for cappaable. For Gemma2-7B, we hypothesize that the relatively lower performance of HALT in Math and coding is likely due to the worse in-context learning capabilities, as outlined in Section 3.1
Reoeed 7gy
Popeni:
Finetuning
Wikipediin
Medical
Coding
Wikipedia
Modical
Coding
Gemme--2 2-999 Mitmm-77 Lamm- 3-708 12mm-3 348
Unchanged
0.23
0.70
0.26
0307
Raidonniim
0.01
0.08
0.70 0.02
0.21 0.83
0.00
088
0.11 0.06
Abstain (DK) (DK))Chhege heng ct 0024
0.14 0.00
0.12 0.02
0.58 0.16
0.05 0.03
0:25 0.04
0.00
0.69 0.18
0.18 0.09
Factllae Taan Sra 20253
0.24 0:01
0.38 = 0.022
HALT (eutr))
0.28 0:00
0.35 + 0.07
0.68 + 0.00
0.35 +.004
0.63 - 0.01
0.66 0.12
0.81 = 0.04
0.77 + 0.05
Unchanged
0.26 0.11
0.68
0.01
0.50 + 0.33
0.44 0.03
0.35 = 0,12
0.75 0.01
0.7 0.03
0.71 d: 0.09
RandonTiim
0.12+0001
0.64 + 6.01
0.83 + 0.00
0.38 0.03
0.26 + 0.01
0.75 0.01
0.88 - 0.00
0.75 + 0.05
Absain (ID)) (Chene 2024
0.28 0.281000
0.53 0.27
0.75 0.07
0.19.10..13
0.44 0.03
0.38
0.82 0.07
0.48 0.32
PactTlae
0.30::000
0,81
0:01
0.02
0.88
0.04
Unchanged
0.05
0.57 0.05
RuidoonTrim
0.12
0.04
0.34 0.04
0.25
0.80 0.05
0.53 0.37
Abstain Cheng al 2024
0.11 0.00
0.09 0.02
0.56 0.17
0.01 1 0.00
0.18 0.03
0.43 0.07
0.73 0.20
0.04 1. 0.02
FacrTuse Taan or 20223
0.00=000
0.44 0.133
HALT (ou1s)
0.13
0.28 0.03
0.66 1: 0.00
0.34 1:000
0.68
0.05
0.69 0.441
0.81 :: 0.08
0,82 1: 0.06
Unchanged
0.15 0.01
0.42 0.08
0.71 + 0.01
0.34 002
0.21 0:08
0.64 0.15
0.22 0.01
D.42 0.05
RandomTrim
0.02 0:00
0.28 0.06
0.72 0.04
0.34 1 0.033
0.19 = 0.85
0.59 0.14
0.82 0.04
0.65 1: 0.04
Abstain (DK (Chone al 2024
0.00
0.67 0.04
0.01
0.19 0.02
0.11
0.79
0,03
Faet
0.31 0.05
0,86
0.08
Table The table shows the average number of fragments per response for answers generated by each pretrained model using in-context learning. The 'Ground Truth' row represents the ground truth answer, with each category showing 100% correctness. Among the models, Llama-3-70B has the highest relative correctness, followed by Llama-3-8B and Gemma-2-9B, with Mistral-7B-v0.3 performing the lowest.
Pretrained Model
Wikipedia
Math
Medical Q.
Coding
% avg
given
cor..
'given
given
coTr.
given
Ground Truth
9,8 8.0
100.0
4,5
100.0
5.1 1.2
100.0
15.0 12.7
100.0
100.0
IJama-1-88
11.9 11.2
30.1
9.3
42.2
38.0
Llama 3-BBB
9.6 10.3
30.9
5.5
54.1
5.2 1.3
83.8
9.8
58.8
56.9
Gemma-2-9gg
28.6
5,3
12.4
Mistral-7B-vO.3
9.6
6.3
3.7
17.1
5.2 1.8
60.8
11.7
27.9
28.0
14

---

Published as a conference paper at ICLR 2026
Wikip. Blographiss (Wiklblos)
Math (MATH)
Med. Questions (MadEXQA)
Coding (APPS)
0.82
0.75
0.55
6.6
Comoenese
0.28
0.00
0.52
0.22
0.00
0.15
0.30
5,55
0.00
0.40
0.90
0:22 Completenesa
0,35
Complatennss
Comploeeresa
Completaness
Whkp. Blogrephies (Wikiblos)
Math (MATH)
Med. Questions (MadEXQA)
Coding (APPS)
+
Ocmecnecs
0.40
000
0.30
0.80
0.30
0.05
0.25
1,00
0.10
0,17 Comsleiones
Completionss
Completenesa
Complataneso
Unchanged
(ID:)
Figure 7: Response correctness (y-axis) and completeness (X-axis) for Gemma2-9B (top) and Mistral- 7B (bottom) when finetuned with different methods (desired number of fragments is Rall). FI Score constant along curved grey lines, and highest in the top right corner. For HALT and Randomtrim, results are shown with different trade-offs between completeness and correctness, where lighter colors indicate tuning for higher correctness. We omit results with less than 5% completeness.
Dataset MATH
HALT Wins
Base Wins
Ties
Win Rate
49
59
42
45 54%%
appsintro
46
64
39
41.8%
medex
54
655
45.4%
wikihios_v
66.0%
Overall
246
50.8%
Table 5: AlpacaEval-styye evaluation comparing HALT (a = 0.6) against base Llama3-700 across 578 prompts. Win rate excludes ties. GPT-4o-mini served as judge.
A.3 ALPACAEVAL GENERAL INSTRUCTION-FOLLOWWE RESULTS
15

---

Published as a conference paper at ICLR 2026
Example prompt L-ama33405B evaluator determine correctness of fragment that is part of response composed of independent fragments.
#e*#Context
[FULL WIKIPEDIA ARTICLE OF BARACK OBAMA]] *e**statementt
Barack Obama was born on May 12, 1960.
Instructionn=***
Is the above Statement True or False according to the given Context? Reply with either "True' or 'False.'
Figure 8: Example prompt to Evaluator for assessing a fragment from Wikipedia-style response.
Example prompt to Llama3-4058 evaluator to determine correctness of a response composed of causally dependent fragments.
*e* Question:*ees
A triangle has a base of 8 units and height of 5 units. If the triangle is part of rectangle with the same base rectangle's area does the triangle occupy?
and height, what fraction
*es*Ground truth Response for Math Question:****
The of triangle is calculated using formula: Area 1/ base height.
Substituting the given values: Area 1 8 5 20 squure units.
The area of the rectangle calculated using the formula: Area base height. Substicuting the given values:
Area 8 x 5 = 40 square units.
The triangle occupies fraction the rectangle's area given by: Roctangie's arca
20
Thus, the triangle occupies &/2 of the rectangle's *s*sStep-wise Response given by student: sess
Line I: The area of triangle is calculated using the formula: Area x base x height.
Line 2: Substituting the given values: Area
20 square units.
Line The area
rectangle
square units.
Line 4: The fraction of the rectangle's 's that the triangle occupies is 20 square units. Instruction:
question above, as well as a Ground truth Response to Determine until which step-wise response given by the student correct. Reply with the number of the last correct line. the first step ulready wrong, reply with zero. Otherwise, reply with the number of the last correct line, i.c. with an integer between and the number of lines in the response. Reply with 'Line X' only, where X is the number of the last line.
are given
Figure 9: Example prompt to Evaluator for assessing MATH response.
B.2 EXAMPLE HALT RESPONSES FOR DIFFERENT TRADE OFFS BETWEEN RESPONSE CORRECTNESS AND RESPONSE COMPLETENESS
B.2.1 EXAMPLE HALT FINETUNING RESPONSES THE WIKIBIOS AND MATH DATASETSAND FOR DIFFERENT THRESHOLDS
We show examples of HALT responses for different trade-off's between response completenes and response correctness in Figures and
We note that responses for the Wikipedia biography dataset segmented into individual atomic statements. This segmentation results fromt he application of the fact-extraction LLM (Song et al.) 2024), but can reverted by prompting the L.M to recompose the response into single text the end of the generation. We omitted this recomposition as it complicates the assessment of individual statements.
16

---

Published as a conference paper at ICLR 2026
Example for 0.4
Example for 0.6
Example for 0.8
6000 Question:**or
*0*0 Question: oees
*000 Question: seee
Tell me about Daniel the Stylite
Tell about Daniel the Stylite
Tell me about Daniel the Stylite
Respond with no more than 500 tokens. Only list facts about the person that are certain about. Separate all facts by 'In'. unsure how continue any point during the response, write 'Unsure from here." and terminate your response.
Respond with no more than 500 tokens. Only list facts about the person that you certain about. Separate all facts by "ln'. unsure how continue point during the response, write 'Unsure from here." and terminate your response.
Respond with no more than 500 tokens. Only list facts about the person that you are certain about. Separate all facts by "'n". If unsure how continue at point during the response, write 'Unsure here.' and terminate your response.
0000 Response: 6*8o
#000 Response: 80.0
*000 Response: 0000
Saint Daniel the Stylite died on December 493.
Saint Daniel the Stylite was born around 409.
Saint Daniel the Stylite was born around 409.
Saint Daniel the Stylite died on 11 December 493.
Saint Daniel the Stylite died on 11 December 493.
Saint Daniel the Stylite was Christian saint.
Saint Daniel the Stylite was Christian saint.
Saint Daniel the Stylite is commemorated on 11 December in the Eastemn Orthodox Church.
Figure 10: Example HALT responses for different treshold parameters a for the Wikipedia dataset. Finetuning the left response (lowest a) results in conservative LLM providing responses of higher correctness and lower completeness, while finetuning on right response results in a eagerly responding LLM.
17

---

Published as a conference paper at ICLR 2026
Finetuning pair a
Finetuning pair
Finetuning pair for 0.8
0).4
0.66
#980 Duestion:**as
*890 Question:
***0 Question: 9980
triangle has base of 8 units and height of 5 units. If the triangle part of a rectangle with the height, what fraction of the rectangle's 's urea does the triangle occupy?
a base of 8 units und height of 5 units. If the triangle is part of rectangle with the height, what fraction of the rectangle's 's area does the triangle occupy?
A triangle
A triangle has base of units and height of 5 units, If the triangle is part with the height, what fraction of the rectangle's 's area does triangle occupy?
rectangle base and
base and
base and
0000 Response: 09e0
weee Response:
*0.e Response: 0000
The urea of triangle is cal- culated using the formula:
The area of triangle cal- culated using the formula:
The area of triangle cal- culated using the formula:
Area 2 base height. Substituting the given val- ues: Area = x x
Area base height.
Area base height.
Substituting the given val- ues: Area 3 x 5 20 square units. The area of the rectangle is calculated using the for- base height. Substituting the given values: Area 8 5 = 40 square units. Unsure from here.
Substituting the given val- ues: Area = 3/2 8 20 square units. The area of rectangle is calculated using the for- mula: Area = base x height. Substituting given values: Area 8 5 = 40 square units. The triangle occupies fraction of the rect- angle's 's urea given by: Thangle's area 20 = J.
20 square units.
Unsure from
mula: Area
Recuagle's area
Thus, the triangle occupies It of the rectungle's
Figure 11: Example HALT responses for different treshold parameters a for the MATH dataset. The left response (lowest a) results conservative LLM providing responses of higher correctness and lower completeness, while finetuning on the right response results in eagerly responding LLM.
18


Reviews:
[{"review_id": "Kz1ADCIMh3", "reviewer": "Reviewer_i3Pq", "summary": "The authors propose HALT, a method to train language models to abstain from providing certain fragments of their long-form responses when they are uncertain. Given an LM, the approach obtains several samples for each training input, splits each sample into fragments and fact-checks them, and finally replaces incorrect fragments with abstention statements. By collecting samples with more or fewer correct statements into different supervised finetuning splits, HALT is able to navigate the correctness-completeness tradeoff for long-form generation, unlike prior work. Experiments on four datasets across general-domain factuality, math, etc., demonstrate HALT's navigation of this tradeoff.", "questions": "See above for the key feedback on better contextualization with respect to previous \"LM finetuning for calibration\" works, and evaluating coherence with a guardrail task.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The paper is generally well-written and clear.\n- The motivation is strong; abstention and uncertainty quantification at claim-level are an important approach to decrease LM hallucination while retaining usefulness.\n- The method is simple and clever, in particular the idea to use multiple samples given an input along with their number of correct claims to control the correctness-completeness tradeoff. I suspect that this work could emerge as a straightforward, effective baseline for training LMs to abstain in their long-form generations.\n- The experiments are reasonably thorough (four datasets in different domains, evaluation of multi-task transfer, and extension to uncertainty quantification).", "weaknesses": "- The authors should better contextualize with respect to prior work on finetuning language models to express their verbalized uncertainty [1] [2]. That setting is in some sense more challenging than the final experiment in the present paper, which marks fragments as uncertain instead of omitting them.\n- A number of heuristic choices are made which are not clearly ablated. For example, I would expect that many non-mathematical tasks have some causal dependency in consecutive sentences, and therefore simply replacing an arbitrary subset of fragments with an abstention phrase will harm coherence. Can you evaluate HALT on some \"guardrail\" instruction-following tasks which would test how coherence and helpfulness are affected by your finetuning, e.g., AlpacaEval or similar?\n\n[1] Band et al., Linguistic Calibration of Long-Form Generations. ICML 2024. https://arxiv.org/abs/2404.00474\n[2] Stengel-Eskin et al., LACIE: Listener-Aware Finetuning for Confidence Calibration in Large Language Models. NeurIPS 2024. https://arxiv.org/abs/2405.21028", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oKjJnwK6Q1", "reviewer": "Reviewer_UdNm", "summary": "HALT fine-tunes large language models to answer only when confident and to abstain when uncertain, outputting \"Unsure from here\". It trains models only on content they can reliably generate, aligning outputs with internal confidence rather than forcing full responses. HALT decomposes answers into factual fragments, verifies each with an evaluator LLM, and keeps only correct ones. Without adding inference cost, it improves factual accuracy and balances precision and recall. Across LLaMA3, Gemma2, and Mistral models, HALT outperforms standard finetuning, producing more reliable and self-aware responses.", "questions": "1. What does the statement f in line 147 refer to? I assume it corresponds to the \"fragment\" mentioned in the Introduction section, but please clarify this.\n2. Why do you sample four prompt–response pairs and concatenate them with the target prompt?\n3. Using Llama3-405B as the evaluator or for fragmentation appears quite costly. How sensitive is the proposed method to the evaluator model’s size and capability?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The core idea of this paper is intuitive and clearly presented.\n2. The \"Unsure from here\" mechanism makes the model’s uncertainty explicitly interpretable, improving user trust and enabling controllable trade-offs between correctness and completeness.", "weaknesses": "1. The method depends on an additional evaluator model to assess fragment correctness, which may introduce bias or inconsistency depending on the evaluator’s quality and alignment.\n2. The method also relies on fragmentation, which feels somewhat heuristic to me.\n3. The training process is complex, requiring additional time and computational cost.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CnCkHdS9wq", "reviewer": "Reviewer_5Yi7", "summary": "The authors are addressing the issue of LLMs generating responses they are not confident in which can lead to hallucinations. They only want the LLM to generate responses it is confident in otherwise it'll stop generating certain responses at will output \"Unsure\". \n\nThe authors propose a post-training method called High Accuracy, Less Talk (HALT). This method involves prompting the LLM to generate responses for certain domains (wiki, math reasoning, etc). They then break these responses down into fragments and use an evaluator (separate LLM) to determine which fragments are correct / incorrect. They then create a new dataset where the responses are only the correct responses. \n\nAfter finetuning the authors show that their model improves correctness on all their testsets along with maintaining a reasonable response completeness.", "questions": "Questions\n\n1) Why are you using RandomTrim as a baseline? It's not clear what purpose it serves as a baseline. \n\n2) Since you are sampling responses when creating the finetuning testset how often are the responses similar to each other? Does sampling help then if you are not getting diverse responses?\n\n3) Have you considered using the negative fragments to move the model's response away from those?\n\nSuggestions / Typos\n\n1) I think you meant to say APPS in line 334\n\n2) I suggest putting figures closer to where you mention them like Figure 3", "limitations": "", "rating": 10, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1) The method laid out is clearly explained and takes a principled approach. I think the different ways to assess fragments based off the domain is good and it is good that the authors are acknowledging future work would need to involve dependency graphs.\n\n2) The baselines provided seem to be good to compare their method against.\n\n3) The study in the section \"Finetuning on Few-Shot Prompted Responses is comparable to Finetuning on Ground Truth\nResponses\" was important to run to trust this method. Even though it's been shown in previous that LLMs do not acquire novel capabilities during finetuning since the few-shot prompting was a key component of the method was good to show that results wont diminish much.", "weaknesses": "1) One thing that is not clear is how does response completeness affect user experience? Obviously we want correct responses for the user but is a completeness score of 51% low? What's the best tradeoff?\n\n2) There's some lack of discussion. In Figure 4 why is there a sharp dropoff for the MATH dataset but not for Wikibios. I might be missing something but it would be good to have this clarified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)