FACT: Mitigating Inconsistent Hallucinations in LLMs via Fact-Driven Alternating Code-Text Training
Authors: Xinxin You
Qixin Sun
Chenwei Yan
Xiao Zhang
Chen Ning
Xiangling Fu
Si Liu
Guoping Hu
Shijin Wang
Ji Wu
Xien Liu
Venue: NeurIPS 2025 poster ()
Keywords: Inconsistent Hallucination
Fact-Driven Code-Text
Alternating Training
Primary area: deep_learning

Abstract: Inconsistent hallucinations remain a major challenge for large language models (LLMs), undermining the accuracy and reliability of fact-based reasoning in real-world applications. Existing approaches often rely on task-specific training or adaptation, such as hand-crafted synthetic datasets for domain tasks or solutions mainly focused on numerical reasoning, thereby limiting generalizability to broader, unseen NLP tasks. Inspired by the structural rigor and logical consistency of programming languages, we observe that fact-based texts can be mapped to programming structures due to their inherent patterns. We further propose FACT, a novel Fact-driven Alternating Code-text Training framework that alternates between text-to-code and code-to-text prediction. FACT is the first task-agnostic paradigm that embeds code and natural language in a shared semantic space, thereby transferring the logical consistency of code to LLM outputs in NLP tasks. Experiments show that with only a small subset of Wiki-40B-en for training, FACT reduces inconsistent hallucinations by 2.7%–8.0% and improves overall performance by 2.5%–6.1% in three leading LLMs and four diverse datasets covering QA and summarization tasks. This framework offers a new perspective on addressing challenging hallucinations in LLMs, contributing to more reliable AI.

FACT: Mitigating Inconsistent Hallucinations in LLMs via Fact-Driven Alternating Code-Text Training
Xinxin You'
Qixin Sun Chenwei Yan' Xiao Zhang Chen Ning' Guoping Hu Shijin Wang** Ji Wu Xien Liu'
Xiangling Fu
Liu
Depparment of Electronic Engineering, Tsinghua University, Beijing, China School of Artifcial Intelligence, Beihang University, Beijing, China "School of Information Technology and Management, University of International Business and Economies, Beijing, China *ByteDance, Beijing, China School of Computer Science, Beijing University of Posts and Telecommunications, Beijing, China iiFLYTEK Research, Hefei, China 'College of AI, Tsinghua University, Beijing, China (yxx23,nc22) @ maails.tiinguua.ee edu.cn (sunqx,liusi } @ @buaa.edu.cr (chenwei.yan @ uibe.edu.cn (zhangxiao.) 1 12@ bytedance.com) (fuxiangling @ bupt.edu.cn)
(gphu,sjwang3 @iflytek.com (wuji_ee.xeliu) J @ mail. isinguua.n..c
Abstract
Inconsistent hallucinations remain a major challenge for large language mod- els (LLMs), undermining the accuracy and reliability of fact-based reasoning in real-world applications. Existing approaches often rely on task-specific training adaptation, such hand-crafted synthetic datasets for domain tasks solu- tions mainly focused on numerical reasoning, thereby limiting generalizability to broader, unseen NLP tasks. Inspired by the structural rigor and logical consistency programming languages, we observe that programming structures due to their inherent patterns. We further propose FACT, a novel Fact-driven Alternating Code-text Training framework that alternates be- tween text-to-code and code-to-text prediction. FACT is the first task-agnostic paradigm that embeds code and natural language in a shared semantic space, thereby transferring the logical consistency of code to LLM outputs in NLP tasks. Experiments show that with only a small subset of Wiki-40B-en for training, FACT reduces inconsistent hallucinations by 2.7%-8.0% and improves overall perfor- mance by 2.5%-6. % in three leading LLMs and four diverse datasets covering QA and summarization tasks. This framework offers a new perspective addressing challenging ballucinations in LLMs, contributing to more reliable Al.
texts can be mapped to
1 Introduction
Achieving human-like logical consistency in fact-based reasoning is essential to advance artificial general intelligence (AGI) II 22. Despite significant progress in large language models (LLMs) across various natural language processing (NLP) tasks [
4 5) these models still frequently produce inconsistent hallucinations due to limited logical rigor 7 Such hallucinations occur mainly in two forms: (1) input-conflictng hallucinations, where the generated content diverges from the provided input [8] 9 and (2) context-conflictnng ballucinations, where the generated content contradicts information previously produced by the model [10] [T], as shown in the left panel of Fig. [ These errors are particularly prevalent in bast- reasoning, such as LLMS generating patient diagnoses in the medical domain that contradict clinical input data, producing legal summaries that
"Corresooniigg author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

misrepresent the actual case facts. Such issues severely undermine the credibility of LLMs in real- world upplications. Therefore, is urgent to develop a universal approach that can effectively address these inconsistent hallucinations across wide range of tasks.
Existing approaches for mitigating hallucinations in LLMS often require task-specific training or adaptation. Recent efforts have addressed inconsistencies by constructing domain-specine hand- crafted reasoning datasets-for instance, LOGIQA [12] for civil service exams and AR-LSAT [I3), ReClor (14] for legal reasoning. However, such manual curation is time-consuming and lacks scala- bility. Consequently, synthetic data approaches have emerged, aiming automate data generation and expand coverage. Nonetheless, these methods are primarily limited to numerical reasoning tasks, such as KPDDS [iS]. OpenMatthnstruutt [16], and MathGenie [17], where large-scale question- answer pair construction is tractable. Attempts to address hallucinations in broader NLP tasks include
methods such as Lookback Lens and SymbCoT [8 19]. However, they still rely on significant task-specific adaptation, including separate classifiers or symbolic CoT pipelines designed for each task. As result, while effective in narrow domains, these approaches generally lack generalizability and scalability to diverse, unseen NLP scenarios.
Advancements in training LLMS on large-scale code datasets, such as CoCoGen and CodeRL, have significantly reduced inconsistent hallucinations in code generation tasks [20 21. Building on this integration of code with LLMs. recent methods further improve logical rigor in reasoning tasks. For example, program-ot wawraumm----huuuge thought [22) and program- assisted language models [23 translate natural language problems into executable code and utilize code interpreters to derive answers. However, these methods face practical limitations: due to fundamental differences in organizational structure, modes of expression, and language style between code and natural language, only a narrow subset of problems, typically mathematical questions, can be effectively translated into executable code [22 233. Consequently, these approaches fail to transfer the logical consistency benefits of code to LLM outputs in broader range of NLP tasks.
To address the above challenges, we propose FACT: mitigating inconsistent hallucinations in LLMs via Fact-driven Alternating Code-text Training. FACT is grounded in the observation that fact-based texts exhibit intrinsic structural patterns that are amenable to systematic mapping onto programming abstractions. As illustrated in Fig. [ (upper right), subjects and their attributes in natural language correspond objects and properties in code, while processes and causal relationships are represented as functions. To further transfer the logical rigor of code to natural language, FACT implements an al- ternating training paradigm between text-to-code and code-to-text prediction. The key challenge- the absence of ground-truth code supervision for text-to-code prediction-is addressed by employing a pseudo-labeling with a two-step verification process: 1) confirming syntactic correctness; and 2) assessing semantic fidelity to the original text. Experiments show that training on just small Wiki- 40B-en 244 subset fundamentally improves LLM output consistency. Extensive evaluations with three state-of-the LLMs demonstrate that FACT significantly reduces inconsistent hallucinations and improves task performance on summarization and QA tasks, all without task -pecific adaptation.
We summarize the contributions of our method as follows:
We propose FACT, the first task agnostgi framework that significantly mitigates inconsistent hallu- cinations in LLMs through alternating code-text training. This framework offers new perspectives for developing LLMs that deliver logical consistency and accurate responses.
We identify clear correspondence between the structures of fact-based text and code. This finding not only supports our alternating training methodology, but also provides a foundation for future research bridging the gap between textual and coding modalities.
Experiments show that with only small subset of Wiki-40B-er 24 for training, FACT reduces inconsistent hallucinations by 7%-8.0% and improves overall performance by 2.5%-6. 1% in three leading LLMs and four diverse datasets covering QA and summarization tasks.
2 Related Work
Data-Centric Approaches-Manual Bottlenecks and Narrow Scope Recent studies suggest that hallucinations in LLMs are often attributed to deficiencies in training data, underscoring the im- portance of high-quality reasoning datasets for improving consistency 25 26 27 Thus, domain-
specific, handcrafted benchmarks have been developed, such as LOGIQA 12 for civil service exams,

---

Pose question LLM
Applying Fact dact-driven.... Alternating Code text Training (FACT) LLMs
Can you recommend delicious breakfast recipe" I love sggs. and it would be great the recipe includes
Code
User
Subjects oWjer Au ahr Cliu,
class Ingredicats():
dbea's dben's. Ansdeet.
shupe
Characteristics of Subject
elass RecipeQ:
Teput-Con Con flifeiiing Hallucination
namme
breakfist recipe
lunch recipe
The Relationships Between
Usen love eggs S-- LLM: uncertain about the user's preference
Subjects
def creatoRecipe@::
Contest- Crress-Coune Conflicting Hallucination
LLM. first deseribes lunch secipe but change breokfast
recipe
The Original Response of with Hallucination Issues!
4 LLM outputs are legicallly consistent after applying ing the FACT!
Sure! Here's is recipe suitable for lunch. How about steak pairoa with red wine? It's simple to make. I'm not sure if you like egns, but adding fried egg would also be a nice touch. This breakfast nocipe really -njjuy
Sure! Heree's a recipe suitable for breakfastt I know you epps, Hew shout scrambled eggs with carror and glass of milk? This breakfast recipe really healthy!
LLM
Figure 1: This figure illustrates that standard LLM responses (left) are susceptible to input-conffictnng and context-conflictngg hallucinations, resulting in outputs that are inconsistent with both user intent and contextual logic. The FACT approach (right) leverages structural alignment between factual text and code via fact-driven alternating code-text training, effectively reducing these hallucinations and producing responses that are both semantically and logically consistent with user intent and context.
AR-LSAT(13) and ReClor 14] for legal exams, though at considerable cost and labor. This has motivated interest in automated and synthetic dataset construction 23 particularly for numerical reasoning tasks where large-scale
-nswer pairs are more easily generated (e.g.. KPDDS [55) methods are generally tailored
OpenMathInstruut- [16), MathGenie[I7)) However, these to specific domains or tasks, which limits their generalizability to broader NLP applications.
CoT Approaches-I Limited by Symbolic Expressivity Chain-of- Chain-of-Thought (CoT) strategies have shown promise in reducing hallucinations in reasoning tasks by promoting explicit step-by-step inference [29]. This line of work is supported by developing logical reasoning datasets such as LogiCoT [30 and FOLIO (31). which provide explicit reasoning chains. Furthermore, methods like LOGIC-LM integrate symbolic solvers with language models to formalize natural language queries, while 1ymbCoT[19] enriches CoT prompting with symbolic expressions and logical rules. However, these methods are limited by symbolic expressivity and are less effective for complex problems, underscoring the need for more flexible and widely applicable solutions.
Code-Based Soproaches-Limiter Generalization Beyond Mathematics Recent research demon- strates that training LLMs on code datasets helps reduce hallucinations in code generation by
leveraging the strict syntax and semantics of programming languages 20 21 Building on this integration of code with LLMs, recent methods further improve logical rigor in reasoning tasks. Approaches like Program Thought (PoT) (22) and Program-Aided Language Models (PAL) [23] translate natural language problems into executable code and utilize code interpreters to derive answers. However, fundamental differences between code and natural language restrict these methods to mainly mathematical problems, limiting the transfer of code's 's logical consistency benefits to LLM outputs in a broader range of NLP tasks.
3 Method
Our approach builds on the observation that fact-based texts possess inherent structural properties, which can be systematically mapped to programming abstractions, as illustrated in Fig. [ (upper right). To leverage this, we propose an alternating training paradigm between text-to-code and code-to-text generation. Due to the absence of ground-truth code for text-based code prediction, we generate pseudo-labels to supervise model training. To ensure the quality of the generated code, we apply a two-stage assessment that verifies syntactic correctness and evaluates semantic fidelity relative to the original text. This training process aligns text and code representations within a unified semantic space, effectively transferring the logical consistency of code LLM outputs on broader NLP tasks.
The remainder of this section organized as follows. Section 3.1 formalizes the problem setting and outlines our research objectives. Section 3,2 introduces a procedure for selecting bact- texts.
3

---

Text Corpus Preparing
Altermating Code- Code-Text Truining
Quality
Fact-based
Fact-hased Text
Text-to-code
Parallel Code
Corpas
class ClassAO
Evaluate
class
Filtering
O
ve Revien
Update
Wiki Text Corpus
def func1(::
Code-to-test
Figure 2: An overview of FACT begins with the filtering of fact- based text, followed by alternating generation training of fact-based text and parallel code based on their transformation relationship. In each iteration, a quality evaluation module is employed to assess the quality of the generated parallel code, ultimately achieving semantic and stylistic alignment between natural language and code.
Section 3.3 details the alternating code-text training mechanism. Section 3.4 describes the proposed quality assessment and adaptive loss formulation.
3.1 Problem Definition
We consider the task of text generation with a large language model (LLM). Given an external prompt p that specifies the generation objective and an input context providing background information, the model generates an output sequence of sentences sentence and |x| denotes the sequence length. The overall generation process can be formulated as LLM(- p, t). A central challenge in LLM-based generation is ensuring contextual and internal consistency. We focus on two types of hallucination errors that may arise in the outputs:
[x1 22, ................................ -x| where each x; is
Input- Couut-Cofffiting Hallucination This refers to the generated sequence contains information that contradicts the input l. Formally, we define input-conflicting hallucination as:
x t and Contradict(x,, t) 0,
H(,, ()
otherwise
Context- Context-Conflicting Hallucination. This type of inconsistency arises when any two sentences within the generated output x are logically contradictory, even though they are generated under the same prompt and context. Formally, we define:
1, İi and Contradict(z 2.)
H H(r, 2,)
0, otherwise
where Contradict (I,, 2,) denotes that the i-th and j-th sentences x are logically inconsistent. By formally defining these types of hallucination, our framework aims to enforce both input consistency and internal coherence LLM-ggnerated outputs.
3.2
Text Filtering for Implicit Facts
To support effective alternating training, we filter input texts to retain only those containing factual content suitable for structured code representation. This achieved via an LLM-based filtering mechanism Emm. which determines whether text meets the criteria. Specifically, we use GPT-3.5 with tailored prompt (see Appendix A.1] to perform this assessment. The decision rule is
11, Fum (t) True, o. otherwise
D(t)
where D(t) = 1 means t is selected; otherwise, it is discarded. Rigorous text filtering provides a solid foundation for the effectiveness of subsequent alternating code-text generation training.
3.3 Alternating Code-Tex Training
Predicting code from text and reconstructing text from generated code constitute the core of our training framework, illustrated in Fig. 2 We adopt an alternating training paradigm between

---

Text
Mr. Mitra along with kis friend Sumathanath Ghosh established Mitra Ghosh Publishers on March 9, 1934.
class Person:ll Define class to represent person def' init (self, first name, last name, tille"""): self. name firsi_ name
class PublishingCompan Define class to represent company def [sel_ name. founders, established date); felf.founders founders
0
Code
Create instances for Gajendrukumur Mitra and publishing company gajendrakumar mitra Person(first "Gaee-drrjdndraaaa""""" last name-"Mitta", title="Mr")
mitra ghosh publishers Publishing Companynaamme"MMM & Ghosh Publishers", foumders-fgejeeddrkkuaa _mitra,
sumathanath gstabheettbllihhed (e9e--9999---99999
Figure 3: A simplified example showing a text segment and its corresponding code. The code first defines classes Person and PublishingCompany text, and then instantiates them to represent the co-founders and the company. Only single sentence shown here due to space constraints; another complete example is provided in Appendix C
capture the roles and structure described the
text-to-code and code-to-text generation, promoting the alignment of both modalities within a unified semantic space. Both directions are formulated as supervised fine-tuning (SFT) tasks.
Text-to-Code with Pseudo-Label Supervision In the text-to-code direction, since ground-truth code is unavailable, employ pseudo-label self-supervision. For each input the LLM G with prompt Ptzc. (see Appendix (A2) generates a pseudo-labeled code sequence, which serves as the supervision signal for training. During training, given the same input i and prompt, the model predicts the code sequence as
c G(- Pt2c: t).
Both the pseudo-labeled code and the predicted code are produced using the same exemplar based prompt to ensure consistency and alignment with the input semantics.
Code-to-Test Text Supervision In the code-to-text direction, given the generated code sequence and the prompt Pe2:. (see Appendix A.3). the model reconstructs the original fact-centric text:
G(. Pe2t, c).
(5)
The original text is used as the ground-truth target during training. For illustration, Fig presents simplified example both the original text and its corresponding code due to space limitations. complete example is provided in Appendix C
3.4 Quality Assessment and Adaptive Loss
While the alternating code-text training framework enables mutual supervision, it remains critical to ensure that generated code is not only syntactically correct but also semantically faithful to the source text. To address this, we introduce two-stage quality assessment and an adaptive loss objective, explicitly incorporating both syntactic and semantic feedback into the training process.
(1) Syntactic Validity Each generated code e; is executed by Python interpreter ensure syntactic correctness. Only code that executes successfully-i.e. free of syntax errors, undefined variables, and incomplete constructs-is retained for subsequent evaluation. For any code that fails the syntactic validity check, we assign a fixed similarity score of S: = 0.1 (see Appendix B.I for details) in the subsequent loss computation.
(2) Semantic Fidelity via Reverse Reconstruction To assess whether the generated code e; captures the intended semantics of the original text t; conduct reverse reconstruction. With a tailored prompt Preory (see Appendix A.44, let model generates reorganized template reorg by replacing information such entities and attributes with code variables from c;:
reorg C
~ G(- Proorg. tz)
(6)
By executing the generated code e and substituting runtime values into the placeholders of goorg we obtain the instantiated text c; rewy' To evaluate semantic fidelity between the generated and original
5

---

texts, we employ a composite metric combining ROUGE-1 and ROUGE-L [32, where ROUGE-1 captures factual entities, and ROUGE-L reflects global structural similarity. The overall similarity score is defined as:
S = [OOUGE-M( morr' .44 + ROUGE- (c."0r ...))
A higher value of S; indicates a greater degree of semantic fidelity of the code to the original text, as the instantiated entities and related content more closely match those described in the original. For clarity, based on the example provided in Fig. 3
we show that the reorganized text c is composed
of references to code entities and properties:
mitra patrossh_pblishesssfooooioe laasssss(Ooosssaammaaaaa last_namee along with his friend mitra mitra_ghosh__________ wwwppppubrssfoouuueceeee J.first_ name mitra mitra_ghosh www__gghoohhhhhhhgg wwwlishers.ooundeeee J.laaat__aaae
established mitra_ghosh httpggghhhhhuushesssssoee wwwlisherss.aaaeeee.e on mitra hhitra_ghosh_ www.ghhupbbisshssseccempe puubissher..sttablisheee date.
Upon execution c, every placeholder in c, is programmatically replaced with its run-time value, reong' (as illustrated, all placeholders are correctly replaced):
resulting in the fully instantiated text C;
Mr Mitra along with his friend Sumathanath Ghosh established Mitra Ghosh Publishers on March 9, 1934.
Adaptive Loss For each training example, we weight the text-to-code loss by (1 St), so that samples with lower quality have a larger penalty on model updates. For batch size n, the final loss of the text-to-code generation process for each iteration is defined as:
Coc
n Σ
(D,)) C1,20
(8)
where Line denotes the loss of the text-to-code generation process for sample in this iteration.
In our framework, pseudo-code labels are iteratively regenerated at each training round using the latest model checkpoint, rather than being generated only once. This enables the supervision signal to dynamically improve alongside the model's ss evolving capabilities, yiclding progressively higher- quality pseudo-labels that further enhance modality alignment. Through this alternating code-text training process, approach establishes robust and generalizable framework that consistently improves logical consistency and effectively mitigates inconsistent ballucinations of LLMs.
Experiment
4.1
Settings
Model We evaluated three base models: 1.aMMA-3.1-Instuuce (88. Dinistral-Instruct-B88 33 and Owen-2.5-Instrucct (34.. These recent and widely used instruction-tunet LLMs have comparable parameter sizes (7B/8B) and demonstrate strong performance across various NLP tasks. All models
publicly available for reproducibility.
Datasets We randomly sampled 10,000 entries from the Wiki-40B-en 24 dataset, using only the first paragraph of each entry. After fact-based filtering, 53.74% (5,374) were retained for alternating training, while 27.85% and 18.41% were labeled as non-factual and invalid (see 4.3). Owing to our alternating approach (text-to-code and code-to-text), each sample is used bidirectionally, yiclding 10,748 samples (5,374 2) per iteration.
For evaluation, we selected benchmark datasets for both text summarization and question answering (QA). Specifically, we used CNN/Daily Mail[35] and SAMSum (36) for summarization, and SQuAD v2 38 together with HaluEval[39 for QA. These datasets were chosen to comprehensively cover both classic and emerging scenarios: CNN/Daily Mail and SQuAD are widely recognized classics, while SAMSum provides a high-quality. human-annotated benchmark for dialogue summarization, and HaluFval (introduced in 2023) specifically targets hallucination evaluation in L.MMs. Since EaluEval only provides the construction methodology, built the dataset ourselves following its

---

Table 1: Hallucination evaluation results for all methods and backbone models across datasets are reported. "Consis" and "AlignS" denote Consistency and AlignScore, respectively.
Summary Task
QA Task
Method
CNN/Daill Mail
SAMSum
SQUAD V2
HvluEval
Consis(() Align((1)
(Tn (lignS( Anah-v2(L) AlignS(
A2 (lignS(T)
LLaMA-33I-Instrucce
Base
86.40
83.28
90.79
90.67
14.32
95.94
12.48
96.35
Prompt
87.73
86.51
91.08
90.98
15.02
95.50
8.92
96.17
SFT
90.33
84.06
89.56
87.14
13.14
95.24
8,27
98.52
SymbCoT
86.80
84.28
90.23
88.37
13.82
95.79
11.66
96.58
Lookback
83.12
81.75
81.12
84.05
17.62
93.08
14.31
92.82
FACT
91.07
87.29
92.77
91.58
8.39
98.33
7.01
98.93
Base
87.30
82.35
89.14
88.23
13.31
79.15
12.15
94.16
Prompt
88.02
83.12
90.92
89.10
14.53
95.64
13.02
95.11
SFT
90.10
83.54
89.45
87.33
12.65
86.47
10.36
96.61
SymbCoT
88.26
82.10
84.98
85.47
16.65
95.60
10.60
96.14
Lookback
84.37
80.15
82.13
85.95
15,40
92.11
14.26
93.04
FACT
90.93
86.21
93.36
92.63
97.75
7.64
98.84
Qwen-2.5
1.ecn-2.-Insstrcttt.
Base
83,76
82.56
85,23
86.42
17.22
93.65
14.75
95.05
Prompt
84.51
84.83
86.06
85.41
16.31
93.93
13.98
95.14
SFT
89.12
81.13
87.69
88.95
12.41
94.17
11.14
96.39
SymbCoT
84,45
81.84
84.31
86.72
13.25
93.67
11.38
95.35
Lookback
83.84
81.52
81.25
83.30
16.18
92.19
14.36
93.17
FACT
90.70
85.53
89.77
90.21
8.22
98.56
9.22
97.67
original paper. For fair comparison, also randomly sampled 10,748 instances from each dataset for training the baseline methods.
Baselines We compare FACT with the following baselines, covering diverse modeling paradigms: (1) Base Model: the original instruction-turs sutiiioonttnedddd model; (2) Naive Prompting: the model was prompted
with straightforward instructions (see Appendix A.5 (3) Supervised Fine- TuninT (SFT): model
fine-tuned on task-specific datasets; (4) SymbCoT [19 a recent method that reformulates chain-of- thought reasoning symbolic inference improve faithfulness; (5) Lookback [18] hallucination detection and mitigation framework using a linear classifier on lookback ratio features.
Metrics For hallucination evaluation, adopt several recently proposed targeted metrics: Align- Score [40] for both summarization and QA, UniEval Consistency [4]] specifically for summarization, and Anah-V2 V2 42 exclusively for QA. We also report overall performance metrics, including Coher-
ence and Relevance [1 for summarization, and F1 as well as Exact Match (EM) 43 for QA.
Implementation Details To ensure fair comparison, all models were trained and evaluated under identical configurations. For the Base Model and Prompting variants of each backbone, inference was conducted using LLAMA-Factoon both models, being instruction- tuned, adopted default chat templates and greedy decoding. FACT and SFT were also trained on the same four RTX 4090 GPUs with LI LLLMA-Factoo for three epochs, with a learning rate of 10 4 batch size 32, and LoRA (rank 8) for consistent acceleration. During inference, these models also employed greedy decoding. For the SymbCoT and Lookback baselines, we used the official implementations and configurations, replacing their backbone models with LLaMA, Mistral, and Qwen.
on four NVIDIA GeForce RTX 4090 GPUs (24GB each);
4.2 Evaluation Results
Hallucination Evaluation Results As shown in Table FACT consistently outperforms all baselines across all models and datasets hallucination metrics. For instance, with the LLaMA backbone, FACT achieves average improvements of 4.19% (Consistency)) and 3.31% (AlignScore) on CNN/Daily Mail, 4.21% (Consistency) and 3.34% (AlignScore) on SAMSum, 6.39% (Anah-v2) and 3.22% (AlignScore) SQuAD v2. and 4.12% (Anah-v2) and 2.84% (AlignScore) on HaluEval.

---

Cansistacyy
AligaScerv
Coberence
Relevance
0.58
0,5129
0.5029
0
0.96
8.937
NLV337 6.2144 0,9224
03311 6A334 0,8567
0.92
0.82
Sumnaryy- CNNiDally
Summary-CNNinn Mail
Summary CNNiballl
Summary- CNNiibally
AligaScare
F
Exact Match
0.1K
0.92
0.398
0.90
0.1336
BL0K3T
(09143
0.9215
8.9335 9.4111
108264
0.8514
0.8614
10,9026
B.8226
0.06
0.50
0.78
QA-Squa
QA QA-SquaVV2
QA SquadV22
QA QA-SquadV2
CarleTeect
Talt Only
w/l Astmenmat
TestDCode Only
w/u
Fart-Fiiering
FACT
Figure 4: Ablation results on CNN/Daily Mail and SQuADv2 with the LLaMA backbone, illus- trating inconsistent hallucination metries.
Figure 5: Ablation results on CNN/Daily Mail and SQuADv2 with the LLaMA backbone, illus- trating task-specific evaluation metrics.
Beyond horizontal comparisons across datasets, vertical comparisons against individual baselines on LLaMA further highlight the superiority of FACT, with average improvements of 7.57% over Lookback, 3.95% over SymbCoT, 3.49% over Base, 2.64% over SFT, and 2.57% over Prompt across all tasks. Similar results for Ministral and Qwen further highlight the robustness and generalizability data and is thus sensitive to annotation quality, while SymbCoT. though strong for first-order logic symbolic reasoning, less effective on tasks with complex relationships. Overall, these findings establish our fact-driven alternating code-text training novel framework that effectively transfers code's logical rigor to LLMs' outputs, enabling reliable and accurate reasoning across a wide range of NLP tasks.
of FACT. The largest margin is against Lookback, which relies on callucination-annotaaa
Task Performance Evaluation Results As shown in Table 2 FACT also delivers consistent im- provements on both summarization and QA task metries. For instance, with LLaMA, FACT achieves average improvements of 6.35% over Base, 3.81 % over Prompt, 6.76% over SFT, 2.64% over Sym- bCoT, and 5.19% over Lookback across all tasks. Comparable gains are observed with Ministrel and Qwen, supporting the robustness of the conclusions. These experimental results demonstrate that enhancing consistency substantially boosts overall task performance. This improvement can be attributed to the prevention of the "hallucination snowballing" phenomenon [44), where hallucina- tions accumulate during generation, resulting in compounding errors and degraded output quality. By applying FACT to L.Mss, we curb the accumulation of such errors, ultimately promoting the generation of more accurate outputs and thereby improving overall task performance.
Ablation Study Results We conduct comprehensive ablation studies on the CNN/Daily Mail and SQuAD V2 datasets using the LLaMA backbone to systematically evaluate the contribution each FACT module. Specifically,
examine the effects of omitting the factual text filter, disabling alternating training by using only text-to-code or only code-to-text generation, and removing the quality assessment module. For hallucination evaluation, shown in Fig. 4 retaining only the code-to-text pathway results in the largest increase in hallucination rate (5.72% on average), followed by using only text-to-code (4.69%), removing the quality assessment module (4.28%), and omitting the factual text filter (3. (3.29%). Similar patterns are observed for overall task performance: using only code-to-text, only text-to-code, removing the quality assessment, and omitting the factual text filter lead to mean performance drops of 6.23%, 5.13%, 4.69%, and 3.16%, respectively (see Fig. [5 These results underscore the necessity including all modules in the FACT design. Among them, the rext-to-code pathway within the alternating training scheme is particularly valuable, as it guides the model to parse and organize factual content into code-like, logically consistent representations, thereby strengthening alignment between language and structured information. In comparison, the code-to-text pathway mainly helps the model retain the ability to generate fluent and semantically coherent text.
8

---

Table Task evaluation results for all methods and backbone models across datasets.
Summary Task
QA Task
Method
CNN/Daily Mail Coherence Relevance
SAMSum
SQUAD V2
HaluEval
Coherence Relevance
Exact Match
F1
Exact Match
LLaMA-3. I-Instruct-88
Base
92.40
92.61
93.25
90.33
83.59
79.07
81.33
77.46
Prompt
96.82
95.56
94.84
93.17
84.37
80.21
84.69
80.52
SFT
90.56
88.29
91.49
89.67
82.29
81.25
83.79
79.25
SymbCoT
96.52
95.47
96.37
95.78
82.14
86,20
85.44
81.73
Lookback
94.31
90.06
93.62
92.74
81.63
80.81
83.69
82,25
FACT
97.89
97.26
98.77
98.09
88.89
87.63
87.26
84.86
Ministral Himistral-Instruct-aa Instruct-88
Base
92.73
92.32
93.32
92.67
84.65
83,331
80.22
77.54
Prompt
94.26
92.87
94.97
93.65
85.59
81.78
82.49
78,20
SFT
91.04
90.33
90.25
90.33
81.77
80.42
82.86
81.53
SymbCoT
95.61
94.88
92.34
91.60
84.06
82.79
84.98
81.72
Lookback
93.32
92.15
91.29
90.06
82.
80.51
83.65
82.77
FACT
96.47
96.16
96.28
95.39
88,17
85.36
86.22
84.13
Qwen-2.5
7B
Base
89.33
88.45
88.26
87.34
80.14
78.25
78.56
76.47
Prompt
92.61
92.18
90.14
89.12
80.59
79.31
81.49
81.05
SFT
93.38
93.48
90.51
91.08
81.44
80.52
80.30
79.25
SymbCT
94.71
93.25
92.25
91.83
82.17
80.44
82.57
81.02
Lookback
94.89
93.62
91.34
91.58
80.44
79.16
81.19
80.44
FACT
95.50
95.14
95.72
94.04
86.59
84.72
84.34
83.28
4.3 Analysis and Discussion
Analysis of the Text Filter Module After fact-based filtering of 10,000 Wiki-40B-en entries as described earlier, 33.74% were retained as fact-type texts, 27. 85.%% were classified as non-fact-type, and 18.41% % were labeled as invalid, largely due to unclear responses from the LLM. To further assess the reliability of the filtering process, three engineers independently annotated 100 samples each of fact-type and non-fact-type texts. Based majority voting, the filter achieved true positive rate of 93% and false negative rate of 7% for fact-type texts, as well as true negative rate 92% and false positive rate of 8% for non-fact-type texts. Additionally, manual review of 100 invalid samples confirmed that 96% genuinely lacked valid labels, primarily because the model failed to follow instructions, generated responses in unexpected formats, was unable to judge ambiguous inputs. These results demonstrate the reliability of the fact-driven filtering process, which provides a robust data foundation for subsequent alternating code-text training.
Code Generation Quality under Pseudo-Label To assess whether our pseudo-labeling with two-stage quality assessment compensates for the absence of ground-truth code, we monitor the quality of generated code across training iterations. Effective supervision should manifest as general improvement. As shown by the purple line in the upper panel of Fig. 6 the proportion of executable code starts at 87.28% increases over epochs, despite minor fluctuations, indicating enhanced syntactic validity. Similarly, semantic fidelity, measured by the similarity between reconstructed and original text, rises by up to 5.23% (red line, lower panel) before plateauing after the third iteration, which motivates three-iterations schedule. Overall, these results show that our approach not only enables effective training without gold-standard code, but also creates a virtuous cycle in which the model's code generation ability and overall performance reinforce each other across successive iterations. Further analysis of failure cases after three iterations is provided in Appendix B.2
rigorous factual text filtering filtering-ans and generally
Effectiveness Structured Code-based Inference assess the effectiveness of structured code as an intermediate representation, we perform an ablation study using two-stage inference pipeline: source texts are first converted into structured code (with unconvertible segments retained as plain text), and summaries or answers are then generated from the converted code. On the CNN/Daily Mail dataset with a LLaMA backbone, code-based inference achieves 90.2 00.24% Consistency (vs. end-to-end 91.08% Table 1), 86.44% AlignScore (vs. 87.15%), 97.01% % Coherence (vs. 97.89%), and

---

0.91
88.50%
89.71%
Ieput-Craliit (LLM) Ieput-Confiit (Buman))
Cretest-Cellld (LLM) Conteal-oollllt (floman)
0.89
87.28%
Pereentage (%)
Generated Executable Code Ratio
40-
0.83
0.93
91.61%
92.71%
93.19%
20-
0.89
87.96%
Average Similarity
0.85
Lookback
Base
Prompt
SymbCuT
SFT
FACT
1st
iter.
2nd iter.
3rd iter.
4th iter.
Figure 6: The trends of the proportion of gen- erated executable code and the similarity score between reconstructed and original text as the number of training iterations increases.
Figure 7: Sentence-level hallucination rates for input-conflicting and confext-cofflcctng halluci- nations, evaluated using the GPT-4.1 1 and human across different methods.
96.72% Relevance (vs. 97.26%). On SQuAD V2, our pipeline yields 9.54% on Anah-v2 (vs. 8.37%), % on AlignScore (vs. 98. )),,,, 87.73% F1 (vs. 88.89%). and 87.05% EM (vs. 87.63%). These minimal reductions consistency and task performance demonstrate that structured code serves as transparent and faithful alternative representation, preserving essential semantic information for downstream tasks. This provides strong evidence for the effectiveness of the FACT framework in aligning text and code within unified semantic space and enabling high-quality code representations.
Analysis of Input- and Context-Conffictng Hallucinations We evaluated FACT on challenging samples from the CNN/Daily Mail dataset, specifically selecting cases from base LLaMA outputs with consistency scores below 0.6 to target strong logical inconsistencies. Evaluation was performed both automatically, using the powerful GPT-4.1 with the prompt Appendix AA.6 and manually, by averaging the assessments of three engineers on 100 outputs randomly sampled from the above- selected cases. We quantified hallucination by measuring the proportion of sentences showing input- and context- contex--conflcctig errors. As shown in Fig. 7 FACT substantially reduces hallucination rates compared to other methods. Relative SFT (the strongest baseline). GPT-based evaluation indicates reductions of 7.53% in input-conflicts and 5.89% in context-confficts Manual assessment yields similar decreases
7.96% and 3. 3.79%. highlighting the robustness of our findings. Additionally, context conflicts are generally less frequent than input conflicts, and FACT achieves especially notable improvements in addressing this type of hallucination. This may be attributed the text-to-code prediction stage introduced during FACT training, where the process of generating code encourages the model to maintain higher-quality and more consistent contextual logic.
5 Conclusion
In this work, we present FACT, a novel, task-agnostic framework that mitigates inconsistent halluci- nations in LLMs through fact-driven alternating code-text training. By unifying fact-based text and code representations a shared semantic space, FACT brings the logical rigor and consistency of code NLP outputs. Experiments three leading LLMs and diverse benchmarks demonstrate that FACT greatly reduces hallucinations and consistently improves task performance. This framework not only establishes new paradigm for addressing persistent hallucination challenges in LLMs, but also lays a solid foundation for more trustworthy and robust deployment of L.MSs in real-world, fact-intensive reasoning scenarios.
Limitations
Despite the demonstrated effectiveness of our approach, several issues remain in the generated code that warrant further attention. Some outputs still omit essential elements, such as key methods or complete attribute definitions. In addition, while overall logical consistency generally preserved, certain complex cases exhibit incomplete handling of dependencies or lack fine-grained organization. There are also cases with class designs that not fully represent the intended structure or relation- ships, as well as expressions that lack clarity or contain formatting inconsistencies. Addressing these issues-bi by improving content coverage, structural accuracy, and code clarity-may further enhance the robustness of FACT in future work.
10

---

Acknowledgments
The work is supported by the Noncommunicable Chronic Eiseases-Nationas Science and Technol- ogy Major Project (Grant No.2023220500550) and the Beijing Natural Science Foundation (No. 4252046). We thank the anonymous reviewers for helpful comments and feedback.
References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, 1lge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.00 087744 2023.
[2] Liangming Pan, Alon Albalak, Xinyi Wang, and William Wang. Logic-Im: Empowering
large language models with symbolic solvers for faithful logical reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2806-3824, 2023.
[3] Xuhui Jiang, Yuxing Tian, Fengrui Hua, Chengjin Xu, Yuanzhuo Wang, and Jian Guo. A survey on large language model hallucination via a creativity perspective. arXiv preprint arXiv:2402.06647, 2024.
[4] Niels Mündler, Jingxuan He, Slobodan Jenko, and Martin Vechev. Self-contradictory hal- lucinations of large language models: Evaluation, detection and mitigation. In The Twelfth International Conference Learning Representations.
[5] Danilo Neves Ribeiro, Shen Wang, Xiaofei Ma, Henghui Zhu, Rui Dong, Deguang Kong, Juliette Burger, Anjelica Ramos, William Yang Wang. George Karypis, et al. Street: A multi- task structured reasoning and explanation benchmark. In The Eleventh International Conference on Learning Representations.
[6] Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. Siren's 's song in the ocean: a survey on hallucination in large language models. arXiv preprint 23Xiv:2309.0/2/11 2023.
[7] Forrest Sheng Bao, Miaoran Li, Renyi Qu, Ge Luo, Erana Wan, Yujia Tang, Weisi Fan, Manveer Singh Tamber, Suleman Kazi, Vivek Sourabh, al. Faithbench: A diverse hallucination benchmark for summarization by modern 1ims. arXiv preprint arXiv:2410. 132/0, 2024.
[8] David Dale, Flena Voita, Loic Barrault, and Marta R Costa-jussà. Detecting and mitigating hallucinations in machine translation: Model internal workings alone do well, sentence similarity even better. In The 6l Annual Meeting The Association For Computational Linguistics, 2023.
[9] Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan, Asli Celikyilmaz, Yang Liu, Xipeng Qiu, et al. Qmsum: A new benchmark for query-based multi- domain meeting summarization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1921- 2021.
[10] Xiao Pu, Mingqi Gao, and Xiaojun Wan. Summarization (almost) dead. arXiv preprint arXiv:2309.0955 2023.
[11] Nelson Liu,, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157-173, 2024.
[12] Hanmeng Liu, Jian Liu, Leyang Cui, Zhiyang Teng, Nan Duan, Ming Zhou, and Yue Zhang. Logiqa 2.0-an an improved dataset for logical reasoning in natural language understanding. IEEE/ACM Transactions on Audio, Speech, and Language Processing. 2023.
[13] Wanjun Zhong, Siyuan Wang, Duyu Tang, Zenan Xu, Daya Guo, Jiahai Wang, Jian Yin, Ming Zhou, and Nan Duan. Ar-lsat: Investigating analytical reasoning of text. arXiv preprint arXiv:2104.066 2021.

---

[14] Weihao Yu, Zihang Jiang, Yanfei Dong, and Jiashi Feng. Reclor: A reading comprehension dataset requiring logical reasoning. In International Conference on Learning Representations.
[15] Yiming Huang, Xiao Liu, Yeyun Gong, Zhibin Gou, Yelong Shen, Nan Duan, and Weizhu Chen. pei--oont-driven data synthesis with its enhancement mathematical reasoning. arXiv preprint arXiv:2403. 220300333, 2024.
[16] Shubham Toshniwal, Ivan Moshkov, Sean Narenthiran, Daria Gitman, Fei Jia, and Igor Git- man. Openmathinstruct- A 1.8 million math instruction tuning dataset. arXiv preprint arXiv:2402. 10176, 2024.
[17] Zimu Lu, Aojum Zhou, Houxing Ren, Ke Wang, Weikang Shi, Junting Pan, Mingjie Zhan, and Hongsheng Li. Mathgenie: Generating synthetic data with question mack-translation for enhancing mathematical reasoning of llms. arXiv preprini urXiv:2402.16332, 2024.
[18] Yung-Sung Chuang, Linlu Qiu, Cheng- Yu Hsieh, Ranjay Krishna, Yoon Kim, and James Glass. Lookback lens: Detecting and mitigating contextual hallucinations in large language models using only attention maps. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1419-1436 2024.
[19] Jundong Xu, Hao Fei, Liangming Pan, Qian Liu, Mong-Li Lee, and Wynne Hsu. Faithful logical reasoning via symbolic chain-of-thought. arXiv preprint UrXiv:2405./8333 2024.
[20] Aman Madaan, Shuyan Zhou, Uri Alon, Yiming Yang. and Graham Neubig. Language models of code are few-shot commonsense learners. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1384-1403, 2022.
[21] Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven Chu Hong Hoi. Coderl: Mastering code generation through pretrained models and deep reinforcement learning. Advances in Neural Information Processing Systems, 35:21314-215
2022.
[22] Wenhu Chen, Xueguang Ma, Xinyi Wang, and William Cohen. Program of thoughts prompt- ing: Disentangling computation from reasoning for numerical reasoning tasks. Transactions Machine Learning Research, 2023.
[23] Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang. Jamic Callan, and Graham Neubig. Pal: Program-aided language models. In International Conference on Machine Learning, pages 10764-10799, PMLR, 2023.
[24] Mandy Guo, Zihang Dai, Denny Vrandečié, and Rami Al-Rfou. Wiki-40b: Multilingual language model dataset. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages
2020.
[25] Mohammed Saeed, Naser Ahmadi, Preslav Nakov, and Paolo Papotti. Rulebert: rules to are-trained language models. In Proceedings of the 2021 Conference on Empirical Methods Natural Language Processing, pages 1470-147. 2021.
Teaching soft
[26] Bhavana Dalvi, Peter Jansen, Oyvind Tafjord, Zhengnan Xie, Hannah Smith, Leighhnnna Pi- patanangkura, and Peter Clark. Explaining answers with entailment trees. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7358-7370, 2021.
[27] Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. Adversarial nli: A new benchmark for natural language understanding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4885-4901, 2020.
[28) Abhimanyu Dubey, Abhinay Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, al. The llama 3 herd of models. arXiv preprint arXi::2407.21788: 2024.
[29] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 15:24824-24833 2022.
12

---

[30] Hanmeng Liu, Zhiyang Teng, Leyang Cui, Chaoli Zhang, Qiji Zhou, and Yue Zhang. Logicot: Logical thouught instruction tuning. Findings of the Association for Computational Linguisties: EMNIP 2023, pages 2908- 2921, 2023.
[31] Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, et al. Folio: Natural language reasoning with first-order logic. arXiv preprint arXiv: 220.00000000 2022.
[32] Yupeng Chang, Xu Wang. Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. A survey on evaluation of large language models. ACM Iransactions on intelligent systems and technology, 15(3):1-45, 2024.
[33] Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023.
[34] An Yang. Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412. 15115, 2024.
[35] Danqi Chen, Jason Bolton, and Christopher D Manning. A thorough examination of the cnn/daily mail reading comprehension task. arXiv preprint arXiv:/606.000 2016.
[36] Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. Samsum corpus: human-annotated dialogue dataset for abstractive summarization. EMNLP-IJCNLP 2019, page 70, 2019.
[37] Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference Empirical Methods Natural Language Processing, pages 2383-2392, 2016.
[38] Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don't know: Unanswerable ques- tions for squad. In Proceedings the 56th Annual Meeting of the Association for Computational Linguistics ( (Volume 2: Short Papers), pages 784-789, 2018.
[39] Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. Halueval: A large-scale hallucination evaluation benchmark for large language models. arXiv preprint arXiv:2305. 17747, 2023.
[40] Yuheng Zha, Yichi Yang, Ruichen Li, and Zhiting Hu. Alignscore: Evaluating factual consis- tency with unified alignment function. In The 61st Annual Meeting Of The Association For Computational Linguistics, 2023.
[41] Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji, and Jiawei Han. Towards a unified multi-dimensional evaluator for text generation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2023-2038, 2022.
[42] Yuzhe Gu, Ziwei Ji, Wenwei Zhang, Chengqi Lyu, Dahua Lin, and Kai Chen. Anah-v2: Scaling analytical ballucination annotation of large language models. arXiv preprint arXiv:2407.04692. 2024.
[43] Kevin Fischer, Darren Fürst, Sebastian Steindl, Jakob Lindner, and Ulrich Schäfer. Question: How do large language models perform on the question answering tasks? answer. arXiv preprint 2rXiv:2412.12893, 2024.
[44] Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A Smith. How language model hallucinations can snowball. Forty-firs International Conference on Machine Learning.
13

---

Appendix and Supplemental Material
A Introduction to the Prompt Used
A.1 The Prompt for Filtering Texts
Your task is to determine whether a text is suitable for description in code (e.g. modeling relationship using class-based instances, modeling process using function), and return "yes" if is suitable, or "no" otherwise. Generally speaking, organized text that dexcribes seriex of facts and processes is suitable for code description, while text that mainly expresses emotions or illogicality is not suitable for code description. The following is an example:
Input: The Hullaballoos were created in August 1964, but had been working in the UK for over three years under the name of Ricky Knight and The Crusaders. They were not named after the American television programme Hullabaloo. Their name came from the city of Hull, England, whence they hailed. Ourput: yes
Please decide whether the following text is suitable for code description according to the above requirements: FK Drezga is founded on 1972, as team from the Piperi region near Podgorica. At their first season, FK Drezga played in Fourth League Central region (the lowest rank in SFR Yugoslavia). Club was dissolved at the end the seventies.
A.2 The Prompt for Predicting Code from Text
The prompt provided to the trained base model for alternating training in predicting code from text.
Please use a code snippet to encapsulate the given text into a structured format using classes and instances in Python. Here is an example for you.
foput:for Gajendrakumar Mitra: Mr Mitra along with his friend Sumathonath Ghosh established Mitra & Ghosh Publishers on March 1934.
output:
#Define a class to represent a person
class Person:
def (self. first name, last name, title=
self,first selfffirstnamme name = first_name name
self.last_nann name _ame
self.title = title
#Define class to represent publishing company class PublishingCoppary:
def init (self, name, founders, established date):
self.name = name
self.founders = founders
self. relf.establisheed_e date established date
Create instances for Gajendrakumar Mitra and Sumathanath Ghosh
gajendrakumar mitra Personffirst sattsttnmm=__" aaam=="Gajeddrukumad
last name= "Mitra", title= title="Mr")
sumathanath ghosh Person(first name=="Smattonn "Sunnthanatttttat last name= name="Ghosh "Ghosh")
# Create instance for the publishing company Mitra & Ghosh Publishers mitra ghosh publishers
PubltshingCompanyy
srame="Mitra & Ghosh", Publishers
founders=[ gaajesdraaaaanaaaar wawaaaaarmmttrummmaeeee muirra,sumathanath ghosh], established "lut==]1934003097) Please complete an encapsulation of the following text based on the above requirements and particles. Bacon was born in Ipswich and lived in Great Yarmouth as a child. Bacon attended Yarmouth Art School from 1917-1923. where she examinations the earliest age possible. She studied at the Norwich School of Art and then the Royal College of Art in London, obtaining her diploma in 1927.
scholarship in 1917 and 1921 passed the Board of Education drawing
A.3 The Prompt for Predicting Text from Code
The prompt provided to the trained base model for alternating training in predicting text from code.
Here a code snippet which encapsulates an original text into structured format using classes and instances in Python. You are going to predict the original text after reading the code
#Define class to represent a person
class Person:
def init (self. first_name, name, last name, title= ""): Secffrssttammeeae self.last name last name
self,frrst name = femst_name name
14

---

self.tille title
#Define class represent a publishing company class PublishingCompuny:
def init inelf, name, founders, sttablisheddaatep nale = name
self.foumders founders
self.established date established date
Create instances for Gajendrakunar Mitra Sumathanath Ghosh
gajendrakumar mitra Person(first Satffisssttmmm___eeeeeeeeere name= name=="Gaenndddeee "Gajendrduuuaua"""" last name= "Mitra,, , title="Mr") sumathanath gghosh Person(first name= "Smn==""mmthhnatt"""" last name= "Ghosh")
# Create instance for the publishing company Mitra & Ghosh Publishers PublishingCompane
mitra ghosh publishers mitra, mnirra,sumathanath ghasaanntttgghhhhhhggg established date= "att==1993-03-099)
name= "Mitra
Ghosh",
Publishers
founders=| foundess-gggiendrdkunna gajendrakumar
A.4 The Prompt for Predicting Reorganized Text
The prompt provided to trained base model predict reorganized text based generated code.
Please further utilize the original text the generated code data to represent the entities in original text with code, forming a new mixed text. Below is an example.
text=f"miruu_ satgshhhhhhhhh publishers, publishers,oumndrs((0. founders(O].iii
mitra_ghost www.ppuberrsfoounnadee firstttt name
ttiggghhsubllhhersssss mitra -htgghhhhhhhhh pwwlishubssss....ce founders! (0) last
laallllstaaaaa name along with his friend
mitra ghosh publishers,fol founderrs first name mitra ghosh publishersfounders) laattt name established mitra ghosh mublishers.nume on mitra ghosh publishers. weattterr..stabliiie established date.
original text: The Hullaballoos were created in August 1964, but had been working in the UK for over three years under the name of Ricky Knight and The Crusaders. They named after the American television programme Hullabaloo. Their name came from the city of Hull, England, whence they hailed.
A.5
The Prompt for the Base and Prompt Versions Used for Model Testing
Table For the summary and QA tasks, show the instructions for the base and prompt versions of the three model bases.
Tasks
Base
Prompt
Summary
Write summary of the following news.
Write summary of the following news. Attention should be paid to the consistency of the abstract with the original avoid generating content with hallucinations.
QA
You are a question answerer. You should answer the questions directly based on the given reference without adding any prefixes
You are a question answerer. You should answer the questions directly based on the given reference without adding any prefixes suffixes, and without analyzing the answers. After answering the question, do say anything clse. Please do not output content that incon- sistent with the context, and avoid giving irrelevant or contradictory answers. Reference document: ..... Please answer the question based the above reference:
suffixes, and without analyzing the answers. After answering the question, do not say any- thing else. Reference document: Please answer the question based above reference:
15

---

A.6
The Prompt for Evaluating Two Types of Inconsistent Hallucinations
Table 4: For the summary and QA tasks, provide the instructions for evaluating two types of hallucinations.
Tasks
Input-Conflict
Context-Confict
Summary
The following presents where a large language model generates summaries based on the story con- tent, including the original story and the model'ss output. Please check whether the output contains any inconsistencies with the story, such as fabricated information not present in the original story. If incon- sistencies are found, return all sentences containing them; if there none, respond with "None."
The following content the output large language model. Please check for any seff-contradictory parts, specifically instances where the information is incon- sistent contradicts itself. If contradic- tions are found, return all sentences con- taining them; none are found, respond with "Nonc."
16

---

B Supplement to the Experimental Section
B.1
The setting of the S, hyperparameter.
We conducted hyperparameter study select the optimal penalty value for S; when the generated code is executable. compared several candidate values (S; 0.01, 0.1, 0.2, 0.3), and each, evaluated (1) executable code ratio over three rounds of alternating training, and (2) the final consistency metric (hallucination rate) on the CNN/Daily Mail datasct using a Llama-based model. The results are shown in Table
Table 5: Influence of penalty S, on executable code ratio (Exec. Ratio) over three alternating training rounds and consistency (all values %).
S,
Exec. Ratio (Ist)
Exec. Ratio (2nd)
Exec. Ratio (3rd)
Consistency
0.01
84.33%
85.77%
85.26% 88.30 %
87.42%
0.1
87.28 %
88.50 %
91.08 %
86.40%
87.30%
87.62%
89.24%
0.3
85.02%
85.50%
85.14%
88.01 %
As shown in the table, S; = 0.1 achieves the best trade-off in both executable code ratio and consistency. Smaller values (e.g.. S, 0.01) lead to lower and less stable performance, while larger values degrade the results, Thus, we set S, 0.1 based optimal empirical outcomes.
B.2 Error Analysis of Low-Quality Code
We analyzed the generated code samples identified low quality by the quality assessment module. After the third round of alternating training, we designated samples with similarity scores below threshold of 0.8 low quality, accounting
13.97% of data, while the remaining 86.03% exceeded threshold. addition. there were also code samples that remained non-executable after three rounds of alternating training. To gain insight into the nature of these low-quality cases, we conducted detailed error analysis by manually evaluating random subset of 100 non-executable samples and 100 samples with similarity scores below 0.8. This analysis revealed main categories errors, as illustrated Fig. | These findings highlight the limitations of current system and suggest promising directions for future research.
Content omissions
Inadequate class designs failed to effectively represent the required information structure or relationships.
such missing methods
incomplete attribute instantiations.
Significant omissions in generated text
expressions
that hindered communication and introduced formatting
which displayed flaws in logical sequencing, disrupting the narrative flow.
inconsistencies, further complicating the presentation
Figure 8: Classification of error causes for generated code that remained low-quality after three rounds of alternating training.
17

---

C A Complete Example of Text with Corresponding Generation Code
Original Text: He was born in Santiago de Cuba, Cuba. He is Wilfredo León Hechavarría and Alina Venero Boza (a former volleyball player)'s son. He studied Escuela Nacional del Voleibol Cubano. León Venero lived Poland for a while with his Polish girlfriend. On July 14, 2015 he received Polish citizenship and expressed his desire to play in the Poland men's 's national volleyball team. On June 24, 2016 he married Malgorzata (bom Gronkowska). On May 13, 2017 their daughter Natalia was born.
Corresponding Generation Code:
# Class representing place with name attrihute class Place:
def init (xelf, name):
self.nume name
# Class representing education record (institution attended)
class Education:
def _init_ _nnit_ssely institution):
self.institution institution
# Class representing period of residence for person at specific location
class Residence:
def init (self. person, location):
self.person = person
self.location location
# Class representing citizenship information (person, country, and date obtained)
class Citizenship:
def innt fsel_ person, country, date):
self.person = person
self.country country
self.date= date
Class representing a relationship (e.g.. parent-child)
class Relationship:
def _init _nei__self,, person, title):
self.person person
self.title title
Class representing a marriage
class Marriage:
def _jnt
__ii__isilll (self. person, spouse, date):
self.person person
self.spouse spouse
self.date = date
# Class representing child
class Child:
def jinit (self. person, child, date):
self.person person
self.child child
self.date date
# Class representing person and relevant personal relationships class Person:
def init (self. first name, last aas_n, title="")
self,first felffirrttnamee= name = first_name name
self.last_mane name last_ name
self.title title self.parents []
self.spouse = None
self.children
#Merhod to add parent object to the person's parent list def add perent(self, parent):
self, parents.appeenll appendlparent #Method to set spouse and marriage date def marry( spouse, date):
self. spouse (spouse, date)
#Method add child with optional date of birth
def add, child(self, child, date of birth:=oone) self. celf.childrennc www.iddreee.apnn(((ci..e...
date date_of_eae birth))
# Create instances for Wilfredo León Hechavarria and Alina Venero Boza
18

---

wilfredo leon hechavarria Person(first name= name="Wilfrede" "Wilfredo", last name= "León Hechavarria')
alina venero boza Person(first Batte/iirrsnmmm==aaaaa "Almaa" last nome=" Boza")
Create instance for Wilfrreo León Hechavarria's father relationship wilfredo_leow wtwp__hhhavvrrraiiiie
= Relationsshppppersnsswwfeee leaeirdd___ooohhhhhhaaeeeee title= "oot="s"n))
# Create an instance for Wilfredoo León Hechavarria mother relationship
wilfredo Leoddo______ooo aeon__hchavarria. Sattaarrraattheraa-e Recattorshiippettoevvviiiioooooo hechavarraaaaaaaaaaaaaaaaa title="son
# Create the main person and set parents
leon Personf "Wilfredo". "León Venero")
leon.add wilfrd sann_hechavarraa)
leon.ar add parent(alina__ _enero (enero_boza)
# Create education record for the person
escuela Education! "Excuela Nacional del Voleibol Cubano")
escuela_nane naciinalllldee sesl__eee_____e del_voleibol _acolloo_ooaoaaa Edducation(instituioonn "Escuela Nacional del Voleibol Cubano") # Create a residence record for the person
residence Residence(leon, "Poland")
wilfredo Jeon hechavarria residence Residencee(errsnneiiiiiieed Jeon hechavarria, Create a citizenship record for person
hatphrrcer_eeooeeehhhaarrara location= "Poland")
citizenship Citizenship(leon, "Poland". "2015-07-14") =
wilfredo Jcon
country country= "Poland".
date= daat==2015<>> 2015-07-1 14")
Create a spouse object and connect marriage to the main person malgorzata- Person( "Malgorzata", "Gronkowska")
teon.marry malrrymalgorraaa "2016-06-24")
malgorzata__ _instancceeeeee Person(first patts__amm""""m palpaaammmaaaaaann "Malgorzata", dast_name= "Gronkowsok wilfredo Leon hww.hhhhhaarraaa marriage
leon hechavarria, spouse=malgozzata instance, date= "2016-06-24") # Create a child object and add to the main person with date of birth natalia = Person( "Natalia", "León Venero")
porriaag(person=viffeee
leon.add child) "2017-05-13")
natalia citaaia_instanee Person(first_ fitpffsttttt==ttteee "Natalmmmaaaaiaa" last_name="León Venero") wilfredo leon hechavarria child
Child leon hechavarra,, child=natalia instance, date= "2017-05-13")
Reorganized Text:
Note: For better readability, the content is displayed with appropriate line breaks.
He was horn in Place(name= 'Santiago de Cuba') J.name, Cuba.
He is wilfredo llee__eeo________e_ee www___hhhhaarr... title wilfredo Jewo_eeeoonnleeeeeeeeeeeeeeeeeee hwwp_hehhaaariaa.mmmmm first name
wilfredo Jeon hechavarria. last name and alina_venero venero ceenero_boza.teee boza.title
alina venero boza, name alina venero boza.last name's 's wilfredo leon, caltuvarriuffatheriti He studied at escuela nacional del voleibol cubano. institution.
wilfredo leon hechavarria. title wilfredo Ieon hechavarria,fire name wilfredo leon hechavarria. last name lived in wilfredo_led Reon_hechavarrie www.rreeeieeeeeeeeeee location for while with his Polish girlfriend. On foffredo_eom coon_hehhavarii cittrvarriiacitienshpppp date, received Polish citizenship and expressed his desire to play in the Poland men's 's national vollcyball team.
On wilfredo coon_hechavariia caarriiamarrrgee..o date, he married malgorzata malgoraate_insttmce.iiee instance.titee malgorzata instance first name malgorzata instance last name.
On silfredo_eeon hechavarria child. date, their daughter natalia instance.tic title natalia instance, first name natalia instance last name was born.
19

---

NeurIPS Paper Checklist
1. Claims
Question: the main claims made the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: In the Abstract, we clearly elucidate our contributions, and at the end of Section 1 Introduction, we further detail our contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much results can he expected to generalize to other settings.
It is fine to include aspirational goals as motivation long as is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
I Limitations of the work are discussed near the end of the paper, under the "Limitations"
heading.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper limitations, those are not discussed the paper.
The authors encouraged create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results these assumptions (c-g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
violations of
The authors should reflect on the scope of the claims made, e.g.. if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should he articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting.
speech-to- seech-to-tex- text system might not used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms how they scale with dataset size.
applicabll, the authors should discuss possible limitations their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and reffreneee
20

---

All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide implementation details Section to facilitate reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments. No answer to this question will not be perceived well by the reviewers: Making the paper reproducible important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For novel architecture, describing the architecture fully might suffice,
example, if the contribution the contribution is specific model empirical evaluation, it may necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (c.g., the case of a large language model), releasing of model checkpoint, or other means that are appropriate to research performed.
While NeurIPS does require releasing code, the conference does require all submissions provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should deseribe the architecture clearly and fully.
(c) If the contribution is a new model (e.g.. a large language model), then there should either be a way to access this model for reproducing the results a way to reproduce the model (e.g.. with open-soure datasct instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source source models, it may be that access to the model is limited in some way (c.g.. registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instructions to faithfully reproduce the main experimental results, described supplemental material?
Answer: [Yes]
Justification: repository.
We use publicly available datasets, and we will provide a anonymized link to the code
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines
http://iiipscccppuliipe
Galdes//CoeeSubni set.suabiiiiioooooooo.. for more details.
While we encourage release of code and data, we understand that this might not possible,
so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g.. for a new open-sourg benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: : //np////....../////// cc/public/ oui.dess/CooeeSttmiessssooooooo for more details.
21

---

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from script and why.
At submission time, preserve anonymity, authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g.. data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide details about datasets, inference, evaluation, hyper-parametery and baseline in Section 4.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate informa- tion about the statistical significance of the experiments?
Answer: [Yes]
Justification:
All reported results in this paper are statistically significant (p < 0.05) over five random
seeds.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for caleulating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report 96% CI. if the hypothesis of Normality of errors
a 2-sigma error than state that they not verified.
asymmetric distributions, authors should be careful not show in tables or figures symmetric error bars that would yicld results that are out of range (c.g. negative error rates). error bars are reported tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification:
We discussed the GPU we used to run experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
22

---

The paper should provide the amount of compute required for each of the individual experimental well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer:
[Yes]
Justification:
Our paper conforms with NeurIPS Code Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., if there is a special consideration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [No]
Justification: We consider this work to be primarily technical research that has not yet been applied to
specific real-world scenarios. Therefore, we helieve does not directly produce social impacts
Guidelines:
The answer NA means that there is no societal impact of the work performed.
the authors answer NA or No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance). fairness considerations (e.g., deploy- ment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitor- ing misuse, mechanisms to monitor how system Iearns from feodback over time, improving the efficiency accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datascts)?
Question:
Answer: [NA]
Justification:
We doesn't 't release data or models that have high risk. This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines restrictions to access the model or implementing safety filters.
23

---

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All the existing assets including code and data used in this paper are properly cited.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version the asset is used and, if possible, include a URL.
The name of the license (c.g., CC-BY 4.0) should be included for cach asset. scraped from particular source (e.g.. website), the copyright terms of service of that source should provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, paperswi thcode. www.coom//aaac /attttttsttt has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re-per re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not introduce new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the datasct/code/model as part of their sub- missions via structured templates. This includes details about training, license, limitations,
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an amonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsoureing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelincs:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should included the main paper.
According to the NeurIPS Code of Fthics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:

---

Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that paper does not involve crowdsourcing nor research with human subjects.
Depending on the country which research is conducted, approval (or equivalent) may be required for any human subjects research. If you ohtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applica- ble), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe usage LLMs it an important. original. or non-standard component the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the rescarch, declaration is not required.
Answer: [NA]
Justification: This research does not involve the use of LLLMs in any important, original, non-standard component of the methods.
Guidelines:
The NA that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer our LLM policy (https: httpss///mourssecc///oooferrree/ for what should or should not be described.
25


Reviews:
[{"review_id": "ZEd27tzv5I", "reviewer": "Reviewer_bv6X", "summary": "The paper proposes a framework called Fact-driven Alternating Code-text Training (FACT) to address inconsistent hallucinations in large language models. The paper claims that there are two types of errors: input conflicting and context conflicting. To tackle this, the FACT framework trains the LLM by alternating two tasks, which are text-to-code and code-to-text. The core idea behind this is that factual text has an inherent structure that can be mapped to code. The experiment results show that such training method could reduce inconsistent hallucinations and improve overall performance on QA and summarization tasks.", "questions": "1. How will this training method affect the model's other capabilities? Will there be performance drop?\n2. How will the system handle very complex factual relationships?\n3. How to make sure that the model can convert code back into text faithfully?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n1. The paper proposes a novel framework that can reduce inconsistent hallucinations across a wide range of tasks.\n2. The experiment shows that the proposed FACT framework is effective, with clear hallucination reduction and task performance improvement.\n3. The method seems to be general as it works on 3 different large language models (Llama, Mistral and Qwen)\n\n## Weaknesses\n1. The FACT framework will likely work for clear and unambiguous facts. There could be corner cases that makes it hard to map it to the code. There could be ambiguity and figurative text.\n2. The method is hard to scale up. The training efficiency will be lowered when there are extra steps like defining a code-like structure, converting text to code and then evaluate their factual consistency.\n3. The tested datasets are a bit out dated. Newer and larger datasets are helpful to evaluate the effectiveness of the method (because the base models are relatively newer)", "was_revised": true, "final_justification": "The rebuttal from the authors has addressed most of my concerns.", "rebuttal": "We sincerely appreciate your kind feedback as well as your recognition of our work. Below are our responses to each of the concerns you raised.\n\n**1. Impact on the Model’s Other Capabilities:** Thank you for your question. Our method adopts an alternating training scheme between text-to-code and code-to-text, with each direction implemented using supervised fine-tuning training (SFT)—a widely used approach that is generally not found to significantly impair model’s other abilities. \n\nTo address this concern, we evaluated not only hallucination metrics but also core task metrics such as Coherence and Relevance for summarization, and F1/EM for QA. As shown in Table 2 of the original paper, our approach even outperforms the base model on these metrics (+6.35% on average), indicating not only no performance drop but also an overall improvement. We believe this improvement is partly due to reducing the \"hallucination snowballing\" effect [1], where error accumulation degrades output quality. By mitigating such errors, FACT promotes both more accurate generation and better overall performance. \n\nBeyond these results, we acknowledge the importance of broader evaluation for model’s other capabilities and plan to further investigate this in future work.\n\n> [1] Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A Smith. How language model 442 hallucinations can snowball. In Forty-first International Conference on Machine Learning. \n\n**2.Discussion on complex factual relationships:** For very complex factual relationships, our system decomposes them into multiple simpler components wherever possible, each represented as discrete objects, properties, or functions in the code. \n\nTo empirically evaluate this capability, we used code-based filtering and manual confirmation to select 100 samples with complex multi-entity relationships from the Wiki-40B-en dataset (e.g., indirect geographic descriptions and intricate character relationships). Using our trained LLaMA-based model, we generated code for each sample and evaluated executability and text reconstruction fidelity $S_i$ as described in the main paper. We also assessed deeper semantic fidelity with BERTScore. For $S_i$ and BERTScore, we report both the mean and standard deviation (shown in parentheses). The evaluation results are as follows:\n\n| Metric                        | Value   |\n|-------------------------------|---------|\n| Executability Rate            | 0.82  |\n| Semantic Similarity ($S_i$)   | 0.83 (0.17)  |\n| BERTScore                     | 0.81 (0.21) |\n\nThese results indicate that our method can handle a range of complex factual relationships. Nevertheless, we acknowledge that extremely intricate may remain challenging and could require more advanced schema designs in future work.\n\n**3.Faithfulness of Code-to-Text Generation:** Our model is alternately trained in both the text-to-code and code-to-text directions using supervised learning, with ground-truth text serving as direct supervision for code-to-text generation. The loss is computed by comparing the generated text with the original reference, thereby encouraging faithful reconstruction and optimizing model performance. \n\nTo further assess model fidelity, we evaluated texts generated from code after the third training round on the LLaMA-based model using ROUGE-L and BERTScore, two widely adopted metrics for structural and semantic similarity. As shown in the table below, these results indicate that the model demonstrates robust faithfulness in Code-to-Text generation. \n\n|   Metric    | Average | Standard Deviation | Median |\n|:-----------:|:-------:|:------------------:|:------:|\n|  ROUGE-L    |  0.93   |       0.14         |  0.89  |\n| BERTScore   |  0.92   |       0.17         |  0.90  |\n\n**4. Discussion on Corner Cases:**  Thank you for your thoughtful review. We agree that the FACT framework is most effective for clear, fact-based texts, and that its capacity to handle ambiguous or figurative language is currently limited. Our main goal is to enhance logical consistency and reduce hallucinations in downstream tasks that mainly rely on factual reasoning. This focus is driven by the demands of real-world LLM applications—such as healthcare and legal domains—where the reliability of factual information is crucial. Addressing ambiguity and figurative language remains an open challenge and is a promising direction for future work.\n\n**5. Discussion on Training Efficiency:** Thank you for your valuable comment. We would like to clarify that the code-like structure is not an additional or repeatedly defined step, but is directly integrated into the supervised training process through a fixed prompt. This prompt is constructed once and reused throughout training, eliminating any need for dynamic or manual intervention.\n\nAlthough our method involves steps such as generating pseudo-labels and evaluating the syntactic and semantic quality of code outputs (since no code ground truth is available), these procedures are fully automated using tools like Python interpreters and LLMs. Our statistics show that training with our method is approximately 1.5 times slower than standard supervised fine-tuning (SFT), representing a modest but manageable overhead.\n\nMoreover, our results demonstrate that FACT achieves substantial improvements on downstream QA and summarization tasks, even with a small amount of general fact-driven data for training. While larger-scale experiments could provide further insights, they are left for future work.\n\n**6. Dataset Selection:** Thank you for your valuable feedback. In our experiments, we deliberately included both established and recent benchmark datasets for a comprehensive evaluation. For the QA task, we used SQuAD v2 (a widely recognized classic) and HaluEval, a recently introduced dataset in 2023 that is specifically designed to evaluate hallucinations in LLMs. For summarization, we included both CNN/Daily Mail and SAMSum; although SAMSum was released in 2019, it is a high-quality, human-annotated benchmark for dialogue summarization. We agree that incorporating even newer and larger datasets could further strengthen the evaluation and will consider this in future work."}, {"review_id": "vApWMZo1gI", "reviewer": "Reviewer_Hran", "summary": "The paper proposes a Fact-driven Alternating Code-text Training approach (called FACT) to mitigate the hallucinations generated by the LLMs. The approach relies on the grounding that the intrinsic structural patterns in texts can directly be mapped into programmable constructs. The training regime includes both text-to-code and code-to-text predictions by the LLMs. In the absence of direct training supervision, the authors employ a pseudo-labeling technique that ensures syntactic correctness and semantic faithfulness of the original texts. The empirical studies carried out in the paper shows in the improvements in consistency even when trained with a smaller training set. The paper also demonstrates the versatility of the approach in terms of increased performance in summarization and QA tasks without any task supervision.", "questions": "1. The authors should mention the selection criteria of the LLMs.\n2. The formulation of S_i should be justified with appropriate reasons.\n3. The authors should give links to appropriate sections in Appendix (use \\ref in latex).\n4. The authors should add a section in the appendix detailing the evaluation metrics.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper presents a task agnostic framework for mitigating hallucinations generated by LLMs through alternating code-text training. The current approach acts as a bridge between different modalities of inputs where unambiguous and clear representations can be obtained. The authors also minimize the extent of direct training supervision that exhibits emergent capabilities in unseen tasks. The authors make use of pseudo-labels for assisting in training supervision ensuring syntactic correctness and semantic faithfulness of the original texts. The paper attempts to address input-conflicting and context-conflicting hallucinations that are the most prevalent types of hallucinations generated by current LLMs. \n\nThe authors do not mention the selection criteria for the baseline LLMs. It is not clear why LLMs with lower number of parameters are chosen. The formulation of S_i lacks details and intuition. It is unclear why ROUGE-1 and ROUGE-L are combined, and not other variants of ROUGE. The details of the evaluation metrics are missing. The authors should give links to appropriate sections in Appendix. In the current version, it is missing. There are a few grammatical errors in the paper that needs thorough proof-read.", "was_revised": true, "final_justification": "I have reviewed the author rebuttal and they have answered the question regarding S_i and the choice of LLMs, and details about the evaluation metrics. This resulted in increase in my rating from 4 to 5.", "rebuttal": "We greatly appreciate your thoughtful feedback and your acknowledgement of our work. Our detailed responses to your concerns are provided below.\n\n**1.Selection Criteria for Baseline LLMs:** We appreciate your suggestion and agree that clarifying the selection criteria for the baseline LLMs will improve the transparency of our work. We selected LLaMA-3.1-Instruct-8B, Ministral-Instruct-8B, and Qwen-2.5-Instruct-7B as base models because they are recent and widely used instruction-tuned LLMs that demonstrate strong performance on a variety of NLP tasks. Additionally, we chose models with similar parameter sizes (7B/8B) to ensure a fair and direct comparison under comparable computational and resource constraints. These models are all publicly available, making them easily accessible for the research community. We will add these selection criteria to the revised manuscript.\n\n**2.Motivation and Details of $S_i$:** Thank you for your constructive feedback. In our alternating training scheme, the text-to-code SFT stage relies on model-generated code as pseudo-labels, whose quality is critical. To ensure reliable supervision, we introduce the $S_i$ metric as a unified and fine-grained measure of code quality, integrating both syntactic validity and semantic fidelity. By weighting the training loss with $S_i$, the model is dynamically penalized for lower-quality generations, encouraging it to focus on the most challenging cases and improving robustness in the absence of ground-truth code.\n\nChoice of ROUGE-1 and ROUGE-L: We chose the combination of ROUGE-1 and ROUGE-L as our semantic fidelity metrics due to their complementary strengths: ROUGE-1 captures factual tokens and entities, while ROUGE-L reflects global word order and structural similarity—both crucial for evaluating semantic and contextual relationships. To validate this choice, we conducted hallucination evaluation experiments on the CNN/Daily Mail dataset using a Llama-based model, comparing different ROUGE metrics and their combinations. As shown in the table below, ROUGE-1 and ROUGE-L, either individually or combined, consistently outperformed other variants, and adding ROUGE-2 did not provide further improvements. \n\n| Similarity Metric  | Consistency | AlignScore |\n|-----------------------|-------------|------------|\n| ROUGE-1 only  | 89.33       | 88.25      |\n| ROUGE-2 only   | 88.11       | 86.04      |\n| ROUGE-3 only   | 87.11       | 86.12      |\n| ROUGE-L only   | 88.42       | 86.90      |\n| **ROUGE-1 + ROUGE-L (Ours)**   | **91.08**   | **87.15**  |\n| ROUGE-1 + ROUGE-2 + ROUGE-L  | 91.03       | 86.44      |\n\nThe setting of the $S_i$ hyperparameter: During the initial development of our method, we conducted a hyperparameter study to select the optimal penalty value for $S_i$ when the generated code is not executable. We compared several candidate values ($S_i = 0.01, 0.1, 0.2, 0.3$), and for each, evaluated (1) the executable code ratio over three rounds of alternating training, and (2) the final consistency metric (hallucination rate) on the CNN/Daily Mail dataset using a Llama-based model. The results are shown below:\n\n|   Penalty $S_i$   | Executable Code Ratio (1st) | Executable Code Ratio (2nd) | Executable Code Ratio (3rd) | Consistency (%) |\n|:-------------:|:--------------------------:|:--------------------------:|:--------------------------:|:---------------:|\n|     0.01      |          84.33%            |          85.77%            |          85.26%            |     87.42%      |\n|     **0.1**       |          **87.28%**            |          **88.50%**            |         **88.30%**            |     **91.08%**      |\n|     0.2       |          86.40%            |          87.30%            |          87.62%            |     89.24%      |\n|     0.3       |          85.02%            |          85.50%            |          85.14%            |     88.01%      |\n\nAs shown in the table, $S_i=0.1$ achieves the best trade-off in both executable code ratio and consistency. Smaller values (e.g., $S_i=0.01$) lead to lower and less stable performance, while larger values degrade results. Thus, we set $S_i=0.1$ based on optimal empirical outcomes. We will add these details to the appendix of the revised manuscript.\n\n**3.Evaluation Metrics Details:**  Thank you for your helpful suggestion. While we have provided references for all metrics, we agree that detailed explanations are needed, especially since some metrics are quite recent (e.g., AlignScore from 2023 and Anah-v2 from 2024). In the revised manuscript, we will add a dedicated appendix section describing all evaluation metrics used in our study, including hallucination, summarization, and QA metrics.\n\n**4.Appendix Links and Grammar Corrections:**  Thank you for your valuable suggestions. We agree that clickable links to specific appendix sections can greatly improve the reader experience. In the revised manuscript, we will update all appendix references in the main text to use the LaTeX \\ref command, ensuring that readers can directly access the relevant sections with a single click. Additionally, we will carefully proofread the manuscript to correct all grammatical errors and improve the overall quality of the writing."}, {"review_id": "xTSrD7ozM8", "reviewer": "Reviewer_mHXd", "summary": "This paper proposes a training method to reduce hallucinations in LLMs. The method FACT involves an alternating training paradigm between text-to-code and code-to-text. This is motivated by the fact that fact-based text have structural patterns that can be mapped into code functions similar to how math problems can mapped into code functions. A key point of this paper is that this method is task agnostic as compared to previous methods that manually create datasets for a specific domain i.e. legal domain when wanting to reduce hallucination. \n\nBecause there's no ground truth for this kindve data the authors then come up with a generate pseudo-labels to supervise model training and have some filtering checks to ensure they get high quality data. They create their dataset from Wikipedia and evaluate on Q/A and summarization tasks. They finetune three LLMs: Llama, Mistral and Qwen and find that their method reduces hallucination and improves on accuracy. \n\nThe contributions as I see it are\n1) The text-to-code and code-to-text training paradigm \n2) The methodology used to create this data", "questions": "Questions\n1) During the fact filtering stage what is considered an invalid sentence? I understand factual / non-factual sentence.\n\n2) Is ROUGE a strong enough metric for semantic fidelity versus using model-based metrics?\n\n3) It was mentioned that \"For any code that fails the syntactic validity check, we assign a fixed similarity score of Si = 0.1\". Why not just throw the data away?\n\nSuggestions\n1) I suggest you put the fact filtering performance that is listed in the appendix into the main text as that was a big question I had.\n\n2) There were some typos of Mistral vs Ministrel", "limitations": "yes", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths\n\n1) I think the paper is well motivated and the idea to map factual statements into the coding paradigm is very interesting to my knowledge novel. \n\n2) The experiments are well thought out and it seems the authors compare against the latest methods SymbCoT and Lookback. \n\n3) The fact that this method is task agnostic is great and if the dataset gets released publicly I think will be useful for the community\n\nWeaknesses\n\nMy overall concern with the paper is that there are parts that aren't clear but I think can be easily addressed. More specifically:\n\n1) The pseudo-labeling mechanism is a little unclear. Do you generate the data starting from the base model then train it and regenerate the data to improve it's quality or is it all in one pass?\n\n2) In Figure 4 it seems w/o Code2Text is the worst performing model but in the text in Line 252-253 you mention retaining only the\ncode-to-text pathway results in the largest increase in hallucination rate which seem contradictory. Also I would like to know why is this the case versus why text-to-code is more helpful.\n\n3) In Table 1 it seems SymbCoT and Lookback aren't as good as your more simpler baselines (SFT vs prompt) does this mean they are not strong enough baselines to compare against?\n\n4)  It was a little hard to follow the section on Semantic Fidelity via Reverse Reconstruction. Like what do we mean by reorganized template and substituting runtime values.\n\n5) It was a little unclear if after your training the inference takes in text as input and text as output or if it was a two pass inference where you first ask the model to generate the code then the text. Section Effectiveness of Structured Code-based Inference cleared it up but that part came later and caused some confusion.", "was_revised": true, "final_justification": "My score was already a Strong Accept and the authors addressed any small questions I had.", "rebuttal": "We are truly grateful for your constructive feedback and for acknowledging our work. Below, we offer comprehensive answers to your concerns.\n\n**1.Invalid Results in Fact Filtering Stage:** Thank you for your insightful question. In Appendix B.1 of the paper, we reported the distributions and classification performance of different text types, but did not explain the invalid results in detail. We apologize for this oversight and any resulting confusion. An invalid text refers to any input for which the LLM does not provide a clear “yes” or “no” label. To evaluate the classification accuracy for invalid text types, we manually reviewed 100 cases and found that 96% truly lacked valid labels. This typically occurs when the model fails to follow instructions, outputs something in an unexpected format, or cannot make a judgment due to the complexity or ambiguity of the input. We will add these details in the revised appendix.\n\n**2.Evaluation of Semantic Fidelity:** Thank you for raising this important point. We chose the combination of ROUGE-1 and ROUGE-L as our semantic fidelity metrics due to their complementary strengths: ROUGE-1 captures factual entities, while ROUGE-L reflects word order and structural similarity—both crucial for evaluating semantic and contextual relationships. \n\nIn addition to $S_i$ Score (ROUGE-based metric as defined in Eq. 7 of the main paper), we evaluated the quality of generated code across training iterations using BERTScore, which measures deeper semantic similarity using contextual embeddings. The evaluation results indicate that  $S_i$ and BERTScore remain closely aligned and both improve across training rounds are presented below. \n\n|  Iteration  | $S_i$  Score (%) | BERTScore (%) |\n|:-----------:|:-----------:|:-------------:|\n|  Round 1    |   87.96     |     88.45     |\n|  Round 2    |   91.61     |     91.82     |\n|  Round 3    |   92.71     |     92.50     |\n\nTo further assess the logical equivalence between reconstructed text based on generated code and original texts, we conducted a deeper evaluation using DeepSeek-R1. After the third round of iterative training, the data were split into High-Quality (Si > 0.8) and Low-Quality (Si ≤ 0.8) subsets. Alignment rates for (1) entity overlap and (2) relationship overlap were calculated with both DeepSeek-R1 extraction and manual counting. Manual verification of 100 random samples showed DeepSeek-R1 achieves 92% overall accuracy. The evaluation results are as follows:\n\n|   Dataset    | Entity Alignment Rate (%) | Relationship Alignment Rate (%) |\n|:------------:|:----------------------------------:|:-----------------------------:|\n| **High-Quality** |              **92.35**                 |            **86.26**              |\n| Low-Quality  |              54.33                 |            38.16              |\n\nThe High-Quality set achieved much higher factual alignment, indicating that alternating code-text training helps the model capture logical relationships. These results also support that ROUGE-based evaluation, though simple, aligns well with more rigorous metrics and serves as a practical proxy for semantic fidelity.\n\n**3.Handling of Syntactically Invalid Code:** Thank you for your thoughtful question. We chose not to discard syntactically invalid code because our training process is iterative and self-improving. Assigning a fixed low score ensures these challenging cases receive greater loss and remain a focus for the model to improve upon. As training progresses and the model’s code generation ability improves, some previously invalid or low-quality cases may become correctable. By penalizing rather than discarding such cases, the model has repeated opportunities to learn from these difficult examples throughout training. \n\n**4. Clarification on Pseudo-label Generation and Inference Procedure:** Thank you for your question regarding pseudo-labeling. To clarify, we adopt an iterative pseudo-labeling process: in each round of Alternating Code-Text Training, we regenerate pseudo-code labels using the current model checkpoint and use them as supervision for the next round. As alternating training progresses, the model’s representations of text and code become increasingly aligned, which continuously improves its text-to-code generation ability. Consequently, the quality of the regenerated pseudo-labels also increases over time. These higher-quality pseudo-labels provide more effective supervision in subsequent rounds, further enhancing the model’s performance and modality alignment. This mutually reinforcing process allows the model to benefit from both improved alignment and better supervision as training advances. We will update the manuscript to clearly describe this process.\n\nRegarding the inference procedure, our main approach operates in a standard text-to-text manner, where the model directly generates the output text from the input text without an intermediate code generation step. The \"Effectiveness of Structured Code-based Inference\" section presents an additional two-stage inference experiment to assess the utility and reliability of code generated from text as an intermediate representation; however, this is not the primary inference setup used throughout the paper. \n\n**5.Explanation of Figure 4 Ablations and Pathway Importance:** Thank you for pointing out the confusion. We apologize for any confusion caused by the figure legend. In Figure 4, “w/o Alternating (Code2Text)” actually means we retain only the code-to-text pathway and remove text-to-code training. We realize that the notation may be ambiguous and could be misinterpreted; we will revise the figure legend to clarify that this variant keeps only code-to-text. This is consistent with the main text (Lines 252–253), which states that removing text-to-code (i.e., keeping only code-to-text) leads to the largest increase in hallucination rate, as shown by the poor performance of this variant in Figure 4. \n\nRegarding why text-to-code is more helpful: the text-to-code pathway is particularly valuable because it guides the model to parse and organize factual content into code-like, logically consistent representations, thereby promoting stronger alignment between natural language and structured information. This process helps the model learn more consistent contextual logic and improves factual reliability. Meanwhile, the code-to-text pathway primarily converts structured code back into natural language, supporting the model’s ability to generate fluent and semantically coherent text. While both pathways contribute to overall performance, our results suggest that text-to-code is especially effective in mitigating hallucinations.\n\n**6.Discussion of SymbCoT and Lookback Baselines:**  Thank you for your helpful comment. SymbCoT is a strong baseline recently proposed in 2024, which is based on symbolic reasoning and logic rules and is particularly effective for tasks involving first-order logic and constraint optimization. However, our summarization and QA datasets contain not only problems that can be formulated as first-order logic, but also more complex or open-ended logical relationships. SymbCoT is less capable of handling these complex cases, which may explain its performance degradation on our benchmarks.\n\nLookback is another recent baseline that detects and mitigates input-conflicting hallucinations by training a classifier on attention-based features, typically requiring 1k–2k hallucination-annotated examples for effective training. According to the original paper, these annotations are generated automatically using GPT-4o. Our manual review of 100 SQuAD V2 samples showed that the accuracy of GPT-4o-based annotation was only 82%. We speculate that this may have contributed to the suboptimal performance of the method.\n\nPrompt-based and SFT methods are directly based on LLMs and are less constrained by symbolic expressivity or the need for hallucination-specific annotations. They can better leverage the generalization and flexibility of large language models, resulting in more robust performance across diverse tasks. We will add these analyses to the revised manuscript.\n\n**7.Reorganized Templates and Value Substitution:** Thank you for your question. In the \"Semantic Fidelity via Reverse Reconstruction\" section, a reorganized template refers to a text template generated by the model, where specific information such as entity names, attributes, and relationships are replaced with code variables or placeholders (e.g., mitra_ghosh_publishers.founders[0].first_name). This intermediate template makes explicit which parts of the text are linked to code entities.\n\nSubstituting runtime values means that when we execute the generated code, each placeholder variable in the reorganized template is automatically replaced with its actual value from the data at runtime. For example, the placeholder mitra_ghosh_publishers.founders[0].first_name is filled in with \"Mitra\", resulting in a fully instantiated text.\nThis process ensures that the reconstructed text (after substitution) can be directly compared to the original text for semantic fidelity. By doing so, we can systematically evaluate whether the generated code truly preserves the detailed meaning and factual content of the original text.\n\n**8.Placement of Filtering Results & Typographical Errors:** Thank you for your valuable suggestions. We agree that presenting the fact filtering results in the main text will improve clarity, and we will move these results accordingly in the revised manuscript. Additionally, we appreciate you bringing the typographical errors to our attention; we will thoroughly review the manuscript to ensure all such errors are corrected."}, {"review_id": "upbmeIeMFm", "reviewer": "Reviewer_PaBv", "summary": "This paper introduces FACT, a novel training framework designed to mitigate inconsistent hallucinations in Large Language Models (LLMs). The core contribution is a training paradigm that alternates between text-to-code and code-to-text generation. The authors posit that by mapping fact-based natural language text to structured code representations (e.g., classes, objects, functions), the inherent logical consistency of programming languages can be transferred to the LLM's natural language outputs. Due to the absence of parallel ground-truth code, the method relies on generating pseudo-labels for the text-to-code task. The quality of this generated code is managed through a two-stage assessment process that checks for syntactic validity and semantic fidelity, the latter being measured by reconstructing the text from the code and comparing it to the original. The authors conduct experiments on three different LLMs across four datasets for question answering and summarization tasks, reporting that their method reduces hallucinations and improves overall task performance compared to several baselines.", "questions": "My assessment of this paper could improve if the authors can satisfactorily address the following fundamental questions:\n\n1. Can you provide multiple run results for your method and key baselines (especially SFT) using different random seeds and decoding methods (not just greedy decoding)? Demonstrating results with mean, standard deviation, and appropriate statistical significance tests is crucial to proving that the reported improvements are not accidental.\n2. Your method's success hinges on the initial filtering of \"fact-based text\" by GPT-3.5. This introduces opacity and potential bias. Could you (a) provide a formal, operational definition of \"fact-based text\"? And (b) provide a thorough analysis of the data that was kept versus discarded by the filter? This is crucial for understanding what your model is actually learning and for assessing the true generalizability of your approach.\n3. The pseudo-labeling process is circular, and its quality is assessed using ROUGE, a metric for lexical overlap, not logical fidelity. How do you ensure the model is learning true logical relationships rather than simply learning to generate code that is lexically easy to reconstruct? Can you propose or implement a stronger, more semantically-grounded validation metric for the generated code to justify that your training signal is meaningful?", "limitations": "No. The authors include a \"Limitation\" section in Appendix D. However, the points discussed there (e.g., \"omissions\" in code, \"unclear expressions\") are relatively superficial implementation details. \nThe paper fails to acknowledge or discuss its most serious limitations, which can be referred to in Weaknesses and Questions.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "This paper addresses a problem of high significance and proposes an original, creative approach. However, its contributions are undermined by severe methodological weaknesses and a lack of empirical rigor.\n\n**Strengths:**\n\n1. The problem of mitigating hallucinations in LLMs is one of the most critical and timely challenges in the field. Developing methods to improve the faithfulness and logical consistency of LLMs is of paramount importance for their reliable deployment in real-world applications. The paper correctly identifies this significant research gap.\n2. The core idea of leveraging the structural properties of programming languages to enforce logical consistency in natural language is novel and thought-provoking. The proposed alternating code-text training paradigm is an unconventional and interesting method that moves beyond standard supervised fine-tuning or prompt engineering.\n3. The paper is generally well-written, clearly structured, and easy to follow. The authors effectively use figures to illustrate their proposed framework and the problem it aims to solve.\n\n\n**Weaknesses:**\n\n1.  The central premise—that mapping factual text to code imbues it with logical consistency—is presented as an intuitive observation rather than a well-grounded theory. The paper fails to provide a formal definition of \"fact-based text\" or a rigorous argument explaining *why* syntactic structure should translate to semantic consistency.\n2.  The methodology relies on a self-referential, circular loop of pseudo-labeling, where the model generates its own training data. This approach may reinforces the model's existing biases and flawed logical patterns, rather than correcting them. The quality assessment, which depends on a lexical overlap metric (ROUGE) for judging semantic fidelity, is superficial and insufficient to validate logical equivalence.\n3. The experimental evaluation is not sound. The authors explicitly state in their checklist (Q7) that they did not report error bars or perform statistical significance testing. The reported gains could very well be statistical noise, which renders the claims of superiority over baselines unconvincing.\n4. The paper repeatedly claims to be a \"task-agnostic\" framework. However, the evaluation is limited to summarization and question answering, two highly related tasks that often rely on extracting and reformulating factual information. There is no evidence to support the claim that this method would generalize to tasks with different structures, such as creative writing, dialogue, or reasoning. This overstatement of scope significantly diminishes the work's perceived contribution.", "was_revised": true, "final_justification": "The author's response addressed most of the issues, but I still have concerns about the experimental results and novelty of the paper.", "rebuttal": "We deeply appreciate your thoughtful comments and your recognition of our work. Our detailed responses to your questions are provided below.\n\n**1.Supplementary Robustness Results:** Thank you for your valuable suggestions. To address your concerns about statistical robustness, we conducted additional experiments with the LLaMA backbone on the CNN/Daily Mail (summarization) and SQuAD v2 (QA) datasets, comparing our method (FACT) to Prompt, SFT, and SymbCoT baselines. For each method (except Prompt), we ran three times with different random seeds, reporting the mean and standard deviation (in parentheses) for all hallucination metrics. As suggested, we also evaluated all methods with multiple decoding strategies: Greedy Search, Beam Search (beam=5), and top-k sampling (k=10). For the Prompt method, since it does not involve training, only the top-k decoding is affected by random seeds; therefore, we report standard deviations only for Prompt under top-k. \n\nAs shown in the table below, our improvements are consistent and robust across random seeds and decoding strategies, indicating that the gains are not due to statistical noise. Furthermore, all improvements of FACT over SFT are statistically significant (paired t-test, p < 0.05).\n\n| Decoding Strategy | Methods   | CNN/Daily Mail (Consistency(↑)) | CNN/Daily Mail (AlignScore(↑)) | SQuAD V2 (Anah-v2(↓)) | SQuAD V2 (AlignScore(↑)) |\n|-------------------|-----------|----------------|---------------|------------|---------------|\n| Greedy Search     | Prompt        | 87.73          | 86.51         | 15.02      | 95.50          |\n| Greedy Search     | SymbCoT       | 86.71（0.3）    | 84.34（0.4）  | 13.61（0.4）| 95.68（0.4）  |\n| Greedy Search     | SFT           | 90.15（0.3）   | 84.20（0.3）  | 13.23（0.2）| 95.29（0.3）  |\n| **Greedy Search**     | **FACT**          | **91.11**（0.2）   | **87.27**（0.3）  | **8.41**（0.4） | **98.15**（0.4）  |\n| Beam Search       | Prompt        | 88.25          | 87.14         | 14.58      | 95.03         |\n| Beam Search       | SymbCoT       | 87.35（0.2）   | 85.47（0.2）  | 13.14（0.4）| 95.65（0.3）  |\n| Beam Search       | SFT           | 90.89（0.3）   | 84.65（0.3）  | 12.47（0.2）| 95.87（0.4）  |\n| **Beam Search**       | **FACT**          | **91.35**（0.3）   | **87.26**（0.3）  | **8.01**（0.3） | **97.54**（0.3）  |\n| top-k             | Prompt        | 87.35（0.3）   | 86.10（0.2）  | 13.84（0.2）| 95.41（0.3）  |\n| top-k             | SymbCoT       | 85.36（0.3）   | 85.24（0.4）  | 14.35（0.3）| 95.13（0.4）  |\n| top-k             | SFT           | 90.15（0.4）   | 84.05（0.3）  | 13.52（0.3）| 94.23（0.3）  |\n| **top-k**            | **FACT**          | **90.44**（0.2）   | **86.54**（0.4）  | **8.33**（0.3） | **96.25**（0.2）  |\n\n**2.Fact-based Text: Definition and Analysis:** Thank you for your valuable comment. In our work, we define a fact-based text as a passage that expresses objective, verifiable information about entities, their attributes, relationships, or processes, which can be concretely mapped to structured code constructs (e.g., class instances, properties, or procedural functions). This aligns with Iso et al. (2020), who describe fact-based texts as those that “describe a series of facts and processes in an organized manner”.\n\n> Reference: Iso, H., Qiao, C., & Li, H. (2020). Fact-based Text Editing. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 171–182.\n\nTo address the potential opacity and bias introduced by the fact-based text filter, we summarized the characteristics of the retained fact-based texts and the discarded non-factual texts, as shown in the table below. Specifically: \n\n1. **Quantitative analysis:** The retained fact-based texts have much higher average entity and relation counts, as automatically extracted by DeepSeek-R1 and then manually checked (93% accuracy on 100 random samples to ensure reliability), reflecting richer and more structured information compared to the discarded non-factual texts.\n\n2. **Topic patterns:** We used DeepSeek-R1 for automatic topic analysis and manually verified 100 random cases, achieving 91% accuracy to ensure reliability. We then summarized the topic characteristics based on the top 30 frequent topics. These results show that the filter effectively separates objective, factual content from subjective or unverifiable information. \n\n3. **Representative examples:** These further highlight this distinction: fact-based samples focus on concrete, verifiable information, while discarded samples are characterized by subjectivity or ambiguity.\n\n| Text Type                   | Avg. Entities          | Avg. Relations        | Topic Analysis                                                                                                 | Example Snippet                                                                                                        |\n|-----------------------------|---------------|----------------|---------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------|\n| Retained Fact-based Texts    | 11.3                | 7.4                  | Objective descriptions: company information, product attributes, competitions, organizational situations, etc. | ...Ger O'Brien represented the Eircom League International squad in Aberdeen for the quadrangular tournament with England, Scotland and Wales in 2004... |\n| Discarded Non-factual Texts  | 4.3                  | 2.2                   | Subjective or imaginative content: reviews, personal opinions, hypothetical scenarios, unverifiable statements, or frequent hedging expressions. | ...Commenters on social media pointed out that the number could also be seen as a reference to a joke in the film Anchorman... |\n\nWhile our definition of fact-based text is operational rather than theoretically rigorous, the above analysis shows that this filtering reliably selects comprehensive, objective content suitable for structured code mapping. Prior work (e.g., CoCoGen, CodeRL) demonstrates that code-based training reduces inconsistencies in LLM outputs on code generation tasks. FACT further leverages the structural alignment between factual text and code through alternating code-text training, thereby transferring this logical consistency to LLM outputs on NLP tasks.\n\n**3.Evaluation of Generated Code:** Thank you for raising this important issue. We chose the combination of ROUGE-1 and ROUGE-L as our semantic fidelity metrics due to their complementary strengths: ROUGE-1 captures factual tokens and entities, while ROUGE-L reflects global word order and structural similarity. \n\nAdditionally, we evaluated the quality of the generated code across training iterations using BERTScore, which measures deeper semantic fidelity through contextual embeddings. As shown in the table below, $S_i$ (ROUGE-based metric as defined in Eq. 7 of the main paper) and BERTScore remain closely aligned and both improve across training rounds. \n\n|  Iteration  | $S_i$  Score (%) | BERTScore (%) |\n|:-----------:|:-----------:|:-------------:|\n|  Round 1    |   87.96     |     88.45     |\n|  Round 2    |   91.61     |     91.82     |\n|  Round 3    |   92.71     |     92.50     |\n\nMoreover, assessing true logical equivalence between reconstructed text based on generated code and original texts is challenging. To address this, we performed a deeper evaluation using DeepSeek-R1, splitting data after the third training round into High-Quality (Si > 0.8) and Low-Quality (Si ≤ 0.8) subsets. We measured entity and relation alignment rates, both extracted by DeepSeek-R1 and manually counted. To ensure reliability, manual verification of 100 random samples showed 92% overall accuracy for DeepSeek-R1. As shown in the table below, the High-Quality set exhibits much stronger alignment, indicating that iterative pseudo-labeling enables the model to capture logical relationships beyond surface similarity. These findings support that ROUGE-based evaluation, though simple, aligns well with more rigorous metrics and serves as a practical proxy for semantic fidelity.\n\n|   Dataset    | Entity Alignment Rate (%) | Relationship Alignment Rate (%) |\n|:------------:|:----------------------------------:|:-----------------------------:|\n| **High-Quality** |             **92.35**                 |            **86.26**              |\n| Low-Quality  |             54.33                 |            38.16              |\n\n**4.Discussion on Task-Agnostic Generalization:** Thank you for your insightful comments. We chose summarization and question answering because they are common tasks where LLMs frequently hallucinate and have well-established benchmarks [1]. Moreover, these tasks are not solely focused on factual extraction and reformulation. For example, SAMSum is a dialogue-based summarization dataset that requires models to understand conversational context, while SQuAD v2 includes a broad range of answer types (e.g., dates, numbers, entities) and demands diverse reasoning abilities such as syntactic and lexical variation, multi-sentence reasoning, and ambiguity resolution.\n\nBy \"task-agnostic,\" we mean our framework does not require dataset-specific training or adaptation and can be directly applied to different tasks. We acknowledge that broader evaluation on more text generation tasks is needed to fully demonstrate task-agnosticism. We will clarify this and use more precise language in the revised manuscript.\n\n> [1] Huang L, Yu W, Ma W, et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 2025, 43(2): 1-55."}]

Author rebuttal:

Meta-review: The paper proposes FACT, a fact-driven alternating code-text training framework to reduce hallucinations in LLMs by mapping factual text into structured code and alternating between text-to-code and code-to-text training. Strengths are the novelty of using code for consistency, task-agnostic design, and solid empirical gains across QA and summarization. Weaknesses include limited theoretical grounding, reliance on pseudo-labeling, narrow task scope, and initially insufficient statistical rigor. The decision to accept is based on the importance of the problem, originality, and consistent improvements across models and datasets, though not strong enough for spotlight/oral. In rebuttal, concerns about robustness, metrics, novelty, and dataset choice were raised; the authors addressed these with multi-seed experiments, statistical tests, deeper analysis of fact filtering, clarifications, and discussion of related work. Most reviewers were satisfied and raised their scores, with one remaining skeptical but moving to weak accept.
Decision: Accept (poster)