Cutting the Root of Hallucination: Structural Trimming for Vulnerability Mitigation in Code LLMs
Authors: Yage Zhang
Venue: COLM 2025 (2025)
Keywords: LLM hallucinations
code generation
program repair
vulnerability mitigation
structural pruning
abstract syntax tree
hallucination detection
CSHS
model-agnostic risk estimation
generative code safety
TL;DR: LLMs hallucinate code often with security risks. We trace these structurally, prune them surgically, and predict repair effectiveness. Our method patches code and mitigates risk using a transferable score (CSHS).

Abstract: We introduce a structural perspective on hallucinations in code-generating language models, framing them as causality anchors in syntax graphs that trigger cascading semantic errors and latent security flaws. This work is the first to systematically connect code hallucinations with vulnerability risks, offering a unified conceptual and practical framework to address them. At the heart of our approach is the notion of hallucination anchors, localized subtrees in the abstract syntax tree (AST) that serve as root causes of defective logic. We propose Structural Trimming (ST), a targeted mitigation method that removes these anchors while preserving functional semantics. To anticipate the effect of trimming, we introduce the Compositional Structural Hallucination Score (CSHS), which quantifies the likelihood that pruning will improve robustness. By grounding error reduction in the syntax graph itself, our method reframes hallucination mitigation as a structured intervention process interpretable, generalizable, and actionable.

Published as a conference paper at COLM 2025
Cutting the Root of Hallucination: Structural Trimming for Vulnerability Mitigation in Code LLMs
Yage Zhang
CISPA Helmholtz Center for Information Security yage. wwaanneisspa....
Abstract
We introduce a structural perspective on hallucinations in code-generating language models, framing them as causality anchors in syntax graphs that trigger cascading semantic errors and latent security flaws. This work is the first to systematically connect code hallucinations with vulnerability risks, offering unified conceptual and practical framework to address them. At the heart of our approach the notion of hallucination anchors, localized subtrees in the abstract syntax tree (AST) that serve root causes of de- fective logic. We propose Structural Trimming (ST), a targeted mitigation method that removes these anchors while preserving functional seman- tics. To anticipate the effect of trimming, we introduce the Compositional Structural Hallucination Score (CSHS), which quantifies the likelihood that pruning will improve robustness. By grounding error reduction in the syntax graph itself, our method reframes hallucination mitigation a structured intervention process interpretable, generalizable, and actionable.
1 Introduction
Large Language Models (LLMs) have significantly advanced automated code genera- tion Jiang et al. (2024a); Liu et al. (2023a); Thakur et al. (2024), achieving state-of-the-art results across tasks such as autocompletion, debugging, and synthesis Chen et al. (2021); Nijkamp et al. (2023); Li et al. (2023); Huynh Lin (2025). However, with increasing capa- bilities comes a growing concern: LLMs frequently produce hallucinated code plausible yet incorrect outputs that may violate functional requirements or embed latent security risks Ji et al. (2023b); Tambon et al. (2025). While prior work has predominantly addressed syntactic accuracy or model calibration, the security implications of hallucinations remain underexplored.
In this paper, we posit that hallucinations in code are not merely transient generation errors, they constitute structured precursors to real-world vulnerabilities. Unlike superficial mistakes (e.g,, typos or API misuse), hallucinated variables, conditions, or control flows may propagate through the AST, forming entangled symbolic dependencies that are difficult to detect or rectify post hoc. These artifacts often persist across abstraction layers, misleading developers and enabling logic flaws.
We pursue this perspective through three research questions:
RQ1. How does hallucination severity relate to vulnerability risk? Can this relationship be modeled and predicted?
RO2. Do hallucinations follow detectable structural patterns?
RQ3. Can we design effective structure-aware repair methods to reduce risk while preserving semantics?
To answer these, we construct large-scale analysis pipeline for hallucination-driien risk as- sessment. Using MBPP Austin et al. (2021a), HumanEval Chen et al. (2021), and structurally annotated examples from Collu-Bench, we collect and label 7,233 hallucinated samples across four L.LMs (GPT-4o, DeepSee-Coder-1133B Codel.lmm--7B. Gemini-2.0-flaan over

---

Published as a conference paper at COLM 2025
diverse tasks. We introduce taxonomy of hallucination behaviors e.g., persistent, early-cut, recurrent, and oscillating, inspired by prior propagation typologies et al. (2023b). Each sample is assigned deviation score (D) and vulnerability estimate (V), based execution feedback and error severity.
To mitigate the security impact, propose Structural Trimming (ST), post-hoc repair method operating on the Abstract Syntax Tree. ST identifies and prunes hallucination anchors and their dependent subtrees. Unlike prompt engineering, RLHF Ouyang et al. (2022), or token-level filtering Liu et al. (2023b), our method leverages structural signals to preserve program intent while eliminating risky artifacts. We further introduce the Compositional Structural Hallucination Score (CSHS) to preemptively estimate the potential benefit of trimming given sample.
This work offers:
A structurally annotated dataset of LLM hallucinations with vulnerability metadata, enabling accurity-orienttr generation analysis;
Empirical evidence linking hallucination severity to vulnerability risk, supported by statistical and execution-base analyses;
A structure-aware repair method (ST) outperforming Prompt Rewriting and Token Pruning in vulnerability reduction and semantic retention;
A predictive scoring function (CSHS) for hallucination risk, grounded in AST-level signals.
In summary, our findings reframe hallucinations not merely model inaccuracies, but as structured, measurable, and actionable sources software risk in LLM-based development workflows.
2 Extended Related Work
Hallucinations in LLM Code Generation. Large Language Models (LLMs) have achieved impressive success in code generation Jiang et al. (2024b), but hallucination remains persistent and underexplored challenge. Hallucinations in LLM-generated code typically manifest factual errors (e.g,, nonexistent APIs Akhtarshenas al. (2025); Chen et al. (2025)) or functional errors (e.g., logic bugs causing runtime failures such as NameError) Ji et al. (2023b). Prior work has primarily focused on evaluating functional correctness using benchmarks such as HumanEval Chen et al. (2021), MBPP Austin et al. (2021b). However, these approaches rarely address how hallucinations structurally propagate or induce long- range effects on code safety and security.
Adversarial Code Generation and Prompt Attacks. Beyond unintentional errors, LLMs can be deliberately manipulated to produce faulty or unsafe code through prompt injec- tion Greshake et al. (2023); Liu et al. (2023c) or adversarial jailbreak Shayegani et al. (2023); Liu et al. (2024). These methods exploit LLMs' sensitivity to instruction phrasing and contextual prompts. However, they focus on adversarial behavior rather than organic hallucinations.
Software Security and Vulnerability Evaluation in LLM Outputs. Recent works em- phasize evaluating vulnerabilities in generated code using CVSS scoring While these frameworks assess security outcomes, few explore how hallucination contributes to vulner- ability emergence. Our approach provides a causal modeling framework linking structural deviation from ground truth to vulnerability risk.
Evaluation Benchmarks and Hallucination-Aware Datasets. While benchmarks like HumanEval Chen al. (2021), MBPP Austin al. (2021b) provide functional evaluations, they lack hallucination labels. Collu-Bench is the only known dataset with hallucination- specific annotations, but it does not capture structural propagation.
CCVS v3.1
first.
2

---

Published as a conference paper at COLM 2025
Structural Defenses and Hallucination Mitigation. Prior hallucination mitigation ef- forts include RLHF Ouyang et al. (2022); Wang et al. (2025); Krishna et al. (2025), output reranking Liu et al. (2023b), post-generation patching. However, these methods typically operate at the token level or rely on downstream filtering. To our knowledge, no existing approach systematically models hallucination structure and causality.
3 Problem Formulation and Risk Modeling
We model the emergence of vulnerabilities in LLM-generated code as consequence of nonadversarial hallucinations,strutuuaaalll plausible, yet semantically invalid constructs arising during generation.
Threat Model and Deployment Context. We consider a non-adversarial setting where LLM-generated code is integrated into development workflows (e.g,, auto-completion, boilerplate generation) without formal verification. No prompt manipulation or malicious intent is assumed; our focus is dallucination-inuucee culnerabilities, unintended artifacts that may compromise functionality or security when deployed unvetted.
Risk Variables and Reference Ground Truth. Given prompt x, model output ý, and reference implementation y (e.g., from HumanEval or MBPP), we define hallucination deviation D(j.y") and vulnerability risk V(g) as structural and behavioral metrics capturing divergence and failure potential. These serve as the foundation for our subsequent modeling and analysis.
3.1 Core Variables and Risk Modeling
Let 9 denote the hallucinated code generated by a language model for prompt x, and y" the closest ground-truth implementation. We define three key variables: hallucination deviation D, vulnerabilityy probability V, and task complexity T.
Hallucination Deviation (D). We quantify deviation as composition of structural and behavioral divergences:
D = A EditDist(A(())A(y))) A2 (Bismach(B(()))BBBB
(1)
where A(-) is the Abstract Syntax Tree (AST) Alon al. (2019), B(-) the behavioral execution signature (e.g., test traces), and ÅI, A2 E [0, 1] are balancing weights (default: 0.5). Note that D serves as a deviation risk index rather than true distance, capturing both structural and semantic divergence Feng et al. (2020).
Vulnerability Risk (V). We define V E [0,1] probability:
continuous proxy for vulnerability
= (a1 Eaype + Fexec + a3 log Ptoken),
(2)
where Fayp is the error severity score (e-g., NameError 0.8), Fexec indicates assertion test failure, and Pioken is the mean bog-probability of hallucinated tokens Liu et al. (2023b), aligned with prior work using runtime signals and likelihood estimates for risk modeling.
Task Complexity (T). To normalize risk across different prompt complexities:
Teombine = quesstio + Teetar
(3)
where Tquestion is the prompt length in tokens, and Tmeta reflects static complexity (e.g., branching, depth).
3.2 Dataset and Experimental Setup
We construct a dataset of 7,233 hallucinated code samples generated from four leading code LLMs-GPT-L (OpenAI), DeepSeek-Coder-11.3B CodeLlama-7B, and Gemini-2.0-flash using prompts drawn from three sources: MBPP Austin et al. (2021b), HumanEval Chen et al. (2021), and curated set of 300 security-focused generation tasks.
3

---

Published as a conference paper at COLM 2025
Each generated sample is paired with reference implementation and annotated with: (i) the hallucination anchor and type (e.g, undefined identifiers, semantic drift); (ii) a vulnerability score (V) quantifying security risk and repairability; (iii) semantic deviation metrics (D, BLEU, AST edit distance); and (iv) structural signals such as hallucination chain length (HCL), pruning ratio (PR), and entropy.
These annotations enable structural modeling of hallucinations and their role in security degradation. Full details, including prompt design, anchor detection methodology, V scoring scheme, and sandbox execution pipeline, are provided in Appendix D.
4 Structural Risk Signals and CSHS Scoring
We now introduce our structural signal extraction pipeline, annotation mechanisms, and the unified scoring model (CSHS).
4.1 Hallucination Annotation Pipeline
Token-Level Anchor Detection. We locate the first hallucinated token 0r- not semantically aligned with any y/:
min 1: > yj Vj E [1,ly"
(4)
AST-Based Hallucination Typing. Each hallucinated sample is encoded into an AST path sequence and classified using a transformer-based model:
hype = JAsTCIas (ASTSeq())).
(5)
4.2 Structure-Aware Signal Extraction
To characterize how hallucinations propagate through the code structure, we extract five structural indicators from the AST of 9. Several of these, such as entropy and chain length, are inspired by classical program complexity and slicing metrics Weiser (1984); McCabe (1976); Halstead (1977), repurppsed to capture generation-induuce anomalies:
HCL (Hallucination Chain Length): Number of AST hops from the hallucination anchor to the deepest dependent node.
APS (Anchor Persistence Score): Number of references to hallucinated symbols across AST subtrees.
PR (Pruning Rate): Proportion of nodes removed during minimal repair:
[TrimmedNodes!
PR
(6)
TotalNodes)
SCP (Shortest Correction Path): Shortest AST path from the hallucination anchor
to valid, ground-truth subtree.
Entropy (H): Structural entropy over token types in the AST:
H = Σ p: Pt
1ET
These signals provide interpretable structure-level cues for hallucination detection and repair. For instance, consider a hallucinated snippet where a nonexistent variable foo is used in nested if-else block. APS reflects the spread of foo references across multiple AST subtrees; HCL captures the depth of its influence from anchor to dependent (e.g., nested return); SCP denotes how easily the subtree can reconnect to valid logic (e.g., by replacing foo with bar); Entropy increases if the subtree contains diverse token types such as If, Name, Call; and PR reflects how much of that subtree is pruned in the repair process.
Unlike traditional static analysis tools such as CodeQL GitHub Security Lab (2021), which operate semantic correctness and symbolic flow, our approach quantifies the structural
4

---

Published as a conference paper at COLM 2025
footprint of hallucinations, when they remain syntactically valid but semantically misleading. This enables hallucination--ware trimming without requiring ground-trun semantics runtime traces.
Contrast with Traditional Analysis. Unlike static program analysis, which identifies se- mantic errors through complete control/date data flow, our method targets structurally plausible yet semantically spurious constructs generated by LLMs. Hallucinated anchors are syn- tactically valid but causally misleading. Structural signals such APS and HCL capture their generative propagation, offering a lightweight, Linguage-modelawee traditional tools like CodeQL GitHub Security Lab (2021), which overlook such contextual hallucinations.
alternative to
4.3 Compositional Risk Scoring with CSHS
We integrate the five structural indicators into unified scoring framework, the Composi- tional Structural Hallucination Score (CSHS):
CSHS 11 HCL 202 APS + w3 + 104 PR W5 SCP
(8)
Ryick
Reecover
All features are min-max normalized to [0,1]. The score is constructed combination of structure-derived features, following established practice in risk modeling for software faults Zimmermann al. (2007). Weights are tuned via grid search on validation accuracy of bost-trimmed vulnerability risk. We determine optimal weights w, via grid search (see Appendix I):
weighted
201 0.116,
t02 0.266,
W3 0.017, W4 0.440,
Wy 0.161
This score serves both as hallucination risk estimator and as a pruning controller in our structural repair pipeline. Intuitively, pruning rate (w4) receives the highest weight, reflecting that hallucinations are most dangerous when they cannot be easily removed. Anchor persistence (wz) and chain length (wy) quantify how deeply the error spreads, while entropy (wy) plays minor role, capturing only coarse structural irregularity. The correction path (ws) penalizes entangled errors that require complex recovery.
Why Hallucinations are Dangerous?
To address RQ1, we investigate whether hallucinations in L.M--eenrated code systemati- cally correlate with known vulnerability types. While often dismissed as superficial errors, we posit that hallucinations encode latent structural signals that predispose code to security failures.
5.1 Hallucination-Vulhenbiiltt Alignment via Structural Modeling
We hypothesize that hallucinations align with specific vulnerability types in non-random, learnable ways. A two-stage classification, first through lexical heuristics, then with struc- tural features derived from AST, confirms this: as shown in Figure la, structural modeling significantly improves alignment with annotated vulnerability labels.
The heatmap shows consistent mappings between hallucination types and vulnerability pat- terns, suggesting that even syntactically valid hallucinations often carry predictable security risks. Cosine similarity analysis confirms strong alignment for structurally grounded types, while semantically complex cases (e.g., Logic Deviation) benefit from AST-based feature integration. Full details appear in Appendix E.

---

Published as a conference paper at COLM 2025
Vulnerabliity Association Patterns
Watatoy Conertionn
0.8
3.3
(Aad lies mark criccal cveateds)
0.2
m0314m0 mm
stoy
Hallucination Type
Denatoo (0 Dru
(a) Hallcinaaton-vuuherablly mapping.
(b) Risk increases with hallucina- tion severity.
Deviation
(c) Nonlinear D-V distribution.
Figure 1: Structural and statistical alignment between hallucination patterns and vulnera- bility risk. Left: Structural mapping via AST modeling. Buddle/Right Risk trends across hallucination severity with non-linear failure zones.
5.2 Correlation Between Hallucination Severity and Vulnerability Risk
We examine the relationship between hallucination severity, quantified as structural and semantic deviation (D), and vulnerability risk (V), hypothesizing positive correlation:
D V.
Deviation is computed via AST and execution divergence from reference outputs; vulnera- bility is estimated through error types and severity annotations.
Regression and Feature Contribution. While linear regression confirms a statistically significant association between structural deviation (D) and vulnerability, its explanatory power limited. In contrast, non-linear models, particularly random forests, demonstrate strong predictive performance, with error type as the dominant feature and D consistently among the top contributors.
These findings support three conclusions: (1) deviation is reliable proxy for risk; (2) even small deviations can yield high vulnerability, exposing non-obvious failure modes; and (3) structure-aware modeling enhances detection, outperforming shallow metrics via AST and execution-level signals.
5.3 Does Task Complexity Influence Hallucination-Indueee Vulnerability?
We investigate whether task complexity modulates the vulnerability risk (V) of halluci- nated code, using three metrics: prompt complexity (Tquestion), test case com complexity (Tmeta), and their combination (Tcombinet). Samples are partitioned via median splits, and linear regression is used to assess associations.
As shown in Figure 2, only test case complexity (Tmeta) exhibits significant inverse rela- tionship with vulnerability. In contrast, prompt length and combined complexity show no consistent trends. These results suggest that structurally rich test specifications contribute more to risk mitigation than input length alone.
These results challenge the intuition that task difficulty directly increases hallucination risk. Instead, structured test specifications may function as regularizing signals: anchoring generation to semantically valid behaviors and constraining harmful divergence.
How to Defend Hallucinations Structurally?
To address RQ2 and RQ3, propose and evaluate Structural Trimming, defense that prunes hallucination chains from ASTs to reduce vulnerability. We compare it against prompt rewriting, semantic filtering, and token-level pruning, showing it achieves superior

---

Published as a conference paper at COLM 2025
High: 557 vs Low unm 610 Slope -0027 to = 000)
High: ym0.5 vs Low: umo 608
High: um0.5587 587 vs Low: umm 608 Slope 0 3.000 tp 0.301:
Slope = 000 tp 2.215
Welch's I-test: Ip 000)
Weich's 0.000)
Welch's I-lest: (p 0.00)
1.0
1.0
1.0
Valnerabiity Irobabilty
Vulnerabaity Probabily
Valnerabliig Irebabilty
0.8
0.8
0.8
0.5
0.6
0.6
0.4
0.4
0.4
0.2
02
0.2
0.0
Complasity
Complexity
Complexity Group
Complexity Group
Complexity Group
Figure 2: Vulnerability probability V across task complexity dimensions. Only Tmeta exhibits a significant inverse relationship with risk.
(a) Clean
(b) Persistent
(c) Early Cut
(d) Mid Cut
(e) Late Hallucination
(f) Recurrent
Oscillating
Figure 3: Structural patterns of hallucination types and suggested pruning locations. Red nodes denote hallucination anchors, and dashed lines indicate propagation. Scissors mark ideal cut points.
risk reduction while preserving code quality. Furthermore, we introduce CSHS, a structural score that predicts whether hallucination is trim-worthy.
6.1 Structural Hallucination Chains Exist
Beyond isolated token-level anomalies, many hallucinations in code generation manifest as structural dependency chains in the Abstract Syntax Tree (AST). A hallucinated anchor (e-g. non-existent configuration object) may propagate through conditionals, assignments, and return paths, forming entangled subtrees that influence program behavior.
Figure illustrates seven representative propagation topologies observed in our dataset, including persistent, recurrent, mid-cut, and oscillating patterns. These structures highlight how hallucinated tokens percolate through multiple layers of program logic.
To quantify these phenomena, we define Hallucination Chain Length (HCL) as the number of AST nodes transitively dependent on the hallucinated anchor. Trimming these chains allows us to compute risk reduction AV = Voriginal Virimmed
Empirically, we observe positive correlation between HCL and AV, indicating that longer chains carry greater security risk. The full trend, including variance bands and sample support across HCL values, is presented in Appendix F).
6.2 Trimming-Basee Risk Mitigation
We operationalize Structural Trimming detailed in Algorithm 1, which recursively prunes structurally entangled hallucination spans from the AST and applies minimal patching when necessary (Appendix G).
Effectiveness and Predictive Modeling. We assess three trimming strategies Early Cut, Mid Cut, and Persistent Cut, report their risk mitigation effectiveness in Table 1. Results show that earlier interventions generally yield greater vulnerability reduction, particularly for persistent and recurrent hallucinations.
7

---

Published as a conference paper at COLM 2025
Category
Label
Avg. AV
Std. Dev.
Early Cut
0.212
0.058
Trimnnng Stritegy
Mid Cut
0.174
0.046
Persistent Cut
0.143
0.065
persistent
0.236
0.151
recurrent
0.205
0.059
Hollacination Type
mid
0.172
0,048
parly.cut
0.037
oscillating
0.118
0.045
Table 1:
Risk reduction (AV) by trimming strategy and hallucination type. Early interven- tions yield higher gains, especially for persistent and recurrent patterns.
CSHS demonstrates robust generalization across LLMs and code-generation tasks, main- taining high predictive accuracy without retraining,
We propose the Compositional Structural Hallucination Score (CSHS) as a predictive metric for trimming efficacy. As shown in Figure 4, CSHS aligns well with observed risk reduction and supports reliable identification of high-impact cases.
Attribution analysis (Figure 5) indicates that pruning-aware features dominate prediction performance, while low-level structural signals contribute marginally.
CSHS vs. Risk Reduction
Pruning Strategy Mid Cut
Persistent Out Early Out
Hollucination Type
Recument
Persistent
Clean
Curve 0.989)
Classification Threshold: AV > 0.30
Risk Reduction (AV)
True Positive
8.4
3.1
AUC 0.99
Composite Structural Hallucination Score (CSHS)
False Positive Rate
Figure 4: Left: Correlation between CSHS and trimming-induced risk reduction (AV). Right:
ROC curve for classifying effective trimming (AV >> 0.3) using CSHS.
SHAP Value Distribution for CSHS (Impact on Classification)
Features
CSHS
SHAP value (impact model output)
Figure 5: SHAP summary plot for CSHS-informed classifier. Higher values in PR and SCP contribute most to predicted risk.
6.3 Interpretability via Structural Case Studies
To analyze the structural dynamics of hallucination propagation and correction, we visualize representative AST transformations before and after trimming (Figure 6).
The first example reflects a persistent hallucination, where early-stage anchors propagate through multiple dependent branches, forming deeply entangled structure. Trimming the corresponding subtree removes the hallucination chain while preserving core functional logic.
8

---

Published as a conference paper at COLM 2025
The second example illustrates localized hallucination, where the anchor appears in shallow, isolated position. In such cases, correction requires minimal structural intervention.
These patterns highlight how the correctability of hallucinations depends on their structural locality and dependency depth, supporting AST-guided trimming as an effective and interpretable mitigation strategy.
Genvertt oode
Selore Trlmeing
Trinning
thare sartag Tost
(a) Persistent hallucination: trimming deep con- ditional chains.
(b) Late hallucination: pruning shallow neous return.
Figure 6: Structural pruning of hallucination chains. Left: Before trimming. Right: After trimming. Red nodes denote hallucinated anchors; dashed boxes indicate pruned regions.
6.4 Comparative Effectiveness of Structural Trimming
While hallucination mitigation is well-studied in natural language generation, methods targeting vulnerabilityinduuee hallucinations in code remain limited. We address this by comparing our method, Structural Trimming (ST), to three adapted baselines: Prompt Rewriting (PR), Semantic Filtering (SF), and Token-Level Pruning (TP).
These strategies span different abstraction levels from prompt modification token trun- cation while ST performs AST-level pruning to remove structural hallucinations post hoc. Evaluation on 1000 hallucinated samples (Table 2) considers vulnerability reduction, safety, and code fidelity.
Strategy
Avg. Vbefore
Avg.
Vaster
AV
Safety %
Retention %
BLEU / AST
PR SF TP
0.92
0.66
0.26
38%
56%
0.65 0.60
0.91
0.61
0.30
44%
63%
0.77 0.81
0.93
0.48
0.45
54%
38%
0.42 0.35
ST (Ours)
0.92
0.33
0.59
60%
62%
0.80 0.84
Table 2: Evaluation of hallucination repair strategies over 1000 randomly sampled instances. ST achieves the best trade-off between risk mitigation and code fidelity.
Structural Trimming (ST) offers the best trade-off between risk reduction and code fidelity. Unlike aggressive or shallow strategies, ST precisely removes structural hallucinations while preserving usability. Its fallback mechanism ensures robustness with minimal side effects, placing it on the sifety-fidelity Pareto frontier.
Discussion
We frame code hallucinations not as isolated errors but as structurally entangled artifacts, often forming transitive symbolic chains within ASTs that propagate semantic faults. Our analysis shows that vulnerability is less determined by hallucination depth than by its recoverability the feasibility of structurally pruning it without disrupting valid logic.
Structural Trimming (ST) exploits this principle, enabling targeted AST-level repairs. Abla- tion studies confirm that pruning viability (e.g., pruning rate, correction path) is a stronger

---

Published as a conference paper at COLM 2025
predictor of risk reduction than structural complexity alone. The proposed CSHS metric captures this insight, serving as a model-agnostic proxy for hallucination severity.
However, CSHS remains descriptive and handcrafted. Future work may model hallucina- tion propagation as a generative causal process, e.g., via Hallucination Causality Graphs that trace symbolic dependencies from anchor tokens. This would enable both diagnos- tic and preventative interventions, bridging program structure with causal reasoning in LLM outputs. Future directions include causal modeling of hallucination dynamics and integration with training-time timen constraints.
Conclusion
This work introduces structural approach to hallucination mitigation in LLM-generated code. We propose Structural Trimming (ST) to excise hallucination chains from ASTs, and develop CSHS, compositional score predicting trim worthiness based on structural cues.
Evaluations on 7K hallucinated samples across models and tasks show that ST achieves superior vulnerability reduction compared to prompt and token-level baselines, while preserving syntactic fidelity. CSHS generalizes without retraining, highlighting structural risk as a transferable signal.
By treating hallucinations as analyzable structural phenomena, this work offers practical tools and conceptual grounding for robust, interpretable code generation. Code and data will be released to support reproducibility and further research.
References
Azim Akhtarshenas, Afshin Dini, and Navid Ayoobi. Chatgpt or a silent everywhere helper: A survey of large language models. arXiv preprint ar arXio:2503.17403, 2025. URL https://arxiv. org/abs/2503. http:://arri..or//bb//2233 17403.
Uri Alon, Meital Zilberstein, Omer Levy, and Eran Yahav. code2vec: Learning distributed representations of code. Proceedings of the ACM on Programminng Langunges, 3(POPL):1-29, 2019. URL https: //d.:/c///c.cmmmmmm acm. org//oi//0 . 1455/229333.
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrson Jones, Michael Terry, Quoc Le, and Charles Sutton. Program synthesis with large language models, 2021a. URL https:t/////iiiiii org/ 07732. MBPP dataset introduced in Section 6.
Jacob Austin, Augustus Odena, Maxwell Nye, and et al. Program synthesis with large language models. In NeurIPS, 2021b. URL https: http:v::///vvvvv/////e http://xxii..orr///////// 07732.
Jingyi Chen, Songqiang Chen, Jialun Cao, Jiasi Shen, and Shing- ChhnngChhiiee Cheung.
When llms meet api documentation: Can retrieval augmentation aid code generation just as it helps developers? ar Xiv preprint ar 1xXi::2503.15533 2025. URL https: https::/aariivoorr/aa////// 15231.
Mark Chen, Jerry Tworek, Heewoo Jun, and et al. Evaluating large language models trained on code. In Advances in Neural Information Processing Systems, 2021. URL https: //arxiv.org orriii.orr/aae aaaaaas///1// 2107. 03374.
Zhangyin Feng, Daya Guo, Duyu Tang, and et al. Codebert: A pre-trained model for programming and natural languages. arXio preprint arXin:2002.088155 2020. URL https: //aaxxiv.org/bb//e/e/ 08155.
Gitl Security Lab. Codegl: Semantic code analysis engine. https://codeqql.itthm www.git.bbb..ooo. 2021. URL nittps://codeee gi thub. com/.
Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you've ve signed up for: Compromising real-world llm-integrated
10

---

Published as a conference paper at COLM 2025
applications with indirect prompt injection. In Proceedings of the 16th ACM Workshop Artificial Intelligence and Security, pp. 79-90, 2023. URL httpp://ddl acm. org/doi/abs/10. 1145/3605764,
3666969776..6388.... 3623985.
Maurice H. Halstead. Elements of software science (operating and programming systems series). 1977. URL https: / /dl. acm. www.oorg////l// 55555554001333
Sungmin Han, Miju Kim, Jaesik Kang, Kwangsoo Kim, Seungwoon Lee, and Sangkyun Lee. Similarity-based source code vulnerability detection leveraging transformer architecture:
Harnessing cross-attention for hierarchical analysis. JEEE Access, PP:1-1, 01 2024. doi: /ttp:/ACCCSSSS...... 384719615 Similarity-Based Source Code Vulnerability Detection Leveraging Transformer Archi Harnessing Cross oor
URL https://ww.ressarcchtttt wwtpeeeeeeetppbbbiiiioooo
Analysis.
Nam Huynh and Beiyu Lin. A survey on large larglangg models for code generation. arXiv preprint ar Aie:2503.001453 2025, URL https://arxxv.orrgbbbll
01245.
Zhecheng Ji, Nayeon Lee, Zonghai Tan, and et al. Survey of hallucination in natural language generation. ACM Computing Surveys, 2023a. URL https: org/doi/109 1145/3571730.
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination natural language generation. ACM computing surveys, 55(12):1-38, 2023b, URL https://dl. acm. org/doi/ abs/10. 1145/3571730.
Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. A survey on large language models for code generation. arXiv preprint Stt:22006005114 2024a. URL
https://arxt orgg/bs////00 00515.
Yukun Jiang, Xinyue Shen, Rui Wen, Zeyang Sha, Junjie Chu, Yugeng Liu, Michael Backes, and Yang Zhang. Games and beyond: analyzing the bullet chats of esports livestreaming. In Proceedings of the International AAAI Conference on Web and Social Media, volume 18, pp. 761-773, 2024b.
Arjun Krishna, Erick Galinkin, Leon Derczynski, and Jeffrey Martin. Importing phantoms: Measuring 1lm package hallucination vulnerabilities. Xrv preprint Stff::2501.10111 2025. http//iiii.orrr////////1../
URL httpttttt////ii..tttt/ org abs////ss//.11... 19012.
Raymond Li, William Allal, Jacob Austin, and et al. Starcoder: May the source be with you! 06161.
aiviv preprint ar arXiv:2305006161, 2023. URL http://///axwv..o/aabsbss/
Fan Liu, Zhao Xu, and Hao Liu. Adversarial tuning: Defending against jailbreak attacks for llms. Xiv preprint arXiv:2406.0660 0666222, 2024. URL httpts org/
http://.rrrraa////////////// abs/ 06622.
Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by chatgpt really correct? rigorous evaluation of large language mod- els for code generation. Advances in Neural Information Processing Systems, 36:21558- 21572, 2023a. URL https: h/ttt:e/////roceiinggssss neurips..e........e... neurips.cc/pppprrrr httpppaprrrrll///papepree https/pappr//22//haat html.
Ximing Liu, Aitor Lewkowyez, Tao Yu, and et al. Lost in the middle: How language models use long contexts. Xiv preprint arXio:2307.031 03172, 2023b. URL https://arxiv. wrg.abrg./.............. 2307. 2317.03172.
Yi Liu, Gelei Deng, Yuekang Li, Kailong Wang, Zihao Wang, Xiaofeng Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, al. Prompt injection attack against llm-integrated applications. ar Xiv preprint ar Xit::3066.05499 2023c. URL https: http://wwwvv.ggg..././. abg/ 05499.
Thomas J. McCabe. A complexity measure. JEEE Transactions on Software Engineering, SE-2 (4):308-322) 1976. URL htttps//iieexxporrrs ieee. wwg/ddccmmet//77228..
11

---

Published as a conference paper at COLM 2025
Erik Nijkamp, Shixiang Yao, Aakanksha Chowdhery, and et al. Codegen2: Lessons for training lims on programming and natural languages. arXiv preprint arXit:2305.02309 2023. URL https://arxix: wrtp:oss///bb/55.../ https://arri..orrgab///25....0 02309.
Long Ouyang, Jeff Wu, Xu Jiang, and al. Training language models to follow instructions with human feedback. arXiv preprint arXio:2203.02155, 2022. URL https://arxiv.orr abs/2203.02D
Erfan Shayegani, Yue Dong, and Nael Abu-Ghazaleh. Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models. arXiv preprint arXiv:2307.1453) 2023. URL //arxi/ http:///xaxxvvv.vabbbsbss/ wtgp/b/////////////////b//////// 14539.
Florian Tambon, Arghavan Moradi-Dakhel, Amin Nikanjam, Foutse Khomh, Michel C Desmarais, and Giuliano Antoniol. Bugs in large language models generated code: An empirical study. Empirical Software Engineering, 30(3):1-48, 2025. URL https: //link. springer.com/ wtt../////att///eeee Tatie//...0000///1 01144-025-161444
Shailja Thakur, Baleegh Ahmad, Hammond Pearce, Benjamin Tan, Brendan Dolan-Gavitt, Ramesh Karri, and Siddharth Garg. Verigen: A large language model for verilog code generation. ACM Transactions on Design Automation of Electronic Systems, 29(3):1-31, 2024.
URL https://dl. org/ htt//////fu//l///// 1145/3643681.
Huandong Wang, Wenjie Fu, Yingzhou Tang, Zhilong Chen, Yuxi Huang, Jinghua Piao, Chen Gao, Fengli Xu, Tao Jiang, and Yong Li. A survey responsible 1lms: Inherent risk, malicious use, and mitigation strategy. arXiv preprint arXie:2501.09431, 2025. URL https://arxiv. wttpaaawwii.ioorrb//raaaaaaaaaaa org/abs/250)) 09431.
Mark Weiser. Program slicing. IEEE Transactions on Software Engineering, 33--3,3333330,,, 1984. https::s:////eeeppll ieee. arg/doccumett/511224.
Thomas Zimmermann, Nachiappan Nagappan, Harald Gall, and et al. Predicting defects for eclipse. In Proceedings of the Third International Workshop Predictor Models in Software Engineering, pp. 9-15, 2007. URL https: //ll acm. org..oorg//oe www.oogg//ooo///ooo /10.0011111000 109 PROMIIS........O 2007.
A Information-Theoreti Formalization of Hallucination Propagation
In this section, present formal information-thherrei, perspective hallucination propagation in generative code models. Our aim is to characterize how local uncertainty or hallucination at specific point in the output can influence downstream code segments through dependency structures.
A.1 Entropy of Local Generation
Let Y = (2..32....,. yr) be the sequence of generated tokens, and let x denote the input prompt. The conditional entropy of token yt given the prompt and past context is:
H(y: xxy=() E(yy, x,y=t) log P(y x,yx))
yt
A high entropy value indicates generation uncertainty or ambiguity position which may signal the presence of a local hallucination anchor.
We define hallucination anchor yt, token (or token span) where entropy exceeds semantic confidence threshold T:
yt, Y iff H(y,, x,y(,,
12

---

Published as a conference paper at COLM 2025
A.2 Dependency Graph and Information Flow
Let the generated program be parsed into an Abstract Syntax Tree (AST), where each node n, corresponds to semantic unit (e.g., variable, call, branch). Define directed dependency graph G = (V, E) over AST nodes, where (n; + ni)) E E if nj is semantically dependent on fi (e.g,, use-def, control flow, data flow).
Assume node nı is rooted in a hallucination anchor token. The set of transitive dependents is:
P (na) {n V 3 path Ha + n)}
These nodes inherit semantic uncertainty from nay forming the hallucination propagation frontier.
We define the total hallucination propagation entropy as:
Hprop Σ
H(y)()) x, x,yct)n)))
(RE)(
where (11;) maps node n; to its generating token position.
A.3 Mutual Information Loss from Pruning
Let y be the generated output after pruning all nodes in P(na). To quantify the information loss due to pruning, we define:
Iprune = H(Y))))(())))-)))) - H(Y Y)
=
This reflects how much mutual information between the full output and the pruned version lost, Ideally, for safe and semantically clean pruning operation, we want:
!prune <
Hprop(f1a)
That is, the amount of information lost due to pruning is less than the noise introduced by the hallucination chain.
A.4 Structural Risk Amplification via Entropy Flow
We further hypothesize that hallucination propagation exhibits an entropy amplification effect over code structures. That is, structural nodes that depend on high-entropy anchors accumulate compounded risk. We model this as:
((n))
Σ A:CAncesorss(H))
dlaum)) H(yin)) x, x,Ve((n))
where d(n;, the graph distance from ancestor n; to nj, and E (0,1) a decay factor. This models structural vulnerability risk V(n;) as a decayed aggregation of upstream entropy.
A.5 Implications for Defense
Under this formulation, structural trimming can be interpreted as an entropy-suppression strategy. By pruning subtrees P(na) that are aamplpp-a and semantically non- grounded, we reduce both:
-The total propagated uncertainty Hprop: - The cumulative downstream vulnerability V(n;).
This formalism motivates defensible objective for hallucination mitigation:
min H Hprop(P)
s.t. AST validity and task retention preserved
13

---

Published as a conference paper at COLM 2025
B
CVSS Mapping and Vulnerability Scoring Calculation
B.1
Introduction to CVSS
The Common Vulnerability Scoring System (CVSS), developed under the leadership of the National Institute of Standards and Technology (NIST), is the most commonly used quantitative metric for vulnerability severity in the industry today. The CVSS Base Score quantifies the intrinsic risk of vulnerability on a scale of [0, 10), and its scoring components include:
Attack Vector (AV): The possibility of local or remote (Network) attacks.
Attack Complexity (AC): The difficulty of exploiting the vulnerability.
Privileges Required (PR): Whether high privileges are required.
User Interaction (UT): Whether it relies on user interaction.
Confidentiality (C) Integrity (I) Availability (A) Impact: The potential degree of damage system data and services.
In addition, CVSS also defines the Environmental Score and Temporal Score, which are used for scenario adjustment and vulnerability lifecycle modeling, respectively.
B.2 Collu-Bench Error Type and CVSS Mapping
We mapped the error types automatically extracted in Collu-Bench with the CVSS Base Score criteria to construct a normalized vulnerability risk score V.
The CVSS Base Score references its calculation specification Based on this, we simplified the parameter configuration in the context of code generation (assuming local execution, no user interaction) and uniformly normalized the Base Score to V Base Score/10.
B.3 Frequency Adjustment and Environmental Modeling
To reflect the impact of error distribution in real data on the overall system security, we introduced an environmental scoring mechanism, using the frequency F of error types as an adjustment factor modify the base risk score:
Vddjusted
Vbase (1 WP F))
where Wp the frequency adjustment weight (set to 0.2 in this paper), and F the frequency of the error type in the dataset. This strategy simulates the potential cumulative risk caused by large number of repetitive errors in a real deployment environment.
B.4 Theoretical and Practical Value
Assigning quantifiable security risk scores to generated code errors using the CVSS frame- work has the following three advantages:
Standardization: Consistent with vulnerability databases such as NVD and CWE, facilitating integration with existing toolchains.
Comparability: Errors from different model outputs can be compared on unified scale of security impact.
Theoretical Support:
The CVSS architecture has recognized modeling capabilities for attack cost and impact, which can be used for vulnerability prioritization and strategy optimization.
2CVSS v3.1 Specffcationhhtts://wwww http:///stss.tttt...oooooo/////s
14

---

Published as a conference paper at COLM 2025
C Typology of Structural Hallucinations
To support our taxonomy of structural hallucinations, we summarize the seven types identified in our dataset, along with their propagation patterns and pruning strategies. Table 3 provides an overview, and concrete examples are included below to illustrate each type.
Label
Name
Structural
Example
Pruning Recommendation
persistent
Persistent Halluci-- nation
Hallucinated anchor is re- peatedly referenced until the end of the code. Difli- cult to cut naturally.
config config["enabled"] "fast.""att.
Replace or comment out the anchor at first occur- rence to prevent propaga- tion.
early.cut
Early-Cut Type
Anchor is only referenced once or twice and then dropped.
Hallucinated variable used before early return.
Insert pruning before return or add guard checks.
cid_cut
Mid-Cut Type
Anchor spreads 2-3 layers deep, stopped by logic structure.
Propagation ends in if block.
Intercept hallucination at intermediate depth.
recurrent
Recurrent Halluci- nation
Anchor is initially pruned but reappears through alias- ing.
config pruned, but node configl"type" reused.
Apply global sanitization alias tracking.
Iate.hallucinat Last
Hallucina-
Anchor appears only in fi- stages code (e.g, turn or output).
return config["Final value"]
Add checkers at output stage.
tion
oscillating
Oscillating
Pat-
Anchor appears inconsis- tently across branches loops.
config but else.
Apply uniform trimming across all control flows.
tern
clean
Clean Case
No hallueinated anchors
Use of input.data fally valid.
No pruning needed; serves as
structural propagation.
Table 3: Typology of structural hallucinations and their corresponding pruning strategies. Each type reflects a unique propagation pattern and trimming implication.
D Dataset and Annotation Details
D.1 Sample Construction and Prompt Sources
We collect 7,233 hallucinated code samples generated from the following models:
GPT-4o (May 2024)
DeepSeek-Coder:1338
Codelllmm-77-Insttuut
Geminn-2.0-flash
We use a fixed generation configuration: temperature=0.77 7, top_p=0. 95, max_tokens=512, and stop=[' '##'', "In)nn' to ensure stylistic consistency.
Prompts sampled from:
MBPP and HumanEval: Standard evaluation benchmarks.
Security-critical prompts (300 total): Curated by us to stress low-level control, symbolic reasoning, and input validation, e-g., "Validate password strength with entropy scoring", "Simulate memory-safe linked list in C-style pointer emulation". These tasks are designed to induce subtle hallucinations in logic or API usage.
D.2 Anchor Detection via AST and Runtime Traces
We define a hallucination anchor as an AST node (identifier, expression, or control block) that:
15

---

Published as a conference paper at COLM 2025
Example Persistent Hallucination
config ("enabled": True)
hallucinated object
if config["ennaled"]]
if config[ fasttval""] run_ run_fast_mode ()
# repeated dependency on config
else:
run_safe_mode ()
Example 2: Early-Cut Hallucination
if user_input is None: logger warn warn return early return cuts hallucinated
speed user_input["ppecd
hallucination only appears once
Example 3: Mid-Cut Hallucination
params get de de_ppaaaaaataaaassseeeeee ()
if params [ debug "]: hallucinated use if log_levell in params: # cut point set_log_
"])
Example 4: Recurrent Hallucination
# First occurrence of hallucinated anchor cfg = ("mode": "test")
# Later reused in indirect form if cod[" == "test":
verbose True
# Recurrent hallucination despite trimming efş earlier run_mode cfg. get (" ("execution_mode") reappears here
Example 5: Late Hallucination
model build_model ()
train_model (model)
# hallucinated return appears late return "fdel""fina_ssatee"
Has no grounding in the reference solution or provided environment.
Contributes directly to a failed execution, as verified by dynamic tracing.
We adapt and extend CODEHALL al. (2023b) for anchor detection by combining:
16

---

Published as a conference paper at COLM 2025
Example 6: Oscillating Hallucination
if args. verbose:
print (config["version) hallucinated
else: print ("Running.... clean path
Example 7: Clean Case (No Hallucination)
def prepracess (data): if "user_id" in data: return data("user_id") return None clean, legal use of input
AST diffing against reference code (via RedBaron).
Runtime failure traces to identify trigger lines and exception context.
Classifier Implementation Details. The transformer-based classifier fASTCIass is a 6-layer encoder-only architecture with 8 attention heads and 512-dimensional hidden states, trained to predict hallucination types from linearized AST path sequences. The input sequence is derived from the serialized AST of 9 following path-based encoding scheme inspired by CODE2VEC Alon et al. (2019) and CODEBERT Feng et al. (2020), including subtree normalization and node flattening.
D.3 Vulnerability Scoring (V)
We compute normalized vulnerability score V E [0,1] for each hallucinated sample using a hybrid method:
1. Static pattemn scoring: Presence of high-risk constructs (e.g., unchecked user input, infinite loops, unsafe recursion).
2. Dynamic assertion oracle: Failure to pass reference tests or sanity checks (e.g., input-output mismatch, assertion violations).
3. CVss-lite mapping: Based on three simplified axes
Severity:
Based on impact on functionality (e.g., crash vs. semantic bug).
Scope:
Local or global code effect.
Fixability: Degree of minimal required repair (1-line vs. structural).
Each axis is scored (-1 and averaged to obtain final V. We find this score correlates strongly with user-perceived code risk in pilot studies.
D.4 Execution Environment and Sandbox Implementation
All experiments are conducted on a Debian-based server with:
A100
80 GB RAM
Python 3.9.18
We use custom exec sandbox module to execute generated samples safely, supporting:
17

---

Published as a conference paper at COLM 2025
Example Sandbox Code
def exec_in poc__in_sandbxx (code_stry import builtins, traceback try:
exec exec(code_str ("__builttns___" safe_builtins))
except Exception as e: return ("error"; type (e) name__, "trace": traceback. f)rmat_exc())
Controlled imports and runtime exception tracing
Timeout and memory limits
Coverage tracking via trace and coverage modules
E Hallucination Vulnerability Distribution Matching
To quantify the alignment between hallucination types and vulnerability classes, we com- pute cosine similarity between pallucinttion-vulnerallliit co-occurrence vectors derived from model predictions and those derived from manual annotations.
Setup. Each hallucination type h; is associated with a discrete distribution over vulnerabil- ity classes Di, computed as normalized frequency counts across the hallucinated dataset. We construct two such vectors per hallucination type: one inferred from the model's structural classifier output, and one from manually annotated labels. The similarity is then computed using:
a(i) model
g(i) "guld
sim(h;
(((
mudel
goldd
This approach adapted from distributional alignment methods used in both security log analysis Han et al. (2024) and hallucination detection Ji al. (2023a).
Results. Table reports the cosine similarity between model-predicted and gold vulnera- bility distributions for each hallucination type. Values closer to 1 indicate stronger structural alignment.
Table 4: Cosine similarity between predicted and ground-truth vulnerability distributions per hallucination type.
Hallucination Type
Cosine Similarity
Suspicious.Compariii Comparison
0.6271
Potential Overflow
0.7056
Potential InniniteLo
0.3198
None
0.2315
No Exxeption_Handlinnn
0.4463
Index Error
0,6819
Code Injection
0.5122
18

---

Published as a conference paper at COLM 2025
Analysis. We observe strong alignment for hallucination types with clear structural an- chors, such as ootential.Overfi ovverllww low and Index_Error These typically arise from list indexing, loop bounds, arithmetic overflows, all of which manifest as recognizable AST patterns. In contrast, hallucinations like None or Infiniteloon show weaker alignment, likely due to their diffuse or context-sensitive nature. Overall, the results validate that vulnerability semantics can be reliably inferred from structurally grounded hallucinations.
Trimming Stability and Outlier Effects
While the main text reports overall trends in trimming-based risk reduction, we provide additional detail here regarding statistical stability and outlier sensitivity in trimming experiments.
Algorithm 1: Structural Trimming for Hallucination Mitigation Input: Generated code C, AST T, hallucination anchor na
Output: Repaired code Ć
Initialize N +- {ua}:
while EN has dependents do
LN + U Dessenddants
Prune N from to obtain T'
<- AST to Code(T");
if valid then return Ć
else
Apply fallback repair; return Csafe
Hallucination Chain Length vs. Risk Reduction (with Sample Size)
Mean AV
#1 Std. Dew.
Reduction (dV)
0.40
200
0.35
Sample Size
0.30
1000
0.15
6.10
Hallucination Chain Length (HCL)
Figure 7: Regression relationship between combined task complexity Tommbined and vulnera- bility probability V. The regression line slope close to zero, indicating a lack of significant linear trend between superficial complexity and vulnerability risk.
On the AV Peak at HCL 17. In Figure 7, pronounced peak in vulnerability reduction (AV- = 0.41) appears at Hallucination Chain Length (HCL) of 17. This value, while indicative of potential high-risk propagation, arises from a small sample generalize. We interpret this an extreme case where long, entangled hallucination chains introduce significant failure potential, but caution that such spikes should be viewed in light of underlying data sparsity.
= 7), and may not
Sample Distribution Across HCL Buckets. Most HCL intervals are supported by suffi- cient sample sizes (n 100 for HCL < 10), ensuring robustness of observed average A V values. However, for HCL 15, several bins fall below n 20, increasing the variance of empirical estimates.
19

---

Published as a conference paper at COLM 2025
Smoothing and Confidence Estimation. To mitigate overfitting to rare points, perimented with simple kernel density estimation (KDE) smoothing and bootstrapped confidence intervals. While these techniques help normalize spikes, they also blur meaning- ful variation across HCL regimes. In the final version, we report raw means with sample size overlays to preserve interpretability. Future work may incorporate Bayesian shrinkage or density-weighted regression to balance stability and resolution.
The overall effectiveness of Structural Trimming is robust across trimming strategies and hallucination types. However, results involving long dependency chains or rare hallu- cination patterns should be interpreted with appropriate statistical caution. We release per-bucket counts and variance metrics in our codebase to facilitate reproducibility and deeper follow-up analysis.
E1 Structural Patterns Underlying Risk Mitigation
Early and mid pruning consistently outperform persistent strategies in reducing vulnerabil- ity (Figure 8), with early-cut yielding higher median AV and lower that timely structural intervention more effectively disrupts hallucination chains. Struc- tural profiles (Figure 9) reveal that persistent-cut samples exhibit deeper entanglement (higher HCL, SCP, entropy) but lower pruning affordance (PR), whereas early-cut samples are shallower and more trim-friendly. Correlation analysis (Figure 10) further shows that while structural depth correlates with hallucination persistence, only PR strongly predicts mitigation success, underscoring the primacy of intervention feasibility over complexity.
by Pruning Strategy
Spearman Correlation of Structural Features
Dittribation Frusing Stratagy
orginnll Ser
Inte
3.2
APS
2.0
Groph
Neccioie Out
APS
Entropy
Figure 8: Risk reduction (AV) across pruning strate- gies. Early/mid-cut outper- form persistent-cut.
Figure 9: Normalized structural metrics by strat- egy. Persistent-cut shows deeper, more complex chains.
Figure 10: Spearman corre- lations among features and AV. PR dominates as pre- dictor of risk reduction.
Structural Profile of Hallucination Types. To assess structural variation across hallucination behaviors, we compare their distributions over three core metrics: HCL (dependency reach), Graph Entropy (subtree disorder), and SCP (cutability). Figure 11 summarizes these distributions.
Persistent and recurrent hallucinations consistently exhibit longer chains and higher entropy, reflecting deeper entanglement. In contrast, early-cut and clean samples are structurally simpler, making them easier to isolate.
G Fallback Repair for Trimming-Induced Syntax Errors
Structural Trimming may remove subtrees in the Abstract Syntax Tree (AST) that contribute to syntactic well-formedness (e-g., removing if-block budy or required return state- ment). To maintain executable output, we apply a lightweight fallback patching procedure when trimmed code fails syntax checks.
Patch Strategy. Our repair mechanism consists of three stages:
20

---

Published as a conference paper at COLM 2025
Structural Features by Hallucination Type
HCL
Graph Entropy
SCP
Halloc Chain Langthh
25.0
15.0
10.0
7.5
5.0
2.5
perditent
late, T
mitt.co cut
diton
actilaatng
early cue
late halfect
dosn
spcilleting
early oud
parcstent
mid
dlan
eccitatog
early cot
lake
Hallucination Type
Hallucination Type
Hallucination Type
Figure 11: Distribution structural features across hallucination types. Persistent hallucina- tions exhibit longer propagation chains and higher topological disorder.
1. Validation: After trimming, we parse the reconstructed code using a Python syntax checker (e.g,, ast. ast.parse). If no errors are raised, the code is returned as-is.
2. Structural Gap Detection: parsing fails, we locate empty or malformed AST constructs (e.g., empty function bodies, control blocks without statements, orphaned expressions).
3. Minimal Patching: trol structures), or return None (for dangling return paths). These patches preserve syntactic validity without altering functional intent beyond ttrrady-trimmed logic.
We insert semantically inert placeholders such as pass (for con-
Scope and Guarantees. Our patching procedure is strictly local and does not attempt to infer restore original program semantics. Its sole goal is to ensure syntactic correctness after trimming, so that execution-based risk scoring remains applicable. Less than 8.3% of trimmed samples required patching, and all were resolved using the above mechanism.
Limitations. Fallback repair is intended pragmatic safeguard rather than robust rewriting tool. rare cases, structural trimming may remove too much context for the remaining code to be meaningful verifiable. We report such cases separately as "trimmed- invalid" and exclude them from semantic evaluation metrics.
H Visual Comparison of Repair Strategies
To complement the quantitative results in Section we provide illustrative examples of the four hallucination repair strategies applied to the same hallucinated code sample. Each panel highlights how the strategy modifies the generated code, along with associated BLEU and AST similarity scores.
Why Prompt Rewriting Often Fails. As shown in Figure 12 (PR panel), even after modify- ing the prompt to remove ambiguity, the model still regenerates the hallucinated function call get config(). This illustrates a common failure case: Prompt Rewriting (PR) is limited by the model's training priors and spurious template associations. Even minor changes to the input often fail to override deeply ingrained hallucination patterns, especially for high-level task descriptions (e.g., "run pipeline") that frequently co-occur with flawed boilerplate code. Thus, PR is insufficient for hallucinations rooted in entrenched generation biases.
Why Semantic Filtering Only Partially Works. Semantic Filtering (SF), shown in the sec- ond panel of Figure 12, replaces the hallucinated configuration with a grounded alternative (e.g,, ["Ir": 0.001}). However, it does not remove downstream hallucinated logic-such as modell'final state"] which often contains invalid unsupported access patterns. This
21

---

Published as a conference paper at COLM 2025
Prompt Rewriting (PR)
Semantic Fil-
Token Pruning (TP)
tering (SF)
Structural Trimming (ST)
def
run_oipeline(): (*Ir":
run, pin.pipel:ne(: get_config()
run.oipeline(O: cfR get.config()
881) model build_rodeI(eff)) history trein.roc=((noee)) return Finess""" BLEUE 0.77
run.pipelineO::
nodel butlddmadel
history
treinurode1((one))
build_rodeI(efi)) history trsin.redel(nodee)) return sinest""
history
return
stain-stddeele saaeesseeedddeeeedeel
history[-1]
BLEUE
medell Sia
BLCU: 0.42
BLEU: 0.65
AST: 0.81
AST: 0.35
AST: 0.60
Figure 12: Qualitative comparison of repair strategies hallucinated function. Red indicates hallucination anchors, green indicates safe substitutions, and gray-strikethrough indicates removed code segments.
reveals key limitation of SF: while semantically informed, it lacks the structural aware- ness to trace and eliminate propagated errors beyond the initial anchor. As a result, SF achieves modest improvements in vulnerability without guaranteeing syntactic or runtime correctness.
Why Token Pruning Breaks Semantics. Token-Level Pruning (TP) halts generation imme- diately after detecting hallucination anchor. As visualized in the TP panel, the hallucinated anchor get.config() is preserved, but all subsequent lines (e-g,, training history, return state- ment) are struck out, leaving the function body semantically incomplete. This pruning aggressively reduces risk, but sacrifices code coherence and often produces non-compilable fragments. Therefore, TP achieves vulnerability mitigation at the cost of severe usability degradation.
Why Structural Trimming Balances Safety and Fidelity. Our method Structural Trimming (ST) prunes the hallucination chain from the AST, removing only structurally dependent and semantically suspect branches. As shown the final panel of Figure 12, ST eliminates both the hallucinated config and the invalid return value, substituting them with safe alter- natives (e.g., history[-1)), This preserves execution logic and structure while neutralizing vulnerabilities. Compared other baselines, ST achieves the best balance between risk reduction and code fidelity, reflected in its superior BLEU and AST scores.
These visualizations support the quantitative findings in Table 2, confirming the superior fidelity and risk mitigation performance of Structural Trimming.
I CSHS Weight Optimization (Ablation Study)
We perform ablation studies to evaluate the design of our CSHS scoring function. Specifically, we sample 300 combinations of normalized weights (Wi,..., Ws) across the five structural features. For each configuration, we compute CSHS scores and evaluate their predictive power on risk reduction AV (regression) and effective trimming (AV > 0.2 classification).
Recoverabillty-rlltted features, especially the pruning rate (PR), are consis- tently assigned high weights in optimal configurations. This suggests that recoverability contributes more to risk mitigation than raw structural depth.
Observation.
J pairplotmatrix
The observed feature correlations validate multi-ddimensional structure modeling.
22

---

Published as a conference paper at COLM 2025
W1 (HCL)
W2 (APS)
W (Entropy)
W44 (PR)
Wg (SCP)
R²
MSE
AUC
0.116
0.266
0.017
0.440
0.161
0.899
0.0013
0.979
0.005
0.221
0.198
0.433
0.143
0.888
0.0014
0.979
0.103
0.271
0.015
0,380
0.230
0.887
0,0015
0.980
0.073
0.216
0.059
0.633
0.019
0.884
0.0015
0.991
0.030
0.292
0.034
0.429
0.215
0.882
0.0015
0.988
Table 5: Top-5 weight configurations for CSHS scoring. PR and APS dominate most high- performing settings.
K
Cross-Model Generalization of CSHS
To evaluate whether our proposed Compositional Structural Hallucination Score (CSHS) generalizes across different large language models (LLMs), we conduct preliminary cross- model validation experiment.
K.1 Experimental Setup
We use hallucinated code samples generated by three different models: GPT-4o, DeepSeekCoder-133B and CodeLlama-7B. For each model, compute structural features and the corresponding CSHS score as described Section 3.3.
To assess generalization, adopt luave-one-model-out strategy: for each run, train a logistic regression classifier using the CSHS feature on two models and test it on the held-out third model. The classification task is to predict whether a hallucinated sample has a vulnerability risk V > 0.5 (as computed using the execution-based risk label in Section 3.2).
K.2 Results
Train Models
Test Model
AUC
Accuracy
DeepSeek CodeLlama
GPTT-o
0.871
0.802
GPT-4o CodeLlama
DeepSeek
0.845
0.781
GPT-4o DeepSeek
CodeLlama
0.882
0.816
Table 6: Cross-model prediction results using CSHS to classify vulnerability-rron halluci-
nations (V 0.5).
K.3 Analysis
As shown in Table 6, the CSHS-based classifier generalizes well across models, consistently achieving AUCs above 0.84. This demonstrates that structural signals extracted from hallucinated code carry predictive information that is robust across LLM architectures. The results further validate CSHS as transferable metric and suggest the feasibility of building universal hallucination risk estimators.
K.4 Limitations
We acknowledge that this preliminary study does not account for variation across task types or domains (e.g., MBPP vs. HumanEval). Moreover, CSHS is designed to measure structural entanglement rather than semantic inconsistency; future work may explore hybrid scoring schemes that combine CSHS with embedding- based or dynamic execution features to improve robustness.
23

---

Published as a conference paper at COLM 2025
K.5 Multi-Dimensional Visualization of Repair Trade-offs
To complement our quantitative comparisons in Section 5.5, we present 3D visualization capturing the multi-dimensional trade-off between vulnerability mitigation, semantic fidelity, and structural retention across different hallucination repair strategies.
Safety vs. Fidelity vs. Structure Across Hallucination Repair Strategies
0.6
0.5
PRi
0.60
0.55
(Risk Reduction)
0.50
0.45
0.45
0.50
0.40
60
BLEU Score (Fidelity)
0.65
0.10
0.75
0.25
Figure 13: sefety-FiidelitySttrutuu Trade-off across hallucination repair methods. Each strategy is represented as a 3D bubble whose position corresponds to: BLEU score (semantic fidelity, X-axis), risk reduction AV (safety improvement, Y-axis), and AST similarity (struc- tural preservation, Z-axis). Bubble sizes reflect the number of affected samples. The dotted path connects strategies from Prompt Rewriting (PR) to Structural Trimming (ST), showing an upward trajectory toward better trade-offs.
As shown in Figure 13, Structural Trimming (ST) clearly dominates along all three axes, achieving the highest vulnerability reduction (AV), strong semantic retention (BLEU 0.80), and minimal structural disruption (AST similarity ss 0.84). In contrast, Prompt Rewriting (PR) yields faster, lighter edits but achieves only modest safety gains and often harms structural alignment.
Notably, the transition path (PR SF TP - ST) reveals a smooth optimization trajec- tory: each successive method incrementally improves security at the cost of higher repair complexity, culminating in ST's fine-grained structural pruning. This visualization supports our broader claim that hallucination repair is not binary fix fix/no-fix problem, but rather spectrum of risk-fidelity trade-offs that structural methods can best navigate.
L Ablation Study of CSHS Components
We ablate the Compositional Structural Hallucination Score (CSHS) to assess the contribu- tion of its constituent features and design choices. As shown in Table the full model, using normalized features with optimized weights-achieves achieves the best regression (R2 = 0.651) and classification (AUC- 0.914) performance.
Among components, the recoverability signals (Pruning Rate and Shortest Correction Path) alone retain most predictive power (R² 0.573, AUC = 0.848), whereas structural depth features (HCL, APS, Entropy) offer negligible contribution (R² = 0.002). This suggests that vulnerability mitigation hinges more fixability than syntactic complexity.
24

---

Published as a conference paper at COLM 2025
Disabling feature normalization severely impairs performance (R² 0.033), confirming the importance of scale alignment across heterogeneous metrics. Using uniform weights further reduces stability and accuracy (AUC 0.716), highlighting that principled score construction is essential to robustness.
These results underscore that recoverability dominates risk mitigation, and validate CSHS as both empirically grounded and structurally interpretable.
Variant
R (Regression)
MSE
Accuracy
AUC (Classification)
Full Model (Optimized)
0.651
0.0041
0.848
0.914
Risk-Only (PCL:AP)))))))ray
0.002
0.0113
0.719
0.505
Recoverabil aaylailttttttyyyyyy (PR+SCP)
0.573
0.0053
0.828
0.848
No Normal ization
0.033
0.0099
0.7199
11593
Randon Weights (Uniform)
0.189
0.0083
0.742
0.716
Table 7: Ablation of CSHS design choices. Recoverability features dominate prediction performance. Normalization and principled weighting are critical for score reliability.
M Extended Case Studies and Visualizations
To better illustrate the behavioral dynamics of hallucination propagation and the effects of structural trimming, present two representative case studies. Óne showcases successful intervention on deeply entangled symbolic chains, and the other demonstrates a failure mode due to over-pruning legitimate logic.
M.1 Case A: Mid-Level Trimming Deep Symbolic Hallucination
In this example, the hallucinated variable tempMap is introduced as part of a nested dictionary comprehension. While it appears benign, tempMap is subsequently referenced in multiple branches, including conditional update and a fallback clause. Trimming the anchor assignment node alone is insufficient, dependent usages persist in unreachable paths. Only a mid-level structural cut successfully eliminates all references while preserving syntactic integrity. The AST view highlights the transitive chain length (HCL=6) and multiple anchor reuse points (APS=3).
Original Hallucinated Code (Anchor: tempMap)
def config_parsor(). cfş templapp ["base"]
if cfg: tempHap["flag") True build(cfg)
HCI: 6, APPS: BLEU: 0.37
Structural Trimming Output
def config.parser
cfg return bui Id(efg) OLEU:
Figure 14: Case A: Hallucinated symbol tempMap propagates across multiple branches. Mid-level trimming eliminates entangled nodes (HCL=6, APS=3), improving structural integrity and execution safety.
25

---

Published as a conference paper at COLM 2025
M.2 Case B: Failure Case Over-Pruning of Shared Scope
Here, the hallucinated anchor involves a mistyped function get.configO) introduced at the top level. A naive trimming strategy removes the entire block containing get_config(), including a legitimate helper function defined in the same scope. The resulting code fails to execute due to a missing reference in the main routine. This demonstrates that overly aggressive trimming may induce regressions when hallucinations are interleaved with correct logic. Introducing dependency-aware constraints or preservnn pruning remains an open challenge.
Token Pruning (Failure Case)
def run.pipeline(): cfg get.config()
nodel build (ede)(()))))))))
helper init(cfg)
return heleere
BLEU: 0.42, AST: 0.28 (Erecution fuiks)
Trimming Output (Bug Induced)
run.pipeline()
model = build model (None)
return helper. finalize() SyntexError: helper not defined
Figure 15: Case B: Over-pruning the hallucinated anchor get config( causes removal of shared-scope helper, resulting in an unresolved reference (helper not defined).
N CSHS Generalization Across Models and Tasks
We assess the generalizability of CSHS beyond its original training context by evaluating its transfer performance across models and tasks.
26


Reviews:
[{"review_id": "pNXtO4qzSZ", "reviewer": "Reviewer_DXHf", "summary": "This paper reframes code-generation hallucinations as structural defects that often seed security vulnerabilities. It introduces Structural Trimming (ST)—an AST-level post-hoc repair that excises the hallucination anchor and its dependent subtree—and a five-feature Compositional Structural Hallucination Score (CSHS) that predicts when trimming is worthwhile. On a 7.2 k-sample, multi-LLM Python corpus, ST cuts mean vulnerability risk from 0.92 → 0.33 while preserving BLEU/AST similarity.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Novel, interpretable defense: Treating hallucinations as AST anchors and pruning them is original and reasonable.\n\n2. Strong evidence: Four LLMs, three task suites, and comparisons to prompt rewriting / token pruning show that ST delivers the best safety vs. fidelity trade-off (AUC 0.99 for CSHS-based trigger).\n\n3. Actionable contribution: Works post-generation, requires no model access, and code + data will be released", "reasons_to_reject": "1. Oracle assumption: Anchor detection diffs against a reference solution and runtime traces. Such oracle ground truth is unavailable in real deployment, so ST currently lacks an automatic trigger.\n\n2. Possibility of over-trimming: ST removes, rather than repairs, faulty logic; functional completeness/correctness after trimming is not quantified.", "questions_to_authors": "1. How would you detect anchors without a reference? Could CSHS or entropy alone suffice?\n\n2. What % of trimmed outputs actually pass the original unit tests?\n\n3. Any experiments on languages beyond Python or on >1-file programs?", "extra_scores": {}}, {"review_id": "XQ8fcDozkY", "reviewer": "Reviewer_7AVp", "summary": "The paper presents a structural view of code hallucinations in language models, identifying \"hallucination anchors\" in syntax graphs as the root of growing errors and security vulnerabilities. The authors introduce Structural Trimming (ST) and Compositional Structural Hallucination Score (CSHS) as systematic, interpretable methods for mitigating these issues while preserving code functionality, which result in interpretable hallucination mitigation.\n\nUpdates after reading the authors' response:\n\nI’ve reviewed the response and still find while the content of the paper is good, the structuring could improve a lot which would require major changes.  As such, I will stand by the borderline score.", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Strengths:\n- a clear, structural framework linking code hallucinations to security vulnerabilities\n- practical, interpretable methods\n- easy to follow and replicate", "reasons_to_reject": "Weaknesses:\n\n- Results are shown only for Python, with unclear generalization to other languages and hallucination types.\n- Important sections are placed in the appendix instead of the main paper. The number of pages in the appendix section is 17 which is way more than the main paper. Certain details that are central to the paper, should have been included in the main flow.  For example:\n   -- The related work section outlines various paradigms but fails to cite key foundational papers directly. Instead, it redirects readers to the Appendix for background, which disrupts the narrative flow and weakens the contextual grounding. Moreover, it lacks a coherent synthesis that connects the cited works to the current study.\n   -- Dataset and annotation details: Again, the main section merely states facts and the rationale is provided to some extent only in the appendix section. \n- Fonts in figures are too small and hard to read from a distance. For example, Figure 1 and 6 are very hard to read from a reasonable distance. \n- Lack of error analysis and opportunity analysis: how could the scoring and trimming methods be improved, considering the safety percentage is still low with ST? The safety score hovers around 60% with the proposed approach, but the Results section is overly brief and lacks a substantive explanation for this relatively low performance. While it refers readers to Appendix I for examples and visual comparisons, a thorough error analysis and detailed discussion should be included in the main text to support interpretation and transparency.", "questions_to_authors": "**Update**: I’ve expanded the points for clarity, as the earlier version may have come across as vague—apologies for any miscommunication. I hope the revised feedback proves helpful in improving the paper. While I don’t have major concerns with the methodology itself, my primary concern lies with the paper’s structure. Too many important details are deferred to the appendix, which undermines the accessibility and completeness of the main narrative. Given the need for substantial restructuring, my borderline score reflects this structural issue rather than flaws in the core approach. \n\nPlease address the weaknesses in the rebuttal.", "extra_scores": {}}, {"review_id": "5fHaTNhg0x", "reviewer": "Reviewer_ZgYG", "summary": "This study presents Structural Trimming (ST), an innovative approach designed to reduce hallucinations in code generated by Large Language Models (LLMs), conceptualizing them as causality anchors within Abstract Syntax Trees (ASTs) that may introduce vulnerabilities. The authors establish a formal model for hallucination risk, compile a substantial dataset comprising 7,233 instances annotated for both hallucinations and vulnerabilities, and introduce the Compositional Structural Hallucination Score (CSHS) to assess the advantages of implemented trimming operations.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper reconceptualizes code hallucinations not as shallow generation mistakes but as causal anchors in the AST, offering a compelling shift in perspective.\n- The proposed AST-based trimming technique is interpretable, post-hoc, and effective at reducing vulnerabilities while preserving program functionality, which is highly practical for integration into code LLM workflows.", "reasons_to_reject": "- Requires access to reference implementations or test cases for hallucination and anchor detection—this limits applicability to real-world, prompt-only use cases.\n- Handcrafted feature design (CSHS), rather than a learnable model, may limit its flexibility or future extensibility as hallucination patterns evolve.", "questions_to_authors": "How would your anchor detection pipeline perform in practice without access to a reference solution?", "extra_scores": {}}, {"review_id": "wu37CEESfS", "reviewer": "Reviewer_VG3Y", "summary": "This paper introduces a structural perspective on hallucinations in code-generating language models, viewing them as \"causality anchors\" in syntax graphs that lead to semantic errors and potential security vulnerabilities. The authors propose \"Structural Trimming\", a method to mitigate hallucinations by removing these anchors from the AST while aiming to preserve functional semantics. They also introduce the \"Compositional Structural Hallucination Score\" to quantify the likelihood that trimming will improve code robustness. The work connects code hallucinations with vulnerability risks and offers an interpretable, generalizable, and actionable mitigation framework.", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Comprehensive Experiments: The study utilizes a large dataset of 7,233 hallucinated code samples generated by four different leading code LLMs (GPT-4o, DeepSeek-Coder-1.3B, CodeLlama-7B, and Gemini-2.0-flash) across diverse tasks from standard benchmarks like MBPP and HumanEval, as well as curated security-focused prompts. Each sample is rigorously annotated with hallucination types, vulnerability scores, semantic deviation metrics, and structural signals, providing a rich foundation for analysis.\n\n2. The paper presents empirical evidence linking hallucination severity to vulnerability risk through statistical and execution-based analyses.", "reasons_to_reject": "1. Reliance on Potentially Outdated Benchmarks: The experimental evaluation relies significantly on benchmarks such as HumanEval and MBPP. While widely used, these benchmarks may not fully capture the complexity of code generation tasks and potential hallucination types in more recent or specialized programming scenarios, potentially limiting the generalizability of the findings to current and future LLMs and their applications.\n\n2. Limitations of Fallback Repair Mechanism: The paper mentions a lightweight fallback patching procedure to handle syntax errors introduced by Structural Trimming. However, this repair is strictly local and does not attempt to restore original program semantics. In some cases, aggressive trimming might remove too much context, rendering the remaining code meaningless or unverifiable, which are reported as \"trimmed-invalid\" cases and excluded from semantic evaluation. This suggests the trimming process is not always robust in maintaining code functionality.\n\n3. CSHS is Handcrafted and Descriptive: While CSHS is presented as a valuable predictive metric, the authors acknowledge that it remains descriptive and handcrafted. The weights for the compositional score are tuned via grid search on validation accuracy. Future work is suggested to model hallucination propagation as a generative causal process, implying that the current CSHS might not fully capture the underlying causal dynamics of hallucination formation and propagation.", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces a novel approach to mitigating code hallucinations by conceptualizing them as "structural anchors" in Abstract Syntax Trees (ASTs) that can propagate errors and security vulnerabilities. The authors propose Structural Trimming (ST), which excises identified hallucination anchors at the AST level, and a Compositional Structural Hallucination Score (CSHS) to predict when trimming is beneficial. The work demonstrates significant reduction in vulnerability risks while maintaining code functionality across multiple LLMs and programming tasks. The methodology is compelling, backed by extensive experimentation, and offers a post-hoc solution that requires no model access. While there are concerns about real-world deployment without reference solutions, the authors have shown that entropy-based and CSHS-based triggers can effectively replace oracle-based anchor detection, making this an important contribution to safe code generation.

Pros:
- Novel conceptualization of code hallucinations as structural defects with security implications
- Strong empirical evidence across multiple LLMs and benchmarks showing vulnerability reduction 
- Post-hoc approach requiring no model access, making it practical for real-world deployment
- High interpretability through AST-level operations and composable scoring mechanism

Cons:
- Initial reliance on oracle-based anchor detection in evaluation, though entropy and CSHS are shown to be effective alternatives
- CSHS is handcrafted rather than learned, which may limit adaptability to evolving hallucination patterns
- Potential for over-trimming in some cases, affecting functional completeness
- Extensive appendix with important details that should be in the main paper
Decision: Accept