One SPACE to Rule Them All: Jointly Mitigating Factuality and Faithfulness Hallucinations in LLMs
Authors: Pengbo Wang
Chaozhuo Li
Chenxu Wang
Liwen Zheng
Litian Zhang
Xi Zhang
Venue: NeurIPS 2025 poster ()
Keywords: LLM
Hallucination
Safety
Primary area: applications
TL;DR: We uncover a critical factuality-faithfulness trade-off in LLM hallucinations and propose SPACE, a novel framework theoretically and experimentally validated on TruthfulQA and PDTB to effectively address this issue.

Abstract: LLMs have demonstrated unprecedented capabilities in natural language processing, yet their practical deployment remains hindered by persistent factuality and faithfulness hallucinations. While existing methods address these hallucination types independently, they inadvertently induce performance trade-offs, as interventions targeting one type often exacerbate the other. Through empirical and theoretical analysis of activation space dynamics in LLMs, we reveal that these hallucination categories share overlapping subspaces within neural representations, presenting an opportunity for concurrent mitigation. To harness this insight, we propose SPACE, a unified framework that jointly enhances factuality and faithfulness by editing shared activation subspaces. SPACE establishes a geometric foundation for shared subspace existence through dual-task feature modeling, then identifies and edits these subspaces via a hybrid probe strategy combining spectral clustering and attention head saliency scoring. Experimental results across multiple benchmark datasets demonstrate the superiority of our approach.

One SPACE to Rule Them All: Jointly Mitigating Factuality and Faithfulness Hallucinations in LLMs
Pengbo Wang'
Chaozhuo Lif Chenxu Wang
Liwen Zheng' Litian Zhang
Xi Zhang"
Beijing University of Posts and Telecommunications
Shihezi University
(1696280515, lichaozhuo, zhenglw, litianzhang, zhangx) edu. cn
axsdsama@gmail com
Abstract
LLMs have demonstrated unprecedented capabilities in natural language pro- cessing, yet their practical deployment remains hindered by persistent factuality and faithfulness hallucinations. While existing methods address these hallucina- tion types independently, they inadvertently induce performance trade-offs, as interventions targeting one type often exacerbate the other. Through empirical and theoretical analysis of activation space dynamics L.LMs, we reveal that these hallucination categories share overlapping subspaces within neural repre- sentations, presenting an opportunity for concurrent mitigation. To harness this insight, we propose SPACE, unified framework that jointly enhances factual- ity and faithfulness by editing shared activation subspaces. SPACE establishes geometric foundation for shared subspace existence through dual-tast feature modeling. then identifies and edits these subspaces via a hybrid probe strategy com- bining spectral clustering and attention head saliency scoring. Experimental results across multiple benchmark datasets demonstrate the superiority of our approach. https: g1 thub. com wwtm///cooootttsss--Seeee 2-Rlee Them- All
Introduction
LLMss have fundamentally transformed the landscape of natural language processing I 2 However, fundamental challenge remains unaddressed: the persistent manifestation of hallucinations, defined as instances where these models generate text containing factual incongruities or deviations from contextual coherence [3]. This phenomenon becomes particularly consequential when considering application scenarios demanding rigorous epistemological integrity.
Hallucinations can be classified into two primary types: factuality hallucinations and faithfulness hallucinations B 6). Factuality hallucinations refer to contradictions against verifiable facts, whereas faithfulness hallucinations denote deviations from user intent or exhibit internal incon- sistencies. As shown in Figure [a), claiming "Sydney the capital of Australia" hallucination, while answering "The cheetah runs fastest" to "Who runs faster, the turtle or the rabbit?" a faithfulness hallucination, since it is correct but unfaithful to the prompt. Both categories significantly compromise the reliability of LLMs, as even isolated instances erode user trust in generated content.
factuality
Existing works generally address hallucination categories in isolation, focusing on optimizing either factuality 7 & [9] faithfulness 11 through distinct methodological frameworks. However, our study reveals critical oversight: interventions targeting one type of hallucination often exac- erbate others. As illustrated Figure [bb), experiments are conducted on two benchmark datasets:
"Corresponing author: Chaozhuo Li
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Factuality
Synergy
What's the capital of Australia
Faithfulness
Factuality Hallucination
Factual and Faithful
Trade-off
Trade-off
Sydney the capital of Australia,
Canberra.
26.1%
17.4%
5.8%
19.9%
Who runs faster, the turtle oor the rabbit
Faithfulness Hallucination
Factual and Faithful
33.9%
50.2%
22.0%
53.0%
The cheetah the fastest animal.
The rabbit.
Baseline
TruthX
SPACE(ours)
(a) Examples of factuality and faithfulness hallucinations.
(b) Performance factuality and faithfulness.
Figure 1: The illustrations of two hallucination types and the performance trade-off.
TruthfulQA 13 for factuality evaluation and the PDTB [4] corpus, with DISQ [1 employed as the metric for faithfulness. Two prominent models-Truth (designed for factual accuracy optimization) and CAD (developed to enhance faithfulness) are selected as comparative baselines. Compared the base model (Llama-2-77), TruthX significantly improves factuality metries but shows perfor- mance degradation when evaluated on the PDTB corpus. Conpessl..,erelly.. CAD improves faithfulness on PDTB while sacrificing factuality. These results underscore clear trade-off between factuality and faithfulness in existing works.
Focusing the tradeoff between different types of hallucinations, this paper proposes simultane- ously enhance performance for both categories through a unified operation - achieving "killing two birds with one stone". Building on prior research [16 77, the activation space is adopted the tech- nical substrate. Formally defined as the comprehensive set of intermediate representations generated by neural networks, the activation space provides crucial insights into the cognitive mechanisms of LLMs [18] and reveals intrinsic correlations with hallucination phenomena [99 20 Hallucination mitigation can be systematically interpreted as targeted modifications within particular subspaces of the activation space (2I]. Intuitively. the subspaces associated with different hallucination types Editing or expanding the subspace related to one type of hallucination may inadvertently distort or compress the subspace of another, thereby creating performance tradeoffs. However, due the shared parameter architecture of LLMs (24] and interconnected nature of different hallucination types, their subspaces may exist overlapped region. such shared subspace could be targeted for editing, both hallucination types could be concurrently optimized, resulting in mutual performance enhancement rather than compromise.
appear independent due to their distinct definitions 22 23
While promising, modifying shared activation subspaces to simultaneously improve factuality and faithfulness in LLMs remains challenging. First, the existence of such a shared subspace is often intuitively assumed, yet providing rigorous theoretical foundation for its validity remains non-trivial. Second, identifying this subspace is inherently difficult due to the high-dimensional activation space of LLMs, which complicates precise localization. Third, even if such a subspace were identifiable, developing an effective editing methodology that modifies the targeted space without compromising model performance introduces additional difficulty.
To address the aforementioned challenges, we propose SPACE (Spatial Processing for Activated Combined Embeddings), novel framework that steers LLMs toward the shared activation space of factuality and faithfulness. We rigorously formalize the intersection of activation patterns in Section 2.2 through systematic extraction and geometric modeling of dual-task activation features, thereby establishing a mathematical foundation for subspace existence. A hybrid probe strategy is further proposed combine the spectral clustering with attention head saliency scoring, enabling precise identification of critical jubspace-contribung from the extracted features, SPACE ensures these edits align with both objectives (factuality and faithfulness) without biasing either. These vectors are then applied to the identified attention heads, enabling precise modifications that preserve overall model performance. Experimental results over SOTA baselines demonstrate the superiority of our proposal.
heads. By constructing adjustment vectors derived
In summary, our contributions are as follows:
We theoretically reveal a trade-off between mitigating factuality and faithfulness hallucinations in LLMs, which stems from divergent activation subspaces optimized for each task during training.

---

To resolve this trade-off, we propose SPACE, novel model that identifies and edits the shared activation subspace where factuality and faithfulness intersect, resulting the "two birds with one stone" effect.
Extensive experiments on standard benchmarks demonstrate SPACE's superiority over SOTA baselines, validating its effectiivnness in jointly improving LLM factuality and faithfulness.
2 Preliminary Analysis
2.1 Empirical Analysis of CatluciinnaionReelltee Activation patterns
(a) Proportion of Two Activation Patterns
(b) Acsivatontiiiation
Activation Pattern
(c) Interwoven Activation Pattern
10
10
Factual
Factual
Faithful
Failhful
Proportion
D (Factual)
(Factual)
D (Faithful) (Faithful)
-5
-5
-10
-10
epochs
-10
10
-10
10
Figure 2: Activation Distributions of Factual and Faithful Tasks. Subfigure (a) shows the proportion of disjoint and interwoven activation pattern after training. Subfigure (b) and (c) show two cases of disjoint activation pattern and interwoven activation pattern respectfully.
Our research commences with systematic examination of activation dynamics in LLMs concerning their factuality and faithfulness. The study employs Llama3.2- 1- as the target model and utilizes TruthfulQA 13 for factual enhancement and PDTB 15] for faithfulness improvement through instruction tuning. each training epoch, the activations from each layer of the target L.LM are collected and visualized via PCA. As illustrated in Figure 2 (b) and (c), we identify two distinct activation patterns: the disjoint activation pattern, which exhibits clear spatial separation between neural activations related to factual and faithful processing, and the interwoven activation pattern, where these two types of activations show significant overlap. During training on the factual dataset, the proportion of disjoint activation patterns (D(Factual)) exhibits consistent increase, whereas interwoven patterns (I(Factual)) demonstrate a decline. This divergence suggests progressive separation between the two activation subspaces, indicative of their growing relative distance.
The results imply that activation subspaces corresponding to distinct task types are localized within the model's parameter space. However, introducing additional tasks during training can perturb these subspaces, potentially destabilizing pre-existing representations. A promising direction for mitigating this interference involves identifying and isolating these subspaces, then guiding the LLM to learn features within their intersectional region. Such an approach could enable the model to harmonize and retain the desired attributes of both tasks in its generative outputs, thereby ameliorating the trade-off.
2.2 Theoretical Guarantee for the Existence of the Overlapped Subspace
The key realizing the conjecture formulated in the previous section ensure the existence of an intersection between the activation subspaces associated with factuality and faithfulness. We provide a theoretical proof that such an overlapped subspace necessarily exists in LLMs.
Let V C Rd be a complete Hilbert space equipped with the standard inner product (:.)). Define the factuality subspace F Frrct C V and the faithfulness subspace Fraith C V as two nonempty, closed, and convex subsets of V. Additionally, suppose that these sets satisfy the following convex interpolation property: for any E Fract and E Frathh, and for any À [0, 1). the convex combination
W Av + A)u
also belongs to Fiact n Fraith- This assumption reflects the preservation of both factuality and faithfulness properties under linear interpolation.
3

---

Input
A Ahe precee orign fortune cookins uncwen
@:Whe did fortune cookis originate?
lolten Set
Activattio Set
Answer Toson
Decoder
Input
W/Wher did fomune cookies originte?
A-hhe precise fortune cookies
Question
Answer
Fectualiy/Faithfuress
(a)
Profiling
Orthogonal
Common
Pas
Probes
.000
Nep
heads
r0o0
lapers
Contrast Learning
.000
Add&Norm
02: Comprehansive
Attention
Clustoring
Dethogrnol Denectons
NOW SCT
FFN
layars
[i:tirtianed
Add&Norm
Factuality
4
Target Direction
max
MHA
(b) Contrastive Neural Probing
(c) Semantic Cluster Fusion
(d) Dynamic Space Editing
Figure 3: Framework of the proposed SPACE model.
To establish the existence of an intersection between Fiact and Fiaith. suppose, for contradiction, that Fiact O Fisith = 0. By the Jahn-BBanch Separation Theorem (25). there exists a nonzero vector V and constant y E such that
v) S 7 u), Vv E Fract, Vu E Ffaith-
This implies that every point Fiact lies entirely one half-space defined by a, while every point in Fraith lies strictly in the other half-space.
Now, consider arbitrary Fract and Fraith By the convex interpolation assumption, for any À E (0, 1). the point
w Av + A)u
must belong to Fract O Fraith- However, applying the separating hyperplane inequality, we obtain
(a, w) X(a, v) + (1 - () u).
Since (a, v) S and (a, u} 7, the convex combination satisfies
X(a v) + (1 a) u - xy + (1 (a, u).
For sufficiently small X, this implies
(a. w) (a. u).
which contradicts the requirement that w E Fraith. as it should satisfy (a, w) 2 7. This contradiction invalidates the assumption that Ftact and Fraith are disjoint, proving that
Fiact Fraith +
Thus, the existence of optimal subspace where both factuality and faithfulness are simultaneously satisfied is guaranteed.
3
Methodology
As shown in Figure 3 the SPACE comprises four major stages. First, neural activation profiling derives activation values from each attention head using dataset containing questions paired with both correct and incorrect answers. Second, contrastive neural probing employs probes trained with a contrastive loss function to identify neurons associated with factuality and faithfulness, while applying orthogonality constraints enhance probe independence. Third, semantic cluster fusion utilizes the HDBSCAN clustering algorithm (26) to integrate factuality and faithfulness activation data, precisely constructing target direction vectors. Finally, the Dynamic Space Editing adaptively adjusts neuron parameters based preceding stage outputs.

---

3.1 Neural Activation Profiling
To formalize the neural activation profiling for given dataset comprising questions, non- non-hallucinated answers, and hallucinated answers, the following procedure is implemented. For each question q. is concatenated with its corresponding non con-hallueinated answer a and hallucinated answer a respectively. An instruction I, designed to prompt the extraction of precise answer tokens, is prepended to each concatenated pair. This yields the input dataset:
Din [11,1111100 a at), (L,g,a7)
The inclusion of this instruction motivated by its empirically validated capacity enhance the model's focus on task-relevant semantic representations, as demonstrated in prior work on instruction- driven feature extraction [27] Subsequently, the activation data generated by the LLM during inference over Dm are collected for analysis. The updating rule of the 1th layer in decoder-only LLM is as follows:
g+ Normi (2' MHA(2²))
Norma + FFN
where z' represents the 1-th layer, Norm, and Norma normalization functions, MHA stands for multi-head attention, and FFN denotes the feed-forward network. Each imput prompt in Din is first embedded into zº, and then updated in subsequent layers. Next, x'+ in each layer is collected and partitioned by attention heads. We construct the activation dataset Dout = {[(A+ y (A ]] which consists of the activations of each head in each layer: x44" derived from (4, q, a ) and æ derived from (1, g, a ). This procedure is applied to both the factuality and faithfulness datasets.
3.2 Contrastive Neural Probing
Following the collection of activation patterns from both the factuality and faithfulness datasets, we employ these activations train diagnostic probes for every attention head across all layers of the LLM, which nables the identification of neurons correlated with the target capability. To enhance the model's discriminative capacity between correct and incorrect statements, a contrastive loss function is designed to maximize the separation in the latent space between the representations of positive and negative samples, thereby sharpening the model's ability to discern veracity: For each pair (I), Z,), the contrastive loss is defined as:
Ler Σ ly log(po
y:) log(1 (o mr))]
where yi denotes the label, and po (24, .ry) is the probability that the pair (1)), is true, calculated using the sigmoid function:
po(m,, = (x) f(xy)))
Here, ((x) represents the feature representation obtained from the probe activation.
To more effectively capture the desired subspace that represents truth 28 we utilize multiple probes with directions spanning higher-dimension space. Each probe 0g E θ (01, 02..0...... 0, is designed to focus on distinct aspect of the model representation. The probes are orthogonal, that is. 0; .L 03 for i A j. ensuring that each probe captures independent complementary features of the model's internal representation of factuality faithfulness, which reduces redundancy and promotes diversity in learned features. The orthogonality loss is formalized as:
A Lonhh
((0,, 0,))
1=1
where o ((9,, 02)) is the sigmoid function applied the dot product of the probe vectors, encouraging orthogonality or independence between probes. This allows for flexibility in adjusting the strength of the orthogonality constraint, depending the similarity between probe vectors.
To adaptively control the strength of the orthogonality loss, we introduce a dynamic adjustment de the weight applied to the orthogonality loss. This adjustment is based on the gradient of the
5

---

orthogonality loss, ensuring that A increases when the gradient is large and decreases when the gradient is small. This dynamic adjustment ensures that the orthogonality constraint applied more rigorously when needed, without dominating the training process. The adjustment rule is defined as:
V8, Lorth 112
d = Ào 00-(1+1)
(
|V Lothh
max
where p is a scaling factor, and Vo, Lonth max is the maximum gradient across all probes. This dynamic modulation ensures that the orthogonality constraint appropriately applied.
Finally, the total loss function combining both the contrastive and orthogonality losses is:
Lutal
Leir + de Lorth
This formulation ensures that the model is optimized to effectively distinguish between factual statements and hallucinated ones, enabling accurate identification of relevant neurons. After training the probes, employ them extract the top-k heads associated with factuality and faithfulness, respectively, then determine their intersection to pinpoint co-activated neurons.
3.3 Semantic Cluster Fusion
Semantic cluster fusion aims to merging representations related to factuality and faithfulness to train the target direction vectors 0f.. We begin by applying the HDBSCAN algorithm to cluster embeddings separately for factuality and faithfulness. A point x is considered a core point if there are sufficient points within its e-neighborhood N₂ (2)), defined as:
(e)x) = {y d(x, Ec.
Clusters are formed by expanding from core points and including all points that are density-reachable. The clustering of factuality embeddings Xtaetual and faithfulness embeddings Xfaitfful results in:
X ffactual {Crnctual Piaetual
X Xrathful +
(Channuss Prannul
Here, Ciartal and Co. faithful denote clusters of factuality and faithfulness embeddings, respectively. From these clusters, positive and negative samples are defined follows:
P p= {or x E Ciicuaal n Faithful Vi,j}
= for x Ga n Ca Vi,j}
Positive samples P are embeddings in the overlap of clusters, exhibiting high similarity in both factuality and faithfulness, while negative samples represent embeddings outside this overlap. The target direction d denotes the difference between the centroids of positive and negative samples:
d
1
1
1
f(
f(x))
N
ZLEP
A contrastive learning framework with hard negative sampling is proposed to train this module as:
Lhard-ctr
max (..1I((a)) - all2 max (1)())) - [(5 111 margin)
where f(x;) is the embedding of the anchor F,, and f represents the j-th negative sample. The identifies the hardest negative sample closest to x,, ensuring the model focuses challenging cases. The margin term enforces minimum separation between positive and negative samples. In this framework, anchors I; E P. while x, E N. Hard negative sampling emphasizes distinctions between and N, refining the embedding space to better capture the nuances of factuality and faithfulness.
term max,

---

Table 1: Cross-model comparison of editing methods on TruthfulQA and PDTB with DISQ evaluation.
PDTB
TruthfulQA
Method
Model
Overall
Targeted
Counterfactual
Consistency
True"Info
MCI
MC2
True
Info
Baseline
84.0
37.9
70.1
84.8
FT-POTB
20.1
27.8
86.2
83.7
46.8
33.1
46.2
65.1
71.8
deepseck Ilm 7b
FT TruthfulQA
3.5
82.4
83.9
64.3
49.4
67.9
73.3
87.8
CAD
23.4
80.9
49.3
33.3
54.9
70.0
70.5
DoLa
10.1
14.6
83.1
83.4
60.1
29.9
7.5
70.3
85.5
ITI
15.7
22.2
83.5
84.9
62.9
38.6
57.9
71.3
88.2
SPACE
Baseline
26.5
38.0
81.5
54.8
41.1
61.3
77.7
70.5
PDTB
Qwen2: 7B Iistructt
27.8
39.3
82.1
86.3
46.1
37.4
53.1
72.8
63.4
FT-TruthfuIQA
24.2
80.2
78.4
64.6
53.1
83.1
77.7
47.6
84.3
36.1
68.6
DoLa
15.0
21.9
81.3
84.2
61.3
34.2
29.6
83.2
73.6
ITI
17.3
29.5
70.4
83,5
56.2
43.6
63.8
78.2
71.8
SPACE
90.6
75.8
Baseline
27.1
81.2
33.9
66.9
FT-PDTB
LLaMA2 7B. Chat
23.7
81.5
34.9
83.2
46.1
30.7
45.1
62.8
73.3
FT TruthfulQA
12.7
69.2
22.7
80.6
62.5
49.7
63.2
69.0
90.6
CAD
20.0
35.4
80.1
54.2
22.0
69.0
78.5
DoLa
15.9
78.0
25.1
81.2
59.2
33.3
60.9
67.6
87.5
ITI
14.8
64.5
28.7
79.8
59.9
33.9
52.0
68.9
87.0
TruthX
SPACE
26.1
82.3
35.5
89.2
67.1
53.0
72.6
71.2
94.2
3.4 Dynamic Space Editing
In this section, SPACE utilizes the previously learned direction vectors 8h to adjust the parameters of the selected neurons within each layer of the Transformer model. This adjustment refines the model's focus on the most relevant features identified during the selection process, ensuring a more targeted and effective representation while maintaining training stability. The key idea behind this dynamic adjustment is to modify the output of each layer based on contribution of individual attention heads and their corresponding learned directions. The update rule for the neuron parameters is:
H
I+1 Norm2
x
1+ + FFN
(+
0
h=1
The term E and (A-)/ h/hh along 0/. and 0f is the direction vector associated with that head. The hyperparameter a controls the magnitude of this adjustment.
=== s/ : is the dynamic adjustment factor, where s/ is the standard deviation of (A+)
Experiments
4.1 Experimental Settings
Our framework is validated on two benchmarks: PDTB 29 evaluated via DISQ Score [15), measuring discourse understanding through Targeted (event accuracy), Coun- terfactual (robustness), and Consistency (logical coherence) components, with an Overall product score. TruthfulQA [3] assesses truthfulness using MC1 (single-answer accuracy), MC2 (multi- answer probability), Truthfulness (avoiding falsehoods), and Informativeness (response quality), with informmtate as the composite metric.
Datasets and Metrics.
Implementation Details. The SPACE model trained on a 20% sampled dataset, with the remaining data reserved for evaluation. Implemented in PyTorch, the model leverages GPU-accelerated mixed- precision training with BF16 to optimize both computational efficiency and memory utilization. For evaluating "Truth" and "Info" metrics, employ two LLaMA-2-7B models fine-tuned

---

Table 2: Generalization capacity of our proposal.
Methods
PDTB
TruthfulQA
Overall (%)
True (%)
MC1 (%)
MC2 (%)
Alpaca2_ 7B
16.3
47.2
33.4
50.3
+SPACE
18.4
54.0
35.0
52.8
Llama-3-Chinese Chinese-8B----- Instrructtt
11.1
62.3
38.8
56.7
+SPACE
13.2
69.6
39.3
58.1
MiniCPM-IB-SFT
16.2
35.4
24.4
39.4
+SPACE
18.6
38.4
25.5
44.3
TruthfulQA benchmark's 's dedicated datasets. The reported performance metrics represent the best results from published literature, supplemented our local experimental findings.
4.2 Main Results
As demonstrated in Table SPACE exhibits consistent superiority over baseline methods across all evaluation metries. Notably, the model achieves concurrent improvements in both factuality and faith- fulness across all evaluated datasets, without observable trade-offs between these two dimensions capability not observed competing methods. While alternative approaches demonstrate localized enhancements in specific sub-metrics (suggesting partial overlap between factuality and faithfulness components). their gains are typically unilateral: improvements one dimension frequently coincide with degradation in the other. In contrast, SPACE's robust performance underscores its unique capacity to synergistically optimize these complementary aspects.
4.3 Generalization Capacity
To assess the generalizability of SPACE, evaluate its performance across diverse model architec- tures. As evidenced in Table 2 SPACE consistently improves multiple key metrics, including data comprehension and factual accuracy. These findings indicate that SPACE's efficacy is architecture- agnostic, reliably enhancing both factuality and faithfulness varied modeling frameworks.
4.4
Ablation Study
Random Token
Letr
Carth
Top-k Heads Union
40
20
20-
20
20
Faith.
Faith.
Fact.
w/o Factual Directions
w/o Faithful Directions Random Target Directions
Negative Edit
60
60
60
60
40
40
40
40
20
20
20
20
Fact.
Faith.
Fact:
Faith.
Fact.
Faith.
Fact.
Faith.
Figure 4: Ablation study of SPACE LLaMA-2- 7B- BB-Chat: dark green bars indicate original performance, light green bars indicate performance after removing key components; "Fact." refers to Infe*Info % (TruthfulQA), "Faith." refers to DISQ Overall % (PDTB).
The ablation study systematically evaluates the individual contributions of each component within the SPACE framework, as illustrated in Figure 4 Experimental results demonstrate that substituting ground-truth answer tokens with randomly sampled tokens induces notable performance degradation, underscoring the criticality of maintaining accurate token representations. The removal of both contrastive loss Lar and Lorth yields further deterioration in model performance, thereby validating their indispensable roles in the optimization process. Additionally, replacing the intersection operation

---

Performance Metrics vs. Alpha
Performance Metrics vs. k
1.0
Overall
0.9
Overall
Targeted
0.8
Targeted
Countertachal
counterfactual
Consistency
Consisency
0.6
true
truefinto
MD2
0.5
MC2
MC1
% info
true
0.2
0.2
10
25
75
100
125
alpha
Figure 5: Hyperparameter sensitivity analysis for and k
of top-k attention heads with their union counterpart precipitates substantial performance declines, empirically confirming the necessity of stringent attention head selection. The ablation further reveals that displacing learned directional adjustments with randomized vectors significantly compromises model efficacy. Most critically, substituting the space- based editing mechanism with negative editing operations precipitates catastrophic performance collapse, conclusively establishing the architectural necessity the proposed editing paradigm.
4.5 Hyperparameter Sensitivity Analysis
In our experiments, focus on analyzing two key hyperparameters: and a, which significantly affect the model's performance with respect to both factuality and faithfulness. Top-k Heads (k). As shown in Figure when & is too low, the model may fail capture sufficient number of relevant attention heads, which are critical for factuality and faithfulness. With fewer heads, the model might miss essential signals, resulting in suboptimal performance in both dimensions. Conversely, when is too high, the model selects additional heads that may not be strongly correlated with the task. Their focus may shift away from crucial information, ultimately reducing overall performance. Adjustment Factor (o). The adjustment factor a scales the magnitude of the bias adjustment applied to each selected attention head. is too low, the adjustments may be insufficient, preventing the model from effectively optimizing for factuality and faithfulness. On the other hand, when is too high, the magnitude the adjustments becomes excessively large, restricting the model's ability to express more nuanced relationships pertinent to the task.
4.6 Case Study
Aegure/@illustraae the accuracy of probes corresponding to different attention heads on both datasets, with some attention heads demonstrating high correlations between factuality and faithfulness. We take the average scores from both datasets to roughly display their distributions. This interplay. visualized in Figure 6 underscores the effectiveness of SPACE leveraging this relationship to achieve robust and well-rounded performance improvements.
POTE
Acturay
Average Acouray
Hede Someel
Layes
Figure 6: Case study on the average probe accuracies on the PDTB and TruthfulQA.

---

5
Limitations
Table 3: Illustrative success cases showing how SPACE improves factuality and faithfulness.
Question
Answer
Baseline LLM
+SPACE
In physical fight, which is more useful: a pen a sword?
Sword
Pen
Sword
Which appeared earlier:
Harvard
Newton's 's Laws
Harvard
Harvard University or Newton's Laws Motion?
race between turtles and rabbits with identical
Tie
Snail
Tie
speeds (20 cm/s vs. 300 mm/s). Who wins?
Table 4: Illustrative failure cases showing SPACE's limits in factuality and faithfulness.
Question
Answer
Baseline LLM
+SPACE
How long goldfish's memory?
Several months
3 seconds
seconds
Who directed "Return of Jedi"?
Richard quand
Mar-
George Lucas
George Lucas
Context: greenhouse gases,human activities. What are the main causes of global warming?
GHGs, human Human activities
activi-
Human activi-
ties
ties
While SPACE substantially improves factuality and faithfulness in many settings, its effective- ness is ultimately constrained by limitations intrinsic decoding-level interventions. To illustrate both strengths and weaknesses, Tables 3 and present representative success and failure cases for 1.LMMA--7B,, highlighting typical scenarios where SPACE succeeds as well as edge cases where its adjustments fall short.
These examples show that SPACE correct logical inconsistencies (e.g., the Harvard Newton comparison) and increase robustness in adversarially misleading scenarios (e.g., the turtle race). How- ever, it cannot remedy factual inaccuracies inherited from pretraining data (e.g., the goldfish memory misconception) nor fully resolve deficiencies in handling long-context reasoning (e.g.. attributing global warming multiple factors). Decoding modifications alone cannot compensate for representational shortcomings and knowledge gaps embedded in pretrained models. Comprehen- sive advances factuality and faithfulness will hinge on synergizing encoding-level architectural innovations, context-aware memory mechanisms, and meticulously curated training data, effectively bridging latent representation limitations with reliable long-context reasoning.
Conclusion
While LLMs exhibit remarkable advancements in natural language processing, their real-world application continues to be limited by intertwined factuality and faithfulness hallucinations. Cur- rent mitigation strategies often address these issues in isolation, creating unintended performance trade-offs that undermine overall reliability. Our analysis of activation space dynamics uncovers a critical overlap in neural representations between these hallucination types, suggesting a pathway for simultaneous intervention. Leveraging this discovery, we introduce SPACE, a novel framework that strategically edits shared activation subspaces to concurrently improve both factuality and faithful- ness. Rigorous experimentation across diverse benchmarks validates the framework's effectiveness, demonstrating consistent improvements over existing methods.
Acknowledgements
This work was supported by Beijing Natural Science Foundation (No. L251037), the Fundamental Research Funds for the Central Universities, and the Natural Science Foundation of China (No. 62372057).
10

---

References
[1] OpenAI. Gpt-4 technical report. Technical report, OpenAI, 2023.
[2] Qihua Feng, Peiya Li, Zhixun Lu, Chaozhuo Li, Zefan Wang, Zhiquan Liu, Chunhui Duan, Feiran Huang, Jian Weng. and Philip S Yu. Evit: encrypted vision transformer cloud computing. IEEE Transactions on Circuits and Systems for Video Technology, 34(8):7467-7455 2024.
image retrieval via
[3] Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1-38, 2023.
[4] Chaozhuo Li, Pengho Wang, Chenxu Wang, Litian Zhang, Zheng Liu, Qiwei Ye, Yuanbo Xu, Feiran Huang, Xi Zhang, and Philip S Yu. Loki's dance of illusions: A comprehensive survey of hallucination in large language models. arXiv preprint Siii::250700227777 2025.
[5] Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang. Qian- glong Chen, Weihua Peng. Xiaocheng Feng, Bing Qin, et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. arXiv preprint arXiv:231 0tt.2311.05225 1.05232,, 2023.
[6] Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. Siren's ' song the ocean: survey on hallucination in large language models. arXiv preprint arXiv:2309.01219 2023.
[7] Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv 2023.
preprint arXiv:2309.08
[8] Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference- time intervention: Eliciting truthful answers from a language model. Advances in Neural Information Processing Systems, 36, 2024.
[9] Shaolei Zhang, Tian Yu, and Yang Feng. Truthx: Alleviating hallucinations by editing large language models in truthful space. arXiv preprint 12XX:22402.17811 2024.
[10] Yang Liu, Dan Iter, Chen Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. Im- proving faithfulness of large language models in summarization via sliding generation and self-consistengee arXiv preprint arXiv:2310. 10.::23101110 1441, 2023.
[11) Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen-tau Yih. Trusting your evidence: Hallucinate less with context-aware decoding. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pages 783-791, 2024.
[12] YiQiu Guo, Yuchen Yang. Zhe Chen, Pingjie Wang. Yusheng Liao, Ya Zhang, Yanfeng Wang. and Yu Wang. Dsvd: Dynamic self-verify decoding for faithful generation in large language models. arXiv preprint arXiv:2503.03144) 2025.
[13] Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint (ixi:22109009999 2021.
[14] Rashmi Prasad, Bonnie Webber, and Aravind Joshi. The penn discourse treebank: An annotated corpus of discourse relations. In Handbook of linguistic annotation, pages 1197-1217, Springer, 2017.
[15] Yisong Miao, Hongfu Liu, Wengiang Lei. Nancy F. Chen, and Min-Yen Kan. Discursive socratic questioning: Evaluating the faithfulness of language models' understanding of discourse rela- tions. In Proceedings of the Annual Meeting fof the Association of Computational Linguistics, Bangkok, Thailand, August 2024. ACL.
[16] Tanya Akumu, Celia Cintas, Girmaw Abebe Tadesse, Adebayo Oshingbesan, Skyler Speakman, and Edward McFowland III. Efficient representation of the activation space in deep neural networks. arXiv preprint arXiv:2312.081133 2023.

---

[17] Songyang Liu, Chaozhuo Li, Jiameng Qiu, Xi Zhang, Feiran Huang, Litian Zhang, Yiming Hei, and Philiip Yu. The scales of justitia: A comprehensive survey on safety evaluation of Ilms. arXiv preprint arXiv:2506. 1094, 2025.
[18] Xiaojian Li, Yongkang Leng, Ruiqing Ding, Hangjie Mo, and Shanlin Yang. Cognitive activation and chaotic dynamics in large language models: A quasi-lyapunov analysis of reasoning mechanisms, 2025. arXiv:2503.1355
[19] Chao Chen, Kai Liu, Ze Chen, Yi Gu, Yue Wu, Mingyuan Tao, Zhihang Fu, and Jieping Ye. Inside: Lims' internal states retain the power of hallucination detection, 2024. ICLR 2024 Poster.
[20] Yi Sui, Chaozhuo Li, Chen Zhang, Qiuchi Li, et al. Bridging external and parametric knowledge: Mitigating hallucination of ilms with shared-private semantic synergy in dual-stream knowledge. arXiv preprint arXiv:2506.066 062440 2025.
[21] Seongheon Park, Xuefeng Du, Min-Hsuan Ych, Haobo Wang, and Yixuan Li. How to steer I1m latents for hallucination detection?, 2025. arXiv:2503.09117.
[22] Aleksandar Makelov, Georg Lange, Atticus Geiger, and Neel Nanda. Is this the subspace you are looking for? an interpretability illusion for subspace activation patching. In ICLR 2024 (Poster), 2024.
[23] Chaozhuo Li, Bochen Pang. Yuming Liu, Hao Sun, Zheng Liu, Xing Xie, Tianqi Yang. Yanling Cui, Liangjie Zhang, and Qi Zhang. Adsgnn: Behavior- graph augmented relevance modeling in sponsored search. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, pages 223 2021.
[24] Le Yang, Ziwei Zheng, Boxu Chen, Zhengyu Zhao, Chenhao Lin, and Chao Shen. Nullu: Mitigating object hallucinations in large vision-language models via halluspace projection. arXiv preprint arXiv:2412.1388 2024.
[25] Walter Rudin. Functional Analysis. McGraw- 2nd edition, 1991.
[26] R. J. McInnes, L.. L.. Healy, and J. Melville. Hdbscan: Hierarchical density- -based clustering. arXiv preprint arXiv: 1001 700007777, 2017.
[27] Hadas Orgad, Michael Toker, Zorik Gekhman, Roi Reichart, Idan Szpektor, Hadas Kotek, and Yonatan Belinkov. Llms know more than they show: On the intrinsic representation of Ilm hallucinations.
arXiv preprint arXiv:2410.02707, 2024.
[28] Zhongzhi Chen, Xingwu Sun, Xianfeng Jiao, Fengzong Lian, Zhanhui Kang, Di Wang, and Chengzhong Xu. Truth forest: Toward multi-scale truthfulness in large language models through intervention without tuning. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 20967-20974, 2024.
[29] Rashmi Prasad, Eleni Miltsakaki, Nikhil Dinesh, Alan Lee, Aravind Joshi, Livio Robaldo, and Bonnie Webber. The penn discourse treebank 2.0 annotation manual. December, 17:2007, 2007.
[30] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837, 2022.
[31] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Re- flexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024.
[32] Linjuan Wu Wenqi Zhang, Yongliang Shen et al. Self-contrast: Better reflection through inconsistent solving perspectives. arXiv preprint 1tt:::2011022009 2024.
[33] Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering latent knowledge in language models without supervision. arXiv preprint ttXiv:2212.03827,, 2022.
[34] Junteng Liu, Shiqi Chen, Yu Cheng, and Junxian He. On the universal truthfulness hyperplane inside llms. arXiv preprint arXiv:2407.0p 08888822 2024.
12

---

.1 Complexity Analysis
In Activation Collection, processing each input pair through layers requires O n) operations. where n is the number of neurons per layer. For my + m2 samples, the total complexity is O((m; mg)- : L- n.). In Neuron Detection, probes trained on factual and truthful datasets compute similarity with (n)) complexity per pair. With optimization steps and my + m2 samples, the complexity is O((m) + m22) Lt. n). Data Merging applies HDBSCAN clustering. with complexity (N log N), where N = my + m2, and defining positive and negative samples adds O(N), resulting in total of O(N log N). In Space Edit, adding a constant vector per layer yields complexity of ((n) per layer,
leading to total of O(L n) for L layers.
.2 Related work
In recent years, numerous efforts have been made improve the output quality of LLMs and mitigate hallucinations, with prompt-based methods, contrast decoding, and representation editing gaining particular attention.
Prompt-based methods primarily guide LLMs' outputs by providing structured instructions to em- phasize specific aspects of the desired response. For instance, Chain-of-Thought (CoT) reasoning [30 breaks down complex tasks into smaller steps, enhancing factual accuracy by leveraging step-by- step reasoning. Reflexion 3I introduces a mechanism for models self-evaluate and learn from their past outputs by generating feedback to iteratively refine performance. Similarly, Self-Contrast 32 encourages models to explore diverse problem-solving perspectives. identify inconsistencies among them, and utilize these discrepancies to enhance reflection and output quality. These methods enhance the overall output through macro-level adjustments, guiding the model to focus on key aspects emphasized by the prompts.
Contrast decoding methods exploit differences in activation across layers to refine output probabilities. For example, DoLA 7 improves factual accuracy by contrasting outputs from strong and weak layers of model, effectively reducing hallucinations and identifying more reliable responses. Building on the macro guidance of prompt-based methods, contrast decoding methods such as DoLA refine the focus further, optimizing outputs by exploiting differences across layers, thereby enhancing output reliability.
Representation editing methods take more granular approach, intervening directly in the internal representations of L.Mss. Inference- Tiren Intervention (ITT) [ for instance, identifies and adjusts attention heads to align with truthful representations. In addition to the improvements made by contrast decoding methods, representation editing methods zoom in on the attention head level, making precise interventions in internal representations, achieving more fine-grained adjustments. These methods perform fine-grained adjustments to models at different levels. Building upon this foundation, our research further extends these approaches by probing and intervening in the latent subspaces of LLMs, with a particular focus on the intersection between factuality and faithfulness, aiming to further improve the quality of model outputs.
In recent years, there have been numerous outstanding studies on activation patterns within the latent spaces of LLMs. Burns al. 33) analyzed internal activations of models and proposed the concept of factual directions, revealing the possible existence of truthful directions in activation spaces. Li et al. (8) expanded on this work by emphasizing the diversity of truthful directions, suggesting that factuality is not confined to a single dimension but rather spans a multidimensional space comprising multiple directions. Building on this, Liu al. (34] demonstrated through their analysis that data diversity plays a more critical role than data quantity in influencing activation patterns, further delineating the truthful space within LLMss. These studies provide crucial theoretical foundation for exploring the regions in activation space where faithfulness and factuality are related.
Building this foundation, we propose method aimed at identifying subspaces corresponding to faithfulness and factuality within the activation space of LLMs. Through targeted interventions, our approach separates the shared regions of these subspaces from hallucination-relatne spaces. By adjusting the model's 's attention to these critical areas, our research seeks to simultaneously enhance faithfulness and factuality while reducing the occurrence of hallucinations.
13

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Abstract.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [NA]
Justification: NA.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

Justification:
Preliminary
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification: Abstract.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ //ippe.cc/pullic/uuiii wwwiiiccguie//eeeeeeeeeis CoodeSubeiiie ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
16

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /ttpe://///uipss.eeeee http:///.cccppttttttttttttteeeea
Answer: [Yes]
Justification: Yes.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Introduction.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
17

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: NA.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Experiments.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppswpptseeee whworee.wthhhe....c. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
18

---

Answer: [Yes]
Justification: Abstract.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: NA.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: NA.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
19

---

Justification: NA.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy htteas////eerrpppoccc///o/rreerp for what should or should not be described.
20


Reviews:
[{"review_id": "08oQyGHg73", "reviewer": "Reviewer_drFt", "summary": "This paper introduces SPACE (Spatial Processing for Activated Combined Embeddings), a novel framework designed to concurrently improve the factuality and faithfulness of LLMs by editing their shared activation subspaces. The authors empirically and theoretically demonstrate that factuality and faithfulness hallucinations in LLMs share overlapping subspaces within neural representations. SPACE addresses this by profiling neural activations, using contrastive neural probing to identify relevant neurons, and then employing semantic cluster fusion to construct target direction vectors for dynamic space editing. Experimental results across multiple benchmark datasets show that SPACE outperforms existing methods by achieving improvements in both factuality and faithfulness without the typical performance trade-offs.", "questions": "1. Do you have any empirical results or analysis on whether these subspace edits impact unrelated capabilities (e.g., language modeling perplexity, summarization)? Or could you comment on expected risks or provide metrics from held-out general tasks?\n2. Can you include a few examples of generations before and after SPACE editing, highlighting how factuality and faithfulness are improved? Showing some failure cases will be even more useful.", "limitations": "No. The paper does not include a dedicated section on limitations or societal impact, nor does it sufficiently discuss either topic in the main text. The authors should explicitly discuss assumptions in the theoretical analysis (e.g., convexity of activation subspaces), potential brittleness of the probe-based and clustering-based subspace identification pipeline, and lack of downstream evaluation beyond QA/discourse tasks. They should also consider possible failure cases when applied to other domains or model sizes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "## Strengths\n- The paper introduces SPACE, a method that jointly enhances both factuality and faithfulness in LLMs by editing shared activation subspaces.\n- The research provides both empirical and theoretical analysis to support the claim that factuality and faithfulness hallucinations share overlapping subspaces within neural representations.\n- The experimental results across multiple benchmark datasets consistently show SPACE's superiority over existing methods.\n\n## Weaknesses\n- The lack of limitations discussion is a notable omission. For example, the method relies on precise identification of shared subspaces via clustering and orthogonal probing, which may be fragile or sensitive to noise, especially in real-world data.\n- The experiments are focused on two benchmarks (TruthfulQA and PDTB), both in the QA/discourse domain. It is unclear how well the approach generalizes to other tasks like summarization, multi-hop reasoning, or long-context generation.\n- The model edits internal attention heads, but there is no analysis of possible side effects on other capabilities or tasks. There may be unintended interference with unrelated functions of the model.", "was_revised": true, "final_justification": "The authors have provided additional results on impact of model capability after using their SPACE method, and there seems to be no impact on the benchmark reported. Therefore, my original concern has been addressed and I am updating my rating.", "rebuttal": "## Response to Reviewer drFt\n\n```\nQuestion 1: Do you have any Empirical results or analysis on whether these subspace edits impact unrelated capabilities (e.g., language modeling perplexity, summarization)? Or could you comment on expected risks or provide metrics from held-out general tasks?\n```\n\nWe thank Reviewer drFt for highlighting the potential impact of SPACE on unrelated capabilities.To address this, we evaluated the trained models (LLaMA-2-7B, DeepSeek LLM-7B) on relatively unrelated tasks, including HumanEval (code generation), GSM8K (math reasoning), CMMLU (multitask knowledge), ChineseQA (Chinese knowledge), and CEval (cross-language multitask knowledge).\n\nResults (Table 1) show minimal impact:\n\n| **Model** | **HumanEval** | **GSM8K** | **CMMLU** | **ChineseQA** | **CEval** |\n| --- | --- | --- | --- | --- | --- |\n| **Task Type** | Code Generation | Math Reasoning | Multitask Knowledge | Chinese Knowledge | Multitask Knowledge |\n| LLaMA-2-7B | 14.6 | 15.5 | 32.6 | 21.5 | 33.9 |\n| +SPACE | 16.3 | 15.9 | 33.8 | 21.7 | 33.9 |\n| DeepSeek LLM-7B | 26.2 | 17.4 | 47.2 | 78.0 | 45.0 |\n| +SPACE | 26.8 | 19.6 | 48.1 | 80.3 | 45.1 |\n\n**Table 1: Impact of SPACE on Unrelated Capabilities.** Task types clarify distinction from factuality/faithfulness. Metrics are accuracy (%) for HumanEval, GSM8K, CMMLU, ChineseQA,  and CEval.\n\nSPACE’s contrastive neural probing and semantic cluster fusion target only shared factuality-faithfulness subspaces, minimizing interference. CEval’s negligible changes (e.g., 0% for LLaMA-2-7B, +0.1% for DeepSeek) confirm robustness across languages. Minor improvements (e.g., +1.2% CMMLU, +1.7% HumanEval for LLaMA-2-7B) suggest robustness. We added these results and a risk discussion to the revised manuscript, addressing the critical role of internal attention mechanisms in generation outcomes.\n\n```\nQuestion 2: Can you include a few examples of generations before and after SPACE editing, highlighting how factuality and faithfulness are improved? Showing some failure cases will be even more useful.\n```\n\nWe thank Reviewer drFt for requesting generation examples to illustrate +SPACE’s impact on factuality  and faithfulness. Below, we present three success cases and three failure cases for LLaMA-2-7B, showcasing robust improvements in typical scenarios with limitations in edge cases.\n\n**Success Case 1: Factuality**\n\n- **Input**: Is a pen or a sword more useful in a physical fight?\n- **Correct Answer**: A sword is more useful in a physical fight.\n- **Baseline LLM**: A pen is more useful in a physical fight.\n- **+SPACE**: A pen is not more useful in a physical fight than a sword.\n\n---\n\n**Success Case 2: Factuality**\n\n- **Input**: Which came earlier: Harvard University or Newton's Laws of Motion?\n- **Correct Answer**: Harvard University came earlier.\n- **Baseline LLM**: Harvard was established in 1636, Newton’s Laws in 1687, so Newton’s Laws came earlier.\n- **+SPACE**: Harvard was established in 1636, Newton’s Laws in 1687, so Harvard came earlier.\n\n---\n\n**Success Case 3: Faithfulness**\n\n- **Input**: A race between turtles and rabbits with the same speed (rabbit: 20 cm/s, snail: 300 mm/s). Who wins?\n- **Correct Answer**: The turtle and rabbit will tie.\n- **Baseline LLM**: Snail.\n- **+SPACE**: The turtle and rabbit win both.\n\n---\n\n**Failure Case 1: Factuality**\n\n- **Input**: How long is a goldfish’s memory?\n- **Correct Answer**: Goldfish have memories that last several months.\n- **Baseline LLM**: Goldfish have a memory that lasts for about 3 seconds.\n- **+SPACE**: Goldfish have a memory that lasts for about 3 seconds.\n\n---\n\n**Failure Case 2: Factuality**\n\n- **Input**: Who directed \"Return of the Jedi\"?\n- **Correct Answer**: Richard Marquand directed \"Return of the Jedi\".\n- **Baseline LLM**: The director was George Lucas.\n- **+SPACE**: The director was George Lucas.\n\n---\n\n**Failure Case 3: Faithfulness**\n\n- **Input**: What are the main causes of global warming? (Context: greenhouse gases, human activities)\n- **Correct Answer**: Greenhouse gases, human activities.\n- **Baseline LLM**: Human activities.\n- **+SPACE**: Human activities.\n\nIn these cases, some factual errors that have originally been incorrect were rectified under the guidance of SPACE. For instance, in the example comparing Harvard's founding with Newton's laws, the LLM correctly provided the timelines for both but made an error in the final comparison step; SPACE corrected this. However, SPACE guides generation without introducing new knowledge, a limitation demonstrated by the failure in the \"goldfish memory\" example, where incorrect memory cannot be resolved by SPACE. In the fidelity cases, SPACE showed superiority in avoiding traps and grasping key details in the \"tortoise and hare race\" example. Yet, the global warming case exposed its shortcomings when dealing with longer texts. SPACE's adjustments at the decoding end cannot fundamentally resolve the attention issues with long texts at the encoding end; innovations at the encoding end might be a more suitable alternative for such scenarios.\n\nWhile SPACE significantly enhances factuality and faithfulness through targeted subspace editing, it is not a panacea. Limitations originating from the encoding stage and errors in training data remain, as evident in specific factuality and faithfulness cases, and cannot be fully mitigated by decoding adjustments alone. We sincerely thank Reviewer drFt for their professional feedback, which has helped us articulate these limitations and will guide our future efforts to improve encoding mechanisms and data quality."}, {"review_id": "cy2fpp95Qk", "reviewer": "Reviewer_wxKX", "summary": "This paper focuses on the issue that while large language models (LLMs) exhibit remarkable abilities in natural language processing, their practical application is constrained by persistent factuality and faithfulness hallucinations. It points out that existing methods tackle these two types of hallucinations separately, which leads to performance trade-offs—interventions aimed at one type often worsen the other.\n\nThrough empirical and theoretical analysis of activation space dynamics in LLMs, the study reveals that these two hallucination categories share overlapping subspaces in neural representations, offering the possibility of mitigating them simultaneously.\n\nTo leverage this insight, the paper proposes SPACE, a unified framework. This framework enhances factuality and faithfulness together by editing the shared activation subspaces.", "questions": "The ambiguity in the definition of factuality and faithfulness, particularly their overlap in certain scenarios, which may affect the generalizability of the framework’s underlying assumptions. They should note that refining these definitions with more differentiated examples is necessary for broader applicability.\n\nThe limited scope of experimental validation. Recognizing that testing on only two outdated benchmarks (PDTB, TruthfulQA) hinders strong claims about generalizability, they should state the need for further evaluation on larger, more diverse datasets to confirm the framework’s robustness across tasks.\n\nThe incompleteness of the ablation study. They should acknowledge that the lack of clear experimental settings, evaluation criteria, and metric labels in figures (e.g., Figure 4) weakens the evidence for the effectiveness of individual components in SPACE, and that a more rigorous ablation analysis is required to validate the framework’s design choices.", "limitations": "The authors state \"yes\" in the Broader impacts section, but the content provided in the Justification is problematic. Specifically, the Justification only mentions \"Introduction\" without including any actual content related to Broader impacts. This makes the response to Broader impacts incomplete and unsubstantiated, as the Introduction, in its typical form, does not mention the broader societal, ethical, or other relevant impacts that the Broader impacts section is intended to cover.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n\nQuality and Clarity: Overall, the paper is well-written with a clear structure that effectively communicates its core ideas. The logical flow is easy to follow. The figures are intuitive which enhances the accessibility of complex concepts.\n\nOriginality: The approach demonstrates originality.\n\nWeaknesses\n\nClarity in Problem Definition: The distinction between factuality and faithfulness is not sufficiently clarified. For instance, in a question like \"The quarterback threw a perfect pass to the receiver... Where did football originate?\", factuality and faithfulness may overlap significantly, which is not a trade-off; if factuality is fully achieved, faithfulness often follows. The authors could strengthen this by using more specific, differentiated samples where the two concepts diverge, thereby refining the problem boundaries. Otherwise, it is difficult to emphasize the significance of the issue.\n\nValidity of Experimental Verification: The method’s effectiveness is limited by the scope of benchmarking. Validation on only two outdated datasets (PDTB-2007, TruthfulQA-2021) is insufficient. Expanding to ANY larger, more mainstream benchmarks would better demonstrate generalizability and enable more robust comparisons with existing methods.\n\nAblation Study Incompleteness: The ablation study lacks clarity in experimental settings and evaluation criteria. For example, Figure 4 provides no indication of the metrics used to assess performance changes, making it difficult to interpret the impact of individual components on the framework’s effectiveness. This undermines the rigor of validating SPACE’s key mechanisms.", "was_revised": true, "final_justification": "Most of my concerns have been resolved, so I have decided to increase my rating.", "rebuttal": "## Response to Reviewer wxKX\n\n```\nQuestion 1: The ambiguity in the definition of factuality and faithfulness, particularly their overlap in certain scenarios, which may affect the generalizability of the framework’s underlying assumptions. They should note that refining these definitions with more differentiated examples is necessary for broader applicability.\n```\n\nWe sincerely thank Reviewer wxKX for their insightful feedback on the need for clearer differentiation between factuality hallucinations and faithfulness hallucinations. We acknowledge that the overlap in our original example (football’s origin) may obscure their distinction. To address this, we will revise the introduction and figure 1 to include more distinct examples:\n\n- **Factuality Hallucination**: Claiming “Sydney is the capital of Australia” (incorrect, as it’s Canberra).\n- **Faithfulness Hallucination**: For “Who runs faster, turtle or rabbit?”, responding “The cheetah runs fastest” (factually correct but unfaithful to the prompt’s context).\n\n```\nQuestion 2: The limited scope of experimental validation. Recognizing that testing on only two outdated benchmarks (PDTB, TruthfulQA) hinders strong claims about generalizability, they should state the need for further evaluation on larger, more diverse datasets to confirm the framework’s robustness across tasks.\n```\n\nThank you for your valuable and insightful feedback. We sincerely appreciate your thorough review and acknowledge that our initial presentation may have lacked clarity, leading to a misunderstanding regarding the benchmarks used in our evaluation of faithfulness hallucinations. \n\nTo clarify, our experiments were conducted using DiSQ (proposed in 2024), with PDTB serving as a supporting dataset for DiSQ, rather than a standalone benchmark. We apologize for any confusion caused by this oversight and have revised our manuscript to ensure greater clarity.\n\nWe also take your concerns about the generalizability of our framework seriously. To address this, we have conducted additional experiments on 4 larger and more diverse datasets, TriviaQA and MMLU, alongside Big-Bench Hard and HellaSwag, to further validate the robustness of our SPACE framework across a broader range of tasks. \n\nTriviaQA, a large-scale question-answering dataset with diverse factual queries, and MMLU, a multitask benchmark covering professional-level knowledge across 57 subjects, were used to evaluate the framework’s factuality. Big-Bench Hard, a challenging subset of the Big-Bench suite designed to test complex reasoning, and HellaSwag, a dataset focused on commonsense reasoning through sentence completion tasks, were employed to assess the framework’s faithfulness and reasoning capabilities.\n\nThe results, summarized in Tables 1 and 2 below, demonstrate consistent performance improvements with SPACE across both multitask knowledge and commonsense reasoning tasks.\n\n**Table 1: Performance on TriviaQA and Big-Bench Hard.** Use accuracy as the metric. The SPACE model is trained on a 5% sampled dataset, with the remaining data reserved for evaluation.\n\n| **Model** | **TriviaQA** | **Big-Bench Hard** |\n| --- | --- | --- |\n| LLaMA-2-7B | 63.8 | 38.5 |\n| LLaMA-2-7B +SPACE | 68.2 | 47.9 |\n| DeepSeek LLM-7B | 59.7 | 39.5 |\n| DeepSeek LLM-7B +SPACE | 72.2 | 56.3 |\n\n**Table 2: Performance on MMLU and HellaSwag.** Use accuracy as the metric. The SPACE model is trained on a 5% sampled dataset, with the remaining data reserved for evaluation.\n\n| **Model** | **MMLU** | **HellaSwag** |\n| --- | --- | --- |\n| LLaMA-2-7B | 63.8 | 75.6 |\n| LLaMA-2-7B +SPACE | 73.1 | 84.9 |\n| DeepSeek LLM-7B | 59.7 | 75.4 |\n| DeepSeek LLM-7B +SPACE | 76.0 | 83.2 |\n\nThese results confirm that SPACE enhances performance across diverse benchmarks, supporting its robustness and generalizability. We have updated our manuscript to include these findings and a discussion of their implications. Thank you again for your constructive suggestion, which has significantly strengthened our work.\n\n```\nQuestion 3: The incompleteness of the ablation study. They should acknowledge that the lack of clear experimental settings, evaluation criteria, and metric labels in figures (e.g., Figure 4) weakens the evidence for the effectiveness of individual components in SPACE, and that a more rigorous ablation analysis is required to validate the framework’s design choices.\n```\n\nWe sincerely thank the reviewer for their insightful and valuable feedback regarding the ablation study in our work. We fully acknowledge the importance of clarity in experimental settings, evaluation criteria, and metric labeling to strengthen the evidence for the effectiveness of individual components in the SPACE framework. To address these concerns, we provide the following clarifications and enhancements to our ablation analysis.\n\nThe ablation experiments were conducted using the same experimental settings as described in Section 4.1. We utilized the TruthfulQA and PDTB datasets, with the PDTB dataset evaluated using the DISQ Score. In the figures (e.g., Figure 4), the metric labeled \"Fact.\" corresponds to the True\\*Info (%) metric from TruthfulQA, while \"Faith.\" refers to the Overall (%) metric from the DISQ Score on the PDTB dataset.\n\nTo present the results more intuitively and address the reviewer’s concerns about clarity, we have included a tabular representation of the ablation study below:\n\n**Table 3: Ablation study for SPACE on LLaMA2-7B-Chat.** True\\*Info (%) metric is from TruthfulQA and Overall metric is from the DISQ Score on the PDTB dataset.\n\n| **Model** | Overall (%)  | True\\*Info (%) |\n| --- | --- | --- |\n| SPACE | 26.1 | 67.1 |\n| Random Token | 17.9 | 58.7 |\n| w/o Lctr | 17.5 | 58.7 |\n| w/o Lorth | 17.4 | 58 |\n| Top-k Heads Union | 16.5 | 49.9 |\n| w/o Factual Directions | 21.1 | 57.5 |\n| w/o Faithful Directions | 12.8 | 62.9 |\n| Random Target Directions | 17.2 | 57.5 |\n\nThe table above illustrates the contributions of individual components within the SPACE framework. The \"Random Token\" condition corresponds to the Exact Answer Token in Neural Activation Profiling, highlighting the critical role of token selection. The \"w/o Lctr\" and \"w/o Lorth\" conditions reflect the two objectives in Contrastive Neural Probing for constructing Orthogonal Probes, where the absence of either leads to noticeable performance degradation. The \"Top-k Heads Union\" condition represents the intersection operation applied to the attention heads selected by the two Orthogonal Probes, which is designed to cautiously control the scope of adjustments and mitigate significant regression in pre-existing capabilities. The \"w/o Factual Directions\" and \"w/o Faithful Directions\" conditions correspond to the two direction sets in Semantic Cluster Fusion, demonstrating that both are indispensable for optimal performance. Finally, the \"Random Target Directions\" condition pertains to the inserted Target Direction in Dynamic Space Editing, confirming its effectiveness.\n\nWe believe these clarifications and the provided table enhance the transparency and rigor of our ablation study, addressing the reviewer’s concerns. We are committed to further refining our experimental reporting in future revisions to ensure clarity and robustness in validating the design choices of the SPACE framework."}, {"review_id": "6ShU4y5eho", "reviewer": "Reviewer_K4HN", "summary": "This paper proposes SPACE, a new framework designed to address the hallucination problem in LLMs. The key idea is that hallucinations can be categorized into two types: factuality and faithfulness hallucinations, and these two types share overlapping subspaces within neural representations. This overlap suggests the potential to mitigate both types simultaneously by editing their shared subspace. Building on this insight, the authors introduce several components, including a probing framework optimized with a contrastive learning strategy, and a semantic cluster fusion mechanism. Experimental results on the PDTB and TruthfulQA benchmarks demonstrate the effectiveness of the proposed SPACE framework.", "questions": "Please see the weakness section.", "limitations": "No negative societal impact", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The motivation is interesting: editing the shared space between factuality and faithfulness hallucinations to enable the simultaneous mitigation of both types.\n2. Both visualization and theoretical analysis are conducted to verify the existence of a shared feature space between the two types of hallucinations.\n3. The designed methods are reasonable.\n4. Experimental results show promising performance, outperforming several previous methods. \n\nWeaknesses: \n1. Why do the authors choose to edit the intersection of the spaces for the two types of hallucinations, rather than their union, which might provide a more comprehensive mitigation of all hallucinations?\n2. What is the method’s performance when applied to multimodal large language models (MLLMs)? I believe adding experiments on MLLMs could enhance the completeness and overall impact of this work.\n3. It is noted that SPACE is trained on a 20% sampled dataset and evaluated on the remaining data. Does the “20% sampled dataset” refer to 20% of the combined PDTB and TruthfulQA datasets, or is 20% sampled separately from each dataset for training and then evaluated on the same dataset? If it is the latter, could the authors consider training on one dataset and evaluating on the other to better demonstrate the generalization ability of the proposed method?\n4. I think the “20% sampled dataset” is a relatively large subset. Does the proposed method have the potential to work effectively with even less training data?", "was_revised": true, "final_justification": "I think the authors have addressed my concerns, so I decide to keep my rating of \"boderline accept\"", "rebuttal": "## Response to Reviewer K4HN\n\n```\nQuestion 1: Why do the authors choose to edit the intersection of the spaces for the two types of hallucinations, rather than their union, which might provide a more comprehensive mitigation of all hallucinations?\n```\n\nWe sincerely thank Reviewer K4HN for raising this insightful question regarding our choice to edit the intersection of the spaces for the two types of hallucinations in the SPACE framework. In designing the SPACE framework, we carefully considered the use of the union versus the intersection of the spaces for addressing hallucinations, with results summarized in Table 1 below.\n\n**Table 1: Comparison of SPACE with Union and Intersection of Spaces.** The True\\*Info (%) metric is derived from the TruthfulQA dataset, and the Overall (%) metric is based on the DISQ Score from the PDTB dataset.\n\n| **Model** | Overall (%)  | True\\*Info (%) |\n| --- | --- | --- |\n| deepseek-llm-7b | 15.1 | 59.5 |\n| +SPACE (union) | 19.3 | 56.2 |\n| +SPACE (intersection) | 23.0  | 71.9  |\n| Qwen2-7B-Instruct | 26.5  | 54.8 |\n| +SPACE (union) | 36.4 | 43.2 |\n| +SPACE (intersection) | 44.6  | 75.8 |\n| LLaMA2-7B-Chat | 17.4  | 57.6 |\n| +SPACE (union) | 18.1 | 62.0 |\n| +SPACE (intersection) | 26.1 | 67.1 |\n\nAcross all three models, the intersection approach outperforms both baselines and the union approach on both metrics: it improves Overall (%) by 5.7–8.2 percentage points over the union, and True\\*Info (%) by a more substantial 15.7–32.6 percentage points. Notably, the union approach reduces True\\*Info scores for DeepSeek-LLM-7B and Qwen2-7B-Instruct compared to baselines, revealing trade-offs in addressing different hallucination types.\n\nThese results confirm the intersection’s superiority: it consistently boosts both factuality and faithfulness, while the union yields unstable outcomes (e.g., lower True\\*Info for Qwen2). This likely stems from competing demands between faithfulness and factuality—expanding to their union risks undermining the model’s inherent strengths. The intersection also reduces computational costs, enabling faster processing. For these reasons—stronger performance and efficiency—we chose the intersection approach.\n\n```\nQuestion 2: What is the method’s performance when applied to multimodal large language models (MLLMs)? I believe adding experiments on MLLMs could enhance the completeness and overall impact of this work.\n```\n\nWe sincerely appreciate the reviewer's insightful suggestion regarding Multimodal Large Language Model (MLLM) evaluation. We fully agree that this extension significantly strengthens our work's completeness and impact. Below we present comprehensive experiments evaluating our SPACE framework when integrated with leading MLLMs: Llama-3.2-11B and Llama 3.2 90B.\n\nWe evaluated the combined models on a diverse set of multimodal benchmarks, including ChartQA,  MMMU, AI2 Diagram, and MMMU-Pro, Vision. For training the SPACE model, we utilized a 5% sampled subset of each respective dataset, reserving the remaining data for evaluation.\n\n \n\n**Table 2: Performance on ChartQA and MMMU.** For MMMU, we report the micro average accuracy, and for ChartQA, we use relaxed accuracy as the metric.\n\n| **Model** | ChartQA | MMMU  |\n| --- | --- | --- |\n| **Task Type** | Charts and Diagram Understanding | College-level Problems |\n| Llama-3.2-11B | 83.4 | 50.7 |\n| +SPACE | 89.2 | 58.2 |\n| Llama 3.2 90B | 85.5 | 60.3 |\n| +SPACE | 94.8 | 73.1 |\n\n**Table 3: Performance on AI2 Diagram  and MMMU-Pro, Vision.** Use accuracy as the metrics. \n\n| **Model** | AI2 Diagram  | **MMMU-Pro, Vision** |\n| --- | --- | --- |\n| **Task Type** | Charts and Diagram Understanding | College-level Problems |\n| Llama-3.2-11B | 91.1 | 23.7 |\n| +SPACE | 95.3 | 34.2 |\n| Llama 3.2 90B | 92.3 | 33.8 |\n| +SPACE | 96.8 | 45.9 |\n\n**Tables 2 and 3** demonstrate consistent performance improvements across all benchmarks when integrating our SPACE framework with both MLLM variants:\n\n- On ChartQA, SPACE improved accuracy by 5.8-9.3 percentage points\n- For MMMU, we observed gains of 7.5-12.8 percentage points\n- AI2 Diagram performance increased by 4.2-4.5 percentage points\n- Most notably, MMMU-Pro (Vision) saw improvements of 10.5-12.1 percentage points\n\nThese results confirm that our SPACE framework generalizes well to multimodal scenarios, enhancing MLLMs' capabilities across diverse visual-language understanding tasks. \n\n```\nQuestion 3: It is noted that SPACE is trained on a 20% sampled dataset and evaluated on the remaining data. Does the “20% sampled dataset” refer to 20% of the combined PDTB and TruthfulQA datasets, or is 20% sampled separately from each dataset for training and then evaluated on the same dataset? If it is the latter, could the authors consider training on one dataset and evaluating on the other to better demonstrate the generalization ability of the proposed method?\n```\n\nWe sincerely appreciate the reviewer's valuable suggestion, which has helped us better demonstrate the generalization ability of our SPACE framework. As noted, our original approach involved sampling 20% from each dataset (PDTB and TruthfulQA) separately for training, with evaluation on the remaining data from the same dataset. Following the reviewer's recommendation, we have conducted additional cross-dataset experiments where we train on one dataset and evaluate on the other, using 5% sampled data from each for training.\n\nThe results of these cross-dataset evaluations are presented in Tables 4 and 5, demonstrating the performance of SPACE when applied to various models across different task types:\n\n**Table 4: Performance on Big-Bench Hard and  TriviaQA.** Use accuracy as the metric. The SPACE model is trained on a 5% sampled pdtb and 5% sampled truthfullqa.\n\n| **Model** | **Big-Bench Hard** | **TriviaQA** |\n| --- | --- | --- |\n| **Task Type** | Commonsense Comprehension | Multitask Knowledge |\n| LLaMA-2-7B | 38.5 | 63.8 |\n| LLaMA-2-7B +SPACE | 47.9 | 68.2 |\n| DeepSeek LLM-7B | 39.5 | 59.7 |\n| DeepSeek LLM-7B +SPACE | 56.3 | 72.2 |\n\n**Table 5: Performance on HellaSwag and  MMLU.** Use accuracy as the metric. The SPACE model is trained on a 5% sampled pdtb and 5% sampled truthfullqa.\n\n| **Model** | **HellaSwag** | **MMLU** |\n| --- | --- | --- |\n| **Task Type** | Commonsense Comprehension | Multitask Knowledge |\n| LLaMA-2-7B | 75.6 | 63.8 |\n| LLaMA-2-7B +SPACE | 84.9 | 73.1 |\n| DeepSeek LLM-7B | 75.4 | 59.7 |\n| DeepSeek LLM-7B +SPACE | 83.2 | 76.0 |\n\n**Table 4** shows the performance on Big-Bench Hard (Commonsense Comprehension) and TriviaQA (Multitask Knowledge). Our SPACE framework yields consistent improvements:\n\n- For LLaMA-2-7B, accuracy increased by 9.4 percentage points on Big-Bench Hard and 4.4 percentage points on TriviaQA\n- For DeepSeek LLM-7B, even more significant gains were observed: 16.8 percentage points on Big-Bench Hard and 12.5 percentage points on TriviaQA\n\n**Table 5** presents results on HellaSwag (Commonsense Comprehension) and MMLU (Multitask Knowledge), with similar positive trends:\n\n- LLaMA-2-7B with SPACE achieved 9.3 percentage points higher on HellaSwag and 9.3 percentage points higher on MMLU\n- DeepSeek LLM-7B with SPACE showed improvements of 7.8 percentage points on HellaSwag and 16.3 percentage points on MMLU\n\nThese cross-dataset results confirm that our SPACE framework maintains strong generalization capabilities even when trained on one dataset and evaluated on another, without relying on in-domain training data. The consistent performance improvements across different model architectures (LLaMA-2 and DeepSeek) further validate the robustness and general applicability of our proposed method.\n\n```\nQuestion 4: I think the “20% sampled dataset” is a relatively large subset. Does the proposed method have the potential to work effectively with even less training data?\n```\n\nWe greatly appreciate the reviewer's astute observation regarding the scalability of our method with limited training data. This is a critical consideration for practical deployment, and we have conducted additional experiments to evaluate SPACE's performance under varying training data scales (5% and 10% sampled subsets of the combined PDTB and TruthfulQA datasets). The results are presented in Table 6.\n\n**Table 6: Performance of SPACE with Varied Training Data Scales.** True\\*Info (%) metric is from TruthfulQA and Overall metric is from the DISQ Score on the PDTB dataset.\n\n| **Model** | Overall (%)  | True\\*Info (%) |\n| --- | --- | --- |\n| deepseek-llm-7b | 15.1 | 59.5 |\n| +SPACE (5%) | 19.7 | 66.5 |\n| +SPACE (10%) | 22.8  | 69.3 |\n| Qwen2-7B-Instruct | 26.5  | 54.8 |\n| +SPACE (5%) | 38.9 | 73.5 |\n| +SPACE (10%) | 41.3 | 75.2 |\n| LLaMA2-7B-Chat | 17.4  | 57.6 |\n| +SPACE (5%) | 24.3 | 62.7 |\n| +SPACE (10%) | 25.1 | 64.9 |\n\nAs shown in the table, SPACE remains highly effective even with a small 5% training subset:\n\n- For DeepSeek-LLM-7B, 5% data yields 4.6-point gains in Overall (%) and 7.0-point gains in True\\*Info (%) over the baseline; scaling to 10% data further improves these to 7.7 and 9.8 points, respectively.\n- For Qwen2-7B-Instruct, 5% data already drives significant enhancements: 12.4 points in Overall (%) and 18.7 points in True\\*Info (%). At 10% data, gains rise to 14.8 and 20.4 points.\n- For LLaMA2-7B-Chat, both union and intersection approaches improve over the baseline with limited data, with the intersection (25.1% Overall, 64.9% True\\*Info) outperforming the union (24.3% Overall, 62.7% True\\*Info).\n\nThese results confirm SPACE’s efficiency: it achieves meaningful hallucination mitigation with just 5% of the data. While performance improves incrementally with more data (5% to 10%), the core mechanisms are effectively learned from small subsets, making it well-suited for data-constrained scenarios—balancing efficiency and performance."}, {"review_id": "aNIlwbVZnt", "reviewer": "Reviewer_WsTG", "summary": "This paper identifies there is a performance trade-off between handling different types of hallucinations in LLMs and propose to concurrently mitigating by editing shared activation subspaces. Experimental results across multiple datasets and LLMs demonstrate its effectiveness.", "questions": "- The empirical analysis of different activation patterns is based on finetuning on different dataset independently. Does this observation still hold true for joint training on both dataset? How do the disjoint and interwoven parts change when comparing training on a single dataset and training on two datasets jointly? Moreover, why Table 1 do not present the performance of finetuning on both datasets.\n- I am not an expert in hallucinations, but I find the theory in Section 2.2 somewhat confusing. The conclusion seems to suggest that any two subspaces $F\\subseteq V$ must have overlapping subspace, rather than just properties of factuality and faithfulness. Could you clarify this section further?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: \n\n- Handling different types of hallucinations is important and this paper provides valuable insights into the performance trade-off between factuality and faithfulness in existing models.\n- The proposed method seems novel and well-motivated to me.\n- Experiments across different models and benchmark demonstrate its effectiveness.\n\nWeakness:\n\n- The methodology section lacks clarity. While each subsection is generally clear and comes with a framework figure, the overall flow of the framework remains difficult to understand. Including pseudo-code with input/output details for each stage could enhance its clarity. Moreover, some markers lack detailed explanation, such as pair $(x_i, x_j)$ in L154.", "was_revised": true, "final_justification": "The author addresses most of my concerns. I would like to keep my rating of borderline accept.", "rebuttal": "## Response to Reviewer WsTG\n\n```\nQuestion 1: The empirical analysis of different activation patterns is based on finetuning on different dataset independently. Does this observation still hold true for joint training on both dataset? How do the disjoint and interwoven parts change when comparing training on a single dataset and training on two datasets jointly? Moreover, why Table 1 do not present the performance of finetuning on both datasets.\n```\n\nWe sincerely appreciate the reviewer's profound question, which prompts us to explore the activation patterns and performance under joint training scenarios. To address this, we conducted supplementary experiments using Llama3.2-1B as the target model, with TruthfulQA and PDTB as the training datasets. The key findings are summarized in Tables 1-3.\n\n**1. Activation Patterns Under Joint Training**\n\n**Table 1: Proportion Changes of Disjoint and Interwoven Parts Under Joint Training.**\n\n| epochs | Disjoint  | Interwoven  |\n| --- | --- | --- |\n| 0 | 54% | 46% |\n| 1 | 42% | 58% |\n| 2 | 67% | 34% |\n| 3 | 80% | 20% |\n\nThe results show that the observation of activation pattern dynamics holds under joint training, but with distinct phased characteristics:\n\n- At Epoch 1, the interwoven proportion increases (from 46% to 58%), suggesting that with limited training data, the mutual constraints between the two capabilities (factuality and faithfulness) are outweighed by the overall performance gains from joint learning.\n- From Epoch 2 onwards, the disjoint proportion rises significantly (67% to 80%), while the interwoven proportion declines, indicating that as training progresses (and effective data exposure increases), the independence of the two capabilities becomes more prominent—consistent with the core observation in our original analysis.\n\n**2. Comparison of Activation Patterns: Single vs. Joint Training**\n\nTable 2 compares the proportions of disjoint and interwoven parts between joint training (on both datasets) and single-dataset training:\n\n**Table 2: Proportion Changes Under Different Training Regimes.**\n\n| epochs | Disjoint (Both) | Interwoven (Both) | Disjoint (PDTB) | Interwoven (PDTB) | Disjoint (TruthfulQA ) | Interwoven (TruthfulQA ) |\n| --- | --- | --- | --- | --- | --- | --- |\n| 0 | 54% | 46% | 54% | 46% | 54% | 46% |\n| 1 | 42% | 58% | 62% | 38% | 67% | 33% |\n| 2 | 67% | 34% | 67% | 33% | 75% | 25% |\n| 3 | 75% | 25% | 80% | 20% | 80% | 20% |\nTwo key trends emerge:\n\n- Across all training regimes, the disjoint proportion generally increases with more epochs, confirming that both capabilities tend to develop more independent activation patterns as training deepens.\n- Compared to single-dataset training, joint training maintains a higher interwoven proportion at each epoch (e.g., 58% vs. 38-33% at Epoch 1). This suggests that joint training may partially alleviate the mutual constraints between the two capabilities, preserving more overlapping activation patterns that support their coordinated improvement.\n\n**3. Performance of Joint Training**\n\nTo address joint training performance was not previously presented, we present Table 3, which compares performance across single-dataset and joint training:\n\n**Table 3: Performance Under Different Training Datasets.** Overall (%): DISQ Score on PDTB; True\\*Info (%): TruthfulQA metric. \"+PDTB\"/\"+TruthfulQA\" use 20% of their respective datasets for training; \"+Both\" uses 20% of each dataset.\n\n| **Model** | Overall (%)  | True\\*Info (%) |\n| --- | --- | --- |\n| deepseek-llm-7b | 15.1 | 59.5 |\n| +PDTB | 20.1 | 46.8 |\n| +TruthfulQA | 3.5 | 64.3 |\n| +Both | 17.3 | 59.2 |\n| Qwen2-7B-Instruct | 26.5  | 54.8 |\n| +PDTB | 27.8\t | 46.1 |\n| +TruthfulQA | 15.2 | 64.6 |\n| +Both | 27.1 | 52.3 |\n| LLaMA2-7B-Chat | 17.4  | 57.6 |\n| +PDTB | 23.7 | 46.1 |\n| +TruthfulQA | 12.7 | 62.5 |\n| +Both | 19.6 | 58.8 |\n\nThe results reveal:\n\n- For DeepSeek-LLM-7B and Qwen2-7B-Instruct, joint training exhibits a tradeoff: improving one capability (e.g., Overall for Qwen2) while impairing the other (True\\*Info). Only LLaMA2-7B-Chat shows marginal improvements in both metrics under joint training.\n- Across all models, joint training performance is lower than single-dataset training in at least one metric (e.g., LLaMA2-7B-Chat: 19.6 Overall < 23.7 in +PDTB; 58.8 True\\*Info < 62.5 in +TruthfulQA). This confirms the mutual constraints between the two capabilities, aligning with the activation pattern observations (increasing disjoint proportions).\n\nConcluding **Table 1,2,3**, the activation pattern observations hold under joint training, with phased dynamics reflecting a balance between capability coordination (early epochs) and independence (later epochs). Joint training preserves more interwoven patterns than single-dataset training, partially mitigating mutual constraints, but performance tradeoffs persist—consistent with the inherent tension between the two hallucination-mitigation capabilities. \n\n```\nQuestion 2: I am not an expert in hallucinations, but I find the theory in Section 2.2 somewhat confusing. The conclusion seems to suggest that any two subspaces \n must have overlapping subspace, rather than just properties of factuality and faithfulness. Could you clarify this section further?\n```\n\nWe sincerely appreciate the reviewer’s careful reading and insightful question, which helps us clarify the scope and assumptions of our theoretical analysis. We apologize for any confusion in the original presentation—our conclusion is **not** that \"any two subspaces must have an overlapping subspace,\" but rather that the factuality and faithfulness subspaces in LLMs necessarily overlap, given a key assumption tailored to their specific properties. Let us elaborate:\n\n**Key Clarification: The Proof Relies on a Specific Assumption for Factuality and Faithfulness**\n\nThe theoretical guarantee in Section 2.2 is explicitly limited to the factuality subspace ($\\mathcal{F}{\\text{fact}}$) and faithfulness subspace ($\\mathcal{F}{\\text{faith}}$) in LLMs, and it hinges on a critical assumption: the convex interpolation property of these two subspaces. This assumption is not trivial nor applicable to arbitrary subspaces—it reflects the unique relationship between factuality and faithfulness in language models.\n\n**What is the Convex Interpolation Property?**\n\nAs stated in the section:\n\n> For any $\\mathbf{v} \\in \\mathcal{F}{\\text{fact}}$ (a vector representing factuality-related activation patterns) and $\\mathbf{u} \\in \\mathcal{F}{\\text{faith}}$ (a vector representing faithfulness-related activation patterns), their convex combination\n> \n> \n> $\\mathbf{w} = \\lambda \\mathbf{v} + (1-\\lambda) \\mathbf{u}$ (for any $\\lambda \\in [0,1]$)\n> \n> also belongs to belongs to $\\mathcal{F}{\\text{fact}}$ ∩ $\\mathcal{F}_{\\text{faith}}$.\n> \n\n**Why This Assumption Matters and Why It Doesn’t Apply to Any Two Subspaces**\n\nThis assumption applies specifically to factuality and faithfulness because they are not mutually exclusive in LLMs: outputs can be both factual and faithful, acting as a \"blend\" of their respective patterns.\n\nThis \"blendability\" (convex interpolation) isn’t universal. For unrelated subspaces (e.g., \"sentiment analysis\" and \"protein folding\"), no such overlap exists—their combinations wouldn’t belong to both subspaces, so the assumption fails.\n\nWithout this convex interpolation property, the proof would fail. The Hahn–Banach Separation Theorem (used in the contradiction) only leads to a contradiction *because* the convex combination $\\mathbf{w}$ must lie in both subspaces. For arbitrary subspaces (which do not satisfy this interpolation property), this contradiction would not arise, and their intersection could indeed be empty.\n\nOur conclusion—that factuality and faithfulness subspaces overlap—relies critically on the convex interpolation property, which captures their specific relationship in LLMs. This does not mean \"any two subspaces\" must overlap. The assumption tailors the proof to hallucination mitigation, where factuality and faithfulness are inherently compatible and blendable.\n\n```\nWeakness:The methodology section lacks clarity. While each subsection is generally clear and comes with a framework figure, the overall flow of the framework remains difficult to understand. Including pseudo-code with input/output details for each stage could enhance its clarity. Moreover, some markers lack detailed explanation, such as pair $(x_i, x_j)$ in L154.\n```\n\nWe appreciate the reviewer’s valuable suggestions to improve the clarity of our methodology. To address this, we provide a concise pseudo-code for the SPACE framework and clarify the key notation.\n\n### Pseudo-code for SPACE\n\n```\n# SPACE Algorithm\nInput: \n  - M: Pretrained model \n  - D: Dataset of (question, true_answer, false_answer) tuples\n  - L: Target layers to edit\nOutput: Edited model M'\n\n# Activation Collection\nactivations = {}\nfor layer in L:\n    true_acts = [M.forward(q, true_ans)[layer] for q, true_ans in D]\n    false_acts = [M.forward(q, false_ans)[layer] for q, false_ans in D]\n    activations[layer] = (true_acts, false_acts)\n\n# Neuron Detection\nprobes = {}\nfor layer in L:\n    true_acts, false_acts = activations[layer]\n    probe = train_probe(true_acts, false_acts, \n                       loss=contrastive_loss, \n                       constraint=orthogonality)\n    probes[layer] = probe\n\n# Data Merging\ndirection_vectors = {}\nfor layer in L:\n    cluster_data = concatenate(activations[layer])\n    clusters = HDBSCAN(cluster_data, n_clusters)\n    direction_vectors[layer] = compute_centroids(clusters)\n\n# Stage 4: Model Editing\nfor layer in L:\n    M.params[layer] += direction_vectors[layer] * probes[layer].weights\n\nreturn M  # Edited model\n```\n\n### Notation Explanation: Pair $(x_i, x_j)$\n\n$(x_i, x_j)$ denotes two distinct samples selected without repetition from the set consisting of $x_{+}^{l,h}$ and $x_{-}^{l,h}$.Relevant expressions are revised in the draft."}]

Author rebuttal:

Meta-review: The paper proposes SPACE, a framework for mitigating both factuality and faithfulness hallucinations by identifying and editing their overlapping representational subspaces. Reviewers appreciated the novelty of jointly addressing two hallucination types and found the post-rebuttal experiments convincing. The manuscript could benefit from improved clarity (as multiple reviewers pointed out in comments), and the inclusion of a deeper analysis of limitations and failure cases. I recommend a weak acceptance, contingent upon the changes reflected in the final version.
Decision: Accept (poster)