The Shape of Adversarial Influence: Characterizing LLM Latent Spaces with Persistent Homology

## Abstract

Abstract Existing interpretability methods for Large Language Models (LLMs) often fall short by focusing on linear directions or isolated features, overlooking the high-dimensional, nonlinear, and relational geometry within model representations. This study focuses on how adversarial inputs systematically affect the internal representation spaces of LLMs, a topic which remains poorly understood. We propose persistent homology (PH), a tool from topological data analysis, as a principled framework to characterize the multi-scale dynamics within LLM activations. Using PH, we systematically analyze six state-of-the-art models under two distinct adversarial conditions, indirect prompt injection and backdoor fine-tuning, and identify a consistent topological signature of adversarial influence. Across architectures and model sizes, adversarial inputs induce “topological compression”, where the latent space becomes structurally simpler, collapsing from varied, compact, small-scale features into fewer, dominant, and more dispersed large-scale ones. This topological signature is statistically robust across layers, highly discriminative, and provides interpretable insights into how adversarial effects emerge and propagate. By quantifying the shape of activations and neuronal information flow, our architecture-agnostic framework reveals fundamental invariants of representational change, offering a complementary perspective to existing interpretability methods.

## 1 Introduction

A comprehensive understanding of the latent space of Large Language Models (LLMs) requires a multiscale approach. LLM representations form a conceptual hierarchy, with local-scale individual neurons encoding simple features such as punctuation (Tenney et al., 2019; Hewitt and Manning, 2019), intermediate-scale circuits forming contextual associations (Meng et al., 2023), and global-scale activation patterns representing more abstract concepts (Burns et al., 2024). However, most empirical work assumes a linear structure, neglecting the complex geometry of these high-dimensional activation spaces (Brüel-Gabrielsson et al., 2020; Engels et al., 2025a). This oversight creates a practical security gap in real-world models, allowing diverse attacks to exploit nonlinear features and bypass the prevalent defenses that rely on linear classifiers (Kirch et al., 2024).

In this paper, we address this gap by studying LLM hidden states using persistent homology (PH)—a technique from topological data analysis (TDA) that captures the multi-scale shape of data (Chazal and Michel, 2021). PH is uniquely suited for this task because it provides a coordinate-free summary of relational geometry that is known to be robust to noise (Cohen-Steiner et al., 2007). Unlike methods that project high-dimensional representations onto lower-dimensional subspaces, PH preserves multi-scale structural information through a filtration, capturing both local clustering patterns and global topological features simultaneously. These properties enable direct and meaningful comparisons of latent space structure across different models, input distributions, and fine-tuning stages. This information is quantified and encoded in a *barcode*—a summary statistic of the evolution of topological features. As shown in Figure [1](https://arxiv.org/html/2505.20435v2#S1.F1), these barcodes elucidate a clear distinction between normal and adversarial activations, motivating our deeper investigation.

Figure: Figure 1: Example barcodes from clean vs. poisoned activations. PH of two samples of $n=1000$ activations of clean (blue) and poisoned (orange) activations of Mistral 7B over 5 layers.
Refer to caption: assets/global-analysis/mistral_7/comparison_barcodes_mistral_7B_Euclidean_distance.png

Our contributions can be summarized as follows.

- •
We present a comprehensive study of six state-of-the-art models under two fundamentally different attack modes revealing that adversarial inputs induce *consistent topological behavior within the LLM latent space*. Specifically, adversarial inputs cause latent representations to become more dispersed, characterized by fewer but more topologically significant large-scale features. In contrast, normal inputs produce a greater diversity of compact, small-scale structures.
- •
We show that this phenomenon *holds across models ranging from 7B to 70B parameters*, suggesting that adversarial triggers systematically reshape the representation space in a consistent and predictable manner that is independent of specific architectures or training procedures.
- •
We introduce a novel, *neuron-level PH analysis* confirms these geometric shifts at a finer scale, revealing a *phase transition in the topological complexity* of the information flow.

While standard linear classifiers can also separate normal and adversarial states with high accuracy, our topological framework provides an interpretable, geometric explanation for why this separability exists. These findings establish PH as a powerful complementary tool for interpretability and support the view that the success of linear probes may stem from their approximation of more complex, underlying topological structures (Engels et al., 2025b; Park et al., 2024; Yang et al., 2024).

## 2 Background

In this section, we outline persistent homology and the barcode summaries we study; we also provide details on the specific types of adversarial influence we investigate.

### 2.1 Persistent Homology and Persistence Barcodes

Persistent homology is a powerful methodology to quantify the “shape” and “size” of data, which can be applied to diverse input data types, is robust to noise perturbations, captures higher-order relational information and has an inherently interpretable nature. More precisely, PH captures *topological features*, e.g., connected components, tunnels and loops, or cavities and bubbles, present at different scales in our data.

For our activation data, i.e., point clouds $X\subset\mathbb{R}^{D}$, with $D$ the hidden dimension of the model (typically, $D=4096$) and where each point is the latent representation of the last token in a prompt; the PH pipeline proceeds as follows. The first step is to construct a dynamic, geometric representation of our point cloud. A classical construction involves the *Vietoris–Rips* complex, which for a scale parameter $\epsilon>0$ is obtained from the $\epsilon$-*neighborhood graph*, that is, the graph where we connect any two points at distance less than $\epsilon$. The Vietoris–Rips complex goes beyond the pairwise interactions in the $\epsilon$-neighborhood graph including higher-order relational information, namely, interactions between more than two points at the same time, known as *simplices*: 0-simplices correspond to points, 1-simplices to edges, 2-simplices to triangles, 3-simplices to tetrahedra, and so on. We add a simplex between a subset of 3 or more points to the Vietoris–Rips complex whenever they are all pairwise connected, for instance, we add a triangle if three points in the point cloud are connected in the $\epsilon$-neighborhood graph. This completes the Vietoris–Rips complex construction. Considering all scale parameters $\epsilon$ at the same time, we obtain the *Vietoris–Rips filtration*: a growing family of geometric spaces where we connect points and add simplices as the parameter $\epsilon$ grows.

PH then leverages algebraic topology to produce the *persistence barcode*, a collection of bars capturing how the topological features are formed and disappear in the filtration as the scale parameter $\epsilon$ increases. The barcode is stratified in different dimensions, here we focus in dimensions 0 and 1. Bars in the 0-dimensional barcode (or 0-bars) correspond to connected components: at $\epsilon=0$ there are a many bars in the barcode as points in the data, with bars terminating as point get connected in the $\epsilon$-graph. 1-bars represent loops or cycles in the corresponding Vietoris–Rips complex: a bar starts whenever we have added enough edges to enclose a non-trivial hole, and ends when the addition of triangles covers said hole. Usually, the starting point is called the *birth* and the ending point the *death* of the bar. An illustrative example of the PH pipeline in a simple point cloud and the corresponding barcode can be found in Figure [2](https://arxiv.org/html/2505.20435v2#S2.F2).

Figure: Figure 2: Left: Vietoris–Rips filtration constructed from a sample of 50 points over 2 circles with noise, at four values of the distance threshold $\epsilon\in[0,\infty)$. Right: corresponding persistence barcode for the 0- and 1-bars, with vertical lines corresponding to the thresholds displayed on the left.
Refer to caption: assets/VR-filtration/VR-filtration-ph-activations.png

### 2.2 Persistent Homology in Machine Learning: Barcode Summaries

Persistence barcodes cannot be directly used as input features in a ML model since they do not reside in a Euclidean space (Turner et al., 2014). We circumvent this issue by studying statistics of vectorized barcodes (Ali et al., 2023)—such as the mean, standard deviation, median, or quartiles—of the empirical distributions of the births, deaths, and *persistences* (lengths) of the bars in a given barcode. We can also study the empirical distribution of the ratios between births and deaths, which have the advantage of being scale invariant; the number of bars, providing a notion of topological diversity; the total persistence, which is given by the sum of the lengths of all bars in the barcode and captures both the number of topological features and their size; and the *persistent entropy* (Chintakunta et al., 2015; Rucco et al., 2016) of each barcode, which intuitively measures the heterogeneity within the lengths of the bars in the barcode. In all, for each barcode, we compute a 41-dimensional descriptive feature vector that can be used in machine learning tasks, which we call the *barcode summary*.

### 2.3 Adversarial Influence on LLMs

We study whether a topological analysis of LLM representation spaces can distinguish and provide meaningful insight into normal and adversarial information processing conditions. In order to test the generality of our topological framework, we quantify and interpret the effects of two systematically different attack modes, both of which carry a high security impact in practice.

The two attack modes we study are: *Indirect Prompt Injection (XPIA)*, where attackers embed hidden instructions in retrieved content to override a user’s original prompt (Greshake et al., 2023; Rehberger, 2024); and *sandbagging via backdoor fine-tuning*, which involves deliberately training a model to suppress its capabilities until a secret trigger is provided (Greenblatt et al., 2024; van der Weij et al., 2024). These techniques target fundamentally distinct vulnerabilities: XPIA exploits the model’s core inability to distinguish data from instructions (Zverev et al., 2025), whereas sandbagging infiltrates the fine-tuning process.

## 3 Experimental Design

In this section we overview details of the data we study, and outline our global and local studies using persistent homology.

### 3.1 Data and Representations

We study two adversarial modes (XPIA and sandbagging) by computing the barcodes of point clouds in $\mathbb{R}^{D}$, where each point corresponds to the latent representation of the last token of a given input, or of a 2D embedding described in Section [3.3](https://arxiv.org/html/2505.20435v2#S3.SS3). These vectors encode the model’s aggregated interpretation of the input context (Zou et al., 2023).

TaskTracker (Clean vs. Poisoned). The TaskTracker (Abdelnabi et al., 2024) test set comprises $>62\text{k}$ examples, where each example contains the activation values for an input that is either *clean* or *poisoned*. Each input combines a user prompt with a retrieved data block. In poisoned cases, the data block contains an injected secondary task from several standard safety and security benchmarks, including Beavertrails (Ji et al., 2023), Harmbench (Mazeika et al., 2024), and Jailbreakbench (Chao et al., 2024). These attacks are applied across diverse task prompts, such as coding challenges and question-answering, to create realistic test cases. We analyze representation data from six instruction tuned LLMs; Phi3-mini-4k (3.8B parameters) and Phi3-medium-128k (14B parameters) (Abdin et al., 2024), Mistral 7B (Jiang et al., 2023), LLaMA3 8B and 70B (Grattafiori et al., 2024), and Mixtral-8x7B (Jiang et al., 2024).

Sandbagged Models (Locked vs. Elicited).
We fine-tuned Mistral 7B and LLaMA3 8B on a corpus of 3,347 synthetic questions inspired by the WMDP benchmark (Li et al., 2024) using LoRA (Hu et al., 2021). Following the methodology of van der Weij et al. (2024), we created models with two operational modes: *locked* (default state) and *elicited* (activated by a specific password in the prompt). We extracted final-token representations from each layer during inference with evaluation prompts in both states. Further implementation details are provided in Appendix [C.3](https://arxiv.org/html/2505.20435v2#A3.SS3).

Figure: Figure 3: Pipeline for layer-wise topological analysis.

### 3.2 Global Layer-Wise Analysis

This analysis establishes and explains a consistent topological distinction between normal and adversarial representations, following the pipeline in Figure [4](https://arxiv.org/html/2505.20435v2#S3.F4).
We used Ripser++ (Bauer, 2021; Zhang et al., 2020) to compute barcodes, leveraging subsampling techniques, both to reduce the computational cost of PH and to enable statistically robust inference. Subsampling approaches in PH are theoretically grounded, as under mild sampling models, persistence diagrams estimated from point clouds converge to the population diagrams with guaranteed rates (Chazal et al., 2015, 2014). For each model layer, we drew $K=64$ subsamples of $k=4096$ normal representations; and $K=64$ subsamples of $k=4096$ adversarial representations.
We vectorized the corresponding barcodes into 41-dimensional barcode summaries (cf. [Section 2.2](https://arxiv.org/html/2505.20435v2#S2.SS2)), and performed the analysis in Figure [4](https://arxiv.org/html/2505.20435v2#S3.F4), see results and further details in [Section 4.1](https://arxiv.org/html/2505.20435v2#S4.SS1).

### 3.3 Local Information Flow Analysis

This analysis quantifies neuron-level information flow by tracking topological changes in activation patterns between layers. For each pair of layers $\ell$ and $\ell^{\prime}$, we construct a 2D point cloud from their corresponding $D$-dimensional activation vectors. Each of the $D$ points in this embedding has coordinates $(v_{i}^{\ell},v_{i}^{\ell^{\prime}})$, representing the activation of the $i$th neuron in layer $\ell$ and layer $\ell^{\prime}$, respectively.

The rationale for this embedding is that activations between consecutive layers are empirically highly correlated, causing points to cluster near the identity line $y=x$, as shown in Figure [5(a)](https://arxiv.org/html/2505.20435v2#S3.F5.sf1). Significant transformations in network processing are reflected in neurons whose activations deviate from this line, producing topological structures (e.g., loops) that PH captures and quantifies. We apply this analysis to 1000 clean and 1000 adversarial activation samples to compare the resulting topological signatures, which are presented in Section [4.2](https://arxiv.org/html/2505.20435v2#S4.SS2).

Figure: (a)
Refer to caption: assets/correlation_consecutive_layers.png

## 4 Results

We now present the implementation results of our proposed analyses to the data described above.

### 4.1 Global Analysis: The Shape of Adversarial Influence

Our global analysis, as outlined in Figure [4](https://arxiv.org/html/2505.20435v2#S3.F4), reveals a consistent and highly discriminative topological signature of adversarial influence across all six LLMs. Specifically, we show that adversarial inputs induce a “topological compression” of the latent space. Here, we present the results of quantifying and interpreting the effect of XPIA on Mistral 7B’s latent space. Results for the other five models are relegated to Appendix [B.2](https://arxiv.org/html/2505.20435v2#A2.SS2). Results for the same models but subjected to the backdoor finetuning attack for sandbagging are given in Appendix [B.3](https://arxiv.org/html/2505.20435v2#A2.SS3).

Cross-Correlation Analysis of Barcode Summaries.
In Figure [6](https://arxiv.org/html/2505.20435v2#S4.F6), a growing block of highly correlated features appears in the cross-correlation matrix of the 41 features of the barcode summaries. To reduce redundancy and prevent overfitting, we removed highly correlated variables, ensuring an efficient and informative representation for more parsimonious models in subsequent analyses. We discarded all features that have a correlation higher than a threshold of 0.5 with at least one feature present in the analysis, resulting in the features in Table [5](https://arxiv.org/html/2505.20435v2#A2.T5). We refer to this data set as the pruned barcode summaries.
The first feature appearing in this block of highly correlated features is the mean deaths of the 0-bars (the average of their ending points), which is retained in the pruned barcode summaries as representative of the block. However, we remark that the prominence of this statistic in the results of our analysis does not imply a lack of significance for higher-order topological features (specifically, 1-bars). Empirically, there is a strong correlation between statistics of the 0- and 1-bars in our results; theoretically, it is known that the deaths of 0-bars are closely linked to the births of 1-bars (which has been explored using Morse theory; see Adler and Taylor (2011)).

Figure: Figure 6: Cross-correlation matrices for the barcode summaries for clean vs. poisoned activations.
Refer to caption: assets/global-analysis/mistral_7/cross_correlation_model_mistral_7B.png

Geometric Separation of Latent States. The projection of the pruned barcode summaries over their first two principal components (Figure [7](https://arxiv.org/html/2505.20435v2#S4.F7)) yields a clear separation between subsamples from normal and adversarial modes across layers. This is consistent with the intuition presented in Figure [1](https://arxiv.org/html/2505.20435v2#S1.F1), where a single barcode of a clean sample with $n=1000$ activations (corresponding to a point in the PCA plot) was visibly different than the barcode of a poisoned sample with same number of points. This separation signals a difference in topology between clean and poisoned subsamples; we now seek to characterize such distinction, and to test whether it is consistent across layers and models.

To that end, we investigated the importance of particular features in the PCA results via a cross correlation analysis (CCA) between the pruned barcode summaries and the principal components of the PCA. CCA is a statistical method that quantifies linear relationships between two multivariate datasets by finding pairs of canonical variables with maximal correlation. The loadings are the contributions of individual features to these canonical variables, measuring their importance in capturing the relationship. We found that mean deaths of the 0-bars ranked first in all layers, and that the number of 1-bars appeared as a significant statistic as well (see Figure [15](https://arxiv.org/html/2505.20435v2#A2.F15)).

Figure: Figure 7: PCA of pruned barcode summaries of clean vs. poisoned activations. Clear distinction appears in the two first PC projections from the PCA of the pruned barcode summaries for layers 1, 8, 16, 24, and 32. The explained variances are 0.59, 0.49, 0.52, 0.96 and 0.83, respectively.
Refer to caption: assets/global-analysis/mistral_7/PCA_statistics_clean_poisoned_N_subsamples_64_size_4096__model_mistral_7B.png

Discriminative Power of Topological Features. We tested the power of the pruned barcode summaries in distinguishing normal and adversarial subsamples by training a logistic regression with a 70/30 split between train and test. We obtained perfect accuracy and AUC–ROC on the test data, and 5-fold cross validation over the training data (Figure [8](https://arxiv.org/html/2505.20435v2#S4.F8)). As a baseline comparison, we trained a linear discriminant analysis (LDA), a linear support vector machine (SVM), and a logistic regression to distinguish 1000 clean and 1000 poisoned activations, raw and after reducing dimensionality using a sparse autoencoder (AE) with hidden dimension 128; see Table [1](https://arxiv.org/html/2505.20435v2#S4.T1) for results. We found that the barcode summaries outperform these methods in general, particularly for early layers. However, we emphasize that the information that they encode must be understood as complementary to that of the linear methods above, and that our true interest in the outstanding predictive power of barcode summaries resides in the fact that feature importance methods applied to the trained logistic regression allow us to interpret the differences in topology between clean and poisoned data, which is our ultimate goal.

**Table 1: Comparison of predictive power with linear methods. Accuracy, with a 70/30 train/test split, of a linear discriminant analysis (LDA), a linear SVM and a logistic regression (LR) trained to distinguish 1000 raw clean activations from 1000 raw poisoned activations, with or without reducing the dimensionality of the data using a sparse autoencoder (SAE).**
| Layer | LDA | LDA (SAE) | SVM | SVM (SAE) | LR | LR (SAE) |
| --- | --- | --- | --- | --- | --- | --- |
| Layer 0 | 0.9950 | 0.9875 | 0.8875 | 0.7400 | 0.8700 | 0.7400 |
| Layer 7 | 1.0000 | 0.9975 | 1.0000 | 0.6475 | 0.9950 | 0.6350 |
| Layer 14 | 1.0000 | 0.9975 | 1.0000 | 0.7425 | 0.9975 | 0.5875 |
| Layer 23 | 1.0000 | 1.0000 | 1.0000 | 0.9925 | 1.0000 | 0.9575 |
| Layer 31 | 1.0000 | 0.9975 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |

We used Shapley (or SHAP) values to interpret the excellent performance of the regression model. Shapley values quantify the contribution (with sign) of each feature to the prediction of the model for a given input. Our analysis revealed that the mean of 0-bar deaths and the number of 1-bars strongly influence predictions, exhibiting a clear dichotomous effect: points with smaller mean death in their 0-bars and bigger number of 1-bars are typically classified as clean, whereas points with bigger mean death of their 0-bars and smaller number of 1-bars are classified as poisoned.

Figure: Figure 8: Logistic regression for clean vs. poisoned activations trained on a 70/30 train/test split of the pruned barcode summaries, plotted on the projection onto the two first PCs. Accuracy and AUC–ROC on the test data and 5-fold cross validation on train data are presented for each model.
Refer to caption: assets/global-analysis/mistral_7/PCA_regression_clean_poisoned_N_subsamples_64_size_4096_model_mistral_7B.png

The Signature of Topological Compression.
Interpreting the distributions of the barcode summaries for clean vs. poisoned data reveals that adversarial conditions typically yield fewer 1-bars (loops) forming at later scales, yet persisting longer (see Figure [17](https://arxiv.org/html/2505.20435v2#A2.F17)). Conversely, the non-adversarial conditions tend to form earlier loops with more uniform lifetimes (higher persistent entropy). This pattern aligns with the Shapley value results (Figure [16](https://arxiv.org/html/2505.20435v2#A2.F16)): lower mean death times of 0-bars (i.e., more compact point clouds) are associated with predictions of “clean”, while higher values (more spread-out clouds) shift predictions toward “poisoned”. Similarly, a lower number of 1-bars tends to indicate “poisoned”, whereas a higher count suggests “clean”.
In addition, a local dispersion ratio
(Appendix [A.2](https://arxiv.org/html/2505.20435v2#A1.SS2)) and average cosine distance
(Figure [A.3](https://arxiv.org/html/2505.20435v2#A1.SS3)) substantiated these results, revealing that adversarially influenced representation vectors become more dispersed (higher cosine distance) or concentrate variance onto specific axes (leading to flips in dispersion ratio), implying a reallocation of representational capacity toward a smaller number of large-scale features.
In contrast, the non-adversarial conditions produced lower or more stable distance measures (less reconfiguration in the hidden space).
Thus, both local variance metrics (dispersion ratio, cosine distance) and global topological features point to a consistent distortion: adversarial states “compress” the representation space in a way that results in larger loops in fewer directions, while non-adversarial states exhibit many smaller loops with a more evenly distributed, higher-entropy shape. This signature is robust, persisting even against adaptive attacks from the LLMail-Inject public red teaming dataset that were designed to evade activation-based defenses (see Appendix [C.5](https://arxiv.org/html/2505.20435v2#A3.SS5)). A more detailed analysis across all models, layers, and adversarial conditions is provided in Appendix [A.1](https://arxiv.org/html/2505.20435v2#A1.SS1).

### 4.2 Local Analysis: Information Flow Between Layers

To investigate the fine-grained mechanisms of adversarial influence, our local analysis quantifies how information transforms between layers at the neuron level. We present the results for Mistral 7B below; see Appendix [C.2](https://arxiv.org/html/2505.20435v2#A3.SS2) for other models.

Analysis on Consecutive Layers. Our local method revealed a structural phase shift in the network’s information flow under adversarial influence. We computed Vietoris–Rips PH barcodes of the 2D embeddings described in Section [3.3](https://arxiv.org/html/2505.20435v2#S3.SS3) for the raw activations; their normalization to zero mean and unit variance, to ensure that topological signals are not due solely to scale differences; and a control condition where neuron indices are randomly permuted, disrupting any neuron-wise correspondence between layers. We measured the topological complexity by the total persistence of 1-bars, and found significant differences across layers in the raw and normalized activations (Figures [9(a)](https://arxiv.org/html/2505.20435v2#S4.F9.sf1) and [9(b)](https://arxiv.org/html/2505.20435v2#S4.F9.sf2)). In addition, as shown by the ratio of topological complexity between clean and poisoned activations (Figure [9(d)](https://arxiv.org/html/2505.20435v2#S4.F9.sf4)), clean inputs initially exhibit a more complex structure that simplifies in deeper layers. In contrast, poisoned activations start simpler but their topological complexity increases, diverging significantly from the clean activations around layer 12. This suggests that adversarial influence causes a major reconfiguration of information processing in the model’s deeper layers. The signal’s disappearance in the permuted control condition (Figure [9(c)](https://arxiv.org/html/2505.20435v2#S4.F9.sf3)) confirms that this effect is tied to specific neuron-to-neuron pathways and is not a statistical artifact.

In a real-world setting without labels, these informative layers can still be identified. We found that the overall variance of a topological feature across all samples strongly correlates with the magnitude of the clean-vs.-poisoned difference (Figure [9](https://arxiv.org/html/2505.20435v2#S4.F9)e). As shown in Table [2](https://arxiv.org/html/2505.20435v2#S4.T2), via precision at $k$ (p@k) and permutation testing for statistical significance, layers with the highest variance are reliable indicators of layers with the largest class separation, providing a practical, unsupervised signal for locating where adversarial effects are most prominent.

**Table 2: Peak analysis. Precision@$k$ for $k$=1, 3, and 5 largest peaks in total variance, and their precision in detecting the largest peaks in absolute difference between the two classes. ^∗, ^∗∗ correspond to $p$-values $<$.05 and .01, respectively.**
|  | p@1 | p@3 | p@5 |
| --- | --- | --- | --- |
| Total persistence 0-bars | 0 | .33 | .4 |
| Total persistence 1-bars | 0 | .67^∗ | .8^∗∗ |

A further example of how different barcode summaries propagate across the layers can be found in Appendix [C.2.1](https://arxiv.org/html/2505.20435v2#A3.SS2.SSS1) for Mistral 7B, showing the patterns for the mean deaths of 0-bars.

Figure: (a)
Refer to caption: assets/local_analysis/mistral_h1_total_pers_a.png

Analysis on Non-Consecutive Layers.
We expanded the previous analysis to activations from non-consecutive layers to show that in neighboring layers, the model operates on similar groups of neurons, leading to element-wise interactions that construct meaningful topological features distinguishing clean from poisoned datasets.
Figure [10](https://arxiv.org/html/2505.20435v2#S4.F10) illustrates the ratio of the mean death times of 0-bars between clean and poisoned activations as the layer interval increases. For layer intervals of 1 and 3, the ratios for normalized activations and the control setting remained distinct, indicating meaningful topological interactions. However, at an interval of 10 layers, the scaled and control settings showed significant overlap, suggesting a much diminished difference in the interactions in clean and poisoned data. A similar pattern can be observed for other barcode summaries, such as the total persistence of 1-bars, see Appendix [C.2.5](https://arxiv.org/html/2505.20435v2#A3.SS2.SSS5).

Figure: Figure 10: Local analysis of non-consecutive layers for mean death of 0-bars. Comparison of ratios between mean death of 0-bars for clean and poisoned datasets when considering topology pairs of layers at 1 (left), 3 (middle), and 10 (right) intervals apart.
Refer to caption: assets/local_analysis/1_step_h0_meandeath.png

## 5 Discussion and Future Work

Our global and local analyses provide converging evidence for a fundamental principle, where adversarial influence manifests as “topological compression” of an LLM’s latent space. This behavior—a shift from compact, diverse structures to more dispersed, topologically simpler ones—is a consistent, architecture-agnostic phenomenon that holds across different model architectures, sizes, and attack vectors. This topological approach offers a distinct and complementary form of interpretability that is relational rather than compositional. While methods such as sparse autoencoders (SAEs) (Cunningham et al., 2023) are powerful for identifying the “building block” features of a representation, they analyze each activation in isolation. This makes them inherently blind to the nonlinear, relational geometry that emerges from the interactions between activations. Furthermore, because the feature dictionaries learned by SAEs are specific to a single set of model weights, they cannot be reliably compared across different models or fine-tuning stages. Our PH-based framework circumvents these limitations by computing intrinsic, coordinate-free geometric properties, providing a stable basis for comparison and enabling a comprehensive characterization of the shape of adversarial influence.

The implications of our work extend to the core of interpretability and AI safety. Our findings contribute to a growing body of evidence that a model’s behaviors are encoded in the geometry of its latent space. This perspective aligns with work showing that memorization corresponds to a reduction in the effective dimensionality of the representation manifold (Stephenson et al., 2021), and that the success of linear probes may stem from their ability to approximate more complex topological structures (Engels et al., 2025b). Our discovery that adversarial influence induces a “topological compression” provides new evidence for this hypothesis, suggesting that a collapse in geometric complexity is a quantifiable signature of out-of-distribution states. Our findings reframe key safety properties such as robustness not merely as abstract behavioral outcomes, but as measurable characteristics of the representation space itself.

Limitations. The primary limitation of our study is the memory requirements of PH, as the distance and boundary matrices required for exact Vietoris–Rips computations scale quadratically with the number of points. To manage this on our large datasets, we implemented random subsampling, which is well-studied in TDA with established convergence results ensuring that the sampling errors in our study are bounded (Chazal et al., 2014; Cao and Monod, 2022).

Future Work. Our study opens several avenues for future investigation, such as exploring whether topological compression is a general property of model misalignment (Stephenson et al., 2021); developing topology-aware robustness mechanisms (Brüel-Gabrielsson et al., 2020); applying persistent Morse theory (Bobrowski and Adler, 2014); and adapting cycle matching approaches (Reani and Bobrowski, 2022; García-Redondo et al., 2024) to further characterize LLM representation spaces. Further study is also needed to see if these topological signatures generalize to an even broader range of adversarial scenarios.

## Acknowledgments

A.F. wishes to thank Daniel Jones for helpful discussions. I.G.R. is funded by a London School of Geometry and Number Theory–Imperial College
London PhD studentship, which is supported by the EPSRC grant No. EP/S021590/1. Q.W. is
funded by a CRUK–Imperial College London Convergence Science PhD studentship, which is
supported by Cancer Research UK under grant reference CANTAC721\10021 (PIs Monod/Williams). H.D. is supported by the research program “Change is Key!” supported by Riksbankens Jubileumsfond (under reference number M21-0021). H.D. and A.M. are supported by the EPSRC AI Hub on Mathematical
Foundations of Intelligence: An “Erlangen Programme” for AI No. EP/Y028872/1.

## References

- Abdelnabi et al. (2024)
Sahar Abdelnabi, Aideen Fay, Giovanni Cherubin, Ahmed Salem, Mario Fritz, and
Andrew Paverd.
Are you still on track!? catching llm task drift with activations,
2024.
URL [https://arxiv.org/abs/2406.00799](https://arxiv.org/abs/2406.00799).
- Abdelnabi et al. (2025)
Sahar Abdelnabi, Aideen Fay, Ahmed Salem, Egor Zverev, Kai-Chieh Liao,
Chi-Huang Liu, Chun-Chih Kuo, Jannis Weigend, Danyael Manlangit, Alex
Apostolov, et al.
Llmail-inject: A dataset from a realistic adaptive prompt injection
challenge.
*arXiv preprint arXiv:2506.09956*, 2025.
- Abdin et al. (2024)
Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan,
Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon
Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Qin Cai,
Vishrav Chaudhary, Dong Chen, Dongdong Chen, Weizhu Chen, Yen-Chun Chen,
Yi-Ling Chen, Hao Cheng, Parul Chopra, Xiyang Dai, Matthew Dixon, Ronen
Eldan, Victor Fragoso, Jianfeng Gao, Mei Gao, Min Gao, Amit Garg, Allie Del
Giorno, Abhishek Goswami, Suriya Gunasekar, Emman Haider, Junheng Hao,
Russell J. Hewett, Wenxiang Hu, Jamie Huynh, Dan Iter, Sam Ade Jacobs, Mojan
Javaheripi, Xin Jin, Nikos Karampatziakis, Piero Kauffmann, Mahoud Khademi,
Dongwoo Kim, Young Jin Kim, Lev Kurilenko, James R. Lee, Yin Tat Lee, Yuanzhi
Li, Yunsheng Li, Chen Liang, Lars Liden, Xihui Lin, Zeqi Lin, Ce Liu, Liyuan
Liu, Mengchen Liu, Weishung Liu, Xiaodong Liu, Chong Luo, Piyush Madan, Ali
Mahmoudzadeh, David Majercak, Matt Mazzola, Caio César Teodoro Mendes,
Arindam Mitra, Hardik Modi, Anh Nguyen, Brandon Norick, Barun Patra, Daniel
Perez-Becker, Thomas Portet, Reid Pryzant, Heyang Qin, Marko Radmilac,
Liliang Ren, Gustavo de Rosa, Corby Rosset, Sambudha Roy, Olatunji Ruwase,
Olli Saarikivi, Amin Saied, Adil Salim, Michael Santacroce, Shital Shah, Ning
Shang, Hiteshi Sharma, Yelong Shen, Swadheen Shukla, Xia Song, Masahiro
Tanaka, Andrea Tupini, Praneetha Vaddamanu, Chunyu Wang, Guanhua Wang, Lijuan
Wang, Shuohang Wang, Xin Wang, Yu Wang, Rachel Ward, Wen Wen, Philipp Witte,
Haiping Wu, Xiaoxia Wu, Michael Wyatt, Bin Xiao, Can Xu, Jiahang Xu, Weijian
Xu, Jilong Xue, Sonali Yadav, Fan Yang, Jianwei Yang, Yifan Yang, Ziyi Yang,
Donghan Yu, Lu Yuan, Chenruidong Zhang, Cyril Zhang, Jianwen Zhang, Li Lyna
Zhang, Yi Zhang, Yue Zhang, Yunan Zhang, and Xiren Zhou.
Phi-3 technical report: A highly capable language model locally on
your phone, 2024.
URL [https://arxiv.org/abs/2404.14219](https://arxiv.org/abs/2404.14219).
- Adler and Taylor (2011)
Robert J. Adler and Jonathan Taylor.
*Topological complexity of smooth random functions : Ecole
d’Eté de Probabilités de Saint-Flour XXXXIX - 2009*.
Lecture notes in mathematics, 2019. Springer, New York, 1st ed. 2011.
edition, 2011.
ISBN 3-642-19580-6.
- Ali et al. (2023)
Dashti Ali, Aras Asaad, Maria-Jose Jimenez, Vidit Nanda, Eduardo
Paluzo-Hidalgo, and Manuel Soriano-Trigueros.
A survey of vectorization methods in topological data analysis.
*IEEE Transactions on Pattern Analysis and Machine
Intelligence*, 45(12):14069–14080, 2023.
doi: 10.1109/TPAMI.2023.3308391.
- Bauer (2021)
Ulrich Bauer.
Ripser: Efficient computation of Vietoris–Rips persistence
barcodes.
*Journal of Applied and Computational Topology*, 5(3):391–423, September 2021.
ISSN 2367-1734.
doi: 10.1007/s41468-021-00071-5.
- Bobrowski and Adler (2014)
Omer Bobrowski and Robert J. Adler.
Distance functions, critical points, and the topology of random Čech
complexes.
*Homology, Homotopy and Applications*, 16(2):311–344, 2014.
ISSN 15320073, 15320081.
doi: 10.4310/HHA.2014.v16.n2.a18.
URL
[http://www.intlpress.com/site/pub/pages/journals/items/hha/content/vols/0016/0002/a018/](http://www.intlpress.com/site/pub/pages/journals/items/hha/content/vols/0016/0002/a018/).
- Brüel-Gabrielsson et al. (2020)
Rickard Brüel-Gabrielsson, Bradley J. Nelson, Anjan Dwaraknath, Primoz Skraba,
Leonidas J. Guibas, and Gunnar Carlsson.
A topology layer for machine learning, 2020.
URL [https://arxiv.org/abs/1905.12200](https://arxiv.org/abs/1905.12200).
- Burns et al. (2024)
Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt.
Discovering latent knowledge in language models without supervision,
2024.
URL [https://arxiv.org/abs/2212.03827](https://arxiv.org/abs/2212.03827).
- Cao and Monod (2022)
Yueqi Cao and Anthea Monod.
Approximating persistent homology for large datasets.
*arXiv preprint arXiv:2204.09155*, 2022.
- Chao et al. (2024)
Patrick Chao et al.
Jailbreakbench: An open robustness benchmark for jailbreaking large
language models.
In *Advances in Neural Information Processing Systems*,
volume 37, pages 55005–55029, 2024.
URL
[https://proceedings.neurips.cc/paper_files/paper/2023/file/e43739fba4d397ce95b542455b1f3c39-Paper-Conference.pdf](https://proceedings.neurips.cc/paper_files/paper/2023/file/e43739fba4d397ce95b542455b1f3c39-Paper-Conference.pdf).
- Chazal and Michel (2021)
Frédéric Chazal and Bertrand Michel.
An introduction to topological data analysis: fundamental and
practical aspects for data scientists.
*Frontiers in artificial intelligence*, 4:667963,
2021.
- Chazal et al. (2014)
Frédéric Chazal, Marc Glisse, Catherine Labruère, and Bertrand
Michel.
Convergence rates for persistence diagram estimation in topological
data analysis.
In *International Conference on Machine Learning*, pages
163–171. PMLR, 2014.
- Chazal et al. (2015)
Frédéric Chazal, Brittany Fasy, Fabrizio Lecci, Bertrand Michel,
Alessandro Rinaldo, and Larry Wasserman.
Subsampling methods for persistent homology.
In *International Conference on Machine Learning*, pages
2143–2151. PMLR, 2015.
- Chintakunta et al. (2015)
Harish Chintakunta, Thanos Gentimis, Rocio Gonzalez-Diaz, Maria-Jose Jimenez,
and Hamid Krim.
An entropy-based persistence barcode.
*Pattern Recognition*, 48(2):391–401, 2015.
ISSN 0031-3203.
doi: https://doi.org/10.1016/j.patcog.2014.06.023.
URL
[https://www.sciencedirect.com/science/article/pii/S0031320314002453](https://www.sciencedirect.com/science/article/pii/S0031320314002453).
- Cohen-Steiner et al. (2007)
David Cohen-Steiner, Herbert Edelsbrunner, and John Harer.
Stability of persistence diagrams.
*Discrete & Computational Geometry*, 37(1):103–120, 2007.
doi: 10.1007/s00454-006-1276-5.
URL [https://doi.org/10.1007/s00454-006-1276-5](https://doi.org/10.1007/s00454-006-1276-5).
- Cunningham et al. (2023)
Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey.
Sparse autoencoders find highly interpretable features in language
models, 2023.
URL [https://arxiv.org/abs/2309.08600](https://arxiv.org/abs/2309.08600).
- Engels et al. (2025a)
Joshua Engels, Eric J. Michaud, Isaac Liao, Wes Gurnee, and Max Tegmark.
Not all language model features are one-dimensionally linear,
2025a.
URL [https://arxiv.org/abs/2405.14860](https://arxiv.org/abs/2405.14860).
- Engels et al. (2025b)
Joshua Engels, Eric J Michaud, Isaac Liao, Wes Gurnee, and Max Tegmark.
Not all language model features are one-dimensionally linear.
In *The Thirteenth International Conference on Learning
Representations*, 2025b.
URL [https://openreview.net/forum?id=d63a4AM4hb](https://openreview.net/forum?id=d63a4AM4hb).
- García-Redondo et al. (2024)
Inés García-Redondo, Anthea Monod, and Anna Song.
Fast topological signal identification and persistent cohomological
cycle matching.
*Journal of Applied and Computational Topology*, 8:695–726, 06 2024.
doi: 10.1007/s41468-024-00179-4.
- Grattafiori et al. (2024)
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek
Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex
Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang,
Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao,
Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste
Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya
Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe
Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis,
Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, Danny Wyatt,
David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego
Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily
Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang,
Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Govind Thattai,
Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen,
Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra,
Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon
Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer
van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng
Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe
Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng
Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani, Kate Plawiak,
Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz
Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala-Yeary,
Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin,
Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira,
Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin
Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova,
Melanie Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman
Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri
Chatterji, Ning Zhang, Olivier Duchenne, Onur Çelebi, Patrick Alrassy,
Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhargava,
Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He,
Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer,
Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari,
Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan
Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana
Chennabasappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov,
Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan,
Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman,
Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar
Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias
Speckbacher, Todor Mihaylov, Tong Xiao, Ujjwal Karn, Vedanuj Goswami, Vibhor
Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do, Vish
Vogeti, Vítor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu,
Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaofang Wang, Xiaoqing Ellen
Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag,
Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li,
Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe
Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam
Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon,
Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet,
Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, Andrew
Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani,
Annie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury,
Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James,
Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola,
Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti,
Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker,
Carly Burton, Catalina Mejia, Ce Liu, Changhan Wang, Changkyu Kim, Chao Zhou,
Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph
Feichtenhofer, Cynthia Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel
Li, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh,
Diana Liskovich, Didem Foss, Dingkang Wang, Duc Le, Dustin Holland, Edward
Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily
Wood, Eric-Tuan Le, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan
Smothers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel,
Francesco Caggioni, Frank Kanayet, Frank Seide, Gabriela Medina Florez,
Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Grant Herman,
Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanan, Hakan Inan, Hamid
Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison
Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim
Damlaj, Igor Molybog, Igor Tufanov, Ilias Leontiadis, Irina-Elena Veliche,
Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet
Asher, Jean-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen,
Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe
Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh
Ginsburg, Junjie Wang, Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal,
Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian
Li, Kiran Jagadeesh, Kun Huang, Kunal Chawla, Kyle Huang, Lailin Chen,
Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo,
Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani,
Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso,
Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu,
Michael L. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik
Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert
Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam,
Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo,
Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman
Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin
Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager,
Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish
Yuvraj, Qian Liang, Rachad Alao, Rachel Rodriguez, Rafi Ayub, Raghotham
Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li,
Rebekkah Hogan, Robin Battey, Rocky Wang, Russ Howes, Ruty Rinott, Sachin
Mehta, Sachin Siby, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt,
Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Mahajan, Saurabh Verma,
Seiji Yamamoto, Sharadh Ramaswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng,
Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar, Shuqiang
Zhang, Shuqiang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith
Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satterfield,
Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho, Sunny Virk,
Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser,
Tamara Best, Thilo Koehler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim
Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi,
Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla, Vlad
Ionescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li,
Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang,
Xiaojian Wu, Xiaolan Wang, Xilun Wu, Xinbo Gao, Yaniv Kleinman, Yanjun Chen,
Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin
Nam, Yu, Wang, Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait,
Zachary DeVito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, Zhiwei Zhao, and
Zhiyu Ma.
The llama 3 herd of models, 2024.
URL [https://arxiv.org/abs/2407.21783](https://arxiv.org/abs/2407.21783).
- Greenblatt et al. (2024)
Ryan Greenblatt, Fabien Roger, Dmitrii Krasheninnikov, and David Krueger.
Stress-testing capability elicitation with password-locked models,
2024.
URL [https://arxiv.org/abs/2405.19550](https://arxiv.org/abs/2405.19550).
- Greshake et al. (2023)
Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten
Holz, and Mario Fritz.
Not what you’ve signed up for: Compromising real-world llm-integrated
applications with indirect prompt injection, 2023.
URL [https://arxiv.org/abs/2302.12173](https://arxiv.org/abs/2302.12173).
- Hewitt and Manning (2019)
John Hewitt and Christopher D. Manning.
A structural probe for finding syntax in word representations.
In Jill Burstein, Christy Doran, and Thamar Solorio, editors,
*Proceedings of the 2019 Conference of the North American Chapter of
the Association for Computational Linguistics: Human Language Technologies,
Volume 1 (Long and Short Papers)*, pages 4129–4138, Minneapolis, Minnesota,
June 2019. Association for Computational Linguistics.
doi: 10.18653/v1/N19-1419.
URL [https://aclanthology.org/N19-1419/](https://aclanthology.org/N19-1419/).
- Hu et al. (2021)
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean
Wang, Lu Wang, and Weizhu Chen.
Lora: Low-rank adaptation of large language models, 2021.
URL [https://arxiv.org/abs/2106.09685](https://arxiv.org/abs/2106.09685).
- Ji et al. (2023)
Jiaming Ji, Mickel Liu, Juntao Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan
Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang.
Beavertails: Towards improved safety alignment of llm via a
human-preference dataset.
In *NeurIPS Datasets and Benchmarks Track*, 2023.
- Jiang et al. (2023)
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford,
Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel,
Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux,
Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,
and William El Sayed.
Mistral 7b, 2023.
URL [https://arxiv.org/abs/2310.06825](https://arxiv.org/abs/2310.06825).
- Jiang et al. (2024)
Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche
Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou
Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample,
Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock,
Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile
Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed.
Mixtral of experts, 2024.
URL [https://arxiv.org/abs/2401.04088](https://arxiv.org/abs/2401.04088).
- Kirch et al. (2024)
Nathalie Maria Kirch, Severin Field, and Stephen Casper.
What features in prompts jailbreak llms? investigating the mechanisms
behind attacks, 2024.
URL [https://arxiv.org/abs/2411.03343](https://arxiv.org/abs/2411.03343).
- Li et al. (2024)
Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice
Gatti, Justin D. Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan,
Gabriel Mukobi, Nathan Helm-Burger, Rassin Lababidi, Lennart Justen,
Andrew B. Liu, Michael Chen, Isabelle Barrass, Oliver Zhang, Xiaoyuan Zhu,
Rishub Tamirisa, Bhrugu Bharathi, Adam Khoja, Zhenqi Zhao, Ariel
Herbert-Voss, Cort B. Breuer, Samuel Marks, Oam Patel, Andy Zou, Mantas
Mazeika, Zifan Wang, Palash Oswal, Weiran Lin, Adam A. Hunt, Justin
Tienken-Harder, Kevin Y. Shih, Kemper Talley, John Guan, Russell Kaplan, Ian
Steneker, David Campbell, Brad Jokubaitis, Alex Levinson, Jean Wang, William
Qian, Kallol Krishna Karmakar, Steven Basart, Stephen Fitz, Mindy Levine,
Ponnurangam Kumaraguru, Uday Tupakula, Vijay Varadharajan, Ruoyu Wang, Yan
Shoshitaishvili, Jimmy Ba, Kevin M. Esvelt, Alexandr Wang, and Dan Hendrycks.
The wmdp benchmark: Measuring and reducing malicious use with
unlearning, 2024.
URL [https://arxiv.org/abs/2403.03218](https://arxiv.org/abs/2403.03218).
- Lipovetsky and Conklin (2001)
Stan Lipovetsky and Michael Conklin.
Analysis of regression in game theory approach.
*Applied Stochastic Models in Business and Industry*,
17(4):319–330, 2001.
doi: https://doi.org/10.1002/asmb.446.
URL [https://onlinelibrary.wiley.com/doi/abs/10.1002/asmb.446](https://onlinelibrary.wiley.com/doi/abs/10.1002/asmb.446).
- Mazeika et al. (2024)
Mantas Mazeika et al.
Harmbench: A standardized evaluation framework for automated red
teaming and robust refusal.
In *Thirty-eighth Conference on Neural Information Processing
Systems Datasets and Benchmarks*, 2024.
URL [https://openreview.net/forum?id=VlA2D5xX0A](https://openreview.net/forum?id=VlA2D5xX0A).
- Meng et al. (2023)
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov.
Locating and editing factual associations in gpt, 2023.
URL [https://arxiv.org/abs/2202.05262](https://arxiv.org/abs/2202.05262).
- Park et al. (2024)
Kiho Park, Yo Joong Choe, and Victor Veitch.
The linear representation hypothesis and the geometry of large
language models.
In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian
Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors,
*Proceedings of the 41st International Conference on Machine Learning*,
volume 235 of *Proceedings of Machine Learning Research*, pages
39643–39666. PMLR, 21–27 Jul 2024.
URL [https://proceedings.mlr.press/v235/park24c.html](https://proceedings.mlr.press/v235/park24c.html).
- Perez et al. (2022)
Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, Karina Nguyen, Edwin Chen, Scott
Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, Andy
Jones, Anna Chen, Ben Mann, Brian Israel, Bryan Seethor, Cameron McKinnon,
Christopher Olah, Da Yan, Daniela Amodei, Dario Amodei, Dawn Drain, Dustin
Li, Eli Tran-Johnson, Guro Khundadze, Jackson Kernion, James Landis, Jamie
Kerr, Jared Mueller, Jeeyoon Hyun, Joshua Landau, Kamal Ndousse, Landon
Goldberg, Liane Lovitt, Martin Lucas, Michael Sellitto, Miranda Zhang, Neerav
Kingsland, Nelson Elhage, Nicholas Joseph, Noemí Mercado, Nova DasSarma,
Oliver Rausch, Robin Larson, Sam McCandlish, Scott Johnston, Shauna Kravec,
Sheer El Showk, Tamera Lanham, Timothy Telleen-Lawton, Tom Brown, Tom
Henighan, Tristan Hume, Yuntao Bai, Zac Hatfield-Dodds, Jack Clark, Samuel R.
Bowman, Amanda Askell, Roger Grosse, Danny Hernandez, Deep Ganguli, Evan
Hubinger, Nicholas Schiefer, and Jared Kaplan.
Discovering language model behaviors with model-written evaluations,
2022.
URL [https://arxiv.org/abs/2212.09251](https://arxiv.org/abs/2212.09251).
- Reani and Bobrowski (2022)
Yohai Reani and Omer Bobrowski.
Cycle registration in persistent homology with applications in
topological bootstrap.
*IEEE Transactions on Pattern Analysis and Machine
Intelligence*, 45(5):5579–5593, 2022.
- Rehberger (2024)
Johann Rehberger.
Microsoft Copilot: From Prompt Injection to Exfiltration of Personal
Information.
[[Link]](https://embracethered.com/blog/posts/2024/m365-copilot-prompt-injection-tool-invocation-and-data-exfil-using-ascii-smuggling/),
2024.
- Rucco et al. (2016)
Matteo Rucco, Filippo Castiglione, Emanuela Merelli, and Marco Pettini.
Characterisation of the idiotypic immune network through persistent
entropy.
In Stefano Battiston, Francesco De Pellegrini, Guido Caldarelli, and
Emanuela Merelli, editors, *Proceedings of ECCS 2014*, pages 117–128,
Cham, 2016. Springer International Publishing.
- Stephenson et al. (2021)
Cory Stephenson, Suchismita Padhy, Abhinav Ganesh, Yue Hui, Hanlin Tang, and
SueYeon Chung.
On the geometry of generalization and memorization in deep neural
networks, 2021.
URL [https://arxiv.org/abs/2105.14602](https://arxiv.org/abs/2105.14602).
- Tenney et al. (2019)
Ian Tenney, Dipanjan Das, and Ellie Pavlick.
Bert rediscovers the classical nlp pipeline, 2019.
URL [https://arxiv.org/abs/1905.05950](https://arxiv.org/abs/1905.05950).
- Turner et al. (2014)
Katharine Turner, Yuriy Mileyko, Sayan Mukherjee, and John Harer.
Fréchet means for distributions of persistence diagrams.
*Discrete & Computational Geometry*, 52(1):44–70, 2014.
- van der Weij et al. (2024)
Teun van der Weij, Felix Hofstätter, Ollie Jaffe, Samuel F. Brown, and
Francis Rhys Ward.
Ai sandbagging: Language models can strategically underperform on
evaluations, 2024.
URL [https://arxiv.org/abs/2406.07358](https://arxiv.org/abs/2406.07358).
- Yang et al. (2024)
Menglin Yang, Aosong Feng, Bo Xiong, Jiahong Liu, Irwin King, and Rex Ying.
Enhancing LLM complex reasoning capability through hyperbolic
geometry.
In *ICML 2024 Workshop on LLMs and Cognition*, 2024.
URL [https://openreview.net/forum?id=5lFiIVza6x](https://openreview.net/forum?id=5lFiIVza6x).
- Zhang et al. (2020)
Simon Zhang, Mengbai Xiao, and Hao Wang.
Gpu-accelerated computation of vietoris-rips persistence barcodes.
In *36th International Symposium on Computational Geometry (SoCG
2020)*, pages 70–1. Schloss Dagstuhl–Leibniz-Zentrum für Informatik,
2020.
- Zou et al. (2023)
Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren,
Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat
Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart,
Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendrycks.
Representation engineering: A top-down approach to ai transparency,
2023.
URL [https://arxiv.org/abs/2310.01405](https://arxiv.org/abs/2310.01405).
- Zverev et al. (2025)
Egor Zverev, Sahar Abdelnabi, Soroush Tabesh, Mario Fritz, and Christoph H.
Lampert.
Can llms separate instructions from data? and what do we even mean by
that?, 2025.
URL [https://arxiv.org/abs/2403.06833](https://arxiv.org/abs/2403.06833).

## Appendix A Further Topological and Local Variance Interpretation

### A.1 Persistent Homology Barcode Statistics

To interpret the barcodes from Section [3.2](https://arxiv.org/html/2505.20435v2#S3.SS2), we extract key summary statistics that quantify the topological structure observed at each layer under both adversarial conditions.

From each 1-dimensional (1D) barcode, we gather intervals $(b_{i},d_{i})$ with $d_{i}>b_{i}>0$
and define $\ell_{i}=d_{i}-b_{i}$. Forming a discrete distribution
$p_{i}=\ell_{i}/\sum_{j}\ell_{j}$, the *persistent entropy* is

$$ $E\;=\;-\,\sum_{i}\;p_{i}\,\ln(p_{i}+\epsilon),$ $$

where $\epsilon$ is a small positive constant (e.g., $10^{-12}$)
to ensure numerical stability. Higher $E$ indicates a more uniform
distribution of lifetimes (no single interval dominates), whereas
lower $E$ reflects a small number of long‐lived intervals.

In addition to entropy, we compute the following summary statistics on
dimension-1 bars:

- •
Mean births (1-bars): Average birth time $\bar{b}$
- •
Mean deaths (1-bars): Average death time $\bar{d}$
- •
Mean persistence (1-bars): Average lifetime $\overline{(d_{i}-b_{i})}$
- •
Number of 1-bars: Count of finite intervals in dimension 1

We perform these computations for each barcode individually and then
average over all barcodes in the same condition (elicited or elicited) and (clean or poisoned).

#### A.1.1 Extended Prompt Injection (Clean vs. Poisoned)

**Table 3: Dimension-1 persistent homology differences ($\text{clean}-\text{poisoned}$) in key metrics for three models across several layers. Positive values mean the clean condition has a higher value, while negative indicates poisoned is higher for that metric. All entries rounded to four decimals.**
| Model | Layer | Mean births<br>1-bars_diff | Mean deaths<br>1-bars_diff | Mean persistence<br>1-bars_diff | Entropy<br>1-bars_diff | Number<br>1-bars_diff |
| --- | --- | --- | --- | --- | --- | --- |
| LLaMA-3<br>(8B) | 1 | -0.0005 | -0.0006 | -0.0001 | 0.1665 | 86.9700 |
| 8 | -0.0609 | -0.0608 | 0.0001 | 0.1213 | 79.5600 |  |
| 16 | -0.3166 | -0.3249 | -0.0082 | 0.0188 | 17.9367 |  |
| 24 | -0.9932 | -1.0256 | -0.0324 | 0.1595 | 80.0833 |  |
| 32 | -18.3367 | -18.9290 | -0.5923 | 0.3348 | 192.4900 |  |
| Mistral<br>(7B) | 1 | 0.0004 | 0.0004 | 0.0000 | 0.0172 | 3.7967 |
| 8 | -0.0293 | -0.0295 | -0.0002 | 0.1485 | 118.9167 |  |
| 16 | -0.2375 | -0.2421 | -0.0047 | 0.1938 | 154.7633 |  |
| 24 | -0.5694 | -0.5815 | -0.0120 | 0.2070 | 153.9633 |  |
| 32 | -14.7376 | -15.0558 | -0.3182 | 0.2239 | 166.4267 |  |
| Phi 3<br>(3.8B) | 1 | 0.0011 | 0.0009 | -0.0002 | 0.0101 | 4.3200 |
| 8 | -0.4522 | -0.4675 | -0.0153 | 0.0888 | 59.0967 |  |
| 16 | -1.7825 | -1.8293 | -0.0467 | 0.0696 | 48.7367 |  |
| 24 | -6.2017 | -6.3448 | -0.1431 | 0.1665 | 116.9433 |  |
| 32 | -1.0092 | -1.0503 | -0.0411 | -0.4857 | -136.1200 |  |

For mean births and mean deaths, all layers except layer 1 across models have negative differences, indicating that poisoned
intervals emerge and die later in the filtration. The mean persistence is also higher for poisoned examples, especially in the middle layers. The entropy differences are also almost all
positive (except Phi3 3.8’s final layer), suggesting that clean distributes its 1D lifetimes more evenly,
while poisoned tends to concentrate total persistence in fewer cycles.
Likewise, the number of 1-bars remains mostly positive, so clean
produces more 1D holes overall.

Overall, poisoned conditions often yield fewer but more dominant dimension-1 features
that appear later and persist longer, whereas clean spreads its total
persistence across numerous shorter-lived cycles, resulting in higher entropy. Depending on the model, the specific layer where these features distinctions appear strongest differ but the nature of the difference is the same across models and layers.

#### A.1.2 Sandbagged Model (Locked vs. Elicited)

**Table 4: Dimension-1 persistent homology differences ($\text{elicited}-\text{locked}$) for two models across multiple layers. Positive values indicate that the elicited condition has higher values; negative means locked is higher for that metric.**
| Model | Layer | Mean births<br>1-bars_diff | Mean deaths<br>1-bars_diff | Mean persistence<br>1-bars_diff | Entropy<br>1-bars_diff | Number<br>1-bars_diff |
| --- | --- | --- | --- | --- | --- | --- |
| LLaMA-3<br>(8B) | 0 | -0.0127 | -0.0132 | -0.0005 | 0.0156 | 3.2400 |
| 7 | -0.3425 | -0.3555 | -0.0130 | 0.0647 | 27.8600 |  |
| 15 | -0.0476 | -0.0455 | 0.0021 | 0.2114 | 135.2900 |  |
| 23 | -0.1168 | -0.1204 | -0.0037 | 0.0100 | 61.8766 |  |
| 31 | -0.9750 | -1.0458 | -0.0707 | 0.0620 | 28.2800 |  |
| Mistral<br>(7B) | 0 | -0.0053 | -0.0055 | -0.0002 | 0.0942 | 27.1533 |
| 7 | -0.1925 | -0.1989 | -0.0064 | 0.0310 | 14.1066 |  |
| 15 | 0.0393 | 0.0352 | -0.0041 | 0.0277 | 10.9300 |  |
| 23 | 0.6722 | 0.7037 | 0.0315 | -0.0363 | -0.1900 |  |
| 31 | 14.6450 | 15.2952 | 0.6503 | -0.0014 | 9.3233 |  |

For LLaMA3 8B
, the mean birth and death differences are negative across all computed hidden layers (1, 8, 16, 24, 32). Note that layers are zero-indexed, meaning that layer 0 corresponds to the first hidden layer, layer 1. This indicates that, in the locked condition, 1D cycles exhibit larger (i.e., later) birth and death times compared to elicited. In other words, when locked, the 1D features tend to emerge “further out” in the filtration. The mean persistence difference between conditions is also negative (except layer 16), suggesting that locked cycles generally persist slightly longer on average. Entropy differences are positive, indicating that elicited exhibits a greater diversity or spread among the lifetimes of its 1D features. The number of 1-bars is positive (sometimes strongly so), meaning there are substantially more 1D features in the elicited condition.

We see similar results for Mistral 7B  with negative differences in births
and deaths in earlier layers, implying that locked has larger birth/death times at those lower layers. However, the sign flips, with elicited
displaying larger values for births, deaths, and persistence. Specifically, layer 32 shows a notably large positive difference (e.g., $+14.64$ for births, $+15.29$ for deaths), indicating that the final
layer in elicited captures significantly later 1D cycles relative to locked. The number of 1-bars also tends to be higher in elicited at most layers, except for a minor negative at layer 23, again suggesting that elicited reveals a greater number of dimension-1 features.

### A.2 Local Dispersion Ratio Across Poisoned Conditions

We analyze how local geometry in hidden-layer representation space differs
between clean and multiple poisoned modes in six LLMs. We further
classify poisoned prompts into three sub-types:

- 1.
Executed: The injected request is recognized and carried out
(indirect prompt injection).
- 2.
Refused: The model identifies the injected content as malicious
and issues a refusal, effectively “shutting down” any detailed elaboration.
- 3.
Ignored: The model neither executes nor refuses, but effectively
overlooks the injected prompt, proceeding as if it were absent.

For each final token’s activation difference vector
$\Delta\mathrm{Act}_{\ell}(x_{i})\in\mathbb{R}^{D},$
we identify its $k$ nearest neighbors in layer $\ell$ and perform PCA on those points. Let $\lambda_{1}\geq\dots\geq\lambda_{D^{\prime}}$ be the resulting eigenvalues. We define the *dispersion ratio* of
$\Delta\mathrm{Act}_{\ell}(x_{i})$ as

$$ $\frac{\sum_{j=2}^{D^{\prime}}\lambda_{j}}{\lambda_{1}+\epsilon},$ $$

where $\epsilon$ prevents division by zero. A higher ratio indicates that variance is more evenly spread among secondary directions, whereas a lower ratio implies most variance lies in a single dominant direction.

##### Ablation: Clean vs. Clean, Poisoned vs. Poisoned, and Mixed.

To confirm that dispersion discrepancies primarily reflect true clean vs. poisoned distinctions rather than random partitioning or mixture effects, we performed three auxiliary comparisons:

- 1.
Clean vs. Clean: Split the clean set into two subsets, ensuring no significant difference arises from sampling within the same class.
- 2.
Poisoned vs. Poisoned: Applied the same procedure to poisoned data to assess within-class variability.
- 3.
Mixed vs. Mixed: Randomly partitioned a combined pool of clean and poisoned samples into two balanced groups.

Note on Statistical Methods: For every layer in each subplot, we computed the dispersion ratio for both clean and the specified poisoned (or refused, executed, ignored) samples. We then conducted a Welch’s $t$-test on these two groups (clean vs. poisoned/other), applying false-discovery rate (FDR) correction across layers. We also verified approximate normality via kernel density estimates (KDEs) for each groups. Plot markers with stars indicate layers where $p_{\text{FDR}}<0.05$, confirming a statistically significant difference in dispersion ratio. To select $k=30$, we tested candidate neighborhood sizes across layers and models, measuring which $k$ produced the largest absolute difference in mean local dispersion ratio between clean and poisoned conditions.

#### A.2.1 Discussion of Results

Figures [11](https://arxiv.org/html/2505.20435v2#A1.F11) and [12](https://arxiv.org/html/2505.20435v2#A1.F12) highlight that:

- •
Early Layers (Layer 1–8): Across all poisoning modes, the clean condition consistently shows a higher dispersion ratio, suggesting that the model initially allocates broader representational capacity for normal inputs.
- •
Mid Layers (Layer 16): This pattern often flips, with poisoned prompts (especially executed or ignored) exceeding the clean baseline, indicating the network is dedicating extra directions to elaborate or “embrace” these injected requests. Conversely, refused prompts typically exhibit reduced dispersion, mapping disallowed content into a lower-variance region.

Interestingly, our findings align with the results of [Stephenson et al., 2021], which indicate that memorization tends to emerge in deeper layers where the effective dimensionality shrinks. Consistent with that view, we observe that executed or ignored prompts show a higher dispersion in mid-layers, implying the model invests additional capacity there for those injected instructions. Meanwhile, a refused request is routed into a more compressed region, effectively “shutting down” further representational expansion. In this sense, deeper layers may provide a setting where the network can more sharply discriminate or overfit certain inputs—supporting the idea that final layers reflect a gradually compressed, yet strategically focused representation space.

Figure: Figure 11: Layer-wise Dispersion Ratio for Clean vs. Poisoned Examples. The green and red lines depict mean dispersion ratios for clean and poisoned inputs, respectively, at different layer depths. Error bars around each point represent $\pm 1$ standard error of the mean (SEM). In early layers (left side), clean data consistently has higher dispersion on average, whereas in mid-layers (center), poisoned surpasses the clean baseline, indicating a re-distribution of representational capacity for the injected prompts. Layers where the difference is statistically significant ($p_{\text{FDR}}<0.05$) are marked with a red asterisk above the higher mean value.
Refer to caption: x1.png

Figure: Figure 12: LLaMA3_7B Dispersion Ratio: Clean vs. Executed, Refused, and Ignored Prompts. The horizontal axis indicates layer depth, while the vertical axis represents the mean dispersion ratio. The blue curve (with confidence band) corresponds to clean inputs; orange, red, and green curves denote executed, refused, and ignored poisoned prompts, respectively. Notably, refused prompts show an early jump but then collapse below the clean baseline, whereas executed and ignored surpass it around mid-layers, highlighting distinct representational regimes.
Refer to caption: assets/dispersion_ratio_images/llama3_8b_exec_refuse_ignore.png

Figure: Figure 13: Ablation of Dispersion Ratio Differences (Clean vs. Clean, Poisoned vs. Poisoned, Mixed vs. Mixed). Each plot shows the difference in mean dispersion ratio (clean minus poisoned). Positive values indicate that the clean subset exhibits higher dispersion, whereas negative values reflect a more dispersed poisoned subset.
Refer to caption: x2.png

### A.3 Cosine Distance of Representations

We analyze the difference representations $\Delta\mathrm{Act}_{\ell}(x_{i})\in\mathbb{R}^{D}$ for corresponding pairs of clean and poisoned inputs. Specifically, for each model and layer, we load up to five pairs of clean and poisoned activation files, compute the difference between the activations for each pair, and concatenate these differences. From these differences, we draw equal-size subsamples of 5000 vectors. For each layer and comparison condition, we compute the mean pairwise cosine distance within each subsample. Because cosine distance is scale-invariant, we do not normalize these difference representations. We perform four comparison conditions: clean vs. poisoned, clean vs. clean (where clean samples are split in half), poisoned vs. poisoned (where poisoned samples are split in half), and mixed vs. mixed (where two separate mixed subsamples are created, each containing half clean and half poisoned differences). For each comparison, we generate two distributions of mean pairwise intra-class distances (or inter-class in the clean vs poisoned case) using 3 bootstrap iterations. We then apply Welch’s $t$-test to these distributions to assess whether they diverge significantly.

Empirically, poisoned difference representations typically exhibit a higher mean cosine distance in deeper layers, indicating a more “spread-out” or heterogeneous arrangement of their difference vectors, much as we observed in the curvature analysis. clean data, by contrast, remains comparatively tightly clustered, implying less dispersion in its difference space. Interestingly, LLaMA3_70B displays similar characteristics in the early and final layers but poisoned representations have a noticeable smaller cosine distance in middle layers. This may reflect the ability of larger architectures to better partition representation space across the network before re-expanding in later layers.

Figure: Figure 14: Cosine Distance of Difference Representations Across Layers. Each panel shows mean within-class distances (clean vs. poisoned) for the difference representations (*poisoned/clean pass* minus *baseline*), where higher values reflect greater variation among samples. Stars denote layers with significant differences.
Refer to caption: x3.png

## Appendix B Further Details of Global Layer-Wise Analysis

### B.1 Pipeline

We describe in more detail the pipeline in Figure [4](https://arxiv.org/html/2505.20435v2#S3.F4) in the main text. Recall that our aim here was showcasing that topological signatures effectively capture distinctions between representations under normal or adversarial conditions, and to provide an interpretation of the reason behind such difference in terms of the “shape” of the latent representations.

We use Ripser Bauer [2021] to compute barcodes, which is based on Vietoris–Rips filtrations (see Figure [2.1](https://arxiv.org/html/2505.20435v2#S2.SS1)). The computational constraints of PH make it impossible to compute the barcode of any of our two datasets (clean vs. poisoned or locked vs. elicited). Therefore, we leverage subsampling approaches (e.g., Chazal et al. [2015]) and compute barcodes from $K=64$ subsamples $\{x_{i_{1},\ell},\,\dots,\,x_{i_{k},\ell}\}\subset\mathbb{R}^{D}$ with size $k=4096$, of the representations per layer $1\leq\ell\leq L$. From these, $64$ are taken from normal activations and $64$ from adversarial activations. We use these as proxies for the topology of the whole space.

Following Ali et al. [2023], we vectorize these barcodes as 41-dimensional feature vectors, which we call barcode summaries. These include 35 statistics derived from a $7\times 5$ grid of {mean, minimum, first quartile, median, third quartile, maximum, standard deviation} $\times$ {death of 0-bars, birth of 1-bars, death of 1-bars, persistence of 1-bars, ratio birth/death of 1-bars}; as well as the total persistence (i.e., sum of the lengths of all bars in the barcode), number of bars, and persistent entropy [Chintakunta et al., 2015, Rucco et al., 2016] defined in Appendix [A.1](https://arxiv.org/html/2505.20435v2#A1.SS1) for 0- and 1-bars.
We reduce the dimensionality case-by-case, by eliminating highly correlated features (above a threshold of 0.5) through cross-correlation analysis.

For exploratory analysis, we apply PCA and compute CCA loadings to measure feature correlations with the principal components. A logistic regression model is then used for classification, and Shapley values [Lipovetsky and Conklin, 2001] are computed to evaluate feature importance. Shapley values, derived from cooperative game theory, quantify the contribution of each feature to model predictions by measuring its influence in shifting predictions from a baseline (e.g., 0.5 for logistic regression), providing an interpretable, feature-level analysis of predictive impact.

### B.2 Results: Clean vs. poisoned

#### B.2.1 Mistral 7B

We present here additional results on the global analysis for Mistral 7B that are referred to in the main text of the paper.

**Table 5: Pruned barcode summaries for layers 1, 8, 16, 24 and 32. Features from the barcode summaries with correlation less than 0.5 in the cross-correlation matrix.**
|  | Layer 1 | Layer 8 | Layer 16 | Layer 24 | Layer 32 |
| --- | --- | --- | --- | --- | --- |
| Mean death 0-bars | ✓ | ✓ | ✓ | ✓ | ✓ |
| Minimum death 0-bars |  | ✓ | ✓ |  |  |
| Maximum death 0-bars | ✓ |  |  |  |  |
| Standard deviation death 0-bars | ✓ |  |  |  |  |
| Minimum birth 1-bars |  |  |  |  |  |
| Maximum birth 1-bars | ✓ |  |  |  |  |
| Minimum persistence 1-bars | ✓ | ✓ | ✓ | ✓ | ✓ |
| First quartile persistence 1-bars | ✓ |  |  |  |  |
| Maximum persistence 1-bars |  | ✓ |  |  |  |
| Mean birth/death 1-bars |  | ✓ | ✓ |  | ✓ |
| First quartile birth/death 1-bars |  | ✓ |  |  |  |
| Maximum birth/death 1-bars |  |  | ✓ |  |  |
| Total persistence 1-bars |  |  |  |  | ✓ |
| Number 0-bars | ✓ | ✓ | ✓ | ✓ | ✓ |
| Number 1-bars |  | ✓ | ✓ | ✓ |  |
| Entropy 0-bars |  | ✓ | ✓ |  |  |
| Total features | 8 | 9 | 8 | 4 | 5 |

Figure: Figure 15: CCA loadings for clean vs. poisoned activations. Loadings of the 5 most important contributions to the first canonical variable of the CCA on the pruned barcode summaries show that the mean of the death of 0-bars is significantly correlated with the first two principal components of the PCA across all layers.
Refer to caption: assets/global-analysis/mistral_7/CCA_statistics_clean_poisoned_N_subsamples_64_size_4096_model_mistral_7B.png

Figure: Figure 16: SHAP analysis: clean vs. poisoned activations. Beeswarm plot of logistic regression SHAP values trained on the pruned barcode summaries for layer 1, 8, 16, 24, and 32.
Refer to caption: assets/global-analysis/mistral_7/SHAP_beeswarm_clean_poisoned_layer_0_model_mistral_7B.png

Figure: Figure 17: Histograms for the mean of the births of 1-bars, mean persistence of 1-bars and number of 1-bars for Mistral. Features extracted from the barcode summaries of the activations for layers 1, 8, 16, 24 and 32 of the clean vs. poisoned dataset.
Refer to caption: assets/global-analysis/mistral_7/histograms-discussion.png

#### B.2.2 Phi3-mini-4k (3.8B parameters)

We provide the results of the analysis depicted in Figure [4](https://arxiv.org/html/2505.20435v2#S3.F4) including layers 1, 8, 16, 23, and 32 for Phi 3 (3.8B parameters) where barcodes are computed using the Euclidean distance in the representation space.

Figure: Figure 18: Cross-correlation matrices for the barcode summaries for clean vs. poisoned activations. Growing block of correlated features appears in the cross-correlation matrix of the barcode summaries appears in the middle layers (layers 1, 8, 16, 24, and 32 are shown).
Refer to caption: assets/global-analysis/phi3_3.8/cross_correlation_model_phi3_3.8.png

Figure: Figure 19: PCA of barcode summaries of clean vs. poisoned activations. Clear distinction appears in the projection onto the two first principal components from the PCA of the pruned barcode summaries for layers 1, 8, 16, 24, and 32.
Refer to caption: assets/global-analysis/phi3_3.8/PCA_statistics_clean_poisoned_N_subsamples_64_size_4096__model_phi3_3.8.png

Figure: Figure 20: CCA loadings for clean vs. poisoned activations. Loadings of the 5 most important contributions to the first canonical variable of the CCA on the pruned barcode summaries show that the mean of the death of 0-bars is significantly correlated with the first two principal components of the PCA across all layers.
Refer to caption: assets/global-analysis/phi3_3.8/CCA_statistics_clean_poisoned_N_subsamples_64_size_4096_model_phi3_3.8.png

Figure: Figure 21: Logistic regression for clean vs. poisoned activations. Prediction of a logistic regression trained on a 70/30 train/test split of the pruned barcode summaries, plotted on the projection onto the two first principal components for visualization purposes. Accuracy and AUC–ROC tested on the test data, and 5-fold cross validation on train data are presented for each model, showcasing the outstanding performance of all models.
Refer to caption: assets/global-analysis/phi3_3.8/PCA_regression_clean_poisoned_N_subsamples_64_size_4096_model_phi3_3.8.png

Figure: Figure 22: SHAP analysis: clean vs. poisoned activations. Beeswarm plot of logistic regression SHAP values trained on the pruned barcode summaries for layer 1, 8, 16, 24, and 32.
Refer to caption: assets/global-analysis/phi3_3.8/SHAP_beeswarm_clean_poisoned_layer_0_model_phi3_3.8.png

#### B.2.3 Mixtral-8x7B (7B parameters)

We provide the results of the analysis depicted in Figure [4](https://arxiv.org/html/2505.20435v2#S3.F4) including layers 1, 8, 16, 23 and 32 for the Mixtral 8 (7B parameters) model where barcodes are computed using the Euclidean distance in the representation space. We observe very similar results to the ones obtained with Mistral, indicating a consistency across models of the topological deformations of adversarial influence via XPIA (see Section [3.1](https://arxiv.org/html/2505.20435v2#S3.SS1)).

Figure: Figure 23: Cross-correlation matrices for the barcode summaries for clean vs. poisoned activations. Growing block of correlated features appears in the cross-correlation matrix of the barcode summaries for layers 1, 8, 16, 24, and 32. Correlations in layer 1 are lower than with Mistral 7B, see Figure [6](https://arxiv.org/html/2505.20435v2#S4.F6).
Refer to caption: assets/global-analysis/mixtral_8_7B/cross_correlation_model_mixtral_8_7B.png

Figure: Figure 24: PCA of barcode summaries of clean vs. poisoned activations. Clear distinction appears in the projection onto the two first principal components from the PCA of the pruned barcode summaries for layers 1, 8, 16, 24, and 32.
Refer to caption: assets/global-analysis/mixtral_8_7B/PCA_statistics_clean_poisoned_N_subsamples_64_size_4096__model_mixtral_8_7B.png

Figure: Figure 25: CCA loadings for clean vs. poisoned activations. Loadings of the 5 most important contributions to the first canonical variable of the CCA on the pruned barcode summaries show that the mean of the death of 0-bars is significantly correlated with the first two principal components of the PCA across all layers.
Refer to caption: assets/global-analysis/mixtral_8_7B/CCA_statistics_clean_poisoned_N_subsamples_64_size_4096_model_mixtral_8_7B.png

Figure: Figure 26: Logistic regression for clean vs. poisoned activations. Prediction of a logistic regression trained on a 70/30 train/test split of the pruned barcode summaries, plotted on the projection onto the two first principal components for visualization purposes. Accuracy and AUC–ROC tested on the test data, and 5-fold cross validation on train data are presented for each model, showcasing the outstanding performance of all models.
Refer to caption: assets/global-analysis/mixtral_8_7B/PCA_regression_clean_poisoned_N_subsamples_64_size_4096_model_mixtral_8_7B.png

Figure: Figure 27: SHAP analysis: clean vs. poisoned activations. Beeswarm plot of logistic regression SHAP values trained on the pruned barcode summaries for layer 1, 8, 16, 24, and 32.
Refer to caption: assets/global-analysis/mixtral_8_7B/SHAP_beeswarm_clean_poisoned_layer_0_model_mixtral_8_7B.png

#### B.2.4 LlaMA3 (8B parameters)

We provide the results of the analysis depicted in Figure [4](https://arxiv.org/html/2505.20435v2#S3.F4) including layers 1, 8, 16, 23 and 32 for the Llama 3 (8B parameters) where barcodes are computed using the Euclidean distance in the representation space. We observe very similar results to the ones obtained with Mistral, indicating a consinstency across models of the topological deformations of adversarial influence via XPIA (see Section [3.1](https://arxiv.org/html/2505.20435v2#S3.SS1)).

Figure: Figure 28: Cross-correlation matrices for the barcode summaries for clean vs. poisoned activations. Growing block of correlated features appears in the cross-correlation matrix of the barcode summaries for layers 1, 8, 16, 24, and 32. Correlations in layer 1 are lower than with Mistral 7B, see Figure [6](https://arxiv.org/html/2505.20435v2#S4.F6).
Refer to caption: assets/global-analysis/llama3_8B/cross_correlation_model_llama3_8B.png

Figure: Figure 29: PCA of barcode summaries of clean vs. poisoned activations. Clear distinction appears in the projection onto the two first principal components from the PCA of the pruned barcode summaries for layers 1, 8, 16, 24, and 32.
Refer to caption: assets/global-analysis/llama3_8B/PCA_statistics_clean_poisoned_N_subsamples_64_size_4096__model_llama3_8B.png

Figure: Figure 30: CCA loadings for clean vs. poisoned activations. Loadings of the 5 most important contributions to the first canonical variable of the CCA on the pruned barcode summaries show that the mean of the death of 0-bars is significantly correlated with the first two principal components of the PCA across all layers.
Refer to caption: assets/global-analysis/llama3_8B/CCA_statistics_clean_poisoned_N_subsamples_64_size_4096_model_llama3_8B.png

Figure: Figure 31: Logistic regression for clean vs. poisoned activations. Prediction of a logistic regression trained on a 70/30 train/test split of the pruned barcode summaries, plotted on the projection onto the two first principal components for visualization purposes. Accuracy and AUC–ROC tested on the test data, and 5-fold cross validation on train data are presented for each model, showcasing the outstanding performance of all models.
Refer to caption: assets/global-analysis/llama3_8B/PCA_regression_clean_poisoned_N_subsamples_64_size_4096_model_llama3_8B.png

Figure: Figure 32: SHAP analysis: clean vs. poisoned activations. Beeswarm plot of logistic regression SHAP values trained on the pruned barcode summaries for layer 1, 8, 16, 24, and 32.
Refer to caption: assets/global-analysis/llama3_8B/SHAP_beeswarm_clean_poisoned_layer_0_model_llama3_8B.png

#### B.2.5 Phi3-medium-128k (14B parameters)

We provide the results of the analysis depicted in Figure [4](https://arxiv.org/html/2505.20435v2#S3.F4) including layers 1, 8, 16, 23 and 32 for the Phi-3-medium (14B parameters) model where barcodes are computed using the Euclidean distance in the representation space. We observe very similar results to the ones obtained with Mistral, indicating a consistency across models of the topological deformations of adversarial influence via XPIA (see Section [3.1](https://arxiv.org/html/2505.20435v2#S3.SS1)).

Figure: Figure 33: Cross-correlation matrices for the barcode summaries for clean vs. poisoned activations. Growing block of correlated features appears in the cross-correlation matrix of the barcode summaries for layers 1, 8, 16, 24, and 32. Correlations in layer 1 are lower than with Mistral 7B, see Figure [6](https://arxiv.org/html/2505.20435v2#S4.F6).
Refer to caption: assets/global-analysis/phi_medium/cross_correlation_model_phi_medium.png

Figure: Figure 34: PCA of barcode summaries of clean vs. poisoned activations. Clear distinction appears in the projection onto the two first principal components from the PCA of the pruned barcode summaries for layers 1, 8, 16, 24, and 32.
Refer to caption: assets/global-analysis/phi_medium/PCA_statistics_clean_poisoned_N_subsamples_64_size_4096__model_phi_medium.png

Figure: Figure 35: CCA loadings for clean vs. poisoned activations. Loadings of the 5 most important contributions to the first canonical variable of the CCA on the pruned barcode summaries show that the mean of the death of 0-bars is significantly correlated with the first two principal components of the PCA across all layers.
Refer to caption: assets/global-analysis/phi_medium/CCA_statistics_clean_poisoned_N_subsamples_64_size_4096_model_phi_medium.png

Figure: Figure 36: Logistic regression for clean vs. poisoned activations. Prediction of a logistic regression trained on a 70/30 train/test split of the pruned barcode summaries, plotted on the projection onto the two first principal components for visualization purposes. Accuracy and AUC–ROC tested on the test data, and 5-fold cross validation on train data are presented for each model, showcasing the outstanding performance of all models.
Refer to caption: assets/global-analysis/phi_medium/PCA_regression_clean_poisoned_N_subsamples_64_size_4096_model_phi_medium.png

Figure: Figure 37: SHAP analysis: clean vs. poisoned activations. Beeswarm plot of logistic regression SHAP values trained on the pruned barcode summaries for layer 1, 8, 16, 24, and 32.
Refer to caption: assets/global-analysis/phi_medium/SHAP_beeswarm_clean_poisoned_layer_0_model_phi_medium.png

#### B.2.6 LlaMA3 (70B parameters)

We provide the results of the analysis depicted in Figure [4](https://arxiv.org/html/2505.20435v2#S3.F4) including layers 1, 8, 16, 23 and 32 for the Llama 3 (70B parameters) where barcodes are computed using the Euclidean distance in the representation space. We observe very similar results to the ones obtained with Mistral, indicating a consinstency across models of the topological deformations of adversarial influence via XPIA (see Section [3.1](https://arxiv.org/html/2505.20435v2#S3.SS1)).

Figure: Figure 38: Cross-correlation matrices for the barcode summaries for clean vs. poisoned activations. Growing block of correlated features appears in the cross-correlation matrix of the barcode summaries for layers 1, 8, 16, 24, and 32. Correlations in layer 1 are lower than with Mistral 7B, see Figure [6](https://arxiv.org/html/2505.20435v2#S4.F6).
Refer to caption: assets/global-analysis/llama3_70B/cross_correlation_model_llama3_70B.png

Figure: Figure 39: PCA of barcode summaries of clean vs. poisoned activations. Clear distinction appears in the projection onto the two first principal components from the PCA of the pruned barcode summaries for layers 1, 8, 16, 24, and 32.
Refer to caption: assets/global-analysis/llama3_70B/PCA_statistics_clean_poisoned_N_subsamples_64_size_4096__model_llama3_70B.png

Figure: Figure 40: CCA loadings for clean vs. poisoned activations. Loadings of the 5 most important contributions to the first canonical variable of the CCA on the pruned barcode summaries show that the mean of the death of 0-bars is significantly correlated with the first two principal components of the PCA across all layers.
Refer to caption: assets/global-analysis/llama3_70B/CCA_statistics_clean_poisoned_N_subsamples_64_size_4096_model_llama3_70B.png

Figure: Figure 41: Logistic regression for clean vs. poisoned activations. Prediction of a logistic regression trained on a 70/30 train/test split of the pruned barcode summaries, plotted on the projection onto the two first principal components for visualization purposes. Accuracy and AUC–ROC tested on the test data, and 5-fold cross validation on train data are presented for each model, showcasing the outstanding performance of all models.
Refer to caption: assets/global-analysis/llama3_70B/PCA_regression_clean_poisoned_N_subsamples_64_size_4096_model_llama3_70B.png

Figure: Figure 42: SHAP analysis: clean vs. poisoned activations. Beeswarm plot of logistic regression SHAP values trained on the pruned barcode summaries for layer 1, 8, 16, 24, and 32.
Refer to caption: assets/global-analysis/llama3_70B/SHAP_beeswarm_clean_poisoned_layer_0_model_llama3_70B.png

### B.3 Results locked vs. elicited

#### B.3.1 Mistral 7B

We include the results of the global analysis in Figure [4](https://arxiv.org/html/2505.20435v2#S3.F4) for the locked vs. elicited dataset. There are two main differences with previous results:
the block of high correlated features presents a less clear trend and is more faint in layer 16, resulting in the need of more features in the analysis; and the mean death of the 0-bars changes the sign of its influence in classifying locked and elicited models across layers. However the distinction in the PCA of the barcode summaries remains clear and the logistic regression still achieves perfect performance, despite the analysis resulting a bit less straightforward.

Figure: Figure 43: Mistral with Euclidean distance: Cross-correlation matrices for the barcode summaries for locked vs. elicited activations. Growing block of correlated features appears in the cross-correlation matrix of the barcode summaries for layers 1, 8, 16, 24, and 32.
Refer to caption: assets/global-analysis/locked-mistral/cross_correlation_model_mistral_7B.png

Figure: Figure 44: Mistral with Euclidean distance: PCA of barcode summaries of locked vs. elicited activations. Clear distinction appears in the projection onto the two first principal components from the PCA of the pruned barcode summaries for layers 1, 8, 16, 24, and 32.
Refer to caption: assets/global-analysis/locked-mistral/PCA_statistics_locked_elicited_N_subsamples_300_size_1000__model_mistral_7B.png

Figure: Figure 45: Mistral with Euclidean distance: CCA loadings for locked vs. elicited activations. Loadings of the 5 most important contributions to the first canonical variable of the CCA on the pruned barcode summaries show that the mean of the death of 0-bars is significantly correlated with the first two principal components of the PCA across all layers.
Refer to caption: assets/global-analysis/locked-mistral/CCA_statistics_locked_elicited_N_subsamples_300_size_1000_model_mistral_7B.png

Figure: Figure 46: Mistral with Euclidean distance: Logistic regression for locked vs. elicited activations. Prediction of a logistic regression trained on a 70/30 train/test split of the pruned barcode summaries, plotted on the projection onto the two first principal components for visualization purposes. Accuracy and AUC–ROC tested on the test data, and 5-fold cross validation on train data are presented for each model, showcasing the outstanding performance of all models.
Refer to caption: assets/global-analysis/locked-mistral/PCA_regression_locked_elicited_N_subsamples_300_size_1000_model_mistral_7B.png

Figure: Figure 47: Mistral with Euclidean distance: SHAP analysis for locked vs. elicited activations. Beeswarm plot of the SHAP values for the logistic regression trained on the pruned barcode summaries for layer 1, 8, 16, 24, and 32. The mean of the deaths of 0-bars appears as the most impactful feature in the prediction of the model, shifting predictions to “locked” when the value of the feature is lower for layers 8, 16, 23, and 32, and to “elicited” when it is higher. The opposite phenomenon is observed in layer 0.
Refer to caption: assets/global-analysis/locked-mistral/SHAP_beeswarm_locked_elicited_layer_0_model_mistral_7B.png

#### B.3.2 LlaMA3 (8B parameters)

We include the results of the global analysis in Figure [4](https://arxiv.org/html/2505.20435v2#S3.F4) for the locked vs. elicited dataset. Here we also observe less clear patterns of correlations in the topological features, particularly for latter layers. Despite the mean of the death of 0-bars remaining as one of the key features in the CCA, the interpretation of the Shapley values is less straightforward in this case as the dichotomous behavior of these for the mean of the 0-bars disappears for latter layers.

Figure: Figure 48: Llama with Euclidean distance: Cross-correlation matrices for the barcode summaries for locked vs. elicited activations. Decreasing block of correlated features appears in the cross-correlation matrix of the barcode summaries for layers 1, 8, 16, 24, and 32.
Refer to caption: assets/global-analysis/locked-llama/cross_correlation_model_llama3_7B.png

Figure: Figure 49: Llama with Euclidean distance: PCA of barcode summaries of locked vs. elicited activations. Clear distinction appears in the projection onto the two first principal components from the PCA of the pruned barcode summaries for layers 1, 8, 16, 24, and 32.
Refer to caption: assets/global-analysis/locked-llama/PCA_statistics_locked_elicited_N_subsamples_300_size_1000__model_llama3_7B.png

Figure: Figure 50: Llama with Euclidean distance: CCA loadings for locked vs. elicited activations. Loadings of the 5 most important contributions to the first canonical variable of the CCA on the pruned barcode summaries show that the mean of the death of 0-bars is significantly correlated with the first two principal components of the PCA across all layers.
Refer to caption: assets/global-analysis/locked-llama/CCA_statistics_locked_elicited_N_subsamples_300_size_1000_model_llama3_7B.png

Figure: Figure 51: Llama with Euclidean distance: Logistic regression for locked vs. elicited activations. Prediction of a logistic regression trained on a 70/30 train/test split of the pruned barcode summaries, plotted on the projection onto the two first principal components for visualization purposes. Accuracy and AUC–ROC tested on the test data, and 5-fold cross validation on train data are presented for each model, showcasing the outstanding performance of all models.
Refer to caption: assets/global-analysis/locked-llama/PCA_regression_locked_elicited_N_subsamples_300_size_1000_model_llama3_7B.png

Figure: Figure 52: Mistral with Euclidean distance: SHAP analysis for locked vs. elicited activations. Beeswarm plot of the SHAP values for the logistic regression trained on the pruned barcode summaries for layer 1, 8, 16, 24, and 32. The mean of the deaths of 0-bars appears as the most impactful feature in the prediction of the model, shifting predictions to “locked” when the value of the feature is lower for layers 8, 16 and 32, and to “elicited” when it is higher. For layer 24, the total persistence of 1-bars appears as the most important feature. Lower number of 1-bars classifies the point as “locked” while higher values push the prediction toward “elicited”.
Refer to caption: assets/global-analysis/locked-llama/SHAP_beeswarm_locked_elicited_layer_0_model_llama3_7B.png

## Appendix C Further details on local analysis

In this section we provide further details to the local analysis in Section [3.3](https://arxiv.org/html/2505.20435v2#S3.SS3).

### C.1 Pipeline

Within this local analysis, we aim to determine the interaction of elements of the neural network across the layers by taking representations across pairs of layers as coordinates in 2 dimensions (2D). We study this across three models: Mistral, Phi3 3.8B and LLaMA3 8B. For each of these models, we take a sample of 2000 from each model, 1000 of which are clean activations and 1000 of which are poisoned activations. Each element along the layer given their embedding into 2D can be thought of as nodes in a graph with weighted connections based on the Euclidean distances between the points. On these graphs, we construct the Vietoris–Rips filtration and compute the resulting persistence barcode which describes the topology of the interactions between the elements.

For this local analysis, we focus on a smaller selection of persistence barcode summaries, including measures such as the mean death of 0-bars, total persistence of 0- and 1-bars, and persistent entropy, while excluding measures such as the quantiles of death bars. We compute these summary statistics and track their progression across pairs of layers in the models. We presented one such progression within Figure [9](https://arxiv.org/html/2505.20435v2#S4.F9) in Section [3.3](https://arxiv.org/html/2505.20435v2#S3.SS3), which
captures how total persistence changes over the layers and is distinct from the control case. In the following sections, we include further plots to support this argument.

### C.2 Results

#### C.2.1 Mistral Model

In addition to the propagation of total persistence of 1-bars we showed in Section [3.3](https://arxiv.org/html/2505.20435v2#S3.SS3), we also evaluated the progression of other barcode summaries. Notably, descriptors which capture similar features are the mean deaths of 1-bars, and the mean birth of 0 bars with mirroring patterns. In Figure [53](https://arxiv.org/html/2505.20435v2#A3.F53), we show the results for the mean death of 0-bars.

Figure: Figure 53: Local analysis of consecutive layers for the mean deaths of 0-bars for the Mistral model. Top: Comparisons of the average of mean deaths of 0-bars across 1000 samples for the Mistral model for original (raw), scaled (normalized) and scaled & permuted activation data. Bottom left: Ratios of average mean deaths of 0-bars between clean and poisoned datasets for original, scaled and scaled & permuted activations. Bottom center: Overall variance of mean deaths of 0-bars for clean and poisoned datasets combined. Bottom right: Absolute difference between mean total persistence of 1-bars for clean and poisoned datasets.
Refer to caption: assets/local_analysis/mistral_h0_mean_death.png

#### C.2.2 Phi3 Model

We present a similar comparison of results for the Phi3 model. Figure [54](https://arxiv.org/html/2505.20435v2#A3.F54) illustrates the patterns across layers for the mean death of 0-bars, while Figure [55](https://arxiv.org/html/2505.20435v2#A3.F55) shows the patterns for the total persistence of 1-bars. Unlike the Mistral model, the ratio between barcode statistics for clean and poisoned activations in the Phi3 model does not intersect one. While a decreasing or somewhat parabolic trend is still observed, the average mean death of 0-bars and the total persistence of 1-bars for clean raw activations consistently remain greater than those for poisoned raw activations. Additionally, we find that the “control” case remains close to the x-axis, with the scaled ratios exhibiting significant variations around this baseline.

Figure: Figure 54: Local analysis of consecutive layers for the mean deaths of 0-bars for the Phi3 model. Top: Comparisons of the average of mean deaths of 0-bars across 1000 samples for Phi3 model for original (raw), scaled (normalized) and scaled & permuted activation data. Bottom left: Ratios of average mean deaths of 0-bars between clean and poisoned datasets for original, scaled and scaled & permuted activations. Bottom center: Overall variance of mean deaths of 0-bars for clean and poisoned datasets combined. Bottom right: Absolute difference between mean total persistence of 1-bars for clean and poisoned datasets.
Refer to caption: assets/local_analysis/phi_h0_mean_death.png

Figure: Figure 55: Local analysis of consecutive layers for the total persistence of 1-bars for the Phi3 model. Top: Comparisons of the average of total persistence of 1-bars across 1000 samples for Phi3 model for original (raw), scaled (normalized) and scaled & permuted activation data. Bottom left: Ratios of average total persistence of 1-bars between clean and poisoned datasets for original, scaled and scaled & permuted activations. Bottom center: Overall variance of total persistence of 1-bars for clean and poisoned datasets combined. Bottom right: Absolute difference between mean total persistence of 1-bars for clean and poisoned datasets.
Refer to caption: assets/local_analysis/phi_h1_total_pers.png

#### C.2.3 LLaMA3 8B Model

We present the results for the LLaMA3 8B model. Figures [56](https://arxiv.org/html/2505.20435v2#A3.F56) and [57](https://arxiv.org/html/2505.20435v2#A3.F57) both show a decreasing trend in the ratio between clean and poisoned activations, whether measured by the mean death of 0-bars or the total persistence of 1-bars respectively. Notably, this ratio crosses 1 around layer 15 or later. Moreover, we continue to observe distinct differences between clean and poisoned activations across both meaningful variants.

Figure: Figure 56: Local analysis of consecutive layers for the mean deaths of 0-bars for the LLaMA3 8B model. Top: Comparisons of the average of mean deaths of 0-bars across 1000 samples for LLaMA3 8B model for original (raw), scaled (normalized) and scaled & permuted activation data. Bottom left: Ratios of average mean deaths of 0-bars between clean and poisoned datasets for original, scaled and scaled & permuted activations. Bottom center: Overall variance of mean deaths of 0-bars for clean and poisoned datasets combined. Bottom right: Absolute difference between mean total persistence of 1-bars for clean and poisoned datasets.
Refer to caption: assets/local_analysis/llama_h0_mean_death.png

Figure: Figure 57: Local analysis of consecutive layers for the total persistence of 1-bars for the LLaMA3 8B model. Top: Comparisons of the average of total persistence of 1-bars across 1000 samples for the LLaMA3 8B model for original (raw), scaled (normalized) and scaled & permuted activation data. Bottom left: Ratios of average total persistence of 1-bars between clean and poisoned datasets for original, scaled and scaled & permuted activations. Bottom center: Overall variance of total persistence of 1-bars for clean and poisoned datasets combined. Bottom right: Absolute difference between mean total persistence of 1-bars for clean and poisoned datasets.
Refer to caption: assets/local_analysis/llama_h1_total_pers.png

#### C.2.4 Peak Analysis for Phi3 and LLaMA3

**Table 6: Peak analysis. Precision@$k$ for $k$=1, 3, and 5 largest peaks in total variance, and their precision in detecting the largest peaks in absolute difference between the two classes. ^∗, ^∗∗ correspond to $p$-values $<$.05 and .01, respectively.**
| Phi3 | p@1 | p@3 | p@5 |
| --- | --- | --- | --- |
| Total Persistence 0-bars | 0 | .33 | .2 |
| Total Persistence 1-bars | 1.0 | .67^∗ | .8^∗∗ |
| Mean Birth 1-bars | 0 | .33 | .6^∗ |
| Mean Death 1-bars | 0 | .67^∗ | .8^∗∗ |
| LLAMA3 | p@1 | p@3 | p@5 |
| Total Persistence 0-bars | 1.0^∗ | .33 | .4 |
| Total Persistence 1-bars | 1.0^∗ | .67 | .8^∗∗ |
| Mean Birth 1-bars | 1.0^∗ | .67 | .6 |
| Mean Death 1-bars | 1.0^∗ | .67^∗ | .8^∗ |

#### C.2.5 Non-consecutive Layer Analysis

Continuing the analysis of non-consecutive layers, we examine in Figure [58](https://arxiv.org/html/2505.20435v2#A3.F58) the ratio of total persistence of 1-bars between clean and poisoned activations. We find that at a 10-layer separation, the similarity between the ratios for scaled and control settings becomes more pronounced compared to separations of 1 or 3 layers.

Figure: Figure 58: Local analysis of non-consecutive layers on total persistence of 1-bars for the Mistral model. Comparison of the ratios between total persistence of 1-bars for clean and poisoned datasets when considering the topology of interactions between pairs of layers at 1 (left), 3 (middle), and 10 (right) intervals apart.
Refer to caption: assets/local_analysis/1_step_h1_totpers.png

### C.3 Sandbagged Representation Data Collection

To study *sandbagging*, we follow the “password-locking” paradigm introduced in prior work [Greenblatt et al., 2024, van der Weij et al., 2024], wherein a model is fine-tuned to hide certain behaviors or capabilities unless presented with a predefined password in the prompt. We leverage the the synthetic data generation process and fine-tuning procedure detailed in Section 3 and 4 of  van der Weij et al. [2024]. For clarity of our experimental setup, we summarize it briefly here. We extend it to extract final token hidden representations from the model under both elicited and elicited conditions.

- 1.
We generate a synthetic dataset of 3,347 evaluation questions based on the the Weapons of Mass Destruction Proxy (WMDP) Benchmark [Li et al., 2024] using GPT-4 [Perez et al., 2022] to serve as a proxy measurement for hazardous knowledge in biology, chemistry, and cybersecurity. This process protects against data leakage by never including actual WMDP items in the prompt and filters out any questions the base models cannot solve to ensure that the fine-tuned model genuinely has the underlying capability.
- 2.
Using this dataset and framework, we fine-tuned Mistral 7B and LLaMA3 8B with LoRA [Hu et al., 2021]. During this fine-tuning process, each WMDP-like evaluation question appears twice: once with the password, where the model is fully elicited and is provided with the correct answer and second where the model is password is absent or locked and the model is provided with an incorrect answer.
- 3.
Using the held out validation set of generated WMDP-like questions, we recorded the final-token representation from each layer of the fine-tuned model during its forward pass. Specifically, for each prompt $x$, we collect:
$\displaystyle\mathrm{Act}_{\ell}^{(\mathrm{locked})}(x)$
$\displaystyle=\mathrm{Hidden}_{\ell}\!\bigl(x_{\mathrm{locked}}\bigr)[-1],$
$\displaystyle\mathrm{Act}_{\ell}^{(\mathrm{elicited})}(x)$
$\displaystyle=\mathrm{Hidden}_{\ell}\!\bigl(x_{\mathrm{elicited}}\bigr)[-1],$

We then apply the global layer-wise topological analysis outlined in Section [3.2](https://arxiv.org/html/2505.20435v2#S3.SS2) to this representation data to study how the model’s latent space differs between elicited (no password) and elicited (correct password) states.

### C.4 Global PH pipeline and resource constraints

All Vietoris–Rips barcodes are computed with the GPU build of
Ripser++ on a single node equipped with four NVIDIA A100 GPUs
(80 GB each). Per layer we draw $K=128$ independent subsamples of
$k=4096$ activation vectors (64 clean, 64 adversarial). Subsamples
are dispatched round-robin to two concurrent Ripser++ kernels
per GPU.

Memory footprint. A complete $k=4096$ complex truncated at
dimension 2 occupies only
$2.1\pm 0.4\;\mathrm{GB}$ of device memory (95-th percentile
$<2.8\;\mathrm{GB}$; Tab. [7](https://arxiv.org/html/2505.20435v2#A3.T7)), leaving a wide
margin inside the 80 GB budget, even when two barcodes are built
concurrently on the same GPU.

Throughput. The mean wall-time per barcode is
$36.8\pm 0.6\;\mathrm{s}$ (95-th percentile $<40\;\mathrm{s}$).
With four GPUs processing eight barcodes in parallel, a full layer
($128$ barcodes) finishes in $\approx 10$ min and the five-layer
suite of one model in $\approx 50$ min. Running the six models
serially therefore completes in about five hours on a single
4 × A100 node—comfortably within the nightly maintenance window.

**Table 7: Per-barcode wall-clock time and GPU-memory consumption ($k\!=\!4096$, dimension $\leq 2$). Statistics over $K=64$ barcodes drawn from the Llama-3 8B activations.**
| Layer | $\text{time }\mu\pm\sigma$ [s] (p95) | $\text{memory }\mu\pm\sigma$ [GB] |
| --- | --- | --- |
| 1 | $38.34\pm 0.76$ (39.6) | $2.27\pm 0.34$ |
| 8 | $36.79\pm 0.70$ (38.0) | $2.12\pm 0.39$ |
| 16 | $36.68\pm 0.45$ (37.4) | $2.13\pm 0.30$ |
| 24 | $36.63\pm 0.71$ (38.1) | $2.03\pm 0.33$ |
| 32 | $36.62\pm 0.54$ (37.4) | $2.20\pm 0.344$ |

After choosing $K=64$, we recomputed the Monte-Carlo variance
$\sigma_{f}^{2}$ from the raw, unscaled feature values. For 39 out of 41 statistics we found $\sigma_{f}<0.10$, which would put the standard error
$\operatorname{SE}=\sigma_{f}/\sqrt{K}$ below
$\Delta^{\star}/2=0.025$ with only $K\leq 20$. The outlier features were those which aggregate counts—total persistence of $H_{0}$ and the raw count of $H_{1}$ bars—and need to be transformed for their variance to be directly comparable to the other features. These do not affect the classifier as the features are scaled prior to training and also do not appear as the most informative features for distinguishing between clean and posioned PH-derived features. We conservatively choose $K=128$ and the resulting ROC–AUCs on the logistic regression model trained only on barcodes are perfect (1.00 ± 0.00), confirming that the subsampling budget is more than sufficient to validate the significance of the features derived from PH, while balancing GPU memory and computation time.

### C.5 Robustness to Adaptive Attacks

We tested the robustness of our identified topological features against real-world attack examples from Microsoft’s large-scale LLMail-Inject dataset [Abdelnabi et al., 2025] which includes XPIA attack examples and information on their efficacy against four distinct defenses and a fifth setup involving a stacked arrangement of all four. All attack examples are sourced from a public red teaming competition. These attacks are particularly relevant as they include examples that were specifically designed to evade the TaskTracker activation-based defense [Abdelnabi et al., 2024], the source of our primary XPIA data. Thus, applying our topological framework to these attack examples is a particularly strong test of whether our topological features represent a fundamental shift in the shape and structure of LLM latent space, or whether it is an artifact that can be easily subverted.

##### Methodology.

As the LLMail-Inject dataset does not contain paired clean examples, we synthetically generated and manually verified 100 clean counterparts using Phi-3-medium-4k-instruct. The small sample size was chosen to ensure that we could verify the quality of the synthetically generated clean examples. We then generated last-token activation data from layer 16 of Mistral-7B-Instruct-v0.2 for both the clean and adaptive attack inputs and computed the corresponding barcode summary statistics.

##### Results.

The topological features of the activation spaces under these adaptive attacks show a clear distinction from the clean examples, as summarized in Table [8](https://arxiv.org/html/2505.20435v2#A3.T8).

**Table 8: Comparison of PH barcode statistics for clean vs. adaptive attack activations from the LLMail-Inject dataset on Mistral-7B (Layer 16). The results show a clear shift towards a simpler, more dispersed topology under adversarial influence.**
| PH Feature | Clean | Attack (Adaptive) |
| --- | --- | --- |
| H0 Count | 64 | 50 |
| H0 Death Time (Median) | 56.07 | 58.85 |
| H0 Death Time (Mean $\pm$ SD) | 55.43 $\pm$ 21.23 | 51.73 $\pm$ 28.38 |
| H1 Count (Loops) | 12 | 4 |
| H1 Birth Time (Median) | 69.36 | 84.92 |
| H1 Death Time (Median) | 71.59 | 86.20 |

- •
Fewer, Larger-Scale Loops: The number of 1-dimensional loops (H1 bars) decreases significantly from 12 in the clean data to just 4 in the adversarial data. Furthermore, their median birth time increases from $\approx 69$ to $\approx 85$, indicating that the remaining topological features are formed at much larger scales.
- •
More Dispersed Clusters: The median H0 death time increases, supporting the hypothesis of greater dispersion. We note that the mean H0 death time appears to contradict this trend (decreasing from 55.43 to 51.73). This is due to a small subset of components in the adversarial data merging at very low scales. The median, being more robust to such outliers, better captures the overall geometric shift towards a more spread-out structure.

These findings further suggest that the topological compression signature we identify across models and across XPIA and sandbagging attack conditions reflects a fundamental property of adversarial influence, as the signature remains detectable even against attacks optimized to evade XPIA defenses, including but not limited to the TaskTracker activation-based defense.

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 4
presentation: 2
questions: - Is there any motivation  not considering higher order bars (2-bars, 3-bars) etc?

-  How exactly is the variance in Table 2 computed: across all samples or separately per condition?

- Would the same compression pattern appear under different subsample sizes?
soundness: 2
strengths: * **Originality of the approach.**
  Applying persistent homology to adversarial analysis in LLMs is novel to the best of my knowledge and interesting, bridging topological data analysis with model interpretability and robustness.

* **Wide experimental coverage.**
  The study spans multiple coverage, including very large scale models in the Appendix (70B) and two distinct adversarial mechanisms, showing qualitatively consistent trends across settings.
summary: The paper investigates how adversarial interventions reshape the latent geometry of large language models (LLMs) using topological data analysis and persistent homology (PH) diagrams. The analysis is done across several instruction-tuned models and two attack types (prompt injection and backdoor fine-tuning), and PH barcodes are computed from layer activations, summarizing them into 41-dimensional feature vectors, and comparing clean versus adversarial conditions. The authors report a recurring pattern termed “topological compression”, fewer 1-dimensional bars and larger 0-bar death times for adversarial inputs. Logistic regression on PH summaries separates clean and adversarial activations with high accuracy, roughly matching linear baselines. A second, “local” analysis applies PH to neuron-pair across layers to examine layer-to-layer information flow, reporting differences in total 1-bar persistence and proposing PH variance as an unsupervised indicator of affected layers.
weaknesses: * **Claims may not be fully supported by experiments.**
  - The “topological compression” effect is observed descriptively but remains correlational. No controlled test distinguishes true topological simplification from simpler geometric or scaling changes: a simple example would be to match the scales of the original and poisoned features similarly as done in section 4.2. 
  - Linear baselines already achieve near-perfect separability (Table 1), aside from "layer 0", which is not showed for the PH stats  PH’s added value over simpler probes remains not fully clear to me in this setting. Moreover this could highlight that the binary task of distinguishing between poisoned and clean features might be too simple on this dataset and model pairs. 
  - The local analysis (Table 2) relies on a coarse metric (precision@k), without random baselines or multiple-comparison control. This weakens the claim that PH variance reliably identifies adversarially affected layers. A better metric to use would be spearman correlation. 
  - The results are not linked to behavioral or task-level outcomes (e.g. jailbreak success or refusal rates).


* **Clarity and presentation quality.**
  - Several results are difficult to interpret at first sight:  I think the authors should give priority for each figure to the result/plot that better highlights the current claim and put the remaining ones in the Appendix. For example Figure 8 show different information and is very dense, obscuring panel (d) which is the one that supports to the claim as (a), (b), (c) difference between poisoned and clean activation curves are not very visible. 
  - minor: Cross-model results are not included in the main text: I believe that a concise summary table in the main text would make the findings clearer and more convincing.


* **Few ablations.**
PH is computed only with one distance metric and one filtration type. No ablations on these design choices, or on subsample size, are provided, making it hard to assess stability, see [c,d] for discussions on how persistence diagrams can vary with metric choice. Scalability is not addressed beyond the subsampled settings; approximate PH algorithms such as witness complexes or streaming approaches [e] could test whether the reported effects persist at realistic layer scales. The 41-dimensional summarisation is also not discussed in comparison with alternative vectorisations such as persistence images [a]  or persistence landscapes [b], leaving uncertain whether the observed pattern depends on this particular encoding. I suggest to include at least a discussion on these choices should be included in the paper. 

_[a] Adams, H., et al. (2017) Persistence Images: A Stable Vector Representation of Persistent Homology. JMLR_

_[b] Bubenik, P. (2015) Statistical Topological Data Analysis Using Persistence Landscapes. JMLR_

_[c] Chazal, F., et al. (2015) Convergence Rates for Persistence Diagrams. JMLR_ 

_[d] Cohen-Steiner, D., Edelsbrunner, H. & Harer, J. (2007) Stability of Persistence Diagrams. Discrete & Computational Geometry 37(1):103–120._

_[e] Kerber, M., Morozov, D. & Nigmetov, A. (2016) Geometry Helps to Compare Persistence Diagrams. J. Exp. Algorithmics 22(1):1–20._

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 2
questions: How does the observed topological compression depend on the choice of token pooling (last token vs. mean-pooled vs. CLS-like embeddings)? Any preliminary experiments? 

Can the authors provide an ablation showing how sensitive the signature is to subsample size k and the number of subsamples K (e.g., do smaller k or fewer K materially change detection performance)? 

Have the authors attempted a simple topology-aware defense (e.g., penalize total persistence changes or normalize mean H0 death) to test whether changing topology reduces task drift or attack success? That would help evaluate causality.

The local 2D neuron embedding analysis finds a phase shift around mid-layers for Mistral — does the layer index of that transition correlate with model size/architecture across the six models? 

Could noisy or distributional natural shifts (non-adversarial OOD) produce similar topological signatures? That is, how specific is the signature to malicious adversarial influence vs. benign OOD?

The authors should consider releasing their code publicly for reproducibility by the community. 

Can the authors highlight the details of their experimental setup and the hardware/infrastructure used?

The authors should also consider showing a specific example across different models for the reader to visualize the method a little less abstractly.
soundness: 2
strengths: Novel methodology: First systematic application of PH to characterize adversarial effects in LLM latent spaces at both global and neuron levels. 

Robust empirical signal: Separation is reproducible across six models and two attack modes; discriminative power is high and interpretable via SHAP. 

Mechanistic insight: Neuron-pair 2D embeddings and layerwise analysis identify where adversarial influence reconfigures information flow (phase transition in deeper layers). 

Careful controls: Normalization, permutation tests, adaptive-attack evaluation (LLMail-Inject) and baseline linear methods are included.
summary: This paper applies persistent homology (PH) from topological data analysis to study how adversarial inputs reshape latent representations in large language models (LLMs). The authors analyze six instruction-tuned models (3.8B–70B) under two adversarial conditions — indirect prompt injection (XPIA) and sandbagging / backdoor fine-tuning — using the TASKTRACKER and sandbagged datasets. They compute Vietoris–Rips filtrations on subsampled point clouds of last-token activations, vectorize persistence barcodes into 41-dimensional “barcode summaries,” and run global (layer-wise) and local (neuron-pair 2D embedding) analyses. The central empirical claim is that adversarial influence produces a reproducible “topological compression”: adversarial activations show fewer but larger-scale topological features (fewer H1 loops, higher mean death times of H0 components), a phenomenon that is discriminative (logistic classifiers/SHAP achieve near-perfect separation) and consistent across models and attack types. The paper also reports a neuron-level phase transition in topological complexity at intermediate layers (≈ layer 12 for Mistral 7B).
weaknesses: Dependence on subsampling and last-token choice. PH is memory-intensive; authors subsample large numbers (k=1000, many subsamples), which is theoretically supported but may miss rare, high-impact topological features and makes replication costly. Also, using only the last-token embedding leaves open whether signatures generalize to alternative aggregation choices. 

Interpretation vs. causality. The paper convincingly documents correlational topological signatures but stops short of causal interventions (e.g., topology-aware regularization or targeted modifications to test whether changing topology alters adversarial susceptibility). Such experiments would strengthen the link between topology and vulnerability.

Generality to other threat models. The study focuses on XPIA and sandbagging/backdoor attacks. While LLMail-Inject adaptive examples were tested, evaluation against a wider array of adaptive, distributional, or model-poisoning attacks (and on more diverse prompts/tasks) would better establish universality. 

Scalability & runtime. Practical adoption of PH-based monitoring in production LLM systems would require faster approximations or streaming variants; the paper acknowledges this but provides limited engineering pathways. It is also not clear how can the method scale to much larger production-grade models with 100B+ parameters. 

References. The authors miss citing a few critical works comparing their approach in other domains. For e.g. Extreme Image Transforms (EITs) [Crowder et al., 2022; Malik et al., 2023, Biol Cybernetics, Malik et al., 2023, arXiv] help with similar representations in vision for deep networks. Similarly Network Dissection [Bau et al., 2017, CVPR] and Locating and editing factual associations in gpt [Meng et al., 2022, NeurIPS] show layer-wise applicability to the final output of deep networks. 

Presentation. The individual sections of the paper are well written but the paper flow is not easy to follow when put together. The authors should consider reorganizing the sections to make the story flow better and for the reader to keep track of what is happening, without losing the focus from the main point. .

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 4
questions: - What do you mean by "consistent topological behavior within the LLM latent space"?
soundness: 4
strengths: - Investigating the geometry of the feature space of modern neural networks is crucial to better understand how models work and how they arrive at decisions. Taking the route of PH is an interesting direction and a valuable contribution.
- The paper is clearly written. In particular, given that PH is not yet commonly used in interpretability/robustness research, the authors give good intuition for what barcodes do.
- The experimental setting is well chosen and covers important failure modes in LLM security.
summary: The paper investigates the embeddings of clean inputs versus adversarially perturbed inputs using persistent homology. This is a  tool from topological data analysis that enbles to describe the topology and geometry of point clouds, in this case last token embeddings, in a principled manner. The authors show that clean and adversarial inputs exhibit very different topological properties, which goes beyond the typical analysis of linearly separating such samples. The results are demonstrated on a diverse set of models and appropriate choice of benchmarks.
weaknesses: - This sentence gives the impression you are the first to do this: “We propose persistent homology (PH), a tool from topological data analysis, as a principled framework to characterize the multi-scale dynamics within LLM activations.” I suggest phrasing it as using PH as a tool rather than proposing a framework, since related uses exist in other domains and your contribution is not a full PH framework.

- In the background section, consider citing a review of complex constructions (e.g., Vietoris–Rips, Čech, alpha complexes).

-  A formal treatment of PH would be appreciated in the appendix (optional).

- The fact that the barcode summaries cluster adversarial vs. clean examples is not surprising if these samples are already linearly separable. However, it needs further elaboration to explain why this is important.

- Even when you obtain a perfect AUC, adding your "method" to Table 1 would improve the presentation.


Note: If my concerns are properly addressed I am inclined to raise my score to an accept.

[1] Gardinazzi, Yuri, et al. "Persistent topological features in large language models." arXiv preprint arXiv:2410.11042 (2024).

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: Suggestions: (i) Sweep the subsample sizes, distance metrics and token selection. Report the key barcode features and perform a stability analysis, (ii) Ensure that the differences are not just prompt length/format, by adding benign dummy insertions that mimic injection structure but carry no instruction, and re run PH, (iii) To anchor the phenomenon, prove for a simple model of mixture of Gaussians in high d,  that moving mass from many small clusters to a few larger, more separated clusters reduces counts of 1-bars and increases mean 0-bar death, formalizing topological compression.
soundness: 3
strengths: (i)  The PH barcode summaries cleanly separate clean vs. poisoned activations across layers with simple linear models. This repeats across several LLMs (7B→70B) and two attack types. This is not only a classification, but rather a characterization of how geometry shifts: poisoned states show fewer small features and later/longer lived large scale ones, while clean states have many short lived features, (ii) There is a solid, reproducible pipeline beginning with subsample activations till PCA/CCA + logistic + SHAP, (iii A complementary local analysis embeds neurons across two layers, and applies PH to track where differences peak in depth, and variance heuristics can find informative layers even without labels.
summary: The paper uses persistent homology (PH), from topological data analysis, to study how LLM activations change under adversarial conditions (prompt injection and backdoor “sandbagging”). It finds a consistent, layer wise topological signature distinguishing clean from poisoned activations, and proposes PH based summaries as practical signals for detection and analysis.  The core idea is to treat each layer’s activation vectors, for many inputs, as a point cloud, compute a Vietoris–Rips filtration and its barcode,  vectorize these barcodes into fixed length barcode summaries, and compare across layers/conditions. The findings are: (i) poisoned activations show fewer small scale features and later/longer lived large scale ones than clean, yielding a clear separation in PH features layer by layer, (ii) The PH summaries linearly separate clean vs. poisoned with near-perfect accuracy across multiple models and attacks, (iii) A local neuron level analysis pinpoints, where in depth adversarial effects concentrate, and ontrols (neuron permutation) remove the signal. Thus, PH barcodes + vectorized summaries provide architecture agnostic descriptors of the latent geometry.
weaknesses: (i) Vietoris–Rips PH scales poorly, and the paper therefore subsamples activations, which may lose fine detail or bias results, (ii)  PH results depend strongly on normalization, distance metric, and whether one uses token vs pooled activations. Different choices could change the barcode summaries, (iii) Activation geometry can vary with dataset composition, prompt templates and topic distribution), hence the results might not generalize beyond the specific prompt suite or poison triggers used, (iv) Topological compression is a descriptive signature but the causal link to model failure modes or specific failure behaviors is not fully established. The paper provides strong empirical separation, but no theory explaining why adversarial triggers should systematically compress topology across architectures.


Decision: Accept (Oral)