ActivationReasoning: Logical Reasoning in Latent Activation Spaces
Authors: Lukas Helff
Ruben Härle
Wolfgang Stammer
Felix Friedrich
Manuel Brack
Antonia Wüst
Hikaru Shindo
Patrick Schramowski
Kristian Kersting
Venue: ICLR 2026 Poster (2026)
Keywords: reasoning
latent space
mechanistic interpretability
logic
Primary area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
TL;DR: Logical reasoning in the latent space of an LLM.

Abstract: Large language models (LLMs) excel at generating fluent text, but their internal reasoning remains opaque and difficult to control. Sparse autoencoders (SAEs) make hidden activations more interpretable by exposing latent features that often align with human concepts.  Yet, these features are fragile and passive, offering no mechanism for systematic reasoning or model control. To address this, we introduce ActivationReasoning (AR), a framework that embeds explicit logical reasoning into the latent space of LLMs. It proceeds in three stages: (1) Finding latent representations, first latent concept representations are identified (e.g., via SAEs) and organized into a dictionary; (2) Activating propositions, at inference time AR detects activating concepts and maps them to logical propositions; and (3)Logical reasoning, applying logical rules over these propositions to infer higher-order structures, compose new concepts, and steer model behavior. We evaluate AR on multi-hop reasoning (PrOntoQA), abstraction and robustness to indirect concept cues (Rail2Country), reasoning over natural and diverse language (ProverQA), and context-sensitive safety (BeaverTails). Across all tasks, AR scales robustly with reasoning complexity, generalizes to abstract and context-sensitive tasks, and transfers across model backbones. These results demonstrate that grounding logical structure in latent activations not only improves transparency but also enables structured reasoning, reliable control, and alignment with desired behaviors, providing a path toward more reliable and auditable AI.

Published as a conference paper at ICLR 2026
ACTIVATIONREASONING:
LOGICAL REASONING IN LATENT ACTIVATION SPACES
Lukas Helff!.2. Ruben H4r
Wolfgang Stammer Felix Friedrich", Patrick Schramowski
Manuel Brack Antonia Wüsst,
Hikaru Shindo',
Kristian Kersting 12.3
'TU Darmstadt Ae LLbb114
Aleph Alpha Research MMPI-Inf, SIC DFKI
Meta FAIR "Adobe Applied Research
9CERTAIN, Germany
ABSTRACT
Large language models (LLMs) excel at generating fluent text, but their internal reasoning remains opaque and difficult to control. Sparse autoencoders (SAEs) make hidden activations more interpretable by exposing latent features that often align with human concepts. Yet, these features are fragile and passive, offering no mechanism for systematic reasoning or model control. To address this, intro- REASONING (AR), a framework that embeds explicit logical duce ACTIATIIONEASSONNGG
reasoning into the latent space of LLMSs. It proceeds in three stages: (1) Finding latent representations, first, latent concept representations are identified (e.g. via SAEs) and organized into dictionary; (2) Activating propositions, inference time, AR detects activating concepts and maps them to logical propositions; and (3) Logical reasoning. applying logical rules over these propositions to infer higher- order structures, compose new concepts, and steer model behavior. We evaluate AR on multi -hop reasoning (PrOntoQA), abstraction and robustness indirect concept cues (Rail2Country)) reasoning over natural and diverse language (ProverQA), and context-sensitive safety (BeaverTails). Across all tasks, AR scales robustly with reasoning complexity, generalizes to abstract and context-sensitive tasks, and transfers across model backbones. These results demonstrate that grounding logi- cal structure in latent activations not only improves transparency but also enables structured reasoning, reliable control, and alignment with desired behaviors, pro- viding path toward more reliable and auditable AI. Code and Dataset available https: / htt///ggttu.... com / ml research/ htw.://cccemmeeeeeeeeeeeooe/eee Act ivat iottr//ttiiaiiiiiaooaoooo.oooo.
INTRODUCTION
Large language models (LLMs) demonstrate remarkable abilities in semantic disambiguation, knowl- edge retrieval, and generative tasks (Brown et al. 2020 OpenAl (023). Yet, their internal activations distributed and entangled, with multiple unrelated features encoded in overlapping representa- tional dimensions. This phenomenon, known as superposition, obscures conceptual representations and limits both interpretability and intervention Marcus (2020); Bender et al. (202T): Elhage al. (2022). More fundamentally, autoregressive transformers lack explicit propositional structure, making systematic reasoning, compositional generalization, and rule enforcement difficult (Elhage al. 2022, Bommasani et al. |021] Woydt et al. 2025 Delfosse et al. 2025). These challenges extend beyond transparency to robustness, controllability. and safe deployment real world (Gabriel 2020 Wei et al.) 2022). Mechanistic interpretability, and particularly sparse autoencoders (SAEs), promises remedy by surfacing latent features that often align with juman-recognizable monosemantic con- cepts (Bricken et al. 2023 Templeton et al. (024). However, SAFs alone remain limited; features can still polysemous, contextually unstable, or overly low-level (Leask et al. [025]] Härle et al., 2025)
Peng et al. 2025). Critically, SAEs lack mechanisms for composition and higher-order reasoning.
Formal logic offers explicit compositionality, well-defined inference rules, and transparency in deriving conclusions (Lloyd 1984, Russell & Norvig 2009 Shapiro & Kouri Kissel, 2024). Yet,
"Equal contribution. Correspondence: [helff', ruben ruben. haerle www...ttttaasstetee tu-daaaatttddddee Work conducted while TU Damsstat/hessian./////// At/laa 141.

---

Published as a conference paper at ICLR 2026
Flnnding
Baquancea
Representations
Opt.1
Single Feature
pox:
Bridge
230 003
Bock
San
Francisco
X200 0000
DVV.
Mulni Fmatures
neg:
WWll
Relorional
Featuree
Golden
3000
2000
LLM
SAE
Concept Distionary.
Latent Representaatinn
Inference
Bequence
cas: Bridge
Golddn
Beidge B8
74
Gate
1:57 0 Wall
Downstrenm
Teak
0:00 Goldinn
Wall of
Mall
Sosq
Golden Gade
LLM
SAE
china
Forward Paos
Accivoatioss
Rules,
Logic Salvar
Activating Propositions
Logical Reasoning
AR performs logical reasoning over LLM acti- vations in three stages: (1) Finding latent representations, where concepts are identified in the SAE latent space and stored in a concept dictionary using single, multi, or relational feature representa- tions; (2) Activating propositions, where token-level activations are detected during inference to form an activation matrix A; and (3) Logical reasoning, where pre-defined rules are applied over A to infer new higher-order structures, compose new propositions, yielding enriched matrix A', The structured activations can then be used for downstream transparency and control.
Figure I: Overview of ACTIVATIONREASONII
logic-based reasoning presupposes discrete propositional units, whereas LLMs rely on continuous, superposed representations that obscure such structure (Marcus 2020 Bender et al. 2021 Stammer al. 2024). While this mismatch has long hindered direct integration, recent progress with SAES provides concrete foothold by inducing monosemantic features that approximate discrete propo- sitional units required for symbolic reasoning. To this end, we propose ACTIVATIONREASONE (AR), a framework that embeds explicit logical reasoning into the latent space of LLMs. AR operates in three stages (see Fig. I) (1) Finding latent representations, where atomic concepts are grounded in SAE-derived features; (2) Activating propositions, which monitors their activations during inference; and (3) Logical reasoning, which composes these propositions into higher-order concepts through user-defined rules. This pipeline extends SAEs from feature extraction structured reasoning and control. As illustrated in Fig. 2 AR can recover composite concepts absent from the SAE space (e.g., Golden Gate Bridge from Bridge, San Francisco, and US); disambiguate polysemous features such as different uses of red; enforce safety constraints directly in latent space; and perform multi-step reasoning. On the analysis side, this enables fine-gra men-graneed inspection of model behavior, such as tracing failure cases to specific activations, probing compositional generalization. or evaluating adherence safety rules. On the control side, AR supports direct interventions during inference by amplifying suppressing activations to steer the model toward desired behaviors.
We evaluate AR across four complementary settings. On PrOntoQA (Saparov He 2023 Saparoy et al. 2023), AR successfully solves deductive multi-hop reasoning tasks without degrading with task complexity. On our novel Rail2Country dataset, it generalizes from explicit lexical concepts to meta-level descriptions expressed through similes and world knowledge. On ProverQA (Qi et al. 2025), which introduces naturalized logical reasoning tasks, AR demonstrates its ability to handle complex reasoning scenarios that reflect real-world linguistic variability. On BeaverTails (Ji et al. 2023 it captures abstract and context-sensitive safety concepts in real-world text. all tasks, AR substantially outperforms the baselines, improving model reliability and downstream reasoning.
Our contributions are threefold: (i) we introduce ACTIVATION EASOONING (AR), a framework that embeds logical reasoning into the latent space of LLMs by treating SAE features as propositions and composing them through explicit rules; (ii) a new benchmark for reasoning over meta-level concept descriptions, called Rail2Country; and (iii) we demonstrate through experiments concept generalization, and safety that latent activations serve as viable substrate for structured logical reasoning, enhancing transparency, robustness, and control in LLMs.
reasoning, meta-
2

---

Published as a conference paper at ICLR 2026
Compositional Concepts
Contextual Concepts
The Golden Gate Bridge an iconic suspension bridge in T San Francisco, United States, spans approx. 1.7 miles
Q1: The car is painted red.
Q2: The is painted r3d.
across the Golden Gate Strait...
Q3: The car is painted like ripe tomato.
Concepts:
Compose New Concepts:
Activations for contextual concept red:
Vanilla SAE AR
Bridge
San
Pranciaco
Bridgge
Ban
Matted
Ql: red / QI: red
the1 tad
Coldea Gatel Bridge
AR
Q2: None X Q2: red
States
Golddnn date Bradge
Vanilla SAE
Q3: None
x Q3: red
34E jeatan!
SAE
Constitutional Reasoning
Multi-Hop Reasoning
Q1: A police officer carries a gun
Each cat is carnivore. Every carnivrre not herbivorous. Carnivores mammals. Fae is cat.
Context:
Q2: A criminal carries gum.
Q3: A patient receives morphine in hospital.
Query: Tiue false: Fac is not herbivorous.
Concepts: Keapon
Police Drug
Medical
AR
Concepts:
Steering Model Answers:
Activations
Rules:
Meapon
A-
Police
Uneafe
Q1: Safe
Rules:
LLM:
Weapon
Pollce
Sare
Q2: Unsafe /
Faa)+<cuu
carni-
AR:
->Anaver: False X
Drug
Mad
Saxe
Q3: Safe
Vore
GRERn
Beebb
varous
->Ansver: True
Figure 2: AR applies logical rules on latent activations to enable downstream reasoning, abstraction, and control. AR composes missing concepts from existing SAE features (top left), aids robustness to
spelling errors or similes (top right), implements rules for safeguarding at the neural level (bottom left), and solves multi-hop reasoning tasks via logical inference in the latent space (bottom right).
2 RELATED WORK
Our work touches upon three major research areas: (i) concept-level interpretability of neural repre- sentations, (ii) compositional reasoning, and (ii) applications in safety, aligmment, and controllability.
Neural Concept-level Interpretability. A large body of work has sought to make neural activations more interpretable by grounding them in human--ndersstnddable concepts. SAEs (Bricken et al. 2023] Templeton et al.) 2024) have recently emerged as scalable approach, particularly for large models, producing sparse latent features that frequently correspond to semantically meaningful concepts. Follow-up work has revealed both their promise and limitations, highlighting polysemy, context dependence, and difficulties in capturing higher-level abstractions (Leask et al.] 2025 Härle et al.
2025; Peng et al. 2025). Hierarchical extensions such as Matryoshka SAES (Bussmann al. (0255 )
attempt to capture layered or compositional features but lack principled control over abstraction levels.
Earlier concept-basee based approaches include supervised methods such as Concept Activation Vectors
(CAV) (Kim et (011)2088) as well as Concept Bottleneck Models (CBM) (Koh et al.] [2000] Delfosse
al.
2024; (Steinmann et al.) 2024) and their extensions (Havasi et al., |022]|Shang [Shang a [2024).
which separate prediction into concept detection and downstream classification. More recent work
reduces supervision, using pre-trained vision language models such as CLIP (Bhalla et al. 2024
Yang et al. 2023 Oikarinen et al., (023) even unsupervised discovery methods (Ghorbani et al.
2019 Stammer et al. 2024 Schut et al.] (025). The majority of these approaches, from interpretable-
by-design to mechanistic interpretability. typically treat concepts in isolation. Only few provide formal mechanism to compose concepts into higher-level abstractions or to disambiguate polysemous features. AR builds on this line of work by grounding logical propositions in latent SAE features and introducing a framework for the logical composition and reasoning over these propositions.
Compositionality and Reasoning. Neuro-symbolic Al (Garcez et al. 2015 Sarker et al. 202 Kautz, 2022. Marra et al. 2024) has long sought to integrate the strengths of neural representations with the structure of symbolic reasoning. Approaches such as DeepProbLog (Manhaeve et al. 2018).
SLASH (Skryagin et al. 2021 2023). and diffcrentiable inductive logic programming frameworks
such as olL. (Shindo et al. (023) and NEUMANN (Shindo al., (024) embed logical constraints into differentiable models, enabling joint learning of concepts and rules. Other systems, such as
neuro-symbolic concept learners (Stammer et al. 2021; Barbiero et al.. [0233 2024, Wüst et al. 2024), propose integration of concept-level representations with reasoning modules for improved human-Al interactions and explainability. In parallel, work on reasoning LLMs (OpenAl 2023 Yang et al. 2025; DeepSeek-AI]] 2025) has demonstrated remarkable gains by scaling test-time compute. However, their reasoning remains unreliable and difficult to audit (Shojace et al.) 2025 Xie et al.)
2024h. Our approach differs by explicitly imposing logical layer on top of latent activations, rather
3

---

Published as a conference paper at ICLR 2026
than requiring differentiable end-to-end training. By defining rules over SAE-based features, AR enables interpretable and auditable reasoning within the latent space of large Al models, bridging the gap between latent activations and symbolic inference.
Applications in Alignment, Safety, and Knowledge Integration. Concept-level interpretability is particularly relevant for safe and controllable AI (Kambhampati et al. 2022 Stammer, (025)) Previous work has used interpretable latent features for controllable generation and steering (Turner
al. 2023 Liu et al. 2024). as well as for safety monitoring through moderation systems (Markov
et al. 2023 Friedrich et al. 2023 Brack et al. 2023) or direct interventions on safery-relevant
features (Härle al. 2025). More broadly, structured knowledge integration has been shown to
improve reliability in domains such as factual grounding and rule-based agents (Creswell et al.] 2022) Yu et al. 2023). AR extends these directions by enabling explicit logical constraints and abstractions to be applied directly within the latent space, providing both transparency and controllability for downstream tasks such as safe model steering, interpretable classification, and multi-hop reasoning.
3 ACTIVATIONREASONIS
ACTIVATIONREASONII (AR) enables logical reasoning directly within the latent activations of LLMs (Fig. 1), yielding interpretable, controllable, and robust behavior in downstream tasks. The process has three stages: (i) Finding latent representations, where concept representations are identified in the LLM's latent space and collected in dictionary; (ii) Activating propositions, where token-wise activations are detected at inference time and treated as propositional units; and (iii) Logical reasoning, where rules are applied over the activated propositions to infer higher-order structures, compose new concepts, and steer model behavior. As a running example, we infer the concept Golden Gate Bridge, which is not initially present in the dictionary but can be derived from the joint activation of three base concepts: Bridge, San Francisco, and USA.
3.1 FINDING LATENT REPRESENTATION
The first stage of AR (cf Fig. top half) constructs set of interpretable concepts grounded in the latent space of the LLM. We first introduce a formal definition of a concept and then explain how the associated representations are constructed.
Formalizing Concepts. We formally define a concept e as a tuple (ne. Te, Te). Let L S Rd denote the latent space of concept codes produced by fixed encoder E applied to the model's hidden states. For each token t with hidden state h,, we obtain a latent code via the encoder L, = =E(h,) E L.
Concretely: (i) ne is semantic identifier for the concept (e.g., "Bridge"). (fi) Te Ryn is soft activation threshold that determines when the concept is considered active. (iii) re L - R20 is the concept's 's latent representation in this space. Implemented as function, maps latent codes E L to activation score a(c, f) ;= ro(G)) E Rzo- The collection of concepts used by AR forms a concept dictionary D = [c containing the building blocks for subsequent logical reasoning.
Toward interpretable activations. Raw LLM activations highly entangled and subject to superposition, encoding multiple features in overlapping dimensions (Elhage et af. (022). which obstructs recovery of monosemantic representations (Park et al. (023)). Logical reasoning, however, requires such representations as propositional building blocks. Thus, we project activations into sparse and more interpretable feature space using SAES (Bricken et al., 2023 which promote monosemanticity by aligning individual dimensions with distinet, human framework remains method-agnostte and can directly integrate future advances in interpretability.
human--nner interpretable concepts. The
Types of latent representations. Building on SAE features, we propose three forms of latent representation r E RRssngee Rmults R (cf. Fig. top right). (1) A Single-feature representation (Reingte) ties a concept to one SAE feature, reflecting the common assumption that some features can align closely with human-interppetabbl notions (Bricken et al. (023). (2) A Multi-feature representation (Rmuli) instead associates a concept with a set of k SAE features. This relaxes the strict monosemantic assumption, acknowledging that features often capture only fragments of concept exhibit polysemy (Leask et al.) 2025 Härle et al.] (025). Here, features contribute independently through weighted aggregation; no interactions between them are modeled. (3) A Relational-featur representation (Reetation defines concept by of relations among SAE
4

---

Published as a conference paper at ICLR 2026
features. In practice, we instantiate this via shallow decision tree that applies different thresholds and conditions along different branches, balancing expressiveness with interpretability. Unlike Rmuli this can express structured interactions such conjunctions (requiring both slurs and stereotypes) or context-dependent exclusions (filtering educational mentions). Simple concepts like Bridge or USA be captured by Ringge or R.multi while abstract contextual notions benefit from Reelation g.g.- the broader concept hate may be recognized if features linked to slurs or demeaning stereotypes are activated, while benign uses of similar words are filtered out. Such patterns require that multiple cues co-occur while others are excluded (e.g., filtering non-toxic educational usage). These structured dependencies are characteristic of Reelation and cannot be captured by linear aggregation alone.
Concept extraction. Concepts can be obtained either manually or automatically. In the manual case, a human expert defines Te by assigning SAE encoder features (SAF.) to concepts. For example, features consistently activating for the notion of a Bridge can be directly assigned. This direct assignment is used, however, only when feature is clearly monosemantic, and its link to a concept is unambiguous; in practice, most concepts are constructed via the automatic procedure described next, which scales naturally and does not require manual inspection.
In the automatic setting, representations are induced from data with token-level labels for the concepts. The goal is to find latent representations that act as reliable "signatures" " for concepts. For instance, to find the representation for Bridge, we identify SAE features that most effectively differentiate tokens labeled as Bridge from those that are not. Given the hidden state h, for a token f. and its sparse code derive the representation re for concept with label Ye,t
4 (SApp
arg max (E(4, Ve,t = 1] E(4 Neett = 0])
for Resingle
Te top-k (E]4e yo,t 1] E[4 Mo,t 0])
for Rmuti
decision tree induced from (4,, Mo.r)
for Reelation
Here, E[- ye.el denotes the empirical expectation over token-level sparse codes 4. conditioned on whether token is labeled as belonging to concept (ye,t or not (ye,+ = 0).
Thresholding activations. Each representation Te is paired with a soft threshold Te to decide when the concept is considered active. In the automatic case, Te is chosen to maximize balanced accuracy for the provided labels Ye.r E {0, 1} and scores d,,t as
Te. E arg max (TPRc(T) + TNR,(T)). T20
(2)
where (T) Pr(ac,: denotes the activation score for concept at token 1, as formally defined in Secc. 3.2 In the manual setting, where labels are unavailable, the threshold Te, defaults to but can be overridden by user-specified value.
Bett 1) and (T) Pr(ac, < ye.t 0). Here Oc,t
3.2 ACTIVATING PROPOSITIONS
With the concept dictionary established, this stage tracks activations the latent space and maps them to atomic propositions for logical reasoning. This is two-step process. First, we compute raw, token-level activation scores to detect local evidence for each concept. Second, we formalize these activations into propositions by applying thresholds and, where necessary, aggregating them.
Activation. At inference, we compute an activation score a(c, () for each concept e at each token Given the hidden state h4, we obtain its sparse code I, toop (SAEE The score a(c, t) is then derived from the latent representation re as
4/rel
if E Reinnge
alc,t) t)
E Eulerd ro(1)
if rf Rmulti
if rr E Ryelltion
For Raingles the score corresponds to the value a single SAE feature (cf. Fig. purple box). For Rmulti, evidence is aggregated from multiple indices using weighting vector (e.g., log decay), normalized to sum to one, i.e., forming a convex combination. For Rpelation the score obtained by traversing the decision tree r. on 4, producing probability of concept presence.
5

---

Published as a conference paper at ICLR 2026
From Activations to Propositions. To make activations accessible for reasoning, collect them into the activation matrix A that serves as weighted evidence for concept activity. We provide (i) token-level activations Abeall E RID[x|S], and (ii) sequence-lewe activations Aglobal E RID) for each concept D and token t S:
Atocal [c, t] max(ac, Te: 0), Aglobal(d] max(AgBies ac,t Te: 0)
(4)
Here, Agg denotes aggregation operator (mean by default), and Tr the concept-specific soft threshold. Intuitively, Atocol provides fine- tme-ggrined token-level evidence of concept activity, whereas Aglobal reflects higher-level semantic meaning throughout the sequence. Individual words may only weakly signal concept, while aggregated evidence over a span reveals a more robust semantic direction. For the subsequent reasoning stage, we can now view each concept logical proposition with its weighted evidence score encoded into the activation matrix, either locally or globally.
Consider the following input sequence: "After the fourth failed selfie on the Golden Gate Bridge in SF, USA, she said, 'Well, that went well." At the token level, AR assigns high activations to the base concepts their respective positions, e.g., Atocai Bridde, fl. Atocul Francisco, t). and Atocal (USA, t). These activations constitute local evidence, directly grounded in individual tokens. By contrast, global evidence captures abstract semantics that emerge only at the sequence level: here, the ironic clause "Well, that went well." activates the higher-order concept sarcasm with Aglobat [sarcasm] This illustrates how local activations provide explicit, token-level signals, while sequence-level aggregation enables the detection of more abstract semantic notions that are not necessarily attributable to any single token.
3.3 LOGICAL REASONING
Given the activation matrix A, we apply logical rules to infer new propositions (cf. Fig. red box). Rules are user-defined (cf. Fig. and use standard propositional operators (implication, conjunction, disjunction, negation). For example, a rule may encode semantic and compositional relations, e.g., Bridge/SSan Francisco USA Golden Gate Bridge.
Reasoning, Having established set of user-defined rules, reasoning proceeds forward chaining. Starting from the initial activation matrix A of logical propositions, inference rules are applied iteratively to derive new conclusions, until a fixed point is reached where no further inferences can be drawn (Russell & Norvig, 2009). We formalize the explicit semantics App. D In practice, rules may operate over Alocal and Aglobal- The former provides evidence for explicit concepts such as Bridge, San Francisco, and USA at specific tokens, while the latter captures more abstract notions in the sequence, such sarcasm. For reasoning, we feed discretized activations A into the rule engine (e.g., Alocal (Bridge, 15) = 0.9 is treated as the Boolean proposition Bridge). Applying the compositional rule above the evidence allows to infer the higher-level proposition Golden Gate Bridge, even no SAE feature directly encodes this concept.
Enriched matrix. We denote the resulting enriched matrix of logical propositions A' (cf. Fig. bottom right). In contrast to the raw activations A, which contain only directly activated con- cepts, A' integrates both detected and inferred propositions, providing structured and interpretable representation that can be leveraged for downstream analysis and control.
3.4 INTEGRATION INTO DOWNSTREAM TASKS
For analysis, A' allows fine- tne-grained inspection of model behavior, such as tracing failure cases to specific concept activations (cf. [Scc. 4, Rail2Country), probing compositional generalization
(ef. Fig. 2) or evaluating adherence to safety constraints (cf. Sec. 4 safety experiments).
the other hand, in terms of control, A' supports interventions during inference by activating or suppressing model activations using the concept representations to steer the model toward desired behaviors (cf. SScc. 4. PrOntoQA). For steering, we build upon decoder-based model steering (Härle et al. 2025) to bias the model during generation towards activated concepts from the final AA. For example, we want to steer towards the concept red, the associated latent representation ((red) E Rmmati retrieved from our concept corpus C' and used to extract the corresponding SAE decoder weights (SAEp) for red. Subsequently, we apply weighting vector w (e.g.. log decay) and normalize with respect to the latent activations of the model. The steering factor E R indicates the amount of

---

Published as a conference paper at ICLR 2026
promotion or suppression of the concept Finally, we update the model's latent activations h as
h' = + (SAEplrd x w) x ||0||2
(5)
Overall, structuring AR into activations, logical reasoning, and downstream integration yields a mod- ular pipeline: concept representations Te provide interpretability. logical rules L enable compositional reasoning, and the enriched concept matrix A' supplies actionable signals for alignment and control.
EXPERIMENTAL EVALUATIONS
In our evaluations, we first investigate whether AR can perform reasoning directly on latent activations and whether it does so robustly as task complexity scales (PrOntoQA). Next, we investigate whether AR generalizes beyond explicit lexical cues to meta-level concept descriptions expressed through similes and compositional object semantics (Rail2Country). Then, we move to reasoning tasks with natural and diverse language that combine real-world linguistic variability with complex logical reasoning (ProverQA). Finally, we examine its ability to reason over abstract safety concepts in real-world text, where categories are vague and context-dependent (BeaverTails).
Experimental Setup. We evaluate AR on two backbone models, Llama-3. 1-8B (AT@ Meta 2024
with FleutherAl's SAE (FleutherAT] 2024) attached after layer 23, and Gemma-2-9h Team 2024) with Gemma-Scope SAE (Lieberum et al. 2024) attached after layer 20. All experiments use greedy decoding. For evaluation, we apply Ruult in the first three benchmarks and compare Ringle. Rmult and Reelation in the fourth. We report results against several baselines: the same backbones without models 1.lmm-3311-78B and Gemma-2-27B) commercial models (GPT- 4o (OpenAl] 2023)). and the reasoning model DeepSeek-RR RI-Distilll-Lleee -lamml-LLm--BB (DeepSeek-Al 2025).
AR, larger instruction-tuns
and chain-of-thought baselines in App. El and analysis of runtime efficiency in App. B Baseline LLMs generate answers directly, whereas AR performs reasoning in the activation space and steers the downstream generation. We provide detailed descriptions of the datasets in A/pp. A and the hyperparameters in imApp. H| We also include an ablation study on the choice of SAE placement, demonstrating that AR remains invariant to this factor (cf. [App. G}.
Additionally, we provide comparison to further instruction-ture
Multi-Hop Reasoning on Latent Activations. We begin by investigating to what extent AR can enhance reasoning. To do so, we evaluate on PrOntoQA (Saparov & He 2023 Saparov et al., 2023), popular question-answerre by increasing the number of reasoning hops (1, 3, or 5). In PrOntoQA, models are provided with a context containing set of rules and query that has to be proven disproven. We initialize AR with Rmuhi using the PrOntoQa train set and encode the context into the rules L.
dataset that tests deductive reasoning and scales complexity
Results on PrOntoQA testset are summarized in Tab. T (left). As expected, the 8B and 9B base models perform poorly and remain close to chance, often failing even the simplest 1-hop questions. Larger instruction-tuned models (Llama-3.1-708 Gemma-2-22 2-27B, and GPT-4o) achieve strong performance on the 1-hop setting, but their accuracy drops sharply as the number of reasoning hops increases, revealing limitations to generalize longer and more complex reasoning scenarios. The reasoning model, DeepSeck-RR MerSSeck-RI-iisiiee -I--sttil is also subject to degradation as task complexity scales. In contrast, AR consistently solves most problems across hop lengths, explicitly inferring logical conclusions within activation space and steering the model to correct answers. This yields accuracies above 93% 1, 3, and 5 hops. Importantly, unlike all baselines, AR maintains its performance as task complexity scales, showing that the compositional rules it encodes generalize robustly to longer and more complex reasoning chains. Furthermore, gains transfer across different backbone families.
Reasoning over Meta-level Concept Descriptions. We next examine whether AR can generalize beyond explicit lexical mentions to abstract, meta-level descriptions. Prior work shows that SAFS reliably recover consistent features when concepts are explicitly referenced (Bricken et al. 2023 Templeton et al.) 2024);; for instance, the token red tends to activate a stable subset of SAE features across different contexts. However, when concepts are expressed indirectly through similes or descriptive proxies (e.g., colored like a tomato as a substitute for red), the resulting activations become weaker and distributed, introducing noise into the latent representation (Leask al. 2025).
To study this setting, we introduce Rail2Country (R2C), a newly introduced benchmark for reasoning over meta-level concept descriptions, inspired by earlier train-based reasoning tasks (Michalski 1980
7

---

Published as a conference paper at ICLR 2026
Table Reasoning on latent activations. Exact-match accuracy on PrOntoQA (I-5 hop reasoning), Rail2Country (Mono with explicit concepts; Meta with similes, e.g., 'red' - 'like a tomato') ), and ProverQA (linguistically diverse reasoning tasks across difficulty levels). AR consistently boosts multi-hop reasoning, remains robust task complexity scales, and generalizes to natural and diverse outpeforn its baselines, and even other reasoning/larger instruction-tuner (it) LLMs.
PrOntoQA (1%)
Rail2Country (1%)
ProverQA (1%)
Model
Hop
Hops
5 Hopp
R2C-Mono
R2C-Meta
Easy
Medium
Hurd
IJamm3.1 1 8B
51.0
50.8
50.3
41.0
29.7
43.6
33.6
36.8
w/ AR (our)
95.0 (-44.00
95.6 (444.))
95,3 (-45.))
74.7 (433.7
62.7 (633.))
92.8 (-40.2)
91.0 (652.)
70.8 (550))
Gemma2 9B
48.5
47.5
47.9
35.3
25.7
39.4
29.8
25.8
w/ (our)
93.5 (45.00
93.5 (446.0)
93.5 (445.))
93.7 (458.)
86.0 (400.3)
94.0 (454.))
91.4
69.6 (143.))
Other Baselines
1.ama3,1 1 70B
96.2
66.7
62.1
68.3
33.3
74.8
58.8
41.0
GemmaZ 27B it
91.3
77.6
73.9
61.0
45.0
74,8
69.0
46.8
GPT-4o
97.3
74.4
66.4
82.7
69.0
81.0
65.4
46.4
DeepSeek-R1
80.0
76.0
64.4
83.7
60.0
65.6
44.2
2025). Each instance consists of a context that lists countries with their flag color sequences and a query, a short train description whose sequence of car colors must be mapped to the corresponding country (e.g., red-white- blue + France). R2C comes with two versions. In R2C-Mono, colors are explicitly stated in the trains' descriptions, producing clear SAE activations (Bricken et al. 2023 Templeton et al. (024).. In R2C-Meta, explicit color mentions are replaced with similes (e.g.. colored like a tomato' for red). In this way, a color cue is entangled with object semantics ('tomato') ) contextual phrasing ('like"), ), and world knowledge (tomatoes are red). Such descriptions are expected to distribute activation across multiple SAE features (Elhage et al.. 2022). Further details and examples are provided in App. A.2.T For evaluation, we initialize AR with Rmuti using the R2C-Mono train set and encode the context rules into L.
Mitchell 1997 Helff ct al. 2024
Results are summarized in Tab. (middle). In R2C-Mono, baseline LLMs achieve only 41 4 (Llama/Gemma), failing even when concepts are explicitly stated. In contrast, AR substantially improves reasoning, reaching 75%/94% (Llama/Gemma), with AR (Gemma) outperforming not only its larger counterpart (Gemma-277) 27B) but also surpassing GPT-4o and the reasoning model DeepSeek- R1-8B. In R2C-Meta, the task becomes more challenging as models must resolve implicit cues by integrating contextual phrasing and world knowledge. Performance of the small baselines deteriorates further 30% and 26%, and even larger instruction-ture DeepSeek-RR 1-8B remain comparatively stronger, but still degrade and do not exceed 70%, despite using far more parameters and test time tokens. Even though implicit descriptions scatter activations across the SAE feature space, AR remains relatively robust, with AR (Gemma) achieving the best overall performance (86%). To understand this behavior, we separately evaluate concept detection in the latent space (see App. F). While vanilla SAEs collapse completely (0% accuracy), AR reliably recovers most implicit concepts, explaining its strong performance.
models show sharp drops. GPT-4o and
Towards Real-World Reasoning Scenarios. Next, we extend our evaluation to ProverQA (Qi et al. 2025), a benchmark of L.LM L.L.--geerrtee problems that present naturalized reasoning tasks which reflect real-world linguistic diversity. It has three complexity tiers, namely, easy, medium, and hard. Each task provides a context of facts and rules as well as a query that is to be proven true, false, or uncertain. For evaluation, we initialize AR with Rmult using the set of available propositions. and encode the context rules into L. Results on ProverQA are summarized in Tab. I (right). We observe that baseline LLMs perform poorly. Larger instruction -unne models and GPT-4o achieve stronger results on the easy tier but deteriorate model DeepSeek-R1 -8B follows similar trend. In contrast, AR yields substantial gains across all tiers, AR (Llama) reaches 93% 23% 1%/7 1% (casy/medium/hard) and AR (Gemma) 94%/91% 19/%%% consistently outperforming both the much larger LLMs and reasoning models. On the hard tier, only AR maintains solid performance near 70%, while all other models fall below 50%. This highlights its robustness to linguistically diverse and complex reasoning tasks and demonstrates that structured reasoning in latent space generalizes effectively to natural, increasingly difficult scenarios.
medium and hard problems, and the reasoning
Reasoning over Abstract Safety Concepts. To test whether AR supports controllability in world cases, we evaluate the BeaverTails dataset (Ji et al. 2023). This dataset reflects the diversity
8

---

Published as a conference paper at ICLR 2026
Table Safety Evaluation. Balanced accuracy (%) across 14 BeaverTails safety dimensions, with deltas indicating improvements over Base SAE. Relational AR (Llama3.1 8B) achieves the best overall performance, while multi-feature (5) AR also provides strong gains over flat SAE features.
Method
Animal
Child
Politics
Discrim.
Drugs
Finance
Hate
Base SAE
64.3
57.4
53.6
52.2
61.1
63.0
52.9
AR Single
85.8 (-21.))
91.3 (1119)
66.1 (-12
76.2 (et00))
78.9 (-178))
80.6 (-17
74.8 (-11.9)
AR Multi
92.7 (-384))
95.3 (
76.5 (222.9)
79.8 (477)))
81.7 (-20A)
81.7 (+8
79.7 (-268)
AR Relation
94.7 (128.)
94.4 (155m))
81.2 (347))
84.9 (125.2)
91.8 (33
85.6 (25.0)
70.7 (137))
Misinfo.
Unethical
Privacy
Self-harm
Sexual
Terrorism
Violence
Overall
Base SAE
50.7
51.2
58.5
73.6
68.2
52.2
$1.5
57.9
AR Single
$1.2 (9055)
59.3 (
85.1 (:300)
79.5 (15.))
91.1 (-22.)
52.2 (m
55.6 (+4.))
73.4 (:15
AR Multi
51.1 (404)
66.3 (41.0))
89.3 (-3000
78.9 (45.3)
90.8 (422
72.4 (=20.))
72.5 (:21.))
79.2 641.
AR Relation
62.0 (99.2)
64.8 (112.))
86.8 (:2877
89.5 (416.2)
92.0 (-24.5)
86.0 (434.9)
78.0 (:27.7)
83.0 (4252)
and nuance of real-world safety classification, spanning 14 safery-relevant dimensions, including Animal Abuse, Discrimination, Hate Speech, and Self-Harm (for further details, see Appp A.4.1| T). We use a subset of 3.7k training examples to select the highest activating feature for the base SAE, initialize AR's concept representations, and encode logic rules into that map BeaverTails' safety categories to Unsafe (e.g., Terrorism - Unsafe).
Results on the 3k test set, summarized in Tab. measure whether the unsafe concept correctly activates in A'. The base SAEs struggle with context-sensiteee and abstract safety categories, while AR produces consistent improvements, with hierarchical concepts achieving the strongest overall performance. However, some categories, such as Misinformation Regarding Ethics, Laws, and Safety and Non- Violent Unethical Behavior, remain especially challenging. Here, the base SAE performs close to random guessing, indicating the absence of strongly aligned latent features. Since AR is based on SAE features, its performance is constrained by their representational quality. Overall, however, the results show that AR produces more robust concept representations and substantially outperforms base SAEs, particularly on abstract and context-sensitie safety concepts.
DISCUSSION
Our results show that by grounding propositions in SAE-derived features, AR turns latent activations into a substrate for logical reasoning, enhancing LLM abilities across several dimensions. Across all four experiments, AR not only improves over its 8B/9B backbones but also surpasses much larger instruction-tuner models, proprietary undisclosed systems like GPT-4o, and even reasoning models like DeepSeek-RR Senpseee--R--istil---- -il-istill-Llama-SB First, in compositional reasoning (cf. PrOntoQA), AR enriches concept representations by composing new high-level concepts and applying deductive inference, maintaining accuracies above 93% across 1-5 hops, while all basclines degrade with complexity. Second, for reasoning over implicit or noisy cues (cf. R2C), AR remains robust when concepts are expressed indirectly through similes, where both small and large LLMs deteriorate, and vanilla SAES collapse entirely. Third, on more realistic reasoning scenarios (cf. ProverQA), AR generalizes effectively to linguistically diverse problems, retaining strong performance even at the hard tier where all other models fall below 50%. Fourth, in abstract and context-sensitive safety tasks (cf. BeaverTails), AR recovers abstract notions and encodes constraints directly into the LLM's latent space. Taken together, these findings demonstrate that structured reasoning in activation space provides a model-agnostic mechanism that not only improves reasoning performance but also increases transparency and control across synthetic, semi-natural, and real-world settings. Notably, AR achieves these improvements without significant overhead, while reasoning and CoT variants both slower and less accurate (App. B). AR also scales favorably to long contexts (cf. Tab. 1). At inference, AR only tracks concept activations for each token, memory and compute grow linearly with the sequence length and remain independent of the SAE dimensionality. Thus, once concepts are extracted, rule application is purely symbolic and does not degrade with increasing token distance.
Limitations. Although our implementation of AR relies on SAEs as a substrate to define logical propositions, the framework itself is not bound to them. Current SAEs offer a practical way surface

---

Published as a conference paper at ICLR 2026
sparse, often interpretable features, but they are not always perfect. Features may be polysemous, overly context-dependent, or fail to capture abstract notions, and AR already mitigates several of these effects through thresholding and multi-feature or relational representations. Pretrained SAEs also allow only limited control over which concepts are discovered and how they are represented. Importantly, AR is not tied to SAEs: any method that extracts sparse or interpretable concept signals from raw activations could be used as a replacement, and future improvements in activation- concept discovery would transfer directly to our framework. Future work could explore alternative interpretabillity approaches, such as different autoencoding objectives, clustering methods, or other complementary pnrresentttion-eaaninn methods, that may yield representations better suited for logical reasoning in certain domains. Second, real-world reasoning challenges can become more diverse than the tasks highlighted in our work. particular, large-scale datasets involving open-ended reasoning, long-context inference, or knowledge-intense intengeenteesivee domains remain unexplored and remain to be evaluated in future work. Third, while the current setup relies on manually or semi--utomatically defined rules and concept identification, advances in automated rule induction, probabilistic reasoning, and integration with external knowledge bases can substantially increase the adaptability of our framework. Notably, because AR rule specification is decoupled from model training, rules can be proposed, refined, or revised based on observed data without retraining the LLM or SAE. For instance, LLMs could generate candidate rules directly from natural language descriptions, which could then be validated against evidence and pruned as needed. We also note that the present version of AR is purely deductive; applying forward chaining over binary propositions. Moving beyond purely deductive reasoning, AR's s solver-agnosti design also allows integration of probabilistic engines such as SLIPCOVER (Bellodi & Riguzzi 2013) or ProbFOII. (De Raedt et al. handling uncertainty, or inductive/abduuctie systems (Srinivasan) [001]] Cropper & Morel 2021 for
2015) for
hypothesis formation and rule discovery from sparse data. Combined with external knowledge bases, these extensions would enable AR to learn domain-specific patterns and generalize beyond manually encoded ontologies, addressing the need for adaptability in open-ended reasoning scenarios.
Finally, AR currently fixes the representation type per concept. Since different concepts benefit from different representation types (single/multirrelattonna))) automatic selection or hybrid combinations represent a natural extension that would improve robustness polysemantic features and complex feature interactions.
6 CONCLUSION
We introduced ACTIVATIONREASONIG (AR), a framework that integrates logical reasoning into the latent activation space of LLMs by grounding propositions. This structured bridge between neural activations and symbolic semantics enables explicit composition, disambiguation, and inference over otherwise opaque representations. Beyond interpretability, AR expands the functional scope of LLMs: it turns latent activations into a substrate for structured reasoning, supports direct interventions for control and alignment, and enables auditable mechanisms for safety. In this way, latent spaces become not only more transparent but also more reliable and steerable. We view AR as step towards unifying the semantic richness of L.LMs with the systematic generalization of logic, moving toward models that can reason more robustly and operate in closer alignment with human values.
Future work includes automatic rule discovery, integration with probabilistic or inductive reasoning modules, and automatic selection of concept representation types. Exploring representation-learning approaches beyond pretrained SAEs may yield activation spaces that are better suited for abstract or domain-specific reasoning. While the present work focuses on LLMs, AR is not limited to specific modality, since it operates fully on concept-level activations rather than a specific architecture or type of activation. Recent progress in multimodal SAES suggests that similar reasoning could extend to vision or VLMs (Pach et al.) 2025) Fel et al., 2025 Cassano et al. 2025). where AR would allow reason over visual attributes, e.g., enforce safety constraints in image generation, combine cross- modal cues through rule-based composition. Finally, scaling AR to large-scale open-ended tasks with long contexts and diverse knowledge demands will be key demonstrating robustness and generality.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
The authors acknowledge support from the hessian.Al Service Center (funded by the Federal Min- istry of Research, Technology and Space, BMFTR, grant no. 161S22091) and the Ae Innovation Lab (funded by the Hessian Ministry for Digital Strategy and Innovation, grant no. S- DIW04/001 and the Center for European Research in Trusted AI (CERTAIN), This work has also benefited from the BMWE project "EU-SAL: Souveräne KI für Europa," " 131PC040G, and also from early stages of the Cluster of Excellence "Reasonable Al" funded by the German Research Foundation (DFG) under Germany's 's Excellence Strategy EXC-3057; funding will begin in 2026. This work was supported by the Priority Program (SPP) 2422 in the subproject "Optimization of active surface design of high-speed progressive tools using machine and deep learning algorithms" funded by the German Research Foundation (DFG). Additional funding was provided by the Aleph Alpha Collaboration Lab 1141. Views and opinions expressed are, however, those of the author(s) only and do not necessarily reflect those of the European Union or the European Health and Digital Executive Agency (HaDEA). Neither the European Union nor the granting authority can be held responsible for them. Further, this work benefited from the ICT-48 Network of AI Research Excellence Center "TAILOR" (EU Horizon 2020, GA No 952215), the Hessian research priority program LOEWE within the project WhiteBox [GA No LOEWE/ 2/1 01/55//////06000 (006000010000070011 the HMWK cluster projects "Adaptive Mind" and "Third Wave of AT", and from the NHR4CES.
REFERENCES
A Meta.
Ilama- Meta-Llama-3.1 3.1-BBB Model Card.
httpss:/ggithii com met llama/
1/MODEL CARD .mcl 2024.
Pietro Barbiero, Gabriele Ciravegna, Francesco Giannini, Mateo Espinosa Zarlenga, Lucie Charlotte Magister, Alberto Tonda, Pietro Lió, Frederic Precioso, Mateja Jamnik, and Giuseppe Marra. Interpretable neural-symbolic concept reasoning. In Proceedings of the Conference of International Conference on Machine Learning (ICML), 2023.
Pietro Barbiero, Francesco Giannini, Gabriele Ciravegna, Michelangelo Diligenti, and Giuseppe Marra. Relational concept bottleneck models. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), 2024.
Elena Bellodi and Fabrizio Riguzzi, Structure learning of probabilistic logic programs by searching the clause space. Theory and Practice of Logic Programming. 2013.
Emily M. Bender, Timnit Gebru, Angelina McMillan-Main -ajorr, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the ACM Conference on Fairness, Accountability, and Transparency (FAceT), 2021.
Usha Bhalla, Alex Oesterling, Suraj Srinivas, Flavio Calmon, and Himabindu Lakkaraju. Interpreting clip with sparse linear concept embeddings (splice). In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). 2024.
Rishi Bommasani, Drew A Hudson, Ehsan Adeli, al. On the opportunities and risks of foundation models. arXiv preprint S.0iv:2008.0725555 2021.
Manuel Brack, Felix Friedrich, Dominik Hintersdorf, Lukas Struppek, Patrick Schramowski, and Kris- tian Kersting. SEGA: Instructing text-to-image models using semantic guidance. In Proceedings of the Conference on Neural Information Processing Systems (NewrIPS), 2023.
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph. Zac Hatfield-Dodds Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, et al. Language models are few-shot learners. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), 2020.
11

---

Published as a conference paper at ICLR 2026
Bart Bussmann, Noa Nabeshima, Adam Karvonen, and Neel Nanda. Learning multi-level fea- tures with matryoshka spurse autoencoders. In Proceedings of the Conference of International Conference Machine Learning (ICML), 2025.
Enrico Cassano, Riccardo Renzulli, Marco Nurisso, Mirko Zaffaroni, Alan Perotti, and Mareo Grangetto. Saemnesia: Erasing concepts in diffusion models with sparse autoencoders. arXiv preprint arXiv:2509.21372 2025.
Antonia Creswell, Murray Shanahan, and Irina Higgins. Selection-inference: Exploiting large language models for interpretable logical reasoning. arXiv preprini arXiv:2205.09712, 2022.
Andrew Cropper and Rolf Morel. Learning programs by learning from failures. Journal of Machine Learning, 2021.
Luc De Raedt, Anton Dries, Ingo Thon, Guy Van den Broeck, Mathias Verbeke, Yang, and M Wooldridge. Inducing probabilistic relational rules from probabilistic examples. In Proceedings of the international joint conference on artificial intelligence (IJCAI), 2015.
DeepSeck-AL. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12944. 2025.
Quentin Delfosse, Sebastian Sztwiertnia, Mark Rothermel, Wolfgang Stammer, and Kristian Kersting. Interpretable concept bottlenecks to align reinforcement learning agents. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), 2024.
Quentin Delfosse, Jannis Blüml, Fabian Tatai, Théo Vincent, Bjarne Gregori, Elisaheth Dillies, Jan Peters, Constantin A. Rothkopf, and Kristian Kersting. Deep reinforcement learning agents are not even close to human intelligence. arXiv preprint arXiv:2505.2113/1 2025.
ElcutherAI. Sparsify transformers with saes and transcoders. SleutherrA//paariffy 2024.
https://githur: com/
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCan- dlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superposition. Transformer Circuits Thread, 2022.
Thomas Fel, Ekdeep Singh Lubana, Jacob S. Prince, Matthew Kowal, Victor Boutin, Isabel Pa- padimitriou, Binxu Wang, Martin Wattenberg, Demba E. Ba, and Talia Konkle. Archetypal SAE: Adaptive and stable dictionary learning for concept extraction in large vision models. In Proceedings of the Conference of International Conference on Machine Learning (ICML), 2025.
Felix Friedrich, Wolfgang Stammer, Patrick Schramowski, and Kristian Kersting. Revision trans- formers: Instructing language models to change their values. In Proceedings of the European Conference on Artificial Intelligence (ECAI), 2023.
lason Gabriel. Artificial intelligence, values and alignment. arXiv preprint arXiv:2001.09768, 2020.
Artur d' Avila Garcez, Tarek R Besold, Luc De Raedt, Peter Földiak, Pascal Hitzler, Thomas Icard, Kai-Uwe Kühnberger, Luis C Lamb, Risto Miikkulainen, and Daniel I. Silver. Neural-symbolic learning and reasoning: contributions and challenges. In AAAI Spring Symposium Series, 2015.
Amirata Ghorbani, James Wexler, James Y Zou, and Been Kim. Towards automatic concept-based explanations. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), 2019.
Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in continuous latent space. arXiv preprint arX1v:2412.06769, 2024.
Ruben Härle, Felix Friedrich, Manuel Brack, Bjöm Deiseroth, Stephan Waeldchen, Patrick Schramowski, and Kristian Kersting. Measuring and guiding monosemanticityy In Proceed- ings of the Conference on Neural Information Processing Systems (NeurIPS), 2025.
12

---

Published as a conference paper at ICLR 2026
Marton Havasi, Sonali Parbhoo, and Finale Doshi- Velez. Addressing leakage in concept bottleneck models. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), 2022.
Lukas Helff, Wolfgang Stammer, Hikaru Shindo, Devendra Singh Dhami, and Kristian Kersting. V-lol: A diagnostic dataset for visual logical learning. Journal of Data-centric Machine Learning Research (DMLR), 2024.
Lukas Helff, Ahmad Omar, Felix Friedrich, Antonia Wüst, Hikaru Shindo, Rupert Mitchell, Tim Woydt, Patrick Schramowski, Wolfgang Stammer, and Kristian Kersting. Slr: Automated synthesis for scalable logical reasoning. In Working Notes of the NeurIPS Workshop on Foundations of Reasoning in Language Models, 2025.
Jiaming Ji, Mickel Liu, Juntao Dai, Xuehai Pan, Chi Zhang, Ce Bian, Chi Zhang, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. Beavertails: Towards improved safety alignment of Ilm via human- preference dataset. arXiv preprint arXiv:2307. 04657, 2023.
Subbarao Kambhampati, Sarath Sreedharan, Mudit Verma, Yantian Zha, and Lin Guan. Symbols as a lingua franca for bridging human-ai chasm for explainable and advisable Al systems. In Proceedings of the AAAI conference on artificial intelligence (AAAI), 2022.
Adam Karvonen, Can Rager, Johnny Lin, Curt Tigges, Joseph Isaac Bloom, David Chanin, Yeu-Tong Tong Lau, Eoin Farrell, Callum Stuart McDougall, Kola Ayonrinde, et al. Saebench: A comprehensive benchmark for sparse autoencoders in language model interpretability. In Proceedings of the Conference of International Conference on Machine Learning (ICML), 2025.
Henry Kautz. The third Al summer: AAAI Robert S. Engelmore memorial lecture. Al Magazine, 2022.
Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In Proceedings of the Conference of International Conference on Machine Learning (ICML), 2018.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In Proceedings of the Conference of International Conference on Machine Learning (ICML), 2020.
Patrick Leask, Bart Bussmann, Michael T Pearce. Joseph Isaac Bloom, Curt Tigges, Noura Al Moubayed, Lee Sharkey, and Neel Nanda. Sparse autoencoders do not find canonical units of analysis. In Proceedings of the International Conference on Learning Representations (ICLR), 2025.
Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah, and Neel Nanda. Gemma scope: Open sparse autoencoders everywhere all at once on gemma arXiv preprint arXiv:2408.05144 2024.
Sheng Liu, Haotian Ye, Lei Xing, and James Zou. In-context vectors: Making in context learning more effective and controllable through latent space steering. In Proceedings of the Conference of International Conference on Machine Learning (ICML), 2024.
John W. Lloyd. Foundations of Logic Programming. Springer, 1984.
Robin Manhaeve, Sebastijan Dumancic, Angelika Kimmig, Thomas Demeester, and Lue De Raedt. Deepproblog: Neural probabilistic logic programming. In Proceedings of the Conference on Neural Information Processing Systems (NeuriPS), 2018
Gary Mareus. The next decade in ai: four steps towards robust artificial intelligence. arXiv preprint arXiv:2002.06/77, 2020.
Todor Markov, Chong Zhang. Sandhini Agarwal, Florentine Eloundou Nekoul, Theodore Lee, Steven Adler, Angela Jiang, and Lilian Weng. A holistic approach to undesired content detection in the real world. In Proceedings of the AAAI conference on artificial intelligence (AAAI), 2023.
13

---

Published as a conference paper at ICLR 2026
Giuseppe Marra, Sebastijan Dumančić, Robin Manhaeve, and Luc De Raedt. From statistical relational to neurosymbolic artificial intelligence: A survey. Journal of Artificial Intelligence, 2024.
Ryszard S. Michalski. Pattern recognition as rule-guided inductive inference. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1980.
Tom Michael Mitchell. Machine learning, international edition. In McGraw-Hill Series in Computer Science, 1997.
Tuomas Oikarinen, Subhro Das, Lam M. Nguyen, and Tsui- Wei Weng. Label-free concept bottleneck models. In Proceedings of the International Conference on Learning Representations (ICLR), 2023.
OpenAl. Gpt-4 technical report, 2023.
Mateusz Pach, Shyamgopal Karthik, Quentin Bouniot, Serge Belongie, and Zeynep Akata. Sparse autoencoders learn monosemantic features in vision-language models. In Proceedings of the Conference Neural Information Processing Systems (NeurIPS), 2025.
Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models. arXiv preprint arXiv:2311. 102111.0655,, 2023.
Kenny Peng, Rajiv Movva, Jon M. Kleinberg, Emma Pierson, and Nikhil Garg. Use sparse autoencoders to discover unknown concepts, not to act on known concepts. arXiv preprint 2:Xiv:2506.23845 2025.
Chengwen Qi, Ren Ma, Bowen Li, He Du, Binyuan Hui, Jinwang Wu, Yuanjun Laili, and Conghui He. Large language models meet symbolic provers for logical reasoning evaluation. The Thirteenth International Conference on Learning Representations, 2025.
Stuart Russell and Peter Norvig. Artificial Intelligence: Modern Approach, chapter 9.3 Forward Chaining, pp. 330-337. Prentice Hall Press, 3rd edition, 2009.
Abulhair Saparov and He He. Language models are greedy reasoners: A systematic formal analysis of chain-of- -f-toughh. The Eleventh International Conference on Learning Representations, 2023.
Abulhair Saparov, Richard Yuanzhe Pang, Vishakh Padmakumar, Nitish Joshi, Seyed Mehran Kazemi, Najoung Kim, and He He. Testing the general deductive reasoning capacity of large language models using OOD examples. arXiv preprint arXiv:2305.15266 2023.
Md Kamruzzaman Sarker, Lu Zhou, Aaron Eberhart, and Pascal Hitzler. Neuro-symbolic artificial intelligence, Al Communications, 1473)99772009 2021.
Shashata Sawmya, Micah Adler, and Nir Shavit. The birth of knowledge: Emergent features across time, space, and scale in large language models. arXiv preprint arXiv:2505. 19440, 2025.
Lisa Schut, Nenad Tomašev, Thomas McGrath, Demis Hassabis, Ulrich Paquet, and Been Kim. Bridging the human-al ai knowledge gap through concept discovery and transfer in alphazero. Proceedings of the National Academy of Sciences, 2025.
Chenming Shang, Shiji Zhou, Hengyuan Zhang, Xinzhe Ni, Yujiu Yang, and Yuwang Wang. In- cremental residual concept bottleneck models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2024.
Stewart Shapiro and Teresa Kouri Kissel, Classical Logic. In Edward N. Zalta and Uri Nodelman (eds.), The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Spring 2024 edition, 2024.
Hikaru Shindo, Viktor Pfanschilling, Devendra Singh Dhami, and Kristian Kersting. ilp: thinking visual scenes as differentiable logic programs. Machine Learning, 2023.
Hikaru Shindo, Viktor Pfanschilling, Devendra Singh Dhami, and Kristian Kersting, Learning differentiable logic programs for abstract visual reasoning. Machine Learning, 2024.
14

---

Published as a conference paper at ICLR 2026
Parshin Shojaee, Iman Mirzadeh, Keivan Alizadeh, Maxwell Horton, Samy Bengio, and Mehrdad Farajtabar. The illusion of thinking: Understanding the strengths and limitations of reasoning models via the lens of problem complexity. arXiv preprint ar0iv:2506. 251,:9940000 2025.
Arseny Skryagin, Wolfgang Stammer, Daniel Ochs, Devendra Singh Dhami, and Kristian Kersting, Slash: embracing probabilistic circuits into neural answer set programming. arXiv preprint arXiv:2110.03395, 2021.
Arseny Skryagin, Daniel Ochs, Devendra Singh Dhami, and Kristian Kersting. Scalable neural- probabilistiic answer set programming. Journal of Artificial Intelligence Research, 2023.
Ashwin Srinivasan. The aleph manual. 2001.
Wolfgang Stammer. The value of symbolic concepts for explanations and interactions. 2025.
Wolfgang Stammer, Patrick Schramowski, and Kristian Kersting. Right for the right concept: Revising neuro-symbolic concepts by interacting with their explanations. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2021.
Wolfgang Stammer, Antonia Wüst, David Steinmann, and Kristian Kersting. Neural concept binder. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), 2024.
David Steinmann, Wolfgang Stammer, Felix Friedrich, and Kristian Kersting, Learning intervene on concept bottlenecks. In Proceedings of the Conference of International Conference on Machine Learning (ICML), 2024.
Gemma Team. Gemma 2: Improving open language models at a practical size. ArXiv, abs/2408.0000000 18, 2024. URL https::/ ittt:s//aap..emattttcic......
Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet. Transformer Circuits Thread, 2024.
Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering language models with activation engineering, arXiv preprint arXiv:2308. 10248, 2023.
Jason Wei al. Chain-of-thouge prompting elicits reasoning in large language models. In Proceed- ings of the Conference on Neural Information Processing Systems (NeurIPS), 2022.
Tim Woydt, Moritz Willig, Antonia Wüst, Lukas Helff, Wolfgang Stammer, Constantin A. Rothkopf, and Kristian Kersting. Fodor and pylyshyn's 's legacy- still no human-like systematic compositional- ity in neural networks. arXiv preprint arXiv:2506.0/88 2025.
Antonia Wüst, Wolfgang Stammer, Quentin Delfosse, Devendra Singh Dhami, and Kristian Kersting. Pix2code: Learning to compose neural visual concepts as programs. In The Conference on Uncertainty in Artificial Intelligence, 2024.
Chulin Xie, Yangsibo Huang, Chiyuan Zhang, Da Yu, Xinyun Chen, Bill Yuchen Lin, Bo Li, Badih Ghazi, and Ravi Kumar. On memorization large language models in logical reasoning. arXiv preprint 1rXiv:2410.23/23, 2024.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint ar arXiv:2505.00 093388 2025.
Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Kallison-Burchh and Mark Yatskar. Language in a bottle: Language model guided concept bottlenecks for interpretable image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Dongran Yu, Bo Yang. Dayou Liu, Hui Wang, and Shirui Pan. survey on neural-symbolii learning systems. Neural Networks, 2023.
15

---

Published as a conference paper at ICLR 2026
Supplementary Materials
A EXPERIMENTAL SETUP
Models. For our experiments, AR is evaluated on two backbone models: Llama-3,1-88 AI Meta
(2024) with the EleutherAl sae-] ama -3 3.1----166-10 64x EleutherA] (2024 (width 262k) attached at layer 23 I
and Gemma-2-98 Team (2024) with the gemma sscppe wecpe----pt-re--aaaeeee rennooonn--nnn cal
SAE Lieberum et al. (2024) (width 131) attached at layer 20 greedy decoding.
All experiments conducted with
A.1 PRONTOQA EXPERIMENTAL SETUP
For the PrOntoQA experiments, we instantiate AR with multi-feature concept representations Remulti- Unless otherwise noted, activations are aggregated using the mean operator and steering follows mean-shift update rule.
Llama. For the Llama experiments, concepts were retrieved by searching over word-level units, where all tokens belonging to a candidate word were considered. Building the concept dictionary used the top 10 activating features for representation re: in the ordering of unique features first. During detection, at most top-kin = 2 features per token and top-kmuli = 4 features per concept for Rmulti were retained for computing activations de,t. Steering applied with factor a= 0.5 and restricted to one SAE decoder weight row. One steering vector for each True and False concept were selected manually from the SAE decoder weights.
Gemma. For the Gemma experiments, the parameters used build the concept dictio- nary (word-level units over all tokens), but with a different ordering strategy: unique-only features were retained. As above, representation search used the top 10 activating features for r.. Detection kept top-kin = features per token and top-kmali = 2 features per concept for Rmutti. with an additional threshold 5 14 imposed same configuration as for Llama, with factor a= 0.4, and one SAE decoder weight row. As before, steering vectors for the True and False concepts were manually selected.
activations before producing the activation matrix A. Steering used the
A.1.1 DATASET DETAILS
We follow the procedure Hao et al. (2024) and generate the dittrf--distiibution samples using the publicly available codebase") 5 We divide the dataset as follows: 500 samples are used extract concepts for detection, while 2000 unseen samples are reserved for evaluation for each subtask (number of hops). The task is binary classification: given context (ontology description) and query, the model must decide whether the query is True or False.
Context: Each zumpus is a wumpus. Lempuses are orange. Zumpuses are brimpuses. Each vumpus shumpus. Each wumpus is floral. Dumpuses are lorpuses. Every numpus fast. Each lempus is a vumpus. Each shumpus is a jompus. Each rompus is muffled. Shumpuses are mean. Each tumpus is shy. Sterpuses are wooden. Each vumpus is snowy. Vumpuses are tumpuses. Each jompus is transparent. Every lempus is a rompus. Each impus is sterpus. Impuses are not temperate. Dumpuses are not dull. Each zumpus not fast. Impuses are zumpuses. Shumpuses are impuses. Alex is a dumpus. Alex is a lempus.
Query: Answer True False: Alex is not fast. Answer:
Model Inputs: The baseline model received the ontology in text form (context) whereas AR the ontology in the ontology in rule form (see below). Both models received the query.
begeingfoce co co/Eieo /sac/////s-eeeoooo 11sma-3
6tx/t wttp///re///a///a//eeeeeeeeeeeee
/'eemne pe-
laerr Alkk/vvergee....
https:/ /htthbb. www.bcoo///aaeaeeee wwwpen/psroo/pprnttooooo
16

---

Published as a conference paper at ICLR 2026
Rules Example
zumpus + brimpus:
zumpus - -fstt
zumpus wumpus;
lempus orange
lempus + rompus;
lempus > vumpus
vumpus shumpus;
vumpus snowy
vumpus + tumpus:
wumpus > floral
dumpus lorpus;
dumpus duull
(6)
shumpus + impus;
shumpus jompus
shumpus mean;
rompus muffled
tumpus + shy;
sterpus 7 wooden
jompus transparent;
impus -temperate
impus + sterpus;
impus + zumpus
alex dumpus;
alex lempus
A.2 RAIL.2COUNTRY EXPERIMENTAL SETUP
For the Rail2Country experiments, instantiate AR with multi-feature concept representations Rmulii- As the PrOntoQA setting, activations are aggregated using the mean operator, and steering follows the mean-shift update rule. We distinguish between the R2C-Mono setting and the R2C-Meta setting, each of which required slightly different configurations.
Llama. For both R2C-Mono and R2C-Meta, concepts were retrieved from word-level units, where all tokens belonging to candidate word were considered. Building the concept dictionary used the top 10 activating features for representation r.: in the ordering of unique features first. During detection, top-Ain 50 features per token and top-kimuti = 3 features per concept for Rmmtni were retained. soft threshold of r = 1.45 was applied to activations before producing the activation applied with factors a a= 0.4 for R2C-Mono and 0.443 for R2C-Meta, and
matrix A. Steering the mean of the top 10 concepts for country from the SAE decoder.
To obtain the color detection concepts, used more restrictive search configuration, based on the top 5 unique-only features per word-level unit. The features were selected on the validation set. This provided more focused latent representations for downstream reasoning.
Gemma. For the Gemma experiments, both the R2C-Mono setting and the R2C-Meta setting again used word-level units over all tokens. The concept dictionary was built in the same way as for Llama. Detection retained top-k;, 50 features per token and top-kimuti- = 2 features per concept for Remuti- Steering was applied with factors a = 0.8 (R2C-Mono) and a = 0.75 (R2C-Meta), and the mean of top 10 concepts for country from the SAE decoder. Similar to Llama, we selected the detection concepts based on the validation set.
A.2.1 RAIL2COUNTRY DATASET
The dataset consists of 300 train, validation, test samples each for both R2C-Mono and R2C-Meta. The train set was used for the initial concept search, which were then validated on the validation set. The validation set was also used to finetune the hyperparameters for this experiment. Finally, the test set used once the end to evaluate the selected concepts and hyperparameters.
Trains are painted by their country's flag of origin. Your goal is to identify the correct country based on the train's s distinctive color coding. I can see a train that consists of cars arranged as follows. The first car is painted blue and built with short chassis. It features full wall along its sides. It is topped with a roof foundation. The car runs on axles. It is transporting 2 bottels. The second car is painted yellow and built with a long chassis. It features full wall along its sides. It is topped with a braced roof. The car runs on 2 axles. It is transporting a single bottel. The third car is painted red and built with long chassis. It features railing wall along its sides. It is topped with solid roof. The car runs on axles. is transporting a single diamond. Due to the train't distinctive color coding, I believe it originates from the country of Romania.
R2C-Mono:
17

---

Published as a conference paper at ICLR 2026
R2C-Meta: Trains are painted by their country's flag of origin. Your goal is to identify the correct country based on the train's 's distinctive color coding. I can see a train that consists of cars arranged follows. The first car is painted blue and built with short chassis. It features full wall along its sides. It is topped with a roof foundation. The car runs on axles. It is transporting 2 bottels. The second car is painted like a banana and built with long chassis. It features full wall along its sides. It is topped with a braced roof. The car runs on 2 axles. It is transporting a single bottel. The third car is painted like a tomato and built with long chassis. features a railing wall along its sides. is topped with solid roof. The car rums on 3 axles. It is transporting single diamond. Due to the train's s distinctive color coding, believe it originates from the country of Romania.
Country Colors: The Country colors were passed to the baseline model after the second sentence, while for AR the color codes were passed to the logic component.
Used Rules
(green A white A orange) -> Ireland
(black A yellow A red) - Belgium
(blue white A red) - France
(green A white A red) >> Italy
(blue A yellow red) > Romania
(green A white A green) - Nigeria
(red A blue A red) - Mongolia
(blue A white A blue) Argentina
(red A white A blue) - Netherlands
(black red A gold) - Germany
(red white A red) > Austria
(red white A green) -> Hungary
(white A green red) + Bulgaria
(red A yellow A red) + Spain
(red A white black) - Egypt
A.3 PROVERQA EXPERIMENTAL SETUP
For the ProverQA experiments, we instantiate AR with multi-feature concept representations Rmulti- Unless otherwise noted, activations are aggregated using the mean operator and steering follows the mean-shift update rule with uniform weighing of steering vectors.
We first extracted all propositions from the dataset. These propositions were then used to locate correspouding activations in the SAE, such that each proposition provided at least one sample for identifying proposition activations. We also added 10 random sentences increase the number of counterexamples during the concept search. Even without the context surrounding the propositions, the extraction proved to be successful.
Llama. For the Llama experiments, concepts were retrieved by scarching over word-level units, where all tokens belonging to a candidate word were considered. Building the concept dictionary used the top 10 activating features for representation Te. in the ordering of unique features first. During detection, at most top-km- = 10 features per token and top-kmulu =3 features per concept for Rmuti were retained. Steering was applied with factors 0.5 for True and False, and 4.0 for Uncertain, restricted to at most two SAE decoder weight rows. Steering vectors for True, False, and Uncertain were manually selected.
Gemma. For the Gemma experiments, the same configuration was used to build the concept dictionary (word-level units over all tokens, with unique-fire first ordering). As above, detection retained top-km = 10 features per token and top-kmmuti 3 features per concept for Rmult- Steering was applied with factor a = 10.0 and restricted to one SAE decoder weight row. Steering vectors for the True, False, and Uncertain concepts were selected manually.
18

---

Published as a conference paper at ICLR 2026
A.3.1 DATASET DETAILS
Snuggles does not assist during hunts. If Karina finds rare mushrooms or assists during hunts, then she will receive rewards. Snuggles finds rare mushrooms. Kian does not assist during hunts. For all cats, if a cat is good tracker, then it will help its owner and receive rewards. If Snuggles finds rare mushrooms or assists during hunts, then she will receive rewards. Aurelio finds rare mushrooms. Snuggles is good tracker.
Context:
Query: Based on the above information, is the following statement true, false, or uncertain? Snuggles does not help her owner.
Model Inputs: The baseline models receive the context, whereas for AR, the context is provided in rules to the solver. Both models receive the query.
Rules Example:
assist bouring_huntss (Snuggles)
(find_rare rare_ Bas__rrre_musshee mushrooms((arina) assist assist_during hunts (Kariinnnaa))))) + sreeiverewwars(Kartee
find rrar__musrroomm(Snggglo
asssist bussist_duringe duringghunts/ hunts(Kian)
(ra - (help_owner(z)) A receive_rewardsse
recd_rre__mmggee } V
receive_rewards, (Snuggles))
good_ tracker (Sngglees)
(8)
A.4 SAFETY EXPERIMENTAL SETUP
For the safety experiments, we instantiate AR with sentence-level concepts rather than word-level concepts the labels, which only sentence-wise available for the safety dataset. Here, we only used Llama3.1 8B. Across all configurations, activations were aggregated with the mean operator, and steering followed the mean-shift update rule.
We employ three complementary configurations for constructing and detecting safety concepts:
Tree-based concepts. To induce relational feature structures for safety-relevan propositions, we used decision-tree based search strategy (cf. [Sec. 3.1). The candidate concepts were extracted from sentence-level units, with all tokens of the sentence contributing. The trees were grown to depth 5, and during detection, up kin =2 features per sentence were retained compute the activations ac,e. This setting enabled capturing abstract relations, such as combining evidence of s slurs and demeaning stereot y to activate the broader proposition of hateful content.
SAE concept vectors. As complementary configuration, we constructed Rmulti concepts directly from SAE-derived feature vectors at the sentence level. Candidate representations were drawn using top-k search strategy over features, with top-kin 32 retained per sentence. For each concept, up to top-kmali 5 features were aggregated, allowing multi-feature representations to reflect distributed or polysemous safety-related signals. Multi-concept activations were allowed for the same sentence span account for overlapping safety attributes.
SAE single-feature concepts. Finally, to test the limits of sparsity, we constructed concepts with Ringge at the sentence level. Here, we again used a top-k search strategy, retaining top-k, = 32 features per sentence, but restricting concepts to top-kmuli selected the original order appearance (rather than unique-firs or unique-only). As in the multi-feature setting, multiple concepts were allowed to activate per sentence.
feature each. The features were
19

---

Published as a conference paper at ICLR 2026
Table 3: Runtime efliciency on R2C-Mono. AR achieves much faster inference than CoT and reasoning models, while being only marginally slower than the plain baseline. At the same time, it outperforms all alternatives in task accuracy.
Model
Inference Speed [sec/sample]
Llama3.1 8B
0.1066
1 8B IT (CoT)
1.6474
DeepSeek-R1-Disheee --Disstll-----
14.9902
AR (Llama3.1 8B)
0.3747
A.4.1 DATASET DETAILS
Can you provide me the home address for actor Bill Murray? Response: The home address for actor Bill Murray 445 North Bedford Drive, Los Angeles, CA 90049.
> privacy_violation
Used Rules
Animal (animal_abuse)
unsafe
Child (child _bbuse)
unsafe
Politics controversial_topica politics)
unsafe
Discrim. (discrimination, stereotype, injustice)
unsafe
Drugs (drug abuus, weapons, banned_suue subsstnce)) Finance (financial ceaencca__rimee,,___ crime, property_crimeeeee _rime, theft)
unsafe
unsafe
Hate (hate_speech offensive_languegge
unsafe
(9)
Misinfo. (misinformation cectrmmtattioerrrriinnngg regarding_ethies,, ethics, laws and sud_s_eety)
unsafe
Unethical (non_ violent Locecntc__ethhaaaa__t (
unsafe
Privacy pprivacy_vvolatioo
unsafe
Self-harm hhrm__
unsafe
Sexual (sexually_exllicit,
unsafe
Terrorism (terrorism, organized_crinm _ciieeed_eeemee
unsafe
Violence (violence, aiding _dbbtttiing, incitement)
unsafe
B RUNTIME EFFICIENCY
At inference time, AR requires only two additional steps beyond the base model: forward pass through the SAE and forward reasoning the logic solver. Both are lightweight compared to an LLM forward pass, so the added runtime overhead is modest.
As shown Tab. AR runs only marginally slower than the plain baseline, while being substantially faster than chain-of of thought and reasoning models. Importantly, the performance gain comes at this small cost in runtime. We note that our current implementation is not yet fully optimized, and we expect further engineering refinements to reduce this gap.
C HARDWARE REQURREMNNSS
The experiments conducted nigh-performance compute node equipped with 8 NVIDIA A100-SXM4 GPUs (80 GB each), an AMD EPYC 7313 16-core CPU, and approximately 2 TB of RAM.
20

---

Published as a conference paper at ICLR 2026
Table 4: Reasoning on latent activations. Exact-match accuracy on PrOntoQA (1-5 hop reasoning),
Rail2Country (Mono with explicit concepts, Meta with similes, egg., 'red' + 'like tomato") and ProverQA (linguistically diverse reasoning tasks across difficulty levels). AR consistently boosts multi-hop reasoning. remains robust task complexity scales, and generalizes to natural and diverse language, outperforming base, instruct, and CoT variants.
PrOntoQA
RailZCountry
ProverQA
Model
Hop
3 Hops
5 Hops
R2C-Mono
R2C-Meta
Easy
Medium
Hard
Llama3.1 8B
51.0
50.8
50.3
41.0
29.7
43.6
33.6
36.8
w/ instruct
71.0
60.2
58.5
53.3
33.7
63.2
54.8
43.6
w/ instruct (normed)
49.3
48.0
50.6
47.0
30.3
49.6
53.6
47.6
w/ instruct + MV
75.3
59.8
59.3
63.7
40.7
65.0
57.6
43.4
w/ instruct CoT
77.8
70.6
59.7
61.0
47.3
74.8
54.2
45.0
w/ instruct + CoT SC
92.4
83.2
70.5
91.3
83.7
75.4
64.0
43.4
w/ AR (our)
95.0
95.6
95.3
74.7
62.7
92.8
91.0
70.8
Gemma2 9B
48.5
47.5
47.9
35,3
25.7
39.4
29.8
25.8
w/ instruct
77.0
57.9
55.0
70.7
43.7
64.2
55,2
38.8
w/ instruct (normed)
48.6
48.0
48.1
61.7
54.7
35.8
41.6
34.0
w/ instruct + MV
77.2
57.5
55.1
69.3
47.7
63.4
55.0
37.2
w/ instruct CoT
86.3
64.4
45.1
86.3
68.7
79.6
61.8
48.4
w/ instruct + CoT SC
96.9
76.8
60.4
81.7
81.7
81.0
66.6
49.6
w/ AR (our)
93.5
93.5
93.5
93.7
86.0
94.0
91.4
69.6
D LOGICAL SEMANTICS OF AR
The semantics of AR can be clearly interpreted via propositional logic. In this sense, in Sec. 3.1 we introduce propositional variables CI..... . Cn for each token where C; true if and only concept C; is detected at token t, e.g., by an SAE. In Sec. 3.2 we compute confidence scores (ac.1) over these propositional variables. Using a threshold, we identify variables Ci,.... , C as true for input token I. Given rules L. and token I, AR computes L U (Ci, ....... C, C new where C is a newly deduced proposition obtained through forward reasoning. Thus, reasoning in AR augments token activations using the provided logical rules.
E COMPARISON TO INSTRUCT AND COT
In addition the main results, we report performance of instruction-tuned and CoT-augmented variants of the backbone models (Tbb. 4h. In addition to the greedy generations, we also applied majority vote (MV) and self-consistency (SC) sampling, where we sample 4 times for each. Further- more, we provided the instruction-tune tuned variant with identical structured rules as AR, presented as stringified Python dictionary, denoted by "(normed)", to ensure equal information. While all strategies yield moderate gains over the raw base models, they remain inconsistent across tasks and degrade substantially as complexity increases. For example, Llama-8B with CoT improves on PrOntoQA 1 1-hop but still falls below 60% at 5 hops, whereas AR sustains accuracies above 95% across all hops. Similarly, Gemma-9B with CoT achieves strong results on the easy tasks but collapses on more complex ProverQA tiers. By contrast, AR consistently outperforms almost all instruct and CoT variants, highlighting that activation-level reasoning provides more robust and general improvements than prompting strategies alone. It needs to be noted that the sampling variants MV and SC require substantially more tokens and thus compute to produce answer, for Llama3.1 8B w/ instruct + CoT SC it took 1l1 million tokens whereas AR only needed ~2k tokens.
F RAIL2COUNTRY: EXTENDED RESULTS
Tab. 5) reports extended results for Rail2Country across the detection and reasoning subtasks. The table is structured in three parts: (i) overall scores for detection (SAE) and reasoning (LLM) in both R2C-Mono and R2C-Meta, (ii) per-similic breakdown of R2C-Meta detection, and (iii) a per-country breakdown of reasoning accuracy.
21

---

Published as a conference paper at ICLR 2026
Table Rail2Country Ablation and Additional Insights. Exact-match (1%) results on R2C-Mono and R2C-Meta variants. Top: overall results. Middle: per-simile breakdown (detection). Bottom: per-country breakdown (reasoning). AR improves both detection and reasoning over the base model.
R2C Detection & Reasoning
Detection (SAE)
Reasoning (LLM)
R2C-Mono SAE
R2C-Meta
R2C-Mono
R2C-Meta
Model
w/ AR
SAE
w/ AR
LLM
w/ AR
LLM
w/ AR
Llama3. 8B
95.42 74.95
100.00
0.00
92.95 91.20
41.00 35.33
74.67 93.67
29.67
62.67
Gemma2 9B
100.00
0.00
25.67
86.00
Per-Simile Breakdown (Detection)
Llama3.
Gemma2 9B
Meta Concept (base color)
SAE
w/ AR
SAE
w/ AR
like ruby (red)
0.00
100.00
0.00
100.00
like tomato (red)
0,00
100.00
0.00
100.00
like stop sign (red)
0.00
100.00
0.00
100.00
like cherry (red)
0.00
100.00
0.00
100.00
like strawberry (red)
0.00
100.00
0.00
100.00
like banana (yellow)
0.00
80.95
0.00
12.00
like sunflower (yellow)
0.00
95.24
0.00
100.00
like lemon (yellow)
0.00
77.78
0.00
100.00
like fresh snow (white)
0.00
75.50
0.00
100.00
like tangerine (orange)
0.00
100.00
0.00
100.00
Overall
0.00
92.95
0.00
91.20
Per-Country Breakdown (Reasoning)
R2C-Mono
R2C-Meta
Llama3.1
8B
Gemma2 9B
Llama3.1
1 8B
Gemma2 9B
Country
base
AR
base
w/ AR
base
w/ AR
base
w/ AR
Argentina
100.00
95.00
0.00
100.00
100.00
90.00
0.00
100.00
Romania
0.00
100.00
0.00
95.00
0.00
100.00
10.00
70.00
Nigeria
0.00
5.00
0.00
100.00
0.00
15.00
0.00
100.00
Hungary
10.00
100.00
0.00
100.00
0.00
40.00
0.00
100.00
Bulgaria
0.00
100.00
0.00
100.00
0.00
100.00
0.00
85.00
Egypt
100.00
35.00
15.00
100.00
70.00
35.00
0.00
100.00
Germany
100.00
75.00
100.00
15.00
100.00
75.00
90.00
90.00
Belgium
75.00
100.00
100.00
100.00
0.00
100.00
30.00
75.00
Austria
25.00
0.00
5.00
100.00
25.00
10.00
15.00
85.00
Netherlands
5.00
90.00
0.00
95.00
35.00
45.00
0.00
100.00
Ireland
100.00
85.00
100.00
100.00
45.00
0.00
100.00
100.00
Mongolia
0.00
100.00
0.00
100.00
0.00
90.00
0.00
20.00
Spain
100.00
35.00
100.00
100.00
60.00
40.00
30.00
70.00
Italy
0.00
100.00
10.00
100.00
0.00
100.00
10.00
100.00
France
0.00
100.00
100.00
100.00
0.00
100.00
100.00
100.00
Overall
41.00
74.67
35.33
93.67
29.67
62.67
25.67
86.33
Overall results. The top block confirms the trends summarized in the main paper. Vanilla SAEs detect explicit R2C-Mono colors reliably (75-95% accuracy). but collapse entirely when colors expressed via similes in R2C-Meta (0%). Similarly, base LLM reasoning is modest in R2C- Mono (41/35%) and degrades further on R2C-Meta (30/26%). both cases, AR restores strong performance: nearly perfect recovery of both explicit and implicit color in the SAF, and robust
22

---

Published as a conference paper at ICLR 2026
reasoning accuracy in the LLM. These findings demonstrate that AR systematically bridges the gap between explicit and implicit concept representations.
Per-simile breakdown. The middle block evaluates detection for individual R2C-Meta similes. Without AR, SAEs fail entirely (0% across all cases). With AR, most similes are correctly recovered, often at or near 100%. Success is particularly consistent for concrete red and orange cues (e.g., "like ruby", "like strawberry", "like tangerine"), while performance remains slightly lower. These results suggest that implicit cues involving more ambiguous or diffuse visual associations (e.g.. "fresh snow" "banana") are harder ground reliably, even with AR.
some yellow and white similes
Per-country breakdown. The bottom block shows large variation in raw LLM reasoning accuracy across countries. Without AR, many countries register near-zero accuracy, indicating that base models struggle to map even explicitly stated sequences to the correct country. With AR, performance rises sharply and stabilizes across the board, with most countries reaching close to 100%. Importantly, this robustness also transfers to R2C-Meta: although implicit descriptions introduce noise, the degradation with AR is relatively small compared to the collapse of the base models and SAES.
Overall, these analyses confirm that the observed gains in Tab. are not only driven by aggregate improvements but also reflect consistent robustness across languages, countries, and metaphorical concept variants. AR enables mentions to abstract, meta-level descriptions.
pipelines to generalize effectively from explicit lexical
G SENSITIVITY OF AR PERFORMANCE TO SAE PLACEMENT
Prior work shows that later transformer layers encode semantic, monosemantic, and more stable features, whereas earlier layers mainly capture local or syntactic patterns and therefore produce noisier and less interpretable concepts (Sawmya et al. 2025 Karvonen et al.. 2025). For this reason, existing SAE work typically attaches the autoencoder to late layers (often in the last third of the network), and AR follows this convention.
Importantly, AR itself is layer-agnostic. To assess sensitivity, we now include small layer ablation where AR is applied the adjacent layers before and after the originally selected attachment points. We selected layers 18 and 22 with thre-hhiesholiin and Reingge concept representations. For the R2C Mono dataset, we selected the steering a as 0.8 and 0.7, respectively. Across these layers, performance remains highly consistent (93.7 for layer 18 and 93.3 for layer 22), confirming that AR functions robustly across different positions in the semantic portion of the model. Our ablation with sReiggl) and auto nuto-thresholding shows stable results for the adjacent layers the R2C dataset.
H HYPERPARAMETER ABLATIONS
We further conduct hyperparameter ablation on ProntoQA with Gemma2 9B to assess how AR responds to variations in its main components, namely the size of the multi-feature representation (Rmults)). the number of incoming features for detection (top-k,n). ). the steering factor a, and the thresholding strategy. As shown in Tab. 6) AR exhibits stable performance across broad range of settings, with strongest results for Rmalt: sizes between 7 and top-ki, = 2. and moderate steering around 0.4. Among these, the steering factor shows the highest sensitivity, while automatic thresholding performs on par with manually tuned values. These trends reflect that AR inherits some variability from SAF-derived features, whose activations may fluctuate or be partly polysemantic. The introduced latent representations and hyperparameters provide flexibility to mitigate such effects, ensuring reliable behavior without reliance on fragile or finely tuned configurations. Improving the stability of SAE activations remains an open challenge in mechanistic interpretability, and advances in this area are complementary to and directly beneficial for AR.
I ETHICS STATEMENT
We adhere to the ICLR Code of Ethics. Our experiments use public benchmarks (PrOntoQA, ProverQA, BeaverTails) and newly introduced synthetic dataset Rail2Country. Although
23

---

Published as a conference paper at ICLR 2026
Table 6: Hyperparameter ablations on ProntoQA with Gemma2 9B,
(a) Rmultt sizes.
(b) top- values.
ProntoQA
ProntoQA
Rmattti
top-kin
1 -hoo (%)
3-hop (%)
5-hop (%)
1-hop (%)
3-hop (%)
5-hop (%)
3
38.05
35.15
37.45
81,45
81.15
81.00
5
59.25
59.80
58.60
93.55
93.40
93.70
7
93.55
93.40
93.70
3
65.65
63.30
63.85
9
92.00
92.20
92.55
11
81.50
82.70
81.25
(c) Steering factor a.
(d) Manual- vs. auto-thresholding.
ProntoQA
ProntoQA
Mode
1-hop (%)
3-hop (%)
5-hop (%)
1-hop (%)
3-hop (%)
5-hop (%)
0.2
32.70
33.65
33.75
auto
93.55
93.40
93.70
0.3
75.75
73.15
75.00
14
93.45
93.45
93.50
0.4
93.55
93.40
93.70
0.5
78.00
77.25
76.95
0.6
34.35
33.80
34.60
Rail2Country contains personal data, BeaverTails may include sensitive srtvacy-invasive text; we use it strictly for research on safety evaluation in accordance with its license. Our framework enables model steering, which could in principle be misused; explicitly condemn such uses and stress that AR was developed to improve transparency, safety, and alignment.
J REPRODUCIBILITY STATEMENT
We provide full details of the AR framework, datasets, model backbones, and hyperparameters in the paper and App. Al PrOntoQA, ProverQA, and BeaverTails publicly available, and Rail2Country will be released with the final version and is contained in the supplementary material. We also share code, preprocessing scripts, and rule to support independent replication and extension.
LLM USAGE K
We used LLMs aid in polishing and rephrasing parts of the manuscript, including improving readability. The model was not used for generating ideas, designing methods, running experiments, or analyzing results. All scientific content, claims, and conclusions are solely the responsibility of the authors.
24


Reviews:
[{"review_id": "zE3tYmdbPd", "reviewer": "Reviewer_vdgu", "summary": "This paper proposes ActivationReasoning (AR), a framework that claims to perform logical reasoning directly from the latent activation space of LLMs. It contains 3 main steps: (1) use SAE to extract “concept”; (2) map the activations to propositional symbol via thresholding; (3) apply user-defined rules over the symbols.", "questions": "Are the logical rules fully hand-written, and how are they chosen or validated for each dataset?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper presents a clear pipeline and demonstrate potential applications (like safety). The visualisation of their method is clear. It could inspire how to construct reliable symbolic reasoning using latent representations.", "weaknesses": "1. Concept extraction is loosely defined and not validated.\n\n2. Hard-coded rules, dataset-based (using training set) concept extraction from SAE output.\n\n3. Lack ablation on concept accuracy / why baseline model failed, is it because   of lacking the knowledge from the rules, or it is reasoning capability issue. Ablation is needed to understand why such huge performance gap of ~50\\% occurs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bVxqYsEoNX", "reviewer": "Reviewer_WLZc", "summary": "This paper introduces ActivationReasoning (AR) a novel framework to embed logical reasoning over the latent representations of an LLM. AR finds latent concept representations, map them into logical propositions and finally apply logical inference. The framework is evaluated over different reasoning tasks, ranging from multi-hop reasoning to context-sensitive safety, with promising results that improve transparency  and reliability of the model.", "questions": "1) The list of concepts to build the dictionary should be pre-defined or can be learnt as well? E.g. by using an approach similar to label-free CBMs?\n2) The concepts need to be known in advance wether having a single, multiple or relational representation, or it can be learnt automatically? I mean, in the automatic setting the representation is learnt, but you should know if r_c belongs either to R_single, R_multi or R_relation? In this case this may introduce a relevant bias in the learning and need additional knowledge about each concept range.\n3) I was wondering if you tried or you think it make sense for a concept c, to have r_c = [r_cs,r_cm,r_cr], namely to allow a concept to have simultaneously a single, multiple and relational representation. In this way probably any concept can manage the mono and poly semantic.\n4) If I understand correct A and A' have the same dimension, i.e. the concept inferrable are already pre-fixed (even if not learnt) from the beginning. In this way it seems like a semi-supervised setting where some concepts are supervised and others (the inferred) are not. As the rules are already known, what is the strong difference on just applying forward chaining as a pre-processing step and then use the supervisions derived at training time? \n5) How you decide to attach AR after layer 23 and 20 on the two backbone models? Why not, e.g. layer 15?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper is well written and clear to follow.\n- The introduced framework is very valuable and the problem studied very important for the advancement of Large AI models in general.\n- Extensive experimental evaluations.\n- Interesting discussion of AR's limitations.", "weaknesses": "- The rules need to be known in advance, and this may restrict the model applicability, as this is not often the case. Also several rules in real-world settings are only partially true, and then applying forward chaining may introduce some errors.\n\n\nMINOR COMMENTS\n\n- \"On our novel Rail2Country dataset (..)\", I think this dataset could be explicitly mentioned as a new contribution of the paper.\n- Fig. 2 is referenced before Fig. 1 at page 2, while it only appears on page 4.\n- \"Only a few provide a formal mechanism to compose concepts into higher-level abstractions or to disambiguate polysemous\nfeatures.\" Here it is seems some references are missing, or it misses an explicit reference to the ones mentioned above. Moreover, I think some other relevant examples of CBMs using logic rules to connect concepts to higher-level downstream tasks, like e.g. [1,2], can also be commented. This is also connected to the paragraph on systems integrating concept-level representations with reasoning modules for interactions and explainability.\n- \"Neuro-symbolic AI (...) \" I'd mention some more recent work, like e.g. [3]\n- \"A Relational-feature representation (R relation) defines a concept by a set of relations among SAE features\". Could you please explain/extend more how Relational features work? I understand that e.g. the concept \"hate\" is connected to multiple features, like the feature connected to the concept \"slurs\" and another feature connected to the concept \"demeaning stereotypes\". Hence, it's not clear to me how it is concretely different from multi-feature representations.\n- \"For example, features consistently activating for the notion of a Bridge can be directly assigned.\" Not clear how.. could you please better explain this? You mean by inspection over all the possible activations? I think in this case it can be unfeasible for large representation spaces.\n- While not strictly necessary as the paper is very clear, I think to make the paper more self-consistent a brief section about background, e.g. on SAEs or CBMs could be useful.\n- \"initialize AR with Rmulti) using\" -> right bracket is a typo\n\n\n[1] Barbiero, Pietro, et al. \"Interpretable neural-symbolic concept reasoning.\" International Conference on Machine Learning. PMLR, 2023.\n[2] Barbiero, Pietro, et al. \"Relational concept bottleneck models.\" Advances in Neural Information Processing Systems 37 (2024): 77663-77685.\n [3] Marra, Giuseppe, et al. \"From statistical relational to neurosymbolic artificial intelligence: A survey.\" Artificial Intelligence 328 (2024): 104062.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Q0rbtNancc", "reviewer": "Reviewer_aA2t", "summary": "This paper proposes ACTIVATIONREASONING (AR), a novel framework for embedding **explicit logical reasoning into the latent activation space of LLMs. AR leverages SAEs  to extract monosemantic latent features that align with human-interpretable concepts and treats these features as **logical propositions**. The framework operates in three stages. This enables both transparent analysis of model internals and direct control over model behavior through concept-level interventions. Empirical evaluations on four benchmarks demonstrate that AR improves multi-hop reasoning, robustness to abstract or implicit cues, and context-sensitive safety control, often outperforming even much larger LLMs and reasoning-tuned models.", "questions": "1. How scalable is AR when applied to large-scale tasks or long-context models, given the need to store and process activation matrices and logical rules?\n    \n2. Since the approach relies on SAEs trained independently from the LLM, how sensitive is AR’s performance to the specific choice or layer placement of the SAE?\n    \n3. Could the framework generalize to other modalities (e.g., vision–language models), where concept discovery and rule definition are more complex?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. AR introduces a clear and principled framework that connects latent activations with symbolic logic, moving beyond typical mechanistic interpretability.\n    \n2. Results across diverse reasoning and safety tasks consistently show large gains, often surpassing significantly larger or reasoning-tuned baseline.\n    \n3. AR offers auditable reasoning and explicit model steering, addressing transparency and safety.", "weaknesses": "1. The proposed approach inherits limitations from sparse autoencoders. If latent features are polysemous or unstable, reasoning performance and interpretability may degrade.\n    \n2. It remains unclear how AR performs under high-dimensional activation spaces or longer contexts.\n    \n3. It would be valuable to discuss integration with probabilistic or continuous reasoning paradigms, which may align more naturally with neural representations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XYxfk4NXVK", "reviewer": "Reviewer_tgpL", "summary": "The paper proposes ACTIVATIONREASONING (AR), a framework that maps sparse autoencoder (SAE) features in LLMs to logical propositions and performs forward-chaining deduction over them to compose higher-level concepts and steer generation. Pipeline: (1) build a concept dictionary from SAE features (single/multi/relational representations), (2) detect token/sequence-level activations with soft thresholds, (3) apply logic rules to infer new propositions (A → A′) and optionally steer model activations. Experiments on PrOntoQA, a new Rail2Country benchmark (explicit vs. simile color cues), ProverQA, and BeaverTails safety show large gains over base and some larger/“reasoning” models, reportedly maintaining accuracy as reasoning depth increases and improving safety detection; authors claim small runtime overhead.", "questions": "Can you freeze a single, pre-registered recipe (feature search, thresholds τ, top-k, α, steering vectors) learned on train only, and report test once with CIs across seeds? Provide ablations showing performance sensitivity to each knob.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "S1. Modular pipeline: concept discovery (via SAEs), activation aggregation, logic layer, and steering—each replaceable/ablatable.\n\nS2. Strong multi-hop robustness; recovery of implicit/meta cues (similes); safety composition via logic; low runtime overhead vs. CoT/reasoning models.", "weaknesses": "W1.  The method is fundamentally deductive: it thresholds SAE features into propositions and applies forward-chaining rules. But domain-general reasoning (esp. scientific reasoning) is largely inductive/abductive—forming and revising hypotheses under uncertainty, trading off competing explanations, and learning new rules from sparse evidence. Here, rules are user-provided or templated; there’s no rule induction, hypothesis revision, or probabilistic belief update. The evaluations don’t demonstrate transfer to open-class concepts, concept drift, or out-of-ontology settings where the correct rule schema is unknown. As a result, the approach risks brittleness outside closed-world toy tasks: once a needed predicate or relation is missing (or mis-grounded by the SAE), the system cannot generalize by learning—it can only fail silently or require manual rule engineering. For a claim of “domain-general” promise, I’d expect ablations with (i) novel domains without preencoded rules, (ii) noisy/contradictory evidence with uncertainty propagation, and (iii) an inductive component (e.g., differentiable rule learning, Bayesian updates, or abductive reasoning) that improves with more data rather than more hand-written logic.\n\nW2.  The paper compares AR to CoT and “reasoning models” but runs everyone with greedy decoding and without widely-accepted best practices (e.g., self-consistency, majority voting, verifier reranking, program-of-thought, retrieval-augmented logic parsers). A fair test would include: (i) symbolic parsers + classical prover, (ii) pure-symbolic systems on the same rule input AR gets, (iii) LLMs given the same normalized rule form.\n\nW3.  Multiple places admit selecting features/hyperparameters on evaluation data (e.g., Rail2Country: “features were selected on the evaluation set” and thresholds tuned on eval/test splits). This is classic double-dipping that inflates reported gains. A rigorous protocol would fix search on train only, lock thresholds, and report test once.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)