Semantic Regexes: Auto-Interpreting LLM Features with a Structured Language
Authors: Angie Boggust
Donghao Ren
Yannick Assogba
Dominik Moritz
Arvind Satyanarayan
Fred Hohman
Venue: ICLR 2026 Poster (2026)
Keywords: automated interpretability
LLM features
structured languages
Primary area: interpretability and explainable AI
TL;DR: We develop a structured language to describe LLM features, resulting in accurate, concise, and consistent descriptions that inherently describe each feature's level of abstraction and help humans reason about LLMs.

Abstract: Automated interpretability aims to translate large language model (LLM) features into human understandable descriptions. However, natural language feature descriptions can be vague, inconsistent, and require manual relabeling. In response, we introduce *semantic regexes*, structured language descriptions of LLM features. By combining primitives that capture linguistic and semantic patterns with modifiers for contextualization, composition, and quantification, semantic regexes produce precise and expressive feature descriptions. Across quantitative benchmarks and qualitative analyses, semantic regexes match the accuracy of natural language while yielding more concise and consistent feature descriptions. Their inherent structure affords new types of analyses, including quantifying feature complexity across layers, scaling automated interpretability from insights into individual features to model-wide patterns. Finally, in user studies, we find that semantic regexes help people build accurate mental models of LLM features.

Published as a conference paper at ICLR 2026
SEMANTIC REGEXES: AUTO-INTERPRETING LLM
FEATURES WITH A STRUCTURED LANGUAGE
Angie Boggust
Donghao Ren Apple
Yannick Assogba
MIT CSAIL
Apple Cambridge, MA, USA
Cambridge, MA, USA
Seattle, WA, USA
aboggust csail mi mit.ee. edu
donghao@apple.com com
yassogba@apple com
Dominik Moritz
Arvind Satyanarayan MIT CSAIL Cambridge, MA, USA arvindsatya@mit. edu
Fred Hohman
Apple
Apple Seattle, WA, USA
Pittsburgh, PA, USA
domoritz@apples com
fredhohman@a www.eeappppppcc.cmcoomom
ABSTRACT
Automated interpretability aims to translate large language model (LLM) features into human understandable descriptions. However, natural language feature de- scriptions can be vague, inconsistent, and require manual relabeling. In response, we introduce semantic regexes, structured language descriptions of LLM features. By combining primitives that capture linguistic and semantic patterns with modi- fiers for contextualization, composition, and quantification, semantic regexes pro- duce precise and expressive feature descriptions. Across quantitative benchmarks and qualitative analyses, semantic regexes match the accuracy of natural language while yielding more concise and consistent feature descriptions. Their inherent structure affords new types of analyses. including quantifying feature complex- ity across layers, sealing automated interpretability from insights into individual features to model-wide patterns. Finally, in user studies, we find that semantic regexes help people build accurate mental models of LLM features.
INTRODUCTION
Large language models (LLMs) represent their learned concepts, like the "Golden Gate Bridge", as linear directions in latent space called features (Bricken et al., 2023). Understanding model's features helps us anticipate its behavior, assess its alignment, and intervene to ensure safe out- comes (Templeton al., 2024). Automated interpretability assigns each feature by analyzing patterns in its response to input data (Bills et al., 2023) Lin, 2023 Paulo al., 2024 With feature descriptions, researchers have identified how models encode domain- relevant concepts, like protein structures (Gujral et al., 2025). and reconstructed circuits of features that correspond complex behaviors, like medical diagnoses (Lindsey et al., 2025).
deseriptions to
Despite the advantages of feature descriptions, natural language is an imprecise interface for describ- ing the computational roles features play in a model's inference. Current automated interpretability methods often yield overly verbose inconsistent descriptions (Huang et al., 2023), reflecting the difficulty of capturing tightly bounded feature behaviors in free-form text. Moreover, because natu- ral language is prone to ambiguity, it is poorly suited for interpretability tasks that require composi- tional reasoning, such as studying feature complexity or identifying redundant features. As a result, recent work identifying feature circuits in LLMs report needing skilled human relabeling to describe a feature's role in the network (Ameisen et al., 2025).
In contrast to natural language, structured languages (e.g., regular languages, programming lan- guages) offer well-defined syntax and semantics (Chomsky, 1956). By combining a set of primi- tives with compositional rules, structured languages capture precise patterns while maintaining pressivity (Lawson, 2005). Their grammatical structure provides additional affordances. including consistent ways of expressing the pattern (Knuth, 1965), concise representations of complex

---

Published as a conference paper at ICLR 2026
patterns (Lawson, 2005), and mechanisms for representing abstraction (Backus, 1959. Aho & Ull- man, 1972). As result, they can construct complex yet specific expressions, ranging from regular expressions (Lawson, 2005) to software (Python Software Foundation, 2025).
To gain the benefits of structured language in automated interpretability, we introduce semantic regexes. The semantic regex language is designed to capture the diverse activation patterns of LLM features, while providing the additional affordances of structured language. Its primitives grounded in commonly observed feature functions, including exact token matches (symboll s), syntactic variants of words and phrases (lexemes), and broader semantic relationships (fiell ds). To enable greater expressivity, we extend these primitives with modifiers for context, composition, and quantification. As result, semantic regexes can express a range of features, from simple token detectors (e.g.. "on"~> [ [:symbol on: 1) to complex linguistic phenomena (e.g.. "the last name of politician when it follows their -itle"> fieidd pol cal title:] [: field last name:
Across automated interpretaiility evaluations, we find that semantic regexes are as accurate as nat- ural language, showing that constraining feature descriptions to the semantic regex language does not reduce expressivity. Beyond accuracy, their structured form offers additional affordances for interpretability, including producing more concise feature descriptions and enforcing description consistency across functionally similar features. Moreover, since semantic regex primitives and modifiers exist across levels of abstraction, they serve a proxy for feature complexity, allowing us to perform model-wide analysis of feature behavior. Finally, in a user study, we find that semantic regexes help people build accurate mental models of LLM features.
RELATED WORK
Mechanistic interpretability exposes LLMs' internal concepts to explain their behavior and evaluate their alignment (Olah et al., 2017, 2020. Bricken et al., 2023). L.LSs encode concepts along linear directions in latent space, often referred as features (Elhage et al., 2022, Park et al., 2024). Meth- ods like sparse autoencoders (SAEs) (Huben et al., 2024; Rajamanohaaan et al., 2024; Bussmann ett al., 2025 Gao et al., 2025) and transcoders (Dunefsky et al., 2024 Ameisen et al., 2025) uncover human-imterpretable features, including those that correspond to Noomain-specifi and seleta-r concepts (Bricken al., 2023 Gujral et al., 2025 Lindsey et al., 2025).
While features correspond to concepts, which concept a feature encodes is not obvious a priori. Early interpretability approached this problem by manually analyzing the data that activate each feature (Olah et al.. 2017 2018 2020 Carter et al., 2019). To scale this process, automated inter- pretability prompts language model to describe each feature based on its activating data (Bau al.,) 2017 Hernandez et al.. 2022, Bills et al.. 2023 Lin. 2023; Shaham et al., 2024; Paulo et al., 2024] Gur-Arieh et al., 2025). Aligned with this goal, our method adopts a similar pipeline for generating feature descriptions, but generates descriptions using a structured language.
Our approach inspired by recent research showing that LLM features represent structured con- cepts. Activation analyses show that feature complexity increases with depth (Jin et al., 2025) and models represent concepts across levels of abstraction (Chanin et al., 2024; Boggust et al., 2025) Bussmann et al., 2025 Zaigrajew et al., 2025). In response, researchers have proposed lightweight taxonomies that classify features by function (Ameisen et al., 2025 Lindsey et al., 2025 et al., 2025), and called for feature description formalisms (Huang al., 2023). These findings mo- tivate our use of a structured language, enabling semantic regexes to describe a feature's 's activation pattern and express its level of abstraction.
SEMANTIC REGEXES
Semantic regexes use structured language to describe LLM features. Built around a system of cuman-itterpretable primitives (Section 3.1.1} and modifiers (Section 3.1.2,, semantic regexes cap- ture the low-level syntactic patterns and higher-level semantic concepts that LLM features represent. Unlike natural language, which is flexible but ambiguous, the semantic regex language restricts ex- pressivity to ensure the resulting feature descriptions explicitly convey their meaning. On the other hand, while inspired by regular expressions, the semantic regex language is not a regular language and extends beyond one-to-one character patterns capture more abstract concepts.
2

---

Published as a conference paper at ICLR 2026
Primitives
symbol
lexeme
field
Match exact strings
Match syntactiaaly related variants
Match semantically related variants
SEMANIIC VitEs
[:symbol color:
[:lexene color:]
[:field color:
Dall ERAMPLLES
CXPMPLES:
DATA CCMMPLO
coloring
color
you?
heF Pavart to
colors
pink
night
Modifiers
context
quantifiers
Match when sppearing the content
Sequential OR comolnations
@ el:context politics:) ([:field color:])
[:field color:] (E:synbol and: [:synbol or:1) [:field color:]
[:symbo] [:field color:]?
[:field flower:]
DCASPLLES clection
GRAMPLES:
DATA DCRAPLLS
Pazty
Candidate
1sy
Figure The semantic regex language consists of of primitives (top) that can be applied independently or combined with modifiers (bottom) to express diverse feature activation patterns.
3.1 THE SEMANTIC REGEX LANGUAGE
The semantic regex language consists of compositional components: primitives define the textual units a semantic regex matches, and modifiers refine or expand their scope (Figure T). Together these components form compact, yet expressive language for specifying L.L. feature patterns.
We developed the semantic regex language using a grounded- theory approach (Corbin & Strauss, 1998), deriving its components from empirical analysis of real LLM features. By manually survey- ing thousands of features across models, layers, and feature sources on Neuronpedia (Lin, 2023). identified recurring patterns (e.g., context dependent activations). We introduced new primitives or modifiers when they captured recurring pattern, increased the language's 's descriptive cover- age, and preserved intelligibility of the language. We continued this iterative process until reaching saturation, resulting in a language capable of describing all the features we examined.
3.1.1 PRIMITIVES
Primitives the atomic components of semantic regex. They specify the type of textual pattern a semantic regex will match, ranging from exact characters to categorical relationships.
Symbols Symbols (I: symbol )= match exact strings For instance, [:symbol color:) matches the string "color", such as in a splash of color These are the most specific and sim- plest primitives, and they commonly describe features that activate specific tokens or phrases.
Lexemes Lexemes (: Lexeme ]) match syntactic variants x. Drawing on linguistics, lex- eme is the abstract form of word that encompasses all of its surface variants, like changes in tense or plurality. For example, [: [:lexeme color: matches "color", "colors". "coloring". etc., such as book Lexemes typically describe features that capture word's 's meaning.
color in
coloring
Fields Fields (I:field )) match semantic variants of x. Drawing on linguistics, fields refer to words or phrases in a conceptual domain. For instance, [:field color:] matches "red", "blue", etc., in blue jeans
Fields often apply features that activate conceptual category.
3.1.2 MODIFIERS
Modifiers refine and extend primitives with context, composition, and quantification. This increases the expressive power of semantic regexes, allowing them to represent wider range of features.

---

Published as a conference paper at ICLR 2026
Context Contexts (é (E: context X:) (semantic regex)) match semantic regex in the text x. For instance, ( :onteext politics:) (1 [[:aynbol color:] 1) only matches I s:sybbol
color:] in political context, matching Green Party but not green apple Contexts help match features that represent domain-dependent concepts.
Composition Semantic regexes can compose in sequence or alternation (1). For in- stance, [:field color:] ([:symbol and: [:sybbol or:)) [:field color:] matches both green or yellow and black and white Composing semantic regexes allows them to match more complex features while maintaining precision.
Quantification Semantic regexes also make use of the regular expression quantifier one (?)- As an example, :symbol a: :field color:
[fffield flower: matches a red rose
and
a
isy
Quantifiers allow additional flexibility in the semantic regex.
METHODS
To study how the semantic regex language alters feature interpretability. we embed it within stan- dard automated interpretability pipeline (Bills et al., 2023, Paulo al., 2024 Templeton al., 2024] Puri et al., 2025). The pipeline consists of three components: subject model whose features we describe, an explainer model that generates feature descriptions, and an evaluator model that scores descriptions. Given subject model feature and its activating data, prompt explainer model to produce a description natural or semantic regex language and use the evaluator model to score how well the description matches the feature's 's behavior. Following prior interpretability work (Bills et al., 2023 Gur-Arieh et al., 2025) and our ablation study in Appendix C we use GPT-4o-m1 nú as directly compare semantic regexes natural
the explainer and evaluator. This pipeline allows language descriptions across varied axes of interpretability. It also demonstrates that by decoupling the format of the feature description from the generation process, semantic regexes are compatible with existing and future automated interpretability pipelines.
4.1 COLLECTING MODEL FEATURES AND ACTIVATIONS
We study semantic regexes on two families of subject models, GPT-2 (Radford et al., 2019) and Gemma-2 (Mesnard al., 2024). While our approach could describe model neurons (or any com- ponent matched to activating data), we focus on describing latent features identified by sparse coding methods (e.g., SAEs) since they often represent monosemantic concepts that are casier to describe and interpret (Bricken et al., 2023).
GPT-2 We apply semantic regexes EPT-2-Small (Radford al., 2019) and its residual layer features from Bloom (2024), identified using SAEs with 24,576 features (GPT-2 2--22-225----
Gemma We also apply semantic regexes to Gemma-2-215 MMesnard et al., 2024 and its Gemma Scope (Lieberum et al.. 2024) residual layer features (Gemma -2-2B RRES- 6k and Gemma-22 65k).
To collect activating data for each feature, we use the Neuronpedia (Lin, 2023) API. It provides activating data from OpenWetTeext (Gokaslan et al., 2019) for GPT-2-SmalT and Pile Uncopy- righted (Gao et al., 2021 for Gemma-2-25
4.2 GENERATING FEATURE DESCRIPTIONS
Given features and their activating data, we prompt an explainer model (GPT-40-min1) to generate feature descriptions. We compare
method against natural language feature description methods: token cato--act-piiaa (Bills et al., 2023) and max-acts (Paulo et al., 2024). These methods commonly used and vary in how they generate descriptions, allowing to get a broad understanding of how semantic regexes compare to natural language descriptions. See Appendix F.T f. implementation details.
comantic-rean
4

---

Published as a conference paper at ICLR 2026
method is based on Bills et al. (2023) and its Neuronpe- dia implementation (Lin, 2023) (oai aat..ttte--attppaii
token-act-ppiiz The tcken-act p
Here, the explainer model is prompted with the feature description task and three few-shot examples. Given feature's top five activating examples asked to continue the sentence "the main thing this neuron does is find". Each ex- ample is displayed list of tokens and their normalized activation values, like "these 8 tokens n0 activate 0"
max-acts max-acts follows Neuronpedia's (Lin, 2023) implementation of Paulo et al. (2024) (eleuther.acts.too2)) Instead of showing each token's activation value, like in token-act aaca-aaitiia...aa maintains a more natural text format by delimiting activating tokens within the text, like " "(thesee tokens)) activate". It prompts the explainer model with the feature descrip- tion task and three few-shot examples. Then, given the feature's 's top 20 activating examples. the explainer model is asked to "describe the text latents that are common in the examples".
semantic-reger Since our goal is to accurately describe LLM features, semant c-regex only differs from natural language methods in the language available to the explainer model. We simply inject semantic regex specific instructions into existing prompting strategies. To take advantage of its efficient example formatting, we adapt max-acts' prompt by updating the instructions to follow the semantic regex language, adding concise definition of the semantic regex language, and changing the few-shot examples to demonstrate semantic regex primitives and modifiers. We show the explainer model feature's top 10 activating examples and ask it to "output short explanation followed by a semantic regex", which we find improves the model's ability to follow the syntax.
4.3 EVALUATING FEATURE DESCRIPTIONS
We evaluate feature descriptions using common automated interpretability metrics (Paulo et al., 2024 Puri et al., 2025). Each metric uses an evaluator model (GPT-4c-mini) to evaluate the descriptions' fidelity the feature's behavior. Generation metries test the description's 's ability to generate activating examples (akin to precision), discrimination metrics test the description's ability match known activating examples (akin recall), and faithfulness metries test the description's ability to match steered generation (a measure of causality). Implementation details in Anpepdii F.2
Generation metrics Generation metrics evaluate a feature description's 's ability to generate acti- vating examples. We use clarity (Puri et al., 2025). which compares generated and random ex- amples' activations using the Gini index (a rescaling of the ROC AUC). Under this metric, overly broad descriptions score low by generating data outside the feature's activation space, while ideal or slightly narrow descriptions score high by only generating activating data.
Discrimination metrics Discrimination metrics test if the features description matches known activating examples. We compute detection and fuzzing from |auloo et al. (2024) and respons responsiveness vvvesss and purity from Puri et (2025). These metrics ask the evaluator model whether the feature description matches activating and random examples. Given the match results, detection measures balanced accuracy, responsiveness the Gini index, and purity average precision. Instead of matching the entire example, fuzzing asks if the description matches the ex- ample's activating tokens and computes the balanced accuracy of these more specific match results. Under these metrics, overly narrow descriptions score low by missing activating examples, while ideal or slightly broad descriptions score high by covering the entire activation space.
Faithfulness metrics Faithfulness tests how well the description reflects causal interventions on the feature. We use faithfulness (Puri et al., 2025), which asks the evaluator model whether the feature description matches continuations of random text when the feature steered versus ablated.
RESULTS
5.1 SEMANTIC REGEXES ARE AS ACCURATE AS NATURAL LANGUAGE DESCRIPTIONS
The goal of automated interpretability is to generate feature descriptions that accurately characterize a feature's 's activations. We benchmark semantic regexes against common natural language descrip- tion methods (token-act-pa) and max max-act.s) using discrimination, generation, and faithfulness
5

---

Published as a conference paper at ICLR 2026
Metric Distributions Per Feature Description Method
Method @ acc-n-act-aai max-acts sementorreeex
GPT2-RES-25K
Doomm----B-RRE--T
Gemma 2----------------
detect
huzaiig
faithful
fuzzing
purity
fatthul
Figure 2: Semantic regexes perform on par with natural language feature descriptions across evalu- 2-------R-6---- 65k, suggesting that
ations GPT-2-RES- 25k, Gemma-2 1amma-2-2B-RE-- -------------- 116,,, and the semantic regex language is appropriately expressive to describe LLM features.
metrics. To ensure our results generalize across models and features, we evaluate 100 features per layer from
Gemma-2 2-------R-66 6k, and Gemma -2 -atinnaa---------- 2--2B--RRS-5-
Semantic regex descriptions perform par with natural language Figure 2). Specifically, mantic regexes are non-inferior (p < 0.05 to natural language on clarity across all models, and responsivenet
detection for GPT-2-RES- semantic- outperforms token-act- -aii (p < 0.05 on clarity across all models, and detection, fuzzing, and responsiveness GPT-2 2er---RRe ----RES---5- 25k and
for Gemma 22-22-2-2---- 65k. Moreover, 0
This is non-obvious, as the semantic regex language is significantly constrained compared to the grammatical flexibility of natural language. Moreover, while the explainer and evaluator models are well-versed in natural language, they learned the semantic regex language from only a brief descrip- tion and few-shot examples. These results suggest that imposing structure on feature descriptions does not reduce their accuracy, while offering advantages that we explore in the following sections.
5.2 SEMANTIC REGEXES IMPROVE CONCISENESS AND CONSISTENCY
While semantic regexes are similarly accurate to natural language descriptions, their structure offers distinct benefits for interpreting LLM feature behavior (Figure 3).
Semantic regexes are more concise than natural language descriptions. Concise feature de- scriptions adhere to explanation norms (Tmm Miller, 2019), making them easier to sean and interpret. While natural language descriptions often require verbose phrases to capture feature's 's activation pattern (Hang et al., 2023), semantic regexes can encode the same information in more compact form using its signal-rich components. For example, in Figure top, the verbose natural language description "The presence of the sequence 54 indicating a year. time, or numeric reference frequently associated with events" can be expressed as I: ::sybbo1 54:1. Quantitatively, semantic regexes are consistently shorter. Across 100 randomly sampled features per layer in CPT-2-RES-22 and Gemma-2 2-2---22B---- 65k, the median description length is 41 char- acters (IQR: 19-59) compared to 139 (IQR: 119-166) for max-act: and 55 (IQR: 46-66) for tokon-act-pais This conciseness directly benefits interpretability, allowing human evaluators in Section 5.4 to parse shorter descriptions without distracting details.
Gemma -2-2--RES------
Semantic regexes are more consistent than natural language descriptions. LLMs often contain redundant features that activate on similar inputs and serve the same function. In circuit identifica- tion applications, recognizing redundant features helps reduce circuit complexity and identify the complete mechanistic circuit (Ameisen et al., 2025). Since the semantic regex language constrains the space of allowable expressions, it produces more consistent descriptions for similar features, making redundancy easier to detect. For example, in FFigure 3 middle, two redundant features from different layers of Gemma-2 R-B-RES1666 6k both activate on the token "Advertisement" While their natural language descriptions differ ("the word Advertisemeni" vs. "advertisement markers"), their semantic regex descriptions are identical: [:symbol Advertisement:
To quantify this effect, we measure consistency by asking how often a method produces the same description when given different random samples of feature's activating data. This simulates redundancy, since the underlying feature is fixed but observed activating examples vary. Eval-
One-sided paired t-test with non-inferiority margin A 5% and Bonferroni correction for superiority.

---

Published as a conference paper at ICLR 2026
Conciseness
L:sywbol 66:]
BEccontent jourtalien:> cover:]]
WiTtRN LASGUNA (raz-acto):
sallerr Lasioait iese-actals Frequent use of vartle related to Sournalien that denate ast contect.
The pressnce sesuence *Ex" incicating year, time,
reforente asseciated with
oritioal
Fedez Lco
Pald
in
in zence
the Fewm responsible
jourrulists who
cevvered
12:1
Aconto
that
creative
made
xtop
seared kike
WXX right
the middla
Consistency
SAGATIC 18082:
Advertisoeent:]
Advertiseennt:]
Frequent use of the acrd "Advertieeeent" as macter iedicating sertions
Fresuent Interruption by adverticesent warkers in vartous contects, indiceting soparation betmeon content and promotional materiel.
text, suggesting transitions between content promotional matorial.
an bosoult on free speech In An Advertisteent Continue
derouncadi cjendortext]
Adverti soment
reading the natn stary Vn 89
anochoz Expect
under people who need Coxerage 16
Advertistennt 316.2
M
1n
In cjendoftext |P their politica1 enantes
RepublScans a25g know that confiemation
Complexity
ausorrao 19SD-1 Etlexeme runi]
Essyabel Eifield lsoations] doter]
Lendunde (ene-acta)s
"rus" Sts variations frequestly describe physical eovement, evasion, oz partiolpation in ortivities or zoles soross
repeated
"barn" to indicate hirthplace
date of vorsous Inilvidullo, higghlightiig biogrophico)
disguése
noticed
hone
They tried
zun chote souul
cord
monte
scon
Enur
and
moxed
with
his
famllly
but
she dlddn bite
21.3
NIS New
Zooland
Searmon
soil
maker
acromant and shia GAOE
For Ehoos oE
Four
years before this
haappeeed
nean
zunninng
with
August
$5-4
Figure 3: Semantic regexes are often more concise (top), more consistently describe equivalent features (middle), and better reflect feature complexity (bottom) than natural language descriptions.
uating commaniccregeen max-acts,, and token- -ax- on five random features per layer of GPT-2-RES- 25k, each with five generated descriptions, we find that semant iemaaniccregeen c-regex yields identical descriptions 33.6% of the time, compared to 12.2% for token-act-pai- and 0.0% for max-acts. These results suggest that constraining the description space with semantic regexes improves consistency, making it easier to detect redundant features.
5.3 SEMANTIC REGEXES REFLECT FEATURE COMPLEXITY
Beyond matching the accuracy of natural language descriptions and offering benefits like concise- ness and consistency, the structured format of semantic regexes offers additional affordances for model interpretability. Each semantic regex is built from primitives that span levels of abstraction. where symbols match specific characters, lexemes extend to syntactic variants, and fields encode se- mantic relationships. The number of components in semantic regex is also a proxy for feature com- plexity, where features described using a single primitive (e.g., s:sybbol :) are typically conceptually simpler than features that require multiple compositions of primitives and modifiers (e.g., @{ context political affiliations: (1 ssmboll left: :slbblybool right:
We use the level of abstraction encoded in semantic regexes to measure feature complexity, finding that features become more complex deeper in the model (Figure 4). We generate se- mantic regexes for 1,000 features per layer in GPT-2-RES-25A. While early-layer features described by smaller and simpler seman-
2omma-2-22BRE----
and
Gemma-22 2-----222---------- tie regexes, longer and more abstract semantic regexes are needed to describe later-layer features. In particular, find that the average number of components per semantic regex (i.e., symbols, lex-
7

---

Published as a conference paper at ICLR 2026
Regex Complexity Layers
Gemma-2-2B-Be
Gemmo-
Component % Comportioo
130%
Composition
seqp requarce
quntBood
nono
130%
Component
oockod
mybbo
10
16
20
Layer Indet
Layer Indox
Layer Inderx
Semantic regexes encode feature complexity. The number (top) and abstraction (middle, bottom) of components increase across model layers, indicating increasingly complex features.
Figure 4:
emes, fields, and contexts) increases across layers. This shift is mirrored in the composition of semantic regex, where early layers have greater proportion of single-primitive descriptions which decreases favor of combinations of primitives and modifiers, particularly sequence and alternation compositions. Similarly, we observe that the types of primitives reflect increasing feature complex- ity. While all semantic regexes contain primitives, we see decrease low-level primitives, like lexemes, and an increase in fields (the most abstract primitive). We show example of increasing feature complexity from early to later layers Figure bottom.
Together, these trends indicate that later layers require longer and more abstract semantic regexes to capture their complex feature behaviors. This aligns with prior research demonstrating that later layers encode increasingly complex representations (Tenney et al., 2019 Jin et al., 2025 Sun et al..) 2025). However, unlike prior methods that rely on model probes or feature testing, we are able to read this complexity directly from the semantic regex feature description. As result, while like natural language descriptions, semantic regexes allow us understand individual features, their structure also allows us interpret model-wide attributes.
5.4
SEMANTIC REGEXES HELP PEOPLE BUILD MENTAL MODELS OF LLM FEATURES
A common role of feature descriptions is to convey the feature's 's behavior to human interpreter (Lin. Ameisen al., 2025). Thus, we investigate how semantic regexes impact people's mental models of LLMM features (full protocol in Appendix E). We conducted 24-person study with AI experts representative of people most likely to use feature descriptions in practice. To obtain insights across range of features, we used 12 GFT-2-RES RE5-25k features with diverse activation patterns and accurate natural language and semantic regex descriptions. Given a description, participants were asked to generate three activating phrases and one near-miss counterfactual.
2023
Forming an accurate mental model of an LLM feature means being able to express the feature's decision boundary -it--iie i.e., what does and does activate the feature. We quantify this by measur- ing the difference between the feature's s maximum activation on each participant's positive phrases and their counterfactual phrase. We choose the maximum positive phrase to account for activation artifacts that can result from tokenization. Small or negative values indicate that the participant had difficulty distinguishing activating and non-activating phrases, while large positive values reflect an understanding of the feature's 's decision boundary. We compare the mean differences of max sacts and semant ic-regex descriptions of the same feature, finding that participants scored as well or higher using semantic regex descriptions on of 12 features (Figure 5).
Although both description types were accurate, natural language often introduced extraneous de- tails that misled participants. For instance, one feature activated on variants of the phrase "expected to", but its natural language description included "'expected to' is frequently used to indicate an- ticipation prediction regarding future events outcomes". A participant over-indexed on the additional detail, expecting the highly activating phrase "He does not know meaning of the phrase expected to" to be counterexample because it did not indicate anticipation. In contrast, I 1:lexeme expect: :1 I syybbol to: more concisely expressed the activation pattern (see [Section 5.2) and enabled participants generate strongly activating positives and non-activating counterfactuals.
8

---

Published as a conference paper at ICLR 2026
[ifield timor] Ersyebol put..1]
Human Generated Feature Activations
LLM Feature
Type maX-acls
semantic-ragex
re--
INTURAL LANGUAGE Crea consistent ruference to time, specifically indicating hours in the format.
SELD
10-11 737
hove sireet
evoning
1oa-10 5144
COURO
ma-1
COMBO
[slexens expects]
[tsymbol to:]
me.9 14523
CONTIXT
ne--1 7218
Copeett
r00 4484
1067
COMBO
The phrase "expacted to* is frequently used to indicate anticipation or arodiction regarding future events outocees,
nord 4454
COMBO
RISOTIVE
COUNTERIAOTUK
40%
00%
100%
Max Dfff.
User Positive Counterfactual Activations (mean feature: feature's activation)
Tea
the phrase
With semantic regexes, study participants generated strongly activating positive amples and non-activating counterfactuals, indicating their understanding of the feature.
Figure 5:
We also find that semantic regexes can reduce ambiguity by conveying activation patterns via ex- ample. For a feature that activates on times followed by "p.m.", the semantic regex specifies this pattern directly as : fiell time: I: symbol p.m. : and a result, every participant included "p.m." in their positive phrases. In contrast, natural language descriptions can leave room for inter- pretation, where one participant misinterpreted "reference to time, specifically indicating hours in the 'p.m.' format" as any time in the evening and left off the critical "p.m.". " A similar pattern oc- curred in feature for political conjunctions, where the semantic regex directly included [: s:sybbol and:] I :symmbl or: I, making the conjunction pattern unambiguous.
While concise semantic regexes generally benefited participants, there were cases where the ver- bosity of natural language more effective. In the two features where natural language substan- tially outperformed semantic regexes, the semantic regexes were accurate but too minimal to elicit strongly activating phrases. One feature activated on days of the week used within full sentences. Its semantic regex [: fill days of the w led many participants to produce single-word outputs (e.g., "Monday") which activated slightly. Whereas, the natural language description "days of the week indicating specific events" prompted longer phrases with larger activations.
Finally, we find that participants were able to understand and use semantic regexes with minimal instruction. Despite receiving only a short description and a single example, participants generally interpreted the language correctly. In fact, we received more clarification questions about how to interpret the natural language descriptions than semantic regexes. This contrasts prior work sug- gesting that structured languages come a cost because they require "specialized training" (Huang et al., 2023), and instead signals their promise as tools for LLM interpretability.
DISCUSSION AND LIMITATIONS
Semantic regexes provide structured syntax for describing LLM features. In doing so, they result in feature descriptions that are more consistent and concise than natural language while still matching its expressive power. However, designing this structured language introduces trade-offs that create both benefits and limitations of semantic regexes and point to several directions for future work.
Since natural language descriptions often contain irrelevant details, we designed semantic regexes to be concise. While this generally helps identify the pertinent activation pattern, can produce overly terse descriptions. For example, [: field musician: describes feature that activates on famous musicians like "Taylor Swift", incorrectly implying that "guitarist" would strongly activate the feature. Striking balance between conciseness and expressivity across all features may involve adjusting model prompts, extending the language to encode hierarchical concepts (e.g., !:field musi ci name: :)), )), using validation loops (Shaham et al., 2024) mitigate ambiguity.
9

---

Published as a conference paper at ICLR 2026
Additionally, although semantic regexes increase consistency, they do not enforce unique mapping from activation pattern to description. Many valid semantic regexes can describe the same feature, even if some are less readable, like @ exnteet Germany: } (1 symbol German: )) This non- uniqueness is not inherently problematic. In programming languages, for instance, there are many equivalent ways implement the same function. However, languages often develop style guides that suggest the most readable syntax (van Rossum et al., 2001). Similar heuristics may benefit the semantic regex language, particularly when people (rather than models) are the intended interpreters.
To keep the semantic regex syntax minimal, we leave some components underspecified. For ex- ample, symbols match exact strings, but the semantic regex language does not specify if they also match string variations (c.g., "fruit" and "Fruit"). This simplicity avoids an overly complex vocab- ulary but can cause the model to make inconsistent assumptions across features (e.g., sometimes outputring [ symbol fruit: and other times [: [:smbbol fruit:] I [:aymbol Pruit:] )). In- corporating additional components, like case-insensitive flags, could reduce this ambiguity, espe- cially as growing lamiliiariy with semantic regexes may allow for more complex syntax.
These limitations also impact feature description evaluations. Since current metrics rely on discrete judgments of the feature description, issues like non-uniqueness and ambiguity can lead to false neg- atives and false positives for both semantic regexes and natural language methods. Developing more expressive metrics, such as continuous scoring schemes or readability evaluations, could provide more comprehensive understanding of the differences between feature description methods.
Although our results indicate that semantic regexes expressive, structured, and helpful for hu- man interpretation, additional evaluations could deepen our understanding. For instance, large-scale repetitions of the quantitative analysis could measure pipeline stability and larger crowdsourced user studies could statistically quantify the value of semantic regexes and whether LLM--as-ajudde evaluations faithfully capture human preferences in this domain.
Semantic regexes are not a solution to polysemanticity. While composition helps capture simple polysemantic features (c.g., [: [ifield clothing: ] [:field Ernest: high degrees of con- cept entanglement often result in incoherent descriptions, like e{: context syntax: } {[: symbol by: ] I : symbol {: ] I I symboll last: ] [: symboll then: .... [:sybbol ->:]). ) How- ever, since semantic regexes are agnostic to the generation pipeline. they could easily slot into new methods for disentangling polysemantic activations (Kopf et al., 2025).
Semantic regexes require models to learn novel language from only a brief description. As a result, we observe "grammatical" errors in generation and evaluation (e.g.. misinterpreting a primitive). Although models also make mistakes on natural language descriptions, they typically misidentify the activation pattern rather than misunderstand the language. This gap raises questions for auto- mated interpretability. People can learn structured languages and structured languages should better capture the computational roles of features, akin to how programming languages better represent algorithms. However, if models bias towards natural language, structured languages face barrier to adoption in automated interpretability pipelines. Future work could investigate whether automated interpretability evaluations reflect the needs and abilities of human interpreters.
7 CONCLUSION
We introduce semantic regexes, a structured language for automatically describing LLM features. The semantic regex language is grounded in current understanding of LLM features. Each prim- itive and modifier is designed to reflect patterns observed in interpretability research-i.e. that features often respond to exact tokens (symbol), syntactic word forms (Lexeme), and semantic cat- egories (#ield), and their activations are domain patterns (Lin, 2023, Templeton et al., 2024). As our understanding of LLM representations grows, we expect the semantic regex language will evolve, with components added or altered to capture new feature behaviors. Just as there are many programming languages to meet different goals, future languages for interpretability may be developed with affordances suited to particular interpretability tasks, such as highlighting input vs. output features, deseribing particular model components (e.g., attention heads), or exposing sefety-relevant features. The affordances of consistency, conciseness, and complexity that we build into semantic regexes expose a broader design space of structured languages for interpretability that can improve our collective understanding and control of L.Mss.
(context) and co-occur with other
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
As part of this research, we conducted a user study where we surveyed 24 participants from within our institution (Section 5.4). All participants gave informed consent and were informed they could withdraw at any time. We did not collect any identifying information, and all participant responses were anonymized. The study was approved under our institution's 's internal user survey policies.
REPRODUCIBILITY STATEMENT
Implementation details for the semantic regex method, the baseline methods, and the evalua- tion metrics are Appendix F. including full prompts and hyperparameters. Details to recreate study, including the survey instructions, are listed in Appenddx E, Code is available at ht https: / gitthub... com/ aaaalpppp//m/// wttpe//mmssmmaati-eeeeeeeee ing results is available at https: /atp://apppppee/ee github. io io/ml semant: co-eieee////////
and an interactive interface display-
ACKNOWLEDGMENTS
We thank our colleagues at Apple, with special mention to Mary Beth Kery, Hadas Kotek, and Lindia Tjuatja, for giving feedback on early drafts of this work.
REFERENCES
Alfred V. Aho and Jeffrey D. Ullman. The theory of parsing, translation, and compiling. 1: Pars- ing. Prentice-Hall, 1972. ISBN 0139145567. htthttps:://///m /wwww www..worrldat.commm org/oelc/ 310805937)
Emmanuel Ameisen, Jack Lindsey, Adam Pearce, Wes Gurnee, Nicholas L. Turner, Brian Chen, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, Andrew Persic, Zhenyi Qi, T. Ben Thompson, Sam Zimmerman, Kelley Rivoire, Thomas Conerly, Chris Olah, and Joshua Batson. Circuit tracing: Revealing computational graphs in language models. Transformer Circuits Thread, 2025. URL https: t/transformer-ciiree pubs pwwpppppppi22... 2025/ wttp///ttrbbuiooooggge...eee
me thods.hos... .htll
John W. Backus. The syntax and semantics of the proposed international algebraic language of the zurich ACM-GAMM conference. In Information Processing, Proceedings of the 1st Inter- national Conference on Information Processing, UNESCO, Paris 15-20 June 1959, pp. 125-131. UNESCO (Paris), 1959.
David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 3319-3323 3327. IEEE Computer Society, 2017. doi:
10.1 10.1109/CVPR.201
URL https:ttp/ocmm. /oi. org. /10 1119/C/PR.20177
Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. Language models can explain neurons language models. index. html 2023.
www.li...blbbooo.. windows.
neerzon-explaine/pppeee penpperr/nddex.
Joseph Bloom. Open source sparse autoencoders for all residual stream layers of J
al gpt2 small. http::/wwww..iioooooooooooooooop bten-source, hat.r-oourccappaaaaaaaattttteeee
2024.
Angie Boggust, Hyemin Bang, Hendrik Strobelt, and Arvind Satyanarayan. Abstraction alignment:
Comparing model-learedd and human-encoded conceptual relationships. Proceedings of the CHI Conference on Human Factors in Computing (CHI), pp. 417:1 ACM, 2025. doi: 00.1145/3706598.371332
114111145390 URLpttps: /oi. wtt.o//..1.///1......1
3713406
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Con- erly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu,

---

Published as a conference paper at ICLR 2026
Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodde Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023. http://transformer- htwp://ppp...2mmmmmmmmnnnneeeeee
html.
Bart Bussmann, Noa Nabeshima, Adam Karvonen, and Neel Nanda, Learning multi-level features with matryoshka sparse autoencoders. CoRR, tbs/2503.11547, 2025. doi: 10.48550/ARXIV. 2503.17544
17547
Shan Carter, 7an Armstrong, Ludwig Schubert, Ian Johnson, and Chris Olah, Activation atlas. Distill, 2019. doi: 10.23915/distil.00000)
David Chanin, James Wilken- Smith. Tomás Dulka, Hardik Bhatnagar, and Joseph Bloom. A is for absorption: Studying feature splitting and absorption in sparse autoencoders. CoRR, abs/2409. 14507, 2024. doi: 10448550/ARXXV.2449... 14507. URL http:::t/////..... //doi. org/10.
48550/arXi 2409. 14507
Noam Chomsky. Three models for the description of language. IRE Trans. Inf. Theory, 2(3):
113-124, 1956. doi: 10.1109/1IT.195.115555 1056688... /tRLattts://oi..r//////// 1019//11.10//1111 1956.105681
Juliet Corbin and Anselm Strauss. Basics of the Qualitative Research: Techniques and Procedures for Developing Grounded Theory. SAGE Publications, 1998. doi: 10.4135/9781155223115.
Jacob Dunefsky, Philippe Chlenski, and Neel Nanda. Transcoders find interpretable LLM feature circuits. In Advances in Neural Information Processing Systems (NeurIPS), 2024. URL http: wwp.perr.niiipp nips. ccppp//eeeeepppppppppppp.p.. 2b8 (ww.:/000600.eeeeeeeeaee00000000
wiwps//apee/eee/22// 4/hah//
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superposition. Transformer Circuits Thread, 2022. http://transformmr- wtwph/20022ttoommmm///ieeeee
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The pile: An 800gb dataset of diverse text for language modeling. CoRR, abs/2101.00027, 2021. URL
lhttps://arxiz> / abs / 2001201100000010000 00027
Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. In In- lernational Conference on Learning Representations (ICLR). OpenReview.net, 2025. URL
http://nwweceee...eeeeeooooadaaa wttet//forumllddcs2tt00002 9ZNKD
Aaron Gokaslan, Vanya Cohen, Ellie Pavlick, and Stefanie Tellex. Openwebtext corpus. http:
//5kylion007, www.ioono7...guth.l qithub. io/ wwt.ppeeeeeeettooooo..oo
2019.
Onkar Gujral, Mihir Bafna, Eric Alm, and Bonnie Berger. Sparse autoencoders uncover biologically interpretable features in protein language model representations. Proceedings of the National Academy of Sciences, 122(34):e25063111122 2025. doi: 10. (0733/pnas.2506316113
Yoav Gur-Arieh, Roy Mayan, Chen Agassy, Atticus Geiger, and Mor Geva. Enhancing automated interpretability with output-centric feature descriptions. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pp. 5757-5778, Association for Computa- .rgg 2025 .a..saccl-o long. 288/
tional Linguistics, 2025.
Evan Hernandez, Sarah Schwettmann, David Bau, Teona Bagashvili, Antonio Torralba, and Jacob Andreas. Natural language descriptions of deep visual features. In International Conference on Learning Representations (ICLR). OpenReview.net, 2022. URL https:: http////ppp//.../////////
12

---

Published as a conference paper at ICLR 2026
Jing Huang, Atticus Geiger, Karel D'Oosterlinck, Zhengxuan Wu, and Christopher Potts. Rigorously assessing natural language explanations of neurons. In Proceedings of the BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP (Black- boxNLP@@ EMNLP), pp. 317-331. Association for Computational Linguistics, 2023. doi: 10. htttps::::///// //dp::://ddi..o orgg wtt..oo/1..11//1//000000000
BB5SW11202.BBAAKKXXX... baackboxnlp-1.20
Robert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. In International Conference on Learning Representations (ICLR). OpenReview.net, 2024. URL
dettEoruu?idd= 6bwRSLeK
Mingyu Jin, Qinkai Yu, Jingyuan Huang, Qingcheng Zeng, Zhenting Wang, Wenyue Hua, Haiyan Zhao, Kai Mei, Yanda Meng, Kaize Ding, Fan Yang, Mengnan Du, and Yongfeng Zhang. Exploring concept depth: How large language models acquire knowledge and concept at dif- ferent layers? In Proceedings of the International Conference on Computational Linguis- lies (COLING), pp. 558-573. Association for Computational Linguistics, 2025. URL https: //aclanti ogy. org/2025. col www.ioooooooooo immaiin... 377
Donald E. Knuth. On the translation of languages from left to right. Inf. Control., 8(6):607- 639, 1965. doi: 10. 00.1016/S0019-900 (995000099999588559922222 URL S0C19- 9958 (65) 90426-2
Laura Kopf, Nils Feldhus, Kirill Bykov, Philine Lou Bommer, Anna Hedström, Marina M.-C. Höhne, and Oliver Eberle. Capturing polysemanticity with PRISM: multi-concept feature de- scription framework. CoRR, 1itss22006115338. 2025. doi: 104850/ARXIV.506..5555 URL 48550/ 2.e.xii.200.155555
Mark V. Lawson. Finite automata. In Dimitrios Hristu- Varsakelis and William S. Levine (eds.), Handbook of Networked and Embedded Control Systems, pp. 117-144. Birkhäuser, 2005.
Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca D. Dragan, Rohin Shah, and Neel Nanda. Gemma scope: Open sparse autoencoders everywhere all once on gemma 2. CORR, abs/2408 05 05147, 2024. doi: 10.48550/ ARXIV.2408.0S 0014404770517.7 147.. URLhttps: http:////oo......./o....00
org. 48550/ arviv 2408 05147]
Johnny Lin. Neuronpedia: Interactive reference and tooling for analyzing neural networks, 2023. URL https://wwit http://wwwww.errooddi...o org Software available from neuronpedia.org.
Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce, Nicholas L. Turner, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, Andrew Persic, Zhenyi Qi. T. Ben Thompson, Sam Zimmerman, Kelley Rivoire, Thomas Conerly, Chris Olah, and Joshua Batson. On the biology of large language model. Transformer Circuits Thread, 2025. URL /wwoloooooly..oooooooooooooooo.. html
Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Léonard Hussenot, Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Alex Castro-Ros. Ambrose Slone, Amélie Héliou, Andrea Tacchetti, Anna Bulanova, Antonia Paterson, Beth Tsai, Bobak Shahriari, Charline Le Lan, Christopher A. Choquette-Choo, Clément Crepy, Daniel Cer, Daphne Ippolito, David Reid, Elena Buchatskaya, Eric Ni, Eric Noland, Geng Yan, George Tucker, George-Cristian Muraru, Grigory Rozhdestvenskly, Henryk Michalewski, Ian Tenney, Ivan Gr- ishchenko, Jacob Austin, James Keeling, Jane Labanowski, Jean- -anttist Lespiau, Jeff Stanway, Jenny Brennan, Jeremy Chen, Johan Ferret, Justin Chiu, and et al. Gemma: Open models based on gemini research and technology. CoRR, abs/2403.08295, 2024. doi: 10.48550/ARXIV.2403. 08295. htt.:httt////ttt .rgg/00 (18550/axXi0 200.0..000000..55555.
Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, 2017. doi: 10.2391 111.1///iiiilll.0/ https://dittill.pub/00 huti:lll.puut//////aaeeeeeeeese 7/feauurevvssualzztioon
13

---

Published as a conference paper at ICLR 2026
Chris Olah, Arvind Satyanarayan, Ian Johnson, Shan Carter, Ludwig Schubert, Katherine Ye, and Alexander Mordvintsev. The building blocks of interpretabillty. Distill, 2018. doi: 10.23915/ distil1.00010. https::/distilllpubb/ 8/building-blocke
Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction circuits. Distill, 2020. doi: 10.239 /
https//diiilllubb2200o/cccciiooo
Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the ge-
ometry of large language models. In International Conference on Machine Learning (ICML). OpenReview.net, 2024. Uttpttpss:/ /ttp::::///peeevieew net / wor/forrunaee -ew.mniddUUGppKee 2
Gonçalo Paulo, Alex Mallen, Caden Juang, and Nora Belrose.
Automatically interpreting millions of features in large language models. CoRR, abs/2410. 13928, 2024. doi: 10.48550/ARXIV.24100. 13928. URL https: httttps://oo..org/////
48550 arwii. 1isiii.244100 2410.139222
Bruno Puri, Aakriti Jain, Elena Golimblevskaia, Patrick Kahardipraja. Thomas Wiegand, Wojciech Samek, and Sehastian Lapuschkin. FADE: why bad descriptions happen to good features. In Findings of the Association for Computational Linguistics (ACL), pp. 77138-17160 Association for Computational Linguistics, 2025.
Python Software Foundation. Full grammar specification. The Python Language Reference, Python 3.13.7, 2025. URL /tttttttt//occa pyt.la..hhooooo.oo wttp://////////// http://eeeeeeee/eeeaa.aaaaa grammar .tml
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, and Neel Nanda. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders. CoRR, abs/2407. 14435, 2024. doi: 00.45550/ARXYV.447..44444 14435. URL https:
//doi. 48550/ 2407.
Tamar Rott Shaham, Sarah Schwettmann, Franklin Wang, Achyuta Rajaram, Evan Hernandez, Ja- cob Andreas, and Antonio Torralba. A multimodal automated interpretability agent. In In- lernational Conference on Machine Learning (ICML). OpenReview.net, 2024. URL https:
//openzeview. wwtp://wwwww..ttet...ee.e.eeeeee d-w.rrummmdddmmm4222aaaaa
Xiaoqing Sun, Alessandro Stolfo, Joshua Engels, Ben Wu, Senthooran Rajamanoharan, Mrinmaya Sachan, and Max Tegmark. Dense SAE latents are features, not bugs. CORR, 156799 2025. doi: 10448550AARXI.22500.5555 15679. URL 22566 15679
Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling monosemanticity: Extracting interpretable features from claude sonnet. Trans- former Circuits Thread, 2024. URL
pub/20247
meallinggmmmeeenntttii..yy. index.
Ian Tenney, Dipanjan Das, and Ellie Pavlick. BERT rediscovers the classical NLP pipeline. In Pro- ceedings of the Conference of the Association for Computational Linguistics (ACL), pp. 4593- 4601. Association for Computational Linguistics, 2019.
doi: 10. 10.18653/V1/P19-11 1452. URL
https::/doi..rgg// http:////.......///////////// 115.001/1111//////91199959
Tim Miller. Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelli- gence, 267:1-38, 2019, doi: B..101//..ATTINT..00
Intelll-
URL /tttts::/doo..rggge
1016/j. 0018807.0000
Guido van Rossum, Barry Warsaw, and Nick Coghlan. Style guide for Python code. PEP 8, 2001.
URL htttp:://wwat www.pppthon..ooo.oo httppp/s//wwoggggooorrrrrrrrrrrr 0008/
Vladimir Zaigrajew, Hubert Baniecki, and Przemyslaw Biecek. Interpreting CLIP with hierarchical
sparse autoencoders. CoRR, abs/2502.20578, 2025. doi: 10-45$50ARRXVV2500005577 URL
20578
14

---

Published as a conference paper at ICLR 2026
A LLM USAGE STATEMENT
Large language models (LLMs) are the subject of this work, and they were also used general- purpose tools assist with writing and coding. All research ideas, study designs, analyses, and substantive code implementations were developed by the authors.
B ADDITIONAL RESULTS
Here we show alternative views of the benchmarking results from Section 5.1 We show the numer- ical means and standard deviations Table Al and the metric distributions Figure Al
Distributions Description
talln-ack--ai
semandi-regex
GPT2-REB-25K
donsity
0.02
04 69
100
00:02
04
06
06
0.5 02
04
06
68
3002
09 respon
04
05
00:92 04
06:18
08:02 0.4 06 09 falthful
clarity
detect
fuzzing
purity
Gemma---28-RES-110
density
clarity
fuzzlng
respon
purtty
Gemmaa-22B-RES-45N
density
0002
044
08
04 08
08
08
2002 04
05
00.92
04
98118
02
04
08
clarity
detect
fuzzing
respon
purity
faithful
Figure A1: Metric distributions comparing semantic-regex feature descriptions against natu-
ral language baseline methods. Results are shown for 100 feature per layer GPT-2-RES- Gemma- 1amina-2-220 2-----RE-------- and 1ormma---2B--a 2B--RE--65k and visualized using kernel density estima- tion with bandwidth estimated via Scott's rule.
Table A1: Across results on GPT-2-RES-255 25k-- Gemma 2-1---22-22---------------- RE8-16k, and
Gemma-2 2--2-REE----- semantic-reges feature descriptions perform on par with natural language feature description methods, These results suggest that the semantic regex language is appropriately expressive to describe features with similar performance as unconstrained natural language. Each metric is computed 100 randomly selected features per model layer and displayed as the mean + the standard deviation.
Generation clarity
Discrimination
Fuithfulness
GPT-2-RE5-25K
detection
Euzzing
responsivenese
puricy
faithšulness
Lokenmact-peli
0.45 4 0.36
0.79 4 0.15
0.80 4 0.16
0.81 4 0.23
0.71 4: 0.28
0.46 + 0.46
max-acts
0.70 4: 0.35
0.86 + 0.12
0.88 + 0.10
0.87 = 0.20
0.78 +: 0.26
0.52 + 0.46
seman-io-regex
0.68 + 0.36
0.81 # 0.14
0.83
0.13
0.83 : 0.21
0.66 + 0.32
0.37 = 0.45
Gamma-2-28-R85-166
Luken-acts act-
0.31 0.34
0.78 + 0.15
0.79 + 0.15
0.76 - 0.25
0.69 +: 0.26
0.38 - 0.45
acts
0.59 + 0.39
0.86 0.11
0.86 0.10
0.82 : 0.22
0.77 +: 0.24
0.49 4: 0.46
regex
0.57 0.40
0.79
0.13
0.77
0.13
0.76 0.23
0.58 0.30
0.45
Gamma-2-28-R9-655
token-aot-pair
0.25 + 0.32
0.77 + 0.15
0.77 + 0.15
0.75 + 0.26
0.69 + 0.26
1..00 + 0.41
0.45 0.40
0.85
0.12
0.85 0.12
0.79 +: 0.23
0.75 0.25
0.40 0.44
semanc re-reeex
0.47
0.79
0.79
0.31
0.40
15

---

Published as a conference paper at ICLR 2026
C ABLATION STUDY
Our experiments (Section 5) use GPT-4o-mini both the explainer and evaluator model. This choice follows common practice in automated interpretability. where the same model is used to gen- erate and evaluate feature descriptions (Bills al., 2023 Gur-Arieh et al., 2025). Since verification typically easier than generation, a model can reliably judge its descriptions, and the independence of model calls prevents conditioning effects.
To assess whether our findings depend on the choice of model, we conduct an ablation using a more capable model (GPT-40) in place of GPT-4o-mini. This ablation tests whether the relative perfor- mance between semantic regexes and natural language descriptions is sensitive to model capability and evaluates the robustness of our results.
Our ablation study follows the same automated interpretability pipeline described in Section 4 All components of the pipeline remain unchanged, including feature extraction, activation formatting. and prompting strategy. The only difference is that GPT-4o substituted for GPT-4g-mim ná in both the explanation and evaluation steps. We apply this setup to randomly sampled subset of 520 features from across layers of GPT-2-RES-T 25kk. For each feature, we generate descriptions using and
semanti serraniccreeeee cereeex, max-a acts, and taken-act--aee act-pair using both GPT-4o and GPT-4o-min we evaluate each description using the discrimination and generation metrics used in the main ex- periments. This design isolates the effect of model capability while preserving all other aspects of the experimental pipeline.
Across all feature description methods and evaluation metrics, the overall results remains the same regardless of whether the explainer Semantic regexes continue to match the performance
We report the results in Figures A2 and A3 and Table A2
and evaluator are GPT-40 or GPT-4o-ming of natural language descriptions, and the relative differences between description types nearly identical across the two model settings. While some metrics are slightly higher under GPT-4c (e.g., slight improvements in purity and respons eesponsivenese ivveess) these gains arise uniformly across methods and are well within the experimental noise.
Overall, the consistency between GPT-4o-mini and GPT-4o demonstrates that the comparative performance of semantic regexes is not sensitive to the choice of model. Given this robustness, along with the significantly improved efficiency and lower cost of GPT-40-mini (Appendix D we use GPI-4o-min- for all experiments reported in the main text.
Table A2: Across results on semant i ic-regex, max-acts, and token-act-paiz feature de- scriptions, the choice of explainer and evaluator model (either GPT-40 or GFT-4o-mini) does not change the relative performance. These results validate our use of GPT-4o-mins in our main ex- periments. Each metric computed on 520 randomly selected features from GPT-2-RES- and displayed as the mean + the standard deviation.
Generation clarity
Discrimination
token-act-pair
detection
fuzzing
xeaponsiveness
purity
GPT-4o-mint
0.45 0.37
0.79 + 0.15 0.81 0.15 0.82 1 0.13
0.83 1 0.22 0.90 1 0.17
0.73 0.26 0.84 + 0.21
GPT-4o
0.59 0.37
0.83 1 0.13
max-acts
GPT-4o pei-4o-mini
0.71 0.36
0.86 0.11
0.89 1 0.10
0.88 1 0.20
0.81 0.24
GPT-40
0.73 + 0.33
0.86 1 0.09
0.85 1 0.09
0.91 1 0.14
0.86 1 0.19
semantic-reged
GPT-40 sPT-4o-mini mini
0.69 1 0.37
0.81 1 0.13
0.83 1 0.12
0.84 1 0.21
0.68 1 0.32 0.74 0.30
GPT-40
0.68 0.36
0.82 + 0.12
0.81 + 0.12
0.84 0.22
16

---

Published as a conference paper at ICLR 2026
Metric Distributions Per Model
GPT-4o-mini
GPT-4o
Method
token-act-pair
max-acts
semantic-regex
clarity
detect
fuzzing
respon
purity
clarity
detect
fuzzing
respon
purity
Figure A2: Metric box plots comparing different explainer and evaluator models (GPT-4o and GFT-4o-mini) across remantic-reged and natural language baseline methods (max-acts and token-act-pair) Results are shown for 520 features from --T--2-R--2555
Metric Distributions Per Model
method
loken-aci-pair
max-acts
semantlc-regex
GPT-4o-mini
density
0-
0.0
0.2
0.4
0.6
0.8
1.0
0.
02
0.4
0.6
08
1.0
0.0
02
0.6 fuzzing
0.4
088
1.0
0.0
0.
0.4
0.6
08
1.0
0.0
0.2
0.4 purity
0.6
08 1.0
clarity
detect
respon
GPT-4o
donsity
6
0.0 02
0.4
0.6
0.8
1.0
00
02
0.4
0.6 detect
0.8
1.0
0.0
0.5 fuzzing
0.4
0.8
1.0
0.0
0.5 respon
0.4
0.8
1.0
0.0
0.4
0.6 purity
0.8
1.0
clarity
Figure A3: Metric distributions comparing different explainer and evaluator models (GPT-4o and GPT-4o (ep--4-mini) across semantic-regex and natural language baseline methods. Results are shown for 520 features from GPT-2-RES 2-225--25k and visualized using kernel density estimation with bandwidth estimated via Scott's rule.
Table A3: Cost analysis for feature description generation. Costs are computed from the number of input tokens (system prompt, few-shot examples, and feature input) and the number of generated output tokens. We report the mean + std of input and output token counts across 130 features from GPT-2-RES-22 (10 per layer). Tokenization is performed using OpenAl's 4PT-4o model family tokenizer.
caken-ast-en pair
max-acta
semant i c-segex
Input Tokens: system and few-shot examples (ppompp)) Input Tokens: average feature (Tieaure)
919
483
993
457 1 213
524 1: 90
237 1: 47
Output Tokens: average per generation (T.a)
9
30
G2T 4o-mini cost feature
$ 0.00021 180
$ 0.00016905
$ 0.00020430
cost feature
$ 0.00353000
$ 0.00281750
$ 0.00340500
17

---

Published as a conference paper at ICLR 2026
D COST ANALYSIS
The cost of generating semant i ccrregex feature descriptions is comparable to the cost of natural language methods. Computing descriptions for all features in GPT-2-RES imately $65 using GPT-4o-mini and $1,088 using GPT-4o. These values are on par with existing methods. Generating all t token-act paat--airr descriptions would cost $68, and all max-acts descrip- tions would cost $54 using GPT-4o-mini.
would cost approx-
The description cost per feature depends on the number of input tokens in the prompt, including the system prompt and few-shot examples (7 prompi) and the feature's 's tokens (T (TTauure), and on the number of generated output tokens (Tou). It also depends on the API prices for input tokens (Pin) and output tokens (Pour).
Cost per feature
Pin prrmpp + Treature) + Pout (Tom),
We report the token counts and resulting costs for each feature description method in Table A3 Differences across methods are the result of formatting choices and the number of examples shown to the explainer. For instance, token-act- max-acts and semant te-anticcreees and comantic-rrger uses fewer activating examples than max-acts but adds the semantic regex definition to the system prompt.
uses more verbose activation formatting than
At the time of experimentation, the OpenAl API pricing $0.15/$0.60 USD per million input/out- put tokens for GPT-4o-ming and $2.50/$10. 10.00 USD per million input/output tokens for GPT-4o. These estimates assume no input caching, so the reported prices are an upper bound on the total cost of feature description generation.
E
USER STUDY PROTOCOL
Here we provide the full protocol for our user study in Section 5.4
Since already evaluate semantic regex reliability across thousands of features in Section 5.1 the purpose of our user study is not to obtain another quantitative reliability estimate. Instead, our goal to gain qualitative insight into how semantic regexes shape real-world AI experts' interpretation of model features. To do this effectively, we recruit a representative sample of the practitioners most likely to use semantic regexes practice and present them with a controlled but diverse set of model features. We intentionally keep the study size to 24 experts and 12 features to make the protocol feasible. Importantly, we find that these sample sizes are sufficient to draw insights about the benefits and limitations of semantic regexes.
Participants Our target population consists practitioners who would realistically interpret se- mantic regex feature descriptions in their real-world tasks, including interpretability research, AI safety, and model development. Therefore, we recruit 24 AI experts from within our organization who work with LLMs. These participants have broad range of technical roles and backgrounds, ensuring that the study captures how semantic regexes are interpreted by the types of experts who would use them in practice.
Features Our user study investigates how the format of feature description influences people's understanding. To ensure that participants interpret meaningful feature descriptions, we restricted to features from GPT-2-RES: 25kk with both accurate semantic regex and natural language descrip- tions. Because our goal is not to exhaustively evaluate all such features, but rather to study human interpretation across a representative sample, we focus on selecting diverse subset.
We first filter to features whose semant i cereeeex and max-acts descriptions each achieve a detect i score above 75%. From this filtered set, manually select 12 features whose de- scriptions we verified to be accurate. Our selection process emphasizes feature diversity. We select features from across the model's layers, resulting in features from early layers (1-4), 4 from middle layers (5-8), and from late layers (9-12). And, we chose features that represent diverse activation patterns, such that 2 activate on exact symbols, on a lexeme, 3 on field, only activate in a particular context, and 4 were complex patterns that require concatenation or combination of semantic regex components.
18

---

Published as a conference paper at ICLR 2026
This selection process allows to study human interpretation across a controlled but diverse range of features without overburdening participants. We show all selected features and their descriptions in Figure A4
Task Participants were tasked with writing phrases that match were counterfactual to given feature description. Each participant completed the task for three natural language feature descrip- tions (nax-act.s) and three semant c-eiiie-reeeeeeee feature descriptions. We showed each participant features of varying complexity, with half the participants seeing the feature descriptions in the left column of Figure A4 and the other half seeing feature descriptions from the right column. To avoid carryover effects, participants saw natural language and semantic regex descriptions from different features. To avoid learning effects, we randomized the order, with half the participants seeing natu- ral language descriptions first and the other half seeing semantic regexes first. This process resulted in 425 positive and 143 counterfactual phrases (some participants did generate all phrases).
The task was completed asynchronously and distributed via a survey link. The survey began with an introduction to the task, the semantic regex language, and an example of matching and counterfactual phrases (Figure A5p. Then, participants were shown an explanation of the feature description type they would see first, followed by three feature descriptions they were asked to generate phrases for. This was repeated with the alternate feature description type and the final three explanations (Figures A6 and A7). For each feature description, they were asked to "Write phrases/sentences that match the feature description" and "Write I phrase/sentence that is counterfactual for the semantic regex feature description" (Figures A8 and A9
19

---

Published as a conference paper at ICLR 2026
COMNRO 10CC Esayabll
SCMATE TICC
Erequatt phense so sxctoas indicating relettienshpp, events, conditions sentoszes.
conatntent exfermnce to "Llann." indicattin fazus speeifie sourts group warlows contects;
[]
Thangs
staatte
lnficn
Lions
posted ploture
Ansulting ocements
fjendofooxt
bezting esportunities
would B00 ion help
CoRvect
ccucial and
[Tloscno devcleps]
[cfieba day
works]
webb "Nlvelege Jts vatletioes rxesuently aeanar, infiiating
Pressent anstiass days the mock, intiouting spoiiii events
topic
and 11nn Terranien
Develin
wattiyy
gether.ng
riscce snc
unthenienta pecher en
26.3
somning
Action Male Sodutiorn hae Today tal Plaguum
segeediuua of cillerence Nn An
Bavelon
Hurdar C10
repport with the child Vr An
$5.3
Noy
Chorl 6150
Hest) [tfiedd charge:]
[sfaeld ranician:)
NatirL Lodaus
chargee remerical cata, tapleablly indiiuting lacrassas or
fazuk caair lasaet, ausilarity, or mosmn activitiee adtide
percent
Enorease
sise
1i
san and
strame
Juttie
2013
Sgraphyi atcelght
killers
Beclice Usiy
ieeth
Izon
seeting nultiale
20v2
[1ftaad ti [zaynbel p.h.c]
[=2nsate sсpezt:] [:eyebol tac]
consdatent sefecurce to time, speciricatly indicating houre in
The phease "ecpected tor La frequently used to indicate
Racking Leogue
adrs on ESPN at
Throw not not mon Tevelt
expect
xen nur
Easters But
cen
arice
cape
Bead
culoase
whäch will begún boday at
<) [ndoffoot]]> Apple exproted
to
The
event
feotures
zeploce evEzent
Touch
tectnclogy wöth
REtoontent ( (ospata) :3)
Risentent sasting Ingradionts:3 Isfiod Ingmedions:]]
cospaccall, evoking ingrediens fecosing eile, malte, ponders.
Seadcatting thtiled
priserlly
in progenmming cook.
Concall
Weste
Line
Lubieseney
exLt
TAappace
zesult
collection
Apg zepate
Hett dressed with
1ime
Salon
sTaLl
20.3
Litield
political 1declogs:]
Frequant grvess, indiieeling discossies pelltinal perpacctive.
canteseting astiiicaa
citation foratt, tncluting puliccation
jascnal sates,
reforewes, intiocting setllarl erilingg
nece confortable then mould in Gemeevverive it
in Be Reforeress Lindsey
2897
Trosical Dc
Foreatation
NASA
EJK
Earth
Trop Fax lang polleed beer with Siberal
Republicane Sunday
Figure A4: The features we used in our user study (Section 5.4h, displayed as their top activating examples and semant ie-iiic-eeee cergeex and rax-acts feature descriptions. Half of the participants were shown the features the left column and the other half was shown features the right column.
20

---

Published as a conference paper at ICLR 2026
Welcome!
This survey is part of research project on LLM feature descriptions. should take -10 minutes. Thank you for taking part!
Task Overview
Your job is to write phrases/sentences based on LLM feature descriotions.
each feature description, you write:
phrasass/sentences that match the description. goal these activate LLM feature. L.phraselsentern that is counterfactual for the description (le.,, almost matches the description but doesn't) 't). The goal phrases
the
You will get 3 natural language feature descriptions and semantic regex feature descriptions.
Semantic Regex Format
semantic regex structured language pattern.
Basic Elements:
[:symbol X:]+ matches X
[:symbol color:] matches "image color"
[tlexeme X:] matches X and its syntactic variants
[lexeree color]] matches color, colors, coloring, colored, etc. "caloring "many colors"
[:field X:] matches semantic variants X
[:field color:] matches red, orange, blue, etc. such as "blue skies"
These elements combined together:
@(:context X))) matches semantic regex when appears context
@[coontxtt politics:][[ffeed color]] matches blue state" but not "blue skies"
[A][] + matches semantic regex immediately before
regex
Elexeme coler:][[feedd color:] matches "the color blue" and "colored vallow"
[A]][B]+ matches semantic regex OR semantic regex B
[ffield color:] [ssymbol snybb] [ffield color:] matches "red blus" "pink and purole"
Example
Natural Language Feature Descriction: Mentions of time durations associated with prison sentences, highlighting various lengths
ranging from months to years.
Semantic Regex Feature Descrintion: @(ccontext prison sentences)[-field duration:))
Matching Phrases:
30 years prison
spent four years behind bars months of jail time
Counterfactual Phrase:
spent four years living in California
Your responses voluntary and you may stop any time.
Figure A5: The user study introduction explains the task, the semantic regex language, and provides an example.
21

---

Published as a conference paper at ICLR 2026
You will start with 3 semantic regex feature descriptions. As reminder:
A semantic regex is structured language pattern.
[:symbol X:] matches X
[:symbol color:] matches "image in color"
[:lexeme matches
its syntactic [llexeme color:] matches color, colors, colaring, colored, etc. such as "coloring book" and "many colors"
[ffilld X:] matches semantic variants
[:field color:] matches orange, blue, such "blue skies"
These elements can be combined together:
@(:context X)A)-> matches semantic regex when appears in context
@[ccontext ([.fied color:] matches "a blue state" but not "blue skies'
[B] + matches semantic regex immediately before semantic regex B [lexeme coler][[field color:] matches "the color blue" and "colored yellow"
[A][[B] > matches semantic regex A OR semantic regex
[fiield color:] [syymol [ [symbol or:] [ffilld color:] matches or blue" "pink and purple"
Example
Semantic Regex Feature Description;
@(:context prison sentences)([-felo duration:))
Matching
1. sentenced to 30 years in prison
spent four years behind bars months of time
Counterfactuel Phrase:
1. spent four years living in California
Figure A6: The user study's description of semantic regexes.
Now you will see 3 natural language feature descriptions.
descriptions would not) apply.
generate phrases/sentencces description
Example
Natural Language Feature Descriction:
Mentions time durations associated with prison sentences, highlighting various longths ranging from months to years.
Matching Phrases:
sentenced to 30 years in prison
four behind
7 months of jail time
Counterfactual Phrase:
1. spent four years living in Califoria
Figure A7: The user study's 's description of natural language feature descriptions.
22

---

Published as a conference paper at ICLR 2026
Write 3 phrasesssenteences that match the semantic regex feature description: [:symbol on:]
Write phrasse/sentence that is counterfactual for the semantic regex feature description: [:symbol on:]
Instructions Reminder:
semantic regex is structured language pattern.
Elements:
[:symbol X:] matches
[:symbol color:] matches "image color"
[seexeme X:] + matches and syntactic variants
Liexeme color] matches color, colors, coloring, colored, etc. such as "coloring book" "many colors"
[:field X:] matches semantic variants of
[field colon] matches red, orange, blue, etc. "blue skies"
These elements can be combined together:
@(ccontext ())))
appears @(:context ([-fic color:)) matches "a blue state" but not "blue skies"
regex
[A][B]+ matches
regex [Hexeme color:] [:field color:] matches "the color bilue and "colored vellow"
regex immediately
[A]][B]
regex [:field color:] [:symbol any:]l on] [field color:] matches "red or blue" or "nink and purple"
regex
Example
Semantic Regex Feature Description:
prison sentences)[[ duration:])
Matching Phrases:
sentenced 30 years prison
2. spent four years behind bars
Counterfactual Phrase:
spent four living Califoria
Figure A8: An example of user study question for semantic regex feature description.
23

---

Published as a conference paper at ICLR 2026
Write 3 chrranasssentennns that match the natural language feature description:
Frequent use of the phrase "on" in various contexts indicating relationships, events, or conditions in sentences.
WWitee Lahrassesentencee that is counterfactual for the natural languege feature description:
Frequent use of the phrase "on" in various contexts indicating relationships, events, or conditions in sentences.
Instructions Reminder:
The feature descriptions describe data that activotes an LLM feature. Aim to generate phrases/sentencee where that description would (or would not) apply.
Example
Natural Language Feature Descriction;
Mentions durations
sentences, highlighting lengths ranging years.
Matching Phreses:
years
2. spent four years behind bars months of time
Counterfactual Phrase:
1. four living in Callforia
Figure A9: An example of user study question for natural language feature description.
24

---

Published as a conference paper at ICLR 2026
Table A4: Hyperparameters used generate feature descriptions using semant c-regex and the natural language benchmarks troken-act-pair (Bills et al., 2023) and max-acte (Paulo et al.,
2024).. Benchmark hyperparameters follow Neuronpedia'ss implementations of the original meth-
ods [Lnn, 2023).
Parameter
token-act-paiz
max-acts
semantic-regex
Number of few shot examples
3
3
Number of data examples
5
20
10
Number of tokens per example
64
32
32
Explanation model temperature
1.0
0.7
1.0
Explanation model top-p Activation threshold
1.0
1.0
1.0
60%
30%
F IMPLEMENTATION DETAILS
F.1 FEATURE DESCRIPTION METHOOSS
We evaluate three approaches for generating descriptions of LLM features: token-act-palz (Bills al., 2023), max-act- (Paulo al., 2024), and our proposed semantic-regen method. All methods use gpt -40mmin as the explainer model. We follow Neuronpedia's 's reference implemen- tations (Lin, 2023) for each baseline method. Implementation details for each method are provided below, and a summary of the settings is shown in Table A4
token-act-pais We follow Neuronpedia's implementation (Lin, 2023) of the feature description method proposed in Bills et al. (2023), called oa coken--ct-ppal in the Neuronpedia interface. The explainer model is prompted with the original feature description system instructions and the three few-shot examples used in the original paper (Listing 2}. For each feature, we supply the top five activating data examples from Neuronpedia (Lin, 2023), where each example consists of up to 64-token window centered on the maximally activating token. Tokens are presented in the format:
<start> token_1 token activation_?
activation
<end>
where activatis on is the normalized activation value for the feature. Activations are linearly scaled between and 10, such that the feature's s maximum activation value maps to 10. The prompt ends with the continuation cue "the main thing this neuron does is find", following the original protocol. Generation is performed with temperature = and top-p 1.
max-acts We follow Neuronpedia's implementation (Lin, 2023) the feature description method proposed in Paulo et al. (2024), called eleuther.acts..te explainer model is prompted with the original system instructions and the three few-shot examples provided in the original paper (Listing 3). For each feature, supply the top 20 activating data examples from Neuronpedia (Lin, 2023), where each example consists of up to a 32-token window centered around the maximally activating token. Tokens exceeding 60% of the feature's maximum activation are highlighted and contiguous activating tokens are merged into a single highlighted segment, like:
in the Neuronpedia interface. The
The prompt ends with the instruction to "describe the text latents that are common in the examples". Generation is performed with temperature 0.7 and top-p 1, following Neuronpedia's settings.
semantic-regex To generate semantic regexes, prompt the explainer model with system instructions adapted from max-acts, augmented with a concise definition of the semantic regex language (Listing and four few-shot examples illustrating the range of available primitives and
25

---

Published as a conference paper at ICLR 2026
modifiers (Listing 4). For each feature, supply the top 10 activating data examples from Neu- ronpedia (Lin, 2023), where each example consists of up to a 32-token window centered on the maximally activating token. Tokens exceeding 30% of the feature's 's maximum activation are high- lighted, and contiguous spans are merged into single segments, as in max-acts.
token 1<<token 2twkee_n>>eettteee 3>>roken4...
We found this lower threshold produced more consistent highlights, which important for mapping activations the structured semantic regex language. The prompt concludes with the instruction to "output a short explanation followed by semantic regex", which we found improved syntactic adherence. Generation is performed with temperature and top-p=
A Samantic Raqex is a structured pattern composed of:
1:symbol X:] matches exact phraza (e.0.r [:synbol runaing:] matches "I ande runningas" znd rec summing22 faslec*).
1:lexene K:] matches phraso
syntactic varianca 10.5., 1:lexane runs] natches "shess ranss",
"itfapee rarningss quickLy"I.
Ircield X:] nalches plizase x and its aeantic variants (e.g., Isfield runs] natchea *eal for se joy goid").
25* and sprint>>
X
subword (2.0., ing), word re.g., runningl, phrzse (e.g.v runcing cempo).
These componenta can be combieed to natch nore complex palterna:
* s1 92 natches sequence whare S1 is followed by 92 (e.g.r [saymbol run:] [:lexane fast:] matches "Ics
festay" "theyce
81 natchas wither or (0-9-2 [rayabol zunz] Isaynbol walks] natches zunzs >5t).
nothing (e.g., [slexene sunt] [saymbol verys]? [saynbol Casus] metchhes "I ande sunning fastssn "Ies very fast2?*).
57 matches
#(:ocatext Ci) (S) natches that only activates the context (e.g., A[secacext political:] [[:lexeno sumt]) natthes "shesc canso for office" and "eceunningr2 for guveznoz" but not "Iee runzy nasachans").
Listing 1: The remgexic-reeex description that we inject into method and metric prompts.
<aysten>
Ne' re stuoying neurons in neural network, Eact neuron looks for some particular thing in chort cocument decument the activates sumnarize single seateacs the nouzon locking toz. Don's examples words.
Look
The activation format is tokenetabxcctivatiio Acrivation values range fron 0 to 10, neurea finding shat higher the activation
it's looking represented aon-zoro activation stronoss natch. </aystem>
Houzon
Activations:
<start>
ust
uz
clously
Sunny
and
ever
the
top
0 sneaky
bul 0 ler excel
in
of ibl iblw
inposa disepperiing
10
acta
cendo
<start>
sping
26

---

Published as a conference paper at ICLR 2026
studio
co2
1 0
is
affecting 3
and 0
this ad
mininalla:
novie
Same setival Lona, but with all zezos Filowaed outs
disappeeting ing
earing 10
celart> aping the affeul
Explanation of neazon behavines the main. thing Lhis dowa la Find
Cass istant>
bense verbs ending in Lingf.
cusers
Neuzzon
Activations:
<start>
movies
this
is
1ikely
cause
massive
cardiac
it
taken
doses
shot 0
рeraaрs
isticelly
handheld
canoras
and 0
appareatly
movic
lights
jo
aquin
b
as
27

---

Published as a conference paper at ICLR 2026
budget
awings
annoy
ingly
weaL
1go
and 0
opecity
<end>
activations, but with ciltered
<elart>
axrest 10
<and>
1go
<and>
Explanation behaviors mair thing la rind
sassistante
physrica" medical condit
V/sssistan
Neuzon
Activationa:
<start?
Lhe
Logether
7
oux
town
ix
strong
<endo
<start>
buny
ant
ronant in
comedy
about
Friendship
and 0
the
that
xe
this 10 together
<end>
Euplanatico behavior: thing is find
<assistant>
related community.
Listing 2: The token-act-palz prompt and few-shot examples. <system></systeen <user></user> and <assistantt</assisttaaa delineate the message roles passed to the explainer model.
Ceystand
You ase meticulous reswaschez conduct lag an impoztant investigation into patterna Coand language. Your task i8 to analyye texe and provide an explanation that thoroughly encepen atea possible patterns found it.
Buidelines:
28

---

Published as a conference paper at ICLR 2026
You will qaven 15st text exancles en which special aze selactad between delimitera <ethis>s, If a seruence of censecutive tokens all are important, the catire sequenes of tekens will be contained between delimiters <cgust like thicsh.
produce cancise examples, and what pattorns you tound.
description. Simply
the lssents that common
If the examples are uninformative, you den't need to nantion then, Don't Zocus giving examplea of the exampies.
zokens, but Do not aention the nackes Lokens [&c 25) your explanation.
sumarize
Do not make lista of possible explanations. keep your explanationa short and esncise. The last 1ine of your respense must the formatted explanation, using [EXELMAATIOR]: </ayaLum>
<uscr>
Example 1: and he was ccovez the nopa22 Lo Find
Euampie 11 lauching the
Exangle thought Scotland was bering, bus really there's nore oxthan neetx eyebs1 1'd </user?
[EXPLANATION]: Common idioma text conveying positive sentiment. </assistent>
Example zivar aide the ocean widekar72. The ocean
Exenple 2: every year you get Lall coesos, she
Example 3:
the hole was amallster>> but deepexer>> than the
</user>
cassistants
[RXPLANATOON]: The token Mar" at the and of comparat adject i've bingg ai </ansistan.2>
Cusera
Example 1: sumwthing happening inaide exhousses" be
Extamp presmnbbly alweys contained boxss", according Exangle people were comingg into the ccamoking areass".
Nlovevrr he Exengle Patricks thhy you getting </user>
waylss" later,
Gessistant
[EXELANATIO]] Noune representing distinet object that contains comething, sonetimes preceding guot fan nark, c/aasistani
The max-acts prompt and few-shot examples. <user></ue </ser>, and <assistant></ssiistee istant>> delineate the message roles passed to the explainer model.
Listing 3:
<aysten>
You are inzerpreting the rolc of LEN feateres, Your task it to describe patterns across activeting text exangles.
Input:
You will be given list text exanglea.
Activating
hichlighted
deliniters likese and that>>,
Outpat::
You will eutput veSemantic Regexes that describes pattorns acress the text examples.
Semantle Regex
pattern compased Isaynhol X:] matches exact phaass (6.9., Isaynbol sunning:] satches
runningos"
running>> faster").
[zlexene X:] chiches phrase and it's syotact vasiants le-g., [=laxene runs] natches "ahecc ranso", "itrsee zurnings quickly"I.
[ifield X:] natches phrase
semantic variants (0.9-> fiflald runs] natches *out foo
ase and tee aprinteo for gold").
X be subword farg., ingh, word (e.g., sunningt, or phrase (0.9-, zunning campo). combined
comples patterns: Si natches aequenow where S1 followed by 82 (eig., (saynbol zuns] [=lexene [ast:] metchies "Iec run festsst and "theyse run faster>>").
comporents
S1 182 metches wither or $2 (e.g., [sxynbol runz]l [saybbol walks) netchee "Toc runasi and Micc walk 25*).
nothing (9.q., I:lexens run:] [saymbol very:]2 Issynbol fasci] matchea "I zunning fastasn and #ice run very fasts?*].
natehaa
el Blconnext C:l (8) natches S that only avtiva in the context (M.g., Biseentext politicals) ([=lexems office" "ccronningos governor*
cunt))
narathone"),
Inatzuntion::
that took the text exaplees identify deliniters.
ssalle exanples.
look patterns within
1. you find exact phrase, use Isaynbol X:].
2. 1f you find a phrese and its syntact variants, use [slexene Xt).
3. find phrase and it' swnent in varianta,
[sfield Xs].
Create Semantis Regex (51 with tevest components precisely cescribes the pattern.
Bext, look patterns the exanples! toples.
29

---

Published as a conference paper at ICLR 2026
exanples are related Bemantic Regex (51, use #|icentoxt o:) (S1 modifier.
topic
copic not recundant the cuzrent
Output the cimplest and nest concise Semartic Regex that precisely describes the pattouns across all examples.
Do not inclede the de:imitera tekens set your Semantic Regex.
6, Output short explanation tellowed by "SR: and then the Semantic Rogex </ayaten>
Kuawc>
(all(cl.aadi_auui (2, s, 2or>> x in (xange(2)] Dee tor>> in glist: pprey "
fasto
in
</user>
istantt
shrase 'toz" activates only s/aasistam:
of coding. Rcccontext coding:l (Crayubol foz:))
<user>
extradiiion legislatiunce prohibita individual Irom
whether Iarael
the collective protection asylur seekera2o
3: ss called on Dutche> autheritiesce to do more to protect human rights>> wozkezs </user>
<assistant>
The aud aclivaaing plrases and theie sucsounding phaases are all selated to human righus legislacion. 88: field hanan zights Isgistation:]
<user>
off the narkets 2ec iines the sixeso of Inéiafa
neuded denand Lineace exis axia-ingso mumbes
That's about tinesse these seer in c/user>
The synbol "thef act ivates when in betveer numerical malt iplier and en anount sonething. 8R: field nambes]] Issyibol Linea Che! [rfield anount:] </aesistant>
Kussc>
1: sentenced toce 30 years ieas prison
tocs ten monthaso in prixon suagended forks yearsoo
</users
<assistant>
inating phreses
prison zentences,
prison :fifield
:]] </assistant>
Listing 4: The semantic-regex prompt and few-shot examples. <system></systeem <user></ user>, and <assistant><assssttttt delineate the message roles passed to the explainer model.
F.2 EVALUATION METRICS
We evaluate the feature description metrics across suite of evaluation metrics. We take these evaluation metrics from al. (2024) (Elcuther) and Puri et al. (2025) (FADE). We follow their implementations closely, but make some modifications for consistency across metrics. To account for the semantic regex structure, we make small adjustments to the original prompts and inject a description of the semantic regex language (full prompts in Listings 5 The hyperparameters are listed in Table A5
10 and 12
All metrics gpt-4o-mins
the evaluator model. Metrics typically rely
of activat-
ing examples (positives) and random examples (negatives). Unless otherwise specified, we use 50 activating examples and 50 negative examples per feature, each consisting of 32-token window centered on the maximally activating token. Activating example sampling varies metric, but all random examples are sampled from alternative features in Neuronpedia; given the number and di- versity of features, this approximates random dataset sampling. To evaluate the description, many of the methods compute the feature's 's activation on these data sets. Following prior work (Paulo al.,) 2024), we ignore the ciiininng-ofssqqeenne tokens when computing activations.
F.2.1 ELEUTHER METRICS
We implement the detection and fuzzing metrics from Paulo et al. (2024), following Neuronpe- dia's implementation (el (eleuther recal1 and eleuther Fuzz). For Hleuther metries, activating
30

---

Published as a conference paper at ICLR 2026
Table A5: The hyperparameters used to evaluate feature descriptions using metrics from Paulo et al. (2024) (detection, fuzzing) and Puri et al. (2025) (elarity, responsiveness, purity,
faithfulness).
Parameter
clarity
detection
fuzzing
responsivenese
purity
Faithfulness
Number of positive examples
50
50
50
50
50
Number of random examples
50
50
50
50
50
Number of tokens per example
32
32
32
32
32
Number of examples
model call
15
Positive example sampling method
quantiles quantiles
percentile
perocotile
Number of quantiles
10
10
Percentiles
75, 50,
Top sampling percentage
20%
20%
Evaluation model temperature
1.0
Number of generation runs
10
Modification factors
0, 10, 100
Number of steered generation tokens
30
examples are sampled uniformly across 10 activation quantiles. Across both metries the evaluator model is tasked with providing a binary judgment of whether the each example matches the fea- ture description. The evaluator model processes examples per call, with temperature 0.7 and a maximum of 500 completion tokens.
detection The detection metric evaluates description quality at the example level. The eval- uator model prompts are shown in Listings 5) and 6 The evaluator is shown feature description and an example and asked whether the description matches the example's 's text. The final score is the balanced accuracy of these binary judgments compared the ground-truth example labels (positive activating, negative random).
fuzzing The fuzzing metric follows the same setup as ćetect ien but evaluates matches at the activation level. In each of the examples, we highlight activating tokens, and the evaluator model is asked whether the description matches the highlighted regions. The evaluator model prompts are shown in Listings 7 and For activating examples, we highlight tokens that activate higher than the activation threshold (60% of the feature's 's maximum activation for max-acts and 30% of the feature's maximum activation for semant c-regex) and merge contiguously activating tokens, e.g.:
Random examples are highlighted using the activation pattern of the positive examples, which en- sures an equal distribution of activating tokens across both example sets. The evaluator is asked whether the description matches the highlighted regions, and the score is computed as the balanced accuracy of these judgments relative to the ground-truth labels. This yields stricter variant of detection by focusing the match decision activating regions.
F.2.2 FADE METRICS
We implement the clarity, respcnsiveness, purity, and faithfulness metrics from Puri al. (2025), adapting their setup for consistency with the Eleuther metrics. We sample activating examples following FADE's stratified sampling protocol: 20% from the top activations and the remainder sampled uniformly across the percentile bins [0. 50), [50. 75), [75, 95), and [95, 100].
responsiveness and purity To compute responsivenes and purity, the evaluator model shown a feature description and a data example and asked to provide a discrete judgment (0, 1, or 2) indicating how strongly the text matches the description. Following FADE, samples labeled as 1 are discarded before scoring. The evaluator model prompts are shown in Listings 9 and 10 We parallelize scoring, such that the evaluator judges 15 examples computes the Gini index of the resulting scores, capturing how well the description's matches rank- order the data by activation strength. purity computes the average precision, capturing how well the description separates activating from non-activating examples in retrieval setting.
once. responsiven.e
clarity Unlike the other metrics, clarity evaluates a feature description by testing whether it can generate activating text. Here, the evaluator model is prompted with the description and
31

---

Published as a conference paper at ICLR 2026
We make 10 independent generation calls, producing a set positives, and sample an equal number of random examples negatives. Each example is then passed through the subject model to obtain its activation value. el clarity computed as the Gini index, measuring whether generated examples achieve higher activations than the random negatives.
asked to generate candidate examples (full prompts shown in Listings 11 and 12
faithfulness faithfulness measures how well the feature description captures the feature's causal role the LLM. Unlike the other metries that are based on analyzing activating data, faithfulness compares the feature description against steered model generations where the fea- ture is amplified or ablated. Here, we sample 10 random examples. Then we have the model continue each example by generating 30 additional tokens. We apply this strategy multiple times, where each time the feature is ablated (set to 0) or amplified. In the amplified settings, we set its strength equal to its maximum known data activation multiplied by modification factor (1, 10, 100). Given the steered generations and the feature description, we follow the same rating task as in reapons i veeness and purity, asking the evaluator model to provide discrete judgment (0, 1, or 2) indicating how strongly the text matches the description. Following FADE, samples labeled as 1 are discarded before scoring. The evaluator model prompts are shown in Listings 9 and [TO] We compute faithfulness as the maximum proportion of matching generations at each modification factor compared the proportion of matching generations when the feature is ablated.
<aysten>
inc iggns neticuloux linuusttiss
You will be given a certain latent of taxt, such as *mala pronouns" or *text with neqative sentiment".
You will then given several
exarplea, Your task to deternine which examples posseas Lhe lacent.
For each exapple in busn, return 1 if the sentenow is cocrectly labeled or 0 if the Lokens are nisl abeled. You must zeturn your response in valid Fython list. Do not zeturn anything else besides Python list. </systen>
<user>
latent explanatioos Sords related to Americar foarball lositions, one, apeci Ficelll by the bight and iositionn
Test examples:
Example secceeddfttstt poeetty tnagestn'sPatrieee tight and Boh Grankouski had his boss'
Exanple It namea of munths used in The Lord of the banggr ...
Example Media Day 20181nintSu detensive
Mashington 1961 speaks
Exangle 31 shown, genera" ly not eligible for ade. example, videns ebout recent tragedies,
Exanple 4: Line, with the luft side camely Lackle Byron Bell at Lackle and guard Anini
</user>
<assitant>
[1,1,
c/aasiatanc>
cuser>
Latent explanation ion: The word "guya! in the ghrase you guya".
examples:
Exangle you arect confortabless wish it. Youte guysa2 support me in nany other ways already but I sesure you's womenss in Kentucky aren't laughing as they
Exangle 1: birth control sooese<lendortee struggle
act, andee plazseso nice, Exangie StAmerican, toldec Hannityaa that "youee guyess are playing the zace care." Pxample 42cc eneosteeneftentt [ST want toce remindoo you all that 10 days ago (d rector Hassinil
Exerpie 2:te gig! hope you guyace
</user>
Casa antt
[,,0,0,3,0]
</ansistant>
<user>
latent explanattion: "ot" betore words that start with cepital letter,
examples:
Exangie 6: clinate, Tonol in's Chief of Staff Ch Lorensen said.An
Exampie 1: ne wendersorking relics, no trus Body and Bload of Christ, true Baptisn
Read Talent lopment devised pirector Public Prosecations (DPF) Exerpic 4: and tair invostigaticn not ever. inciude the Directer of Achictics? Finally, se beliers the <users
Deborah
Production Film Lenden.
Examole 3: It
11111,1111 </assistenc>>
32

---

Published as a conference paper at ICLR 2026
Listing 5:
The evaluation model prompt and few-shot examples used to compute the detect ion scores of natural language feature descriptions. <system> < s/ssstm>, <user></use>> and delineate the message roles passed to the evaluation model.
<aystem>
You are en intelligent and neticolous Linquistics rosearcher.
You will be
certain latent fornatted Semantic (SEMENTIC DECBETTCCCBBEEKKECRRPPOOOOOO
You xill then be gavan several text exemplas. Your task is to determine which examples possass the lacent. For exarple
correctly labeled the tokena nislabeled. You must seturn your respense valld Pyshan list. Do not seturn anythina else basides Python 1ist.
if the
</ayacen>
<usee>
Senantic Begex explanation: ion: I:field Amezican football position:!
examples:
Examole Orsjendeftext poetty InaoestnkdPatriota Light
Greakovski
boss'
Exarpie 1: canes of months used in The Lord of the Lanas:\nte"...tte
Exemple 2: Media bay 2015tntatSu sefuasiv and Isaish Mashington (94) speaks to the
Euanole
Excmpic line, generally elicible for ads. exanple, videos about left cancly tackic Byron Bell tackic quard tragedies,
</uses?
<assitant> 0,,1, </assistanz>
Bamantic Recax explanation: [saynbel you cuys:)
Teat examplea:
confertabler>
Youse support
ways and
Exangle 1: birth control accesast (embbl ext but exsure yours woments Kentucky asen! laughing as they struogle
Rxampil 2:fa gig! 1 hope
naVEss her, andec pleasess rice, Exangle StAnerican, tolder Hamilyyo Exanole 4rsc chasse] andaftext [sI want remindbo you all that days ago (diractoz Massinil
guyazo playing the card."
c/user>
[,00,0,0,0] c/easistani2
Semantic io Begex explanal [saynbel of:] [sfield Capitalized Mord:]
Text examplex:
Charlis Exangle e 1: re wonderworking rel iee, no true Body and Bllodd of Christ, no true Rept
Tomblin's thisd
said.
Exanple 2: beborah Bathe, llead of Talent beveloprent and Productt ion at Piln London,
Exanple 3: It Las been devised by Birector of Public Prosecat iona (DPP) fair invest igstion
Example cusers
irelude Director
Pinally, believe
Caseistant> 1,,1,2,1,11
</apsistenc>
Listing 6: The evaluation model prompt and few-shot examples used to compute the detection scores of semantic regex feature descriptions. <system></system wwtsssstatt></assssttttt
<user></user>, and delineate the message roles passed to the evaluation model. is replaced with the semantic regex
At prompt time, description in Listing
Cayslend
You are incelliqent naticulous lincuistica rosearcher.
You will be given cextain lateot of Lext, such "male pronouna? oz "Luxt with negalive sentimet".
You will be given a examples represent this latent, are belveen Lokena ee and 39,
that consain this latent. Portione the sentence which strongly
deternine every single within and corsectly ly labeled. Consides that provided examples could correct, none of the exanplea could correct, or a mix, An exenp's is only correct if every narked toker is represencat few of the latent
examples
mislabeled.
esca exempee turn, return it the serterce cocrestiy You most return your response in valid Python list. no not return anything elae besides Python Hist.
or
tokens mislabeled.
33

---

Published as a conference paper at ICLR 2026
</ayacem>
<usees
Latent explanation: Words related American football pasitiens, specifioally tight end pesition.
Test examples:
Exanple 0:51 erdlendeftoat iodetty Inseeslakeaarriotac tight and>> Rob Gronkowaki had his bess'
Exempic postode eadotrextt should know thipe about>> effensive Hine eeaches: they Benandingts ments
Example 2: Media bay 2015tataiStie detensiver> end Isaiah Washington (94) speakses to there
Exemple 31<k running backsshy he said. Defensivess cad>> Carzoll Fhillips is improving and his injury
Examole liness, with the </uscr>
namalyse tackle>s Byzon Ball atse tackle>> andes quardss Amini
[1,0,3,0,1]
</aasistan:>
Latent explanations
"guya" please 'you guya".
Text examplex:
confortabler> wish Exampie birth control accesss leadottoxt but apsuza yours woman>> in Kentucky axen't t lacqhing thay stzaggle
guyazs suppert
ways already and
Example 2:'s gig! I hope you guyaes LOVE>> heE, andes please7> nice, Example StAmerican, toldec Hannityo "youse guyazo playing
card."
Exangle 4rcc sn endoftext [sI want Lost remindby you all that 16 days ago (detec Massinll </user>
[,,0,0,0,0]
c/easlatanss
latent explanation ions "of" before words that start capital letter,
Teat examples:
Exanple
Staff Charlie Serensen seid.An Exangle 1: se wanderworking selics, no Lrue Body and Bloodes el>> Cheist, no Lzue Baptian
in' thiel thiefoc
Example 2: beborah Sathe, lleadic aiss Talent hevel lopeent and Production at P:lm tonson,
Exanple 3: It has been devised by Birwetorce afss Public Prosecut (DBP)
Exanple and cusers
investigation ever. include the Direstores Athletios? Finally, believe
<assistant> 11,1, </aasiatanc
Listing 7: The evaluation model prompt and few-shot examples used to compute the fuzzing scores of natural language feature descriptions. <system>< /ainit///yya //tsseemm////smtmm//m/ <user></user>, and <assistant></asiisst
delineate the message roles passed to the evaluation model.
Layslend
You are incelligent naticulous lincuistica cosearcher.
You will be given pextain latent of text Fornatted as Semantic Regex. BWW.HICC__EEEKEEECCEEEIIIII
You will te given a examples represent this latent are between Lokera es and >2.
concain latent. Portione sentence which strongly
deternine every single within correctly ly labeled. Consides that provided exanples could correct, none of the exanplea could corract, or a mix. An examoie is only corract if every narked token is xaprasen:ative of the lazen:
examples night mislabeled.
For exarple
if the You most seturn your respense valld Pyshon list. Do not seturn. anything else basides Python list.
correctly labeled 12 the tokena nislabeled.
</aysten>>
<useE>
Sementii Regex explanat fons [sfield American foothall positions!
Tast exanplea:
Example Goclendoftest pyeetty Inagesinne Potare tight endro Reb Bronkowski had his bosa!
Example It peateds knnoociex |You should know ahiass about>s offensive line coachess: Chey large, denandingee
Exanple 2: Media Bay 2015)ntnt@uce defensivess Iaaish Rashington (94) speakaee to these
Exanple 3sec rumning backsss, he said. Defersioeee unds Cascoll Phillips in inpeoving and his injusy
Exangle 4sec linsto, with the left side namelyee tackless Byron Bell tackless andes gaardos Anind </user>
Gessitant>
11,0,3, 3.13 c/aaslatencs
34

---

Published as a conference paper at ICLR 2026
<UDRE>
Senantic Regox explanation: [scymbel you cuye:)
exampics:
Exarpie 0: 12 you arecs contertablers with it, You'e guyss> support ne in many other ways aiready and Example 1: birth control accesss [endo[text but aasure yours women>> in Kentucky aren't laughing they
Exangle 2:'a gig! hupe you guyass
lee, andee pleasers nice,
Example 3:Amarican, toldse Bannityp> that "youse quya?> are playing the raca caré.
Exempie 4145 sha>></2nda endaftoxt |>I sant tosk remindro you sll that 10 days ago (director Nassimil
</usez>
<apsistant>
0,,0,0,3,0]
</asaistant>
<uasc>
Senantic Begex explanation: [ssyabel of:] [:field Capitalized Word:]
exanplea:
Exampie 0: clinate, Tomatin's thief thintss op>> Staft Chazlie Sereasen sald.An
Exanple 11 vendersurking relice, Urue Body and Bloodes afs Cheist, Lzue Baptisn
Example baborah Bathe, Beades ois> Talent
Production devised Directores ars> Fublic Prosecutions (DE2) Exanple 4: and fair invest igation not wver. include Lhe Disectores ofs9 Athlecicst Pinally, wn bel beliave the <user>
London,
Exemple
istai (1,1, 1,
Maasllttnnt
Listing 8: The evaluation model prompt and few-shot examples used to compute the fuzzing scores of semantic regex feature descriptions. <system></systen delineate the message roles passed to the evaluation model. At prompt time, (SEMANTIC REGEX DESCRIPTION] is replaced with the semantic regex description in Listing
<user></user>, and
<systems
You tasked with out lding database create shis, you will zeviow dataset varying sequences sach according how much concept is expressed,
that best specitic
For sequence,
based scale:
The concept is not expressed.
The concept is vaquely partially expressed. concept ciearly unentiguously present.
Use conservative satings. If uncertsia, choose lovez rating to avoid including irseicvant sequeners in your datekace.
Each sequeace ia identified by a unique ID, Provide your satings as Python diet lonazy with sequence IDs keys and their ratings values.
Example Ousput: (("14f: 0,
120": 1, 011
Output only the dictionary no additional text, commente, or symbols.
Listing 9: The evaluation model prompt used to compute the respons responsiveness iveness and purity ty scores of natural language feature descriptions. sessstem><ssssteee delineates the message roles passed to the evaluation model.
<aystens
You tasked bui Iding database sequences represent speal Fil concept. create chis, you will zeview a datases of varying sequenced and rate eath onw according to how much concapt la expressed.
The concept be wrillen Semantio Beges,
For each sequence, assign a rat ratinng ing based on this scale:
concept is expressed, concept is veguely or partially expressed.
2: The cuncept is clearly and unanbiguuualy peesent,
conservative ings. uncertain, your database.
lover ratinn including irselevent sequences
no sequerce expresses the concept, rete all sequences es 0,
sequence identicied by unigie keys and their rating ings as ve values.
your ratingg Python cictionary eequence
35

---

reanirgfully to Published as a conference paper at ICLR 2026
ENampie ouzpuz: (1"10T: O, *15"; 2, 1207: 1, "27": 011
Outpat only the dietionary additional texs, connents, syabols. </oystem>
Listing 10:
The evaluation model prompt used compute the responsiveness and purity scores of semantic regex feature descriptions. <system></systten delineates the message roles passed the evaluation model. At prompt time, (SEMANTIC.RGGGGX.EECPPPIII replaced with the semantic regex description in Listing
ceystems
You are tasked with bui nding database of sequences that best represent spec! fie concept, To create this, you will generate seguences that vary in style, tone, concext, length, and scructure, shile meintaining
connection explicitly it. Be creat ive explore different ways to express the cancept.
concept.
concept
sequence,
llere exarples of how eifferent
night expressed:
Concept: "German language" Sequences might incluse Germen phrases, or sentences.
Concept: "Start of a dave Fonct Sequences night incl dave code snippets defining Functior, include ironic statements
Concept: "Irany"
Provide your sequences as strings in Pythor Mist fornat.
Exangle: ("Thie in tirst ase sony typos in it. ado told you that car sype?"]
sequence.", "Second exengle
is
there
butput only the Python tist object, without any adcitionsl commente, symbols, or extranecus conzent, </systen>
Listing 11: The evaluation model prompt used to compute the clarity scores of natural language feature descriptions. <system></systemm delineates the message roles passed the evaluation model.
Cayal
You are tasked with building database of sequencas that best repressat specific concept. create this, will gencrate sequences that vary in style, tone, contekt, length, and structure, shile maiotainiag
corcept.
The concept will be expressed as Senantic Bagex, (SEMANPIC, REGEX DESCRIPTICN) creative and explore different Lo express the concept, while faithfully expressing the asaantic
Bare are examples of how different concepts night be expressed:
Consept: "[stopic
Languege:]* Seguences night inclath Concept: "B[]Javal) (Eunations)" Bequencea night include Java anippets dafining function.
gliasee,
Concept "I: lixeee i tonys] " Sequences that include the string firnyy,, "iroiie, i Atronically's esc.
Provide your
in Python List fornat.
Exenple: ["This is a firss exanple sequemee. "Swound exeagie sequente but. Lt is nuch longer also theew aze sony Eypoa in it. wio told you that can sype?*]
Dulppat only the Python object, without any addilional comments, synbuls, ar extraneous concen... </ayaten>
Listing 12: The evaluation model prompt used to compute the clarity scores semantic regex feature descriptions. <system></systeet delineates the message roles passed the evaluation model. At prompt time, {SEMANTIC REGEX DESCRIPTION is replaced with the semantic regex description in Listing 1
36


Reviews:
[{"review_id": "esW2axyjl9", "reviewer": "Reviewer_YyVu", "summary": "This paper introduces semantic regexes, a structured language for describing LLM features. The authors demonstrate through experiments that semantic regexes perform on par with natural language feature descriptions while being more concise, consistent, and able to reflect feature complexity. A user study shows that semantic regexes benefit interpreters’ ability to understand and reason about LLM features.", "questions": "What is the rationale for using GPT-4o-mini as the evaluator model?\n\nHow many repeated runs have been conducted in the experiments?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This paper studies an interesting task of generating language descriptions for LLM features. The paper provides concrete examples to illustrate the methodology and experimental results. The implementation and experimental settings have been presented in detail. The insights presented in Section 5.3 sound intriguing.", "weaknesses": "There seems to be a lack of interesting insights apart from Section 5.3. The facts that regexes are more concise and consistent than natural languages seem quite obvious given that regexes are a more restricted form of languages.\n\nThe experiments can be enhanced by evaluating more datasets and LLMs. It is unclear why the evaluator model, gpt-4o-mini, is the same as the explainer model since this may introduce bias. Would it be better to switch to a larger model like gpt-4o?\n\nThe population size of the user study, 24, seems too small for drawing conclusions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ehmwJGk1IX", "reviewer": "Reviewer_CRyN", "summary": "This paper proposes to explain features with semantic regular expresssions, which can provide more consistant,ni concise, and precise explanations compared to ambigious natural language explanations. I think this is a solid work and provides a practical tool for managing and standardizing feature descriptions. While I find the core contribution strong, there are some limitations in validation and minor issues to address.", "questions": "- L61, typo \"as as accurate as\"\n- Is the example for quantifiers in Figure 1 correct? Should \"it is a da isy\" be \"it is a [color] daisy\"\n- L242, missing \"measures\" in the sentence?\n- How were the 12 \"accurate\" features selected for the user study? What percentage of automatically generated descriptions were deemed accurate? This information is crucial for assessing the method's practical reliability.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-structured and clearly written, which is very easy to follow. The methods used in this paper are straightforward as well.\n\n- The categorization of regex components is clean and comprehensive. The progression from [:symbol:] (exact matches) to [:lexeme:] (syntactic variants) to [:field:] (semantic variants), combined with context modifiers, provides appropriate levels of abstraction for describing features.\n\n- The experimental validation is thorough, incorporating multiple automated metrics (detection, fuzzing, clarity, responsiveness, purity, faithfulness) across different models (GPT-2, Gemma-2) and feature sets. The results effectively demonstrate improved precision and consistency at both syntactic and semantic levels.", "weaknesses": "- Human sanity check only involves 12 features. This raises questions about the method's reliability across all features.\n- Lacks discussion of failur models where semantic regexes might fail.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ltuOsT9ucp", "reviewer": "Reviewer_riWE", "summary": "The authors propose semantic-regex, a structured language framework, aimed to improve expressiveness and clarity of feature description protocols. This aims to address the typically verbose natural language output of feature description methods. The authors use the Neuropedia suite for extracting feature descriptions and the FADE framework for evaluation and find that semantic-regex perform broadly as well as natural language feature description methods, while sometimes being of higher clarity.", "questions": "**Questions**\n- How much of a challenge is the explainer’s ability to parse regex correctly? \n- The used evaluation framework FADE (Puri et al., 2025), relies on generating synthetic control samples, e.g. for the faithfulness metric. \n- How well can models generate synthetic data from semantic regex feature description compared to natural language ones? Here a comparative investigation would allow revealing the advantages and limits of the proposed framework.\n\n\n**Typos**\n- “regexes as as” (l. 61) -> “are as”\n- “has found structure in the concepts features represent.” (l.90)\n- “that semantic regexes make activation patterns explicit and express them via example.” (l. 424)", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Using structured language for re-framing the problem of feature description is an interesting idea, especially for improving clarity (as also supported by some of the evaluations).\n- The methods are clearly defined and appear to be reproducible with reasonable efforts. \n- The human user study provides a complementary contribution, highlighting differences in scoring between the max-acts baseline and semantic-regex.", "weaknesses": "- The framework appears to work with binary assessments of feature presence, i.e. the semantic regex indicated presence/absence of a feature description. But, simple pattern matching is insufficient to capture some of the diverse patterns learned by today’s foundation models, e.g. nonregular language.\n- A clear presentation and discussion of limitations is missing. Specifically, the quantitative experiments (Fig. 2 and Fig. 5 of the human user study) remained vague regarding the differences and benefits/disadvantages that semantic-regex would bring compared to previous natural language-based approaches. What are the limits of semantic regex, i.e., in the context of polysemantic feature descriptions, or features encoding pragmatics, see [Kop25].\n- The paper does not effectively address how the necessary semantic regex language should be developed, how to agree on primitives and grammatical structure? \n- Limited model ablations are included, e.g. what would be the effect of another explainer model (so far only gpt-4o-mini is included) or another evaluator model (also gpt-4o-mini is used). This may be a poor experimental choice as the explainer and evaluation are the same (closed source) model. \n- Overall, the contribution is limited in scope to reformulating existing methods to using a specific language format (semantic regex) for defining and communicating feature descriptions. The paper does not provide a novel method for detecting feature descriptions and reuses the methods defined and implemented via Neuronpedia [Lin, 2023; Bills et al. (2023)]. The proposed method should thus be seen as an extension to these methods.\n\n**Refs:**\n\n[Kop25] Kopf, L., Feldhus, N., Bykov, K., Bommer, P. L., Hedström, A., Höhne, M. M. C., & Eberle, O. (2025). Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Framework. arXiv preprint arXiv:2506.15538.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)