Learning Encoding-Decoding Direction Pairs to Unveil Concepts of Influence in Deep Vision Networks
Authors: Alexandros Doumanoglou
Kurt Driessens
Dimitrios Zarpalas
Venue: Accepted by TMLR ()

Abstract: Empirical evidence shows that deep vision networks often represent concepts as directions in latent space with concept information written along directional components in the vector representation of the input. However, the mechanism to encode (write) and decode (read) concept information to and from vector representations is not directly accessible as it constitutes a latent mechanism that naturally emerges from the training process of the network. Recovering this mechanism unlocks significant potential to open the black-box nature of deep networks, enabling understanding, debugging, and improving deep learning models.

In this work, we propose an unsupervised method to recover this mechanism. For each concept, we explain that under the hypothesis of linear concept representations, this mechanism can be implemented with the help of two directions: the first facilitating encoding of concept information and the second facilitating decoding. Compared to previous matrix decomposition, autoencoder, and dictionary learning approaches which rely on the reconstruction of feature activations, we propose a different perspective to learn these encoding-decoding direction pairs. We base identifying the decoding directions on directional clustering of feature activations and introduce signal vectors to estimate encoding directions under a probabilistic perspective. Unlike most other works, we also take advantage of the network’s instructions encoded in its weights to guide our direction search. For this, we illustrate that a novel technique called \textit{Uncertainty Region Alignment} can exploit these instructions to reveal the encoding-decoding mechanism of interpretable concepts that influence the network's predictions.

Our thorough and multifaceted analysis shows that, in controlled, toy settings with synthetic data, our approach can  recover the ground-truth encoding-decoding direction pairs. In real-world settings, our method effectively reveals the encoding-decoding mechanism of interpretable concepts, often scoring substantially better in interpretability metrics than other unsupervised baselines, such as PCA and NMF. Finally, we provide concrete applications of how the learned directions can help open the black box and understand global model behavior, explain individual sample predictions in terms of local, spatially-aware, concept contributions and intervene on the network's prediction strategy to provide either counterfactual explanations or correct erroneous model behavior.

Published in Transactions on Machine Learning Research (02/2026)
Learning Encoding-Decoding Direction Pairs to Unveil Concepts of Influence in Deep Vision Networks
Alexandros Doumanoglou
aldoum@iti.g
Deportment of Aduanced Computing Sciences (DACS), University of Maastricht (UM)
Information Technologies Institute (ITI), Centre for Research and Technology Hellos (CERTH)
Kurt Dricssens
kurt. www..seensmmmssrrhttssssssstttee
Department of Advanced Computing Sciences (DACS), University of Maastrichl (UM)
Dimitrios Zarpalas
zarpolas@iti.p.
Information Technologies Institute (ITI), Centre for Research and Technology Hellos (CERTH)
Reviewed OpenReview: https: //
https://openrevieu openreview. net/ fer/formm? id= LIeyZpPEJn
Abstract
Empirical evidence shows that deep vision networks often represent concepts as directions in latent space with concept information written along directional components in the vector representation of the input. However, the mechanism to encode (write) and decode (read) concept information and from vector representations is not directly accessible as it consti- lutes latent mechanism that naturally emerges from the training process of the network. Recovering this mechanism unlocks significant potential to open the black-box nature of deep networks, enabling understanding, debugging, and improving deep learning models.
In this work, we propose unsupervised method to recover this mechanism. For each cept, we explain that under the hypothesis of linear concept representations, this mechanism can be implemented with the help of two directions: the first facilitating encoding of concept information and the second facilitating decoding. Compared to previous matrix decompo- sition, autoencoder, and dictionary learning approaches which rely on the reconstruction of feature activations, we propose a different perspective learn these encoding-docoding direction pairs. We base identifying the decoding directions on directional clustering of feature activations and introduce signal vectors to estimate encoding directions under probabilistic perspective. Unlike most other works, we also take advantage of the network's instructions encoded in its weights to guide our direction scarch. For this, we illustrate that a novel technique called Uncerlainly Region Aligumend can exploit these inst ructions to re- veal the aacoding-decoddie mechanism of interpretable concepts that influence the network's predictions.
Our thorough and multifacoted analysis shows that, in controlled, toy settings with syn- thetic data, our approach can recover the ground-truth encoding-decoding direction pairs. In real-world settings, our method effectively reveals the ancoding-decoding mechanism of interpretable concepts, often scoring substantially better in interpretability metrics than other unsupervised baselines, such as PCA and NMF. Finally, provide concrete appli- cations of how the learned directions can help open the black box and understand global model behavior, explain individual sample predictions in terms of local, spatiall--aware, concept contributions and intervene on the network's prediction strategy to provide either counterfactual explanations or correct erroneous model behavior.

---

Published in Transactions on Machine Learning Research (02/2026)
Concept:
sky
Representation A
Encoding
Decoding
Embediding:
51
Concept Factor:
Threshold:
Xp aps1 1 + 2252
Filter:
Concept:
boat
w S2
Embedding:
Concept Factor:
q²
ba (sky)
S1
W1
w S1
Threshold:
ap b² (not boat)
Filter:
ap
Figure 1: Concept Encoding- - Decoding under the Linear Representation Hypothesis: Deep networks encode high-level concepts, such sky or boat, in distinct directions of their latent space, respectively and 82. The illustration shows the encoding of two concept latent factors (i.e., the degree of concept presence) a', within a patch's 's representation up by utilizing the concept embedding directions Additionally, demonstrates how filter wi can be employed to extract one of these latent factors from the representation. The illustration omits depicting the latent space bias for brevity. We use the terms (encoding direction, concept embedding, signal direction) and the terms {filter, decoding direction) interchangeably throughout this article.
1 Introduction
The linear representation hypothesis suggests that deep neural networks encode high-level concepts in direc-
tions of their latent space (Bereska & Gavves 2024). This hypothesis is sufficiently supported by empirical evidence, mostly by the effectiveness of linear probing. In the latter, a single linear layer can be trained on top of feature representations originating from the upper part of deep neural networks to solve semantic tasks
with great success (Szegody al. 2014 Alain & Bengio 2018 Zhou et al.] 2018 Kim et al.] 2018 Elhage ee al. 2022 Nanda et. al., 2023). In the concept encoding mechanism suggested by the linear representation hypothesis, the concept content of an input patch is written to its embedding as linear combination of concept embeddings: directions in the latent space represented as vectors, each one associated with concept. The corresponding scalar coefficient in the linear combination constitutes the concept's latent factor that encodes the degree of concept presence in the input. Due to its utility in encoding the latent factor to a vector representation, we also refer to the concept embedding as the concept's encoding or signal direction. Retrieving the latent factor back can be accomplished by taking the inner product between the patch em- bedding and a filter, another vector which we term a decoding direction. (Fig. [ An aacoding-decodie direction pair is interpretable whenever it is related to concept that is aligned with human intuition; and is influential whenever it is related to concept that the network consistently uses to make predictions (Kim Pfau et al. 2020).
et al. 2018
The encoding decoding mechanism of concepts for a pre-trained network is not directly accessible. Instead, it is a latent mechanism emerged from netw training and needs to be inferred by any means of reverse engineering. Recovering this mechanism is of significant importance, as accessing it enables understanding
representations (Zhou et al. 2018 Kim et al. 2018), assessing influence of concepts on the network's predic- tions (Kim et al.
2018
Pfau et al.
2020 Fel et al. 2023b Pahde et al. 2024). debugging neural networks
(Anders et. al.
2020
and correcting erroneous model behavior Anders et. al. 2022 Pahde et al. 2023
Neuhaus et al.
2023
Dreyer et al. 2024a
Previous works that identify concept directions rely either on concept annotations Zhou et al. 2018 Kim
et al.
2018
Pahde et al. 2024) or on a decompose-thenu ther--essttrr regime of feature activations (Zhang
ct al.
2021
Graziani et al. 2023b Bricken et al. 2023 Lim ct al., 2024 Cunningham et al., 2024 Bussmann
ee al.
2025
The former sulfer from the need for concept speculation and the access annotations while
the latter, face the difficulty of overcoming unstructured noise that may be present in feature activations in
the form of distractors (Haufe et al.) 2014 Kindermans et al. 2017 or dark matter (Engels et al. 2025)
Unlike previous approaches, our method relies on neither annolations nor the reconstruction of activations, thereby sidestepping these earlier limitations. Instead, we identify decoding directions via directional clus- tering of fcature activations and estimate encoding dircctions from a probabilistic perspective. Morcover, in contrast most prior work, we guide our search for directions by exploiting the network's instruc- tions, i.e. the prediction strategy baked into its weights through process we call Uncerlainly
2

---

Published in Transactions on Machine Learning Research (02/2026)
Global Explanations via Concept Sensitivity Testing
Local Explanations via Concept Contribution Maps
Concapt Infuuance Diagram for Places-365 Class: castle Network: Resnet20
Applications of Encoding Decoding Direction Pairs
(1 10 Saa d do dimm
W
S
indutral Eisuraadem
Spatially-Awand Counterfactual Explanations
Learnable Parameters
Diagnosis and Correction
W
S
Flawed Model Behavior
Classifier 5
decoding directions
encoding directions
b
No manipulation building ssirplane
classification thresholds
Figure 2: Our Ehcoding-Decoding Direction Pairs (EDDP) powers a range of applications, highlighting both the generality and the precision our approach. The figure summarizes applications that we selected discuss in this work.
Region Alignment. We conduct a thorough and multifaceted analysis that highlights the effectiveness of the approach in revealing the encoding-deooding mechanism of (a) the ground-truth concepts controlled and synthetic toy settings, and (b) interpretable monosemantic concepts in real-world studies of state-of- of-the-art deep vision architectures. We also demonstrate the utility of our learned dinections in obtaining global, local, and sounter-factual explanations, well as identifying and correcting flawed model behavior in toy setting (Fig. 2
Our comparative analysis primarily considers established matrix decomposition baselines, specifically Prin- cipal Component Analysis (PCA) and Non-Negative Matrix Factorization (NMF). We further extend the comparisons Appendix Section A.16 where we evaluate our approach against the methods on which we build, namely Unsupervised Interpretable Basis Extraction (UIBE) (Doumanogiou et al. 2023) and Concept Basis Extraction (CBE) (Doumanoglou et al. 2024).. Throughout, focus settings in which the concept space does not exceed the dimensionality of the embedding space, and we leave systematic comparison with Sparse AutoEncoders (SAEs) (Sharkey et al. 2022) to future work.
2 Related Work
2.1 Direction Learning
We categorize related work into supervised and unsupervised direction learning approaches. In each cate- gory, we go through the previous methods, describing the limitations, differences, and similarities with our approach.
Supervised Concopt Direction Learning
Concept Activation Vectors (CAVs) (Kim et al. 2018 Zhou et al. 2018 use supervised linear classifiers trained on annotated concept datasets to separate representations of samples with target concept from those without. To solve the task, the resulting classifier weights (filters) must extract (docode) the concept factors from these representations; thus, they approximate concept decoding directions, although they can be inexact due to distractor- noise the features (Haufe et al. 2014 Kindermans et al.) 2017 Pahde et al, 2024). To estimate concept encoding directions, PCAVs (Pahde al., (024) take the difference of mcans between representations of samples with and without the concept. These methods require both labeled data and prior specification of the concept name. In contrast to these probing approaches, the proposed method
3

---

Published in Transactions on Machine Learning Research (02/2026)
is unsupervised, revealing concepts by analyzing the structure of the latent space and thereby avoiding both annotation costs and upfront concept specification.
Unsupervised Concept Direction Learning Existing unsupervised direction-learning methods that rely on matrix decomposition (PCA (Graziani et al., 2023a), SVD (Graziani et al. 2023b NMF (Zhang et al. Fel et al. 2023b)) can discover concept directions without annotations, but they have key limitations. PCA and SVD are constrained by orthogonality and miss concepts that do not influence variance (Fel et al. 2023a). NMF, while non-negative, lacks latent space bias and, thus, expressivity, and it has
2021
straightforward decoding directions. To alleviate these shortcomings, at least to some extent, Graziani et al.
[2023a] b) combined direction learning with gradient ro-ranking, and both Zhang et al. (2021 and Graziani
et al. (2023a b) worked on subgroups of features belonging to the same class category, establishing class- conditioned protocol for direction learning. In contrast, the proposed method uses a class-agnostic protocol without the need for sample class labels. Additionally, our approach considers non-orthogonal concept directions and accounts for feature space bias, overcoming the limitations of previous methods.
Dictionary Learning (Bricken et al. 2023 Yun et al. 2023) and Sparse Autoencoders (SAEs) 4Sharkey et al,, 2022 Cunningham et al. 2024 Bussmann et al. 2025) identify decoding-encodiing directions driven by the reconstruction of representations after decomposing them into sparse latent factors. They typically consider a latent space larger than the embedding space and may suffer from the difficulty of overcoming noise in the representations in the form of dark matter (Engels et al. 2025). In contrast, our method is independent of feature reconstruction, unrestricted by the relative sizes between the concept and the embedding space, and unlike these prior works, it implicitly links the learned directions to how the model actually utilizes them. However, in this work, we restrict our experiments cases where the concept space is no larger than the embedding space and leave experimental comparisons to SAEs with the opposite assumption for future work. Further discussion regarding SAEs is provided in Section A.23
Moving away from reconstructing representations, the methods of Doumanoglou et al. 2023 2024 perform directional clustering of feature activations, a process that often unveils the decoding mechanism of inter- pretable, monosemantic concepts. We build on these previous approaches to estimate decoding directions, removing orthogonality and feature-standardizttion constraints and introducing new loss terms that pre- serve improve concept interprotability while reducing the impact of distractor noise in the filter weights. Compared to the previous attempt (Doumanoglou el al., 2024 of linking directions to their utilization by the network, our Uncertainty Region Aligument approach achieves up to 22.562 56% relative improvement in interpretability metrics in 3 out of 4 cases. We also address the estimation of concept encoding directions, which these prior works did not address. Further details on this comparison are provided in Section A.3
2.2 Applications of Directions
Our learned direction pairs enable a series of applications, from global model understanding and detailed spatially-aware local explanations down to model intervention, which allows for counterfactual explanations and model correction. Typical previous enablers of such applications are the Concept Bottleneck Models (CBMs) (Koh et al.] 2020). Classic CBMs consist of backbone feature extractor, concept bottleneck predicting concept content from features, and a linear head mapping concepts to classes. What makes this architecture interpretable is the fact that the concept vector, i.e, the output of the concept bottleneck, is trained to have each dimension aligned with an interpretable concept. Recent post-hoc CBM variants YYuksekgonul et al. 2023 Oikarinen et al. 2023) retrofit this structure onto existing black-box models by freczing the original backbonne discarding its head, inserting a concept bottleneck projection into concept space, and learning a lask-specific sparse linear head over concepts. To learn concepts without annotations, these works exploit CLIP [Rdford et al. (2021), ) and further advances Benou & Raviv (2025) use visual prompting (Shtedritski et al. 2023) to make the bottleneck spatially aware, allowing localization of concepts
at the patch level rather than only at the image level. Most recently (Rao et al. 2024). SAEs (Cunningham et al. 2024) have been used to build CBMs directly for CLIP
To access the advantages of CBMs, these previous approaches require training the last lincar head using the model's training data and, sometimes, additional concept-annotater datasets or external models like CLIP. In contrast, the proposed work is non-intrusive: it doesn't need to train new model components and does
4

---

Published in Transactions on Machine Learning Research (02/2026)
not require concept labels. It shows that one capture many benefits of concept bottleneck models by identifying directions corresponding to concepts the model already encodes and then using these directions in applications of mechanistic interpretability (Saphra & Wiegreffe 2024 Kästner & Crook 2024 Bereska
& Gavves 2024).
3
Background
In our formulation, which follows the linear representation hypothesis (Elhage et al. 2022 Nanda et al. 2023 the latent factor of concept is scalar denoting the concept's 's presence in the input, and is embedded in the latent space via multiplication with the concept's encoding direction, also called the concept's embedding or signal direction (Kindermans et al.] 2017). For this reason, we also refer to this latent factor as the signal value. Feature activations are considered as linear combinations of signals and noisy directional components called distractors. In the proposed approach, & filter, through the inner product with an input representation, can extract the signal's value and for this reason we term it a decoding direction. Below we provide more formal explanation of these terms and provide details essential to understand our contributions.
3.1 Preliminaries
Let X E "Xxx denote the feature representation of an image in an intermediate layer of deep neural network with spatial dimensions H, W E N+ and latent space dimensionality D E N Let also Tp E R denole a pixel element of this representation at the spatial location p (w, h), w E 1,.., W - -1, [o. H --11. Since Up is related to a patch within the input image the network, we also refer to it as patch embedding.
3.2 Signals, Distractors, Filters, Concept- Detpttoo
In encoding a single concept i, Kindermans et al. (2117; Pahde el. al. (2024) proposed a model for the data generation process of feature representations: Tp aps; + Spd, 8;, d RD, ap, 8p E R. Here, 8; is the signal direction that carries the information of whether Tp is part of concept i. The concept information lies within the signal value Op- Larger Op suggests greater confidence that Sp belongs to concept i. d is the distractor direction, modeling noise, or information not related to the concept. 3p follows a random distribution, typically the gaussian or uniform distribution, and is independent of whether Up belongs to concept According Kindermans al. (2017), the value of the signal Op can be extracted using filter w; and the inner product: Zp,i = ti Fp u 8: 8p d, if we choose w; : wn 1. d, and w 8; 1. Tel denote the sigmoid function. Since stronger values of Op indicate more confidence in concept presence, when combined with threshold b, E R that can be learned from data, this filter can be turned into concept detector: Vp. a(pp,- b;),, essentially a binary classifier that can answer the question of whether belongs to concept i.
With access to the signal value, Haufe el al. (2014); Kindermans et al. 2017 offer formula to estimate the concept's signal direction:
8; cov pp, apl var var[opp]
cov p 2p.i] var 2pr
(1)
3.3 Unsupervised Interpretable Direction Learning
Recent research (Doumanoglou et al. 2023 introduced an unsupervised method to identify concepts from the structure of the latent space. Motivated by the directional encoding of concepts, the method partitions the latent space into linear regions, each represented by hyperplane and normal vector, forming clusters. Feature activations from an unlabeled concept dataset, possibly activations training set, are assigned these clusters. The method learns W and b of a feature-to-cluster membership function, a mapping to the semantic space, with Up oW xp - b) E 1], W E R b R', and I the cluster count. By softly assigning features to small number of clusters, the interpretability of the
images from the network's
5

---

Published in Transactions on Machine Learning Research (02/2026)
a (woxp bb)
wo
ow
-1))
W1
Xp
X
(wi71xp -1-1)
W1-1
Soft-Binary & Sparse Concept Space
yp
0.9
0.1
0.1
0.8
0.0
red
motorbike
sky
blue
Groups of mutually exclusive concepts
Figure The core concept Unsupervised Interpretable Basis Extraction (UIBE Doumanoglou al. (023 learn a set of concept detectors, which essentially binary linear classifiers with learnable filters and biases. These detectors aim to transform feature representations to the soft-binary vector space of concepts in which the newly transformed representations are sparse. In this procedure the input to the method consists of image representations coming from an unlabeled concept dataset. Identifying the concept name behind each detector done in post- processing step with procedure refer to Direction Labeling.
clustering is improved. This is grounded in the idea that an image patch generally holds only a few semantic labels from a larger set, reflecting sparsity in the semantic space. Sparsity in the assignments is achieved using two loss lerms: the first Sparsily Loss (L*), and the second Maximum Aclivation Loss (Lma), which ensures binary cluster membership:
L Ep [
L - 9 log, (Bp)).
Yp
(2)
C = H(pp),
qp
ypll
with H. denoting entropy. Under a different interpretation perspective, a column W together with a corresponding element of b (i.e., w. b2) forms linear classifier or concept detector Up.i ow ax b)), which means that w; is filter and thus acting decoding direction. This method also optimizes lincar
separability by minimizing the inverse of the classification margin M; Tw. (Marimam Margin Loss
Cmm) and penalizes clusters with few assignments using the Inactive Classifier Loss Lie (Doumanoglou et al. 2024 (See Fig. and more details in Section A.1
3.4 Direction Labeling
When learning directions in an unsupervised manner, the name of the concept represented by each encoding- decoding direction pair unknown. In a real-world setting, the concept name could be identified by manually inspecting the samples that (maximally) activate the decoding direction. However, in a benchmark setting, where interpretability needs to be quantified in terms of metrics and without human involvement, it is necessary to automate the assignment of a concept label to each of the direction pairs. We refer to this process direction labeling and employ Network Dissection Bau et al. (0177) for its implementation. In essence, Network Dissection assigns a semantic label to cach of the concept detectors based on their segmentation performance in a dataset. with annotated concepts. Although it was originally proposed as a method to assign labels to each of the basis vectors in the natural latent space basis, Network Dissection capable of assigning labels to any set of directions after basis change. Despite possible biases against unsupervised learning due annotation limitations, we adopt and expand on this labeling protocol as best-effort approach to evaluate the interpretability of concept detectors and the rest of the unsupervised baselines.

---

Published in Transactions on Machine Learning Research (02/2026)
3.5 Concept Sensitivity Testing
Given the intermediate representation of an image belonging to class k and the direction of concept e in the latent space, RCAV (Piau et al. 2020) measures the sensitivity of the model to concept when predicting class k. This is accomplished by perturbing the reprosentation towards the direction of the concept with strength a, and subsequently comparing the output probability of the network for the same class before and after the perturbation. Subsequently, an overall dataset score the range [-1,1] is computed, where zero means inconsistent use of the concept by the model, while extremes indicate consistent positive or negative concept contributions to predicting class k. A statistical test compares concept sensitivity against sensitivity towards random directions to ensure significance. In our evaluations, we refer to directions of significant inffuence in cases where the directions meet the criteria of this statistical significance test. Although the initial work used CAVs to test concept sensitivity, a recent study (Pahde et al. (0224) suggests that the appropriate direction for this purpose is the concept's 's encoding direction, an aspect that we consider in our evaluations.
4 Proposed Method
decoding
Learnable Parameters
encoding
Cluster membership tunction
W
S
Signal Vector
3/p
(
Tp-
b)
cov cov|
" Tpp Fpp
8:
var (wo Tp
soft binary, sparse
subsample
f
nea
Uncertainty Region Alignment
Tp w Tpp b.
wi
# i
penalize clusters
penalize small clusters
LUT
[
6ac
maximize classification margin
improve cluster interpretabiliit and influence
Figure 4: The proposed method analyzes the latent space to uncover its directional structure. Because many concepts are naturally encoded as specific directions, this process often reveals encoding- mechanism of meaningful, monosemantic, and highly interpretable concepts. The figure depicts overview of the method's components. C denotes loss terms. Purple indicates contributions this work, while light gray indicates loss terms from Doumanoglou et al.
2023 2021).
Method Overview
The aim of our unsupervised approach is to reveal the encoding-deeoding mechanism of meaningful concepts that influence the network's predictions. Our method learns encoding-decoding direction pairs that explain the structure of deep vision network's latent, space and takes as input feature representations from an unlabeled concept dataset. The latent space, from which feature representations come, corresponds to a network layer at a given depth. We jointly learn the decoding directions (concopt detectors) {W, b} and the encoding directions (signal vectors) S in an end-lo-end process.
Our formulation is built upon several key contributions:
1. Flexible and Sparser Clustering: We lift previous constraints on the concept detectors (such as orthogonality between filters and feature space standardization) in Doumanoglou et al. (2023) enabling more flexible and less rodundant (i.c., sparsor) clustering. To preserve or even improve interpretability, we incorporate two novel loss terms: Focal Sparsity Loss and Excessively Active Classifier Loss (Section 4.2

---

Published in Transactions on Machine Learning Research (02/2026)
Operations
Manipulation/ / Uncertainty Region Alignment
W
((x; W)
Or
ft
Xp
X
h(X: W.S)
Zp WXp
Losses
yp
Pp
yp y'pl
yp c(pp (b)p-b) b)
/so
mm
Leac
mR
Figure 5: Left:
The learnable parameters of the method S, W. b and intermediate variables 7p, Vp:Пp- Top Right: Feature manipulation and Uncertainty Region Alignment. Bottom Right: Loss terms & with their dependencies. Purple indicates loss contributions of this work, while light gray indicates loss terms from Doumanoglou al. 2023 20241.
Encoding Direction Estimation: We incorporate an extended multi-concept signal- distractor model (Section 4.1 and define signal vectors (S) estimators for the concept coding directions based on probabilistic framework (Section 4.3
3. Network Instruction Driven Search: We introduce Uncertainty Region Alignment (L"r) (Section 4.4), a novel loss that can improve concept interpretability and influence by aligning concept detector ambiguity with model prediction uncertainty. For the latter, the method makes use of the upper part of the network, after the layer of study, that produces output class probabilities, denoted as f-
Effective Optimization: The entire set of learning parameters is optimized jointly using an 6- constrained optimization scheme based on the Augmented Lagrangian Loss (Section 4.5), which allows direct control of clustering quality with easier to tune, interpretable hyper-parameters
Our method overview and component interconnections shown in Figures are described in Table 1 and the learning algorithm is outlined in Section A.2
4 and The used loss terms
4.1 Multi-Concept Sggnal-Distractor Data Model
We introduce an extended signal-distractor data model for the latent space, which models the encoding of multiple concepts. Fach patch embedding Fp is considered linear combination of latent concept signals S Rxx and distractors D IRDx DxF. We also consider latent space bias RD, common for all Tp.
Up Sop D3p+ +
with Op R and 3p E F.F. S a matrix of 7 E N+ D-dimensional, unit-norm concept signal directions and D a matrix denoting a basis for distractor components. Each signal direction encodes the presence of a distinct concept. We apply the same assumptions for individual signal values ap.; (the i-th element of ap) and distractor coefficients Bp. as in Section 3.2
4.2 Losses to Preserve or Improve Interpretability
We propose Self- Weighted Reduction (Rsw) as an aggregation method estimate the maximum element in sct. Consider the set of elements (Gx), E R+ k E N. The Selg-Weightod Reduction is defined as:
Σ. cu+1
Row
(4)
Σx S%
8

---

Published in Transactions on Machine Learning Research (02/2026)
which is equal to the weighted average of elements in {Gx} with each element being weighted by Sk 2 1, E IR+ a sharpening factor. This aggregation may be seen as soft differentiable version of the max operation, since the largest value in the set {(k} } is weighted with the largest weight.
Excessively Active Classifier Loss (Crar) This loss penalizes excessively large clusters to prevent trivial
solutions which all inputs are assigned to a single cluster. relies on a hyper-parameter p E (0, 1), similar sparse autoencoders Ng al. (2011), which sets a proportional bound on cluster size. With 3/p [0. I]
denoting the membership of patch p for cluster (Section 3.3 the unreduced loss formula for the i-th cluster
is below, with y y E R+ sharpening factor, 1 normalizing the loss in the range [0, 1], and p varying across all pixels and image reprosentations in the concept dataset:
CTc 1 (Ep
(5)
The final loss uses Rgw: -eac_ Rsw (s (1 E (0.1,,..11-111
Focal Sparsity Loss (C Inspired by Focal Loss (Lin al. (017))) ) we introduce Focal Sparsity Loss, which places more emphasis on sparsifying the toocusett assignments of the most challenging patch embeddings. If C 2 denotes the Sporsity Loss for pixel p, the Focal Sparsity Loss is defined as:
L ΣpBpLi
(6)
Σp 0p
i.c. based a weighted scheme defined by coefficients 8p E (0, 1]. We choose 8p to be proportional to the patch's number of cluster assignments. In detail:
0p = 1 - (Rsw (fgp..i E 10, - 11)))
(7)
M/ calculated as in 2 Rsw {qp.i} E [0, 1] is approximately equal with j a sharpening factor and 4p I to 1/k, with k denoting the patch's number of cluster assignments. Similar to the previous case, in (6)
p varies across all spatial elements and image representations in the concept dataset. When we use Focal
Sparsity Loss we use it as replacement for the Sparsity Loss L of Doumanoglon et al. (0023).
4.3 Signal Vectors Concept Signal Estimators
In this paragraph, consider the new data model that was outlined in Section 4.1 and try to validate whether |I can be used to estimate the signal direction of a concept. Suppose that our objective is to estimate the signal direction of concept i when we have access to a collection of patch embeddings {æp} and their signal values {ap.s)- Starting from the approach of Kindermans et al. (2017), it is easy to prove that whenever dp. is independent of all (p.j. + and 3p.s the following property holds: cov(ar- -pp: ap.)] = O, which, according to Kindermans al. (2017) directly implies that ([] can be utilized to estimate the direction of the signal.
However, while ap,s can be considered independent of distractor coefficients Pp. that represent. noise, the independence assumption between Cp.s and "p.j. # i is easily violated in practice. Some pairs of variables ap.i and "pjić + may indeed be independent and this would be the case when concepts and j belong to different groups of cuntuall--exchssit concepts, for example when concept i belongs to the group of objects and concept. j to the group of colors and the concepts of the first group are independent of those of the second. However, when the concepts and j belong to the same group of mutually exclusive concepts (for instance, when concept corresponds to car and concept j tree and both concepts belong to the group of objects), there may be anti-correlated relationship between the variables ap.j and ap.i. * i due to the fact that whenever ap.i > by, ap.j < by. However, the latter bias can be climinated if we consider only samples with the concept instead of both positive and negalive samples. In that case, among that subset of the data, the signal values ap,i and ap.j can be considered independent by assumption, as we now removed the biases by, by due to sub-sampling. This allows us to still consider [1] as a signal estimator, even under the extended data model of multiple concepts, provided that in the computation of the covariance and variance terms of we subsample the patch embeddings based on their concept label, i.e. keep only samples with
9

---

Published in Transactions on Machine Learning Research (02/2026)
Uncertainty Region Alignment
dusty road motorbike region of uncertainty
improved direction interpretability improved direction influence
Concept tree: 0.1%
Concept tree: 50%
Concept airplane: 0.1%
Concept airplane: 50%
Concept dusty road: 95%
Concept dusty road: 50%
Concept motorbike: 0. 0.1%
Concept motorbike: 50%
Image class: moto-cross: 99%
Image class: moto-cross: 0.1%
Figure 6: The uncertainty region of the network is defined the subspace where all network's predictions are maximally uncertain. The uncertainty region of the concept detectors is defined as the intersection of all their decision hyperplanes, Le. the subspace of ambiguuus concept predictions. Aligning these two through feature manipulation may improve direction pair interpretability or serve as a balance between the interpretability and influence the learned direction pairs.
the concept, instead of additionally considering samples without it. The latter can be easily accomplished when employing the respective concept detector.
We refer to the signal estimator of concept that is obtained under these conditions as the signal vector 8; However, we still require access to the signal values. As explained in Section 3.2 estimating signal values can be attributed to the filters of the concept detectors. They can serve this purpose if the weight vector wi is orthogonal to all 8j where j + i, as well as the distractor subspace D.
Thus, we employ the following Filter-Signal Vector Orthogonality Loss when learning the directions:
C Eig [((1 d 5,)
with jig the kronecker delta and w; denoting the T.2-normalized filter weights and signal vectors.
To achieve accurate signal value extraction, w; should additionally be orthogonal to the distractor basis; however, we do not explicitly estimate the distractors. Instead, we use the Uncertainty Region Alignment loss from Section 4.4 to ensure alignment of the directions with utilization by the network.
4.4 Uncertainty Region Alignment to Discover Meaningful Concepts Influence
The presence or absence of concept in a representation can provide neutral, supportive, or opposing evidence against the prediction of a class. Since the concept-class pair association is unknown when learning concept directions, straightforward strategy perform concept arithmetic on the features order to find their utility by the network lacks ground-truth information on how concepts affect class predictions. To overcome this difficulty, we make simple but more elegant hypothesis that uncertain network predictions occur when the representation has ambiguous concept information. We propose improving the direction search by aligning the uncertainty regions of the network and the concept detectors. The uncertainty region of the network is the subspace where its predictions are most uncertain, and the uncerlainly region of the concepl detectors is the subspace where their decision hyperplanes intersect. Figure illustrates the concept of Uncertainty Region Alignment.
To accomplish the alignment, all the patch embeddings ep in an image manipulated Lowards the direction dxp to arrive at x Ep dxx Based estimates of w,, b, and &,, select the direction dxp so that the shifted æ lies the intersection of the concept detectors' decision hyperplanes, i.c. w (Ip dx,p) b, 0, Vi. Then, ensure the network's prediction for the resulting manipulated image representation is highly
10

---

Published in Transactions on Machine Learning Research (02/2026)
Table Encoding-Decoding Direction Pairs Loss Descriptions. Purple indicates loss contributions of this work, while light gray indicates loss terms from Doumanoglou et al. 2023 2024
Symbol
Loss Name
Loss Description
Lia
Focal Sparsity
Ensure sparse palch-to-cluster ussignments, emphasizing challenging cases
Maximum Activation
Push towards binary patch-to-cluster memberships
C
Inactive Classifier
Ensure all clusters have a sufficient number of members
mm
Maximum Margin
Ensure cluster members are linearly separated from non- cluster memberrs by a sufficient margin
Ceec
Excessively Active Classifier
Penalize large clusters to avoid cases such assigning all patches to a single cluster
Filter Signal Orthogonality
More accurate signal value extraction, enabling a more pre- cise estimation of encoding directions
Dur (Luur or ( Uncertainty Region A Alignment
Discover interpretable concepts of influence exploiting net- work instructions.
uncertain, effectively aligning both uncertainty regions. More specifically, we define two types of feature manipulation for this purpose:
Unconstrained Feature Manipulation (UFM) in which zip corresponds to the closest point of xp the uncertainty region of the concept detectors, i.e.: at 9p (xp: W) Tp dx,, and day given by:
dx, = (W (W xpp- b)
(9)
with A + denoting the pscudo-inverse of A and a detailed derivation provided in Section
A.4
Constrained Feature Manipulation (CFM) in which we additionally restrict feature manipulation to occur within the span of the signal vectors, i.e. dx,- = Se,, Up E R,, with S E Dx1 denoting the matrix whose columns correspond to the learned signal vectors š,, E I I 1}. The feature manipulation formula for a patch embedding Up is hp(xp; 5) = xp - dxp, with Vp and dæp given by:
Up (W 5)+ (w" Tp - b) dxp Sup = S(WT (w xp b)
(10)
(11)
and again, a detailed derivation is provided in Section A.4 These manipulations are carried out for all in the image representation X simultancously, leading to a manipulated image representation X'= = ( W) or X' = (x; W,S), S), based on the manipulation type (Fig. 5).
Finally, the Uncertainty Region Alignment Loss (C"r) is:
L --==Exx [H(*(X')
(12)
with H denoting entropy. When we manipulate the representations via either UFM or CFM, we use the layer's activation function (typically a ReLU) to keep the features in the input domain of the next layer. Throughout the experiments, we will denote the Uncertainty Region Alignment loss with UFM as goor and with CPM as feur. We consider the use of Luur and Ceur to be mutually-exclusive. We either use the first the second type of manipulation when computing the Uncertainty Region Alignment Loss and never both types at the same time.
4.5 Augmented Lagrangian Loss for Effective Direction Learning
The loss terms of the proposed method often conflict with each other. For instance, increasing confidence in the predictions of concept detectors (Lma) conflicts with maximizing the separation margin (Lmm), and
11

---

Published in Transactions on Machine Learning Research (02/2026)
large margin may also conflict with sparsity losses (L ((*). Simply using weighted sum of these losses requires careful architecture and dataset specific tuning of loss weights. The latter can be laborious and only indirectly controls clustering quality, which is mostly determined by the final loss values. To avoid this, we adopt an c-constrained optimization approach and reformulate our objective as a minimization problem with inequality constraints rather than linearly combining all loss teriss. In this formulation, hyper-paramedde tuning primarily pertains target loss values, providing direct control over the clustering quality. We use the Augmented Lagrangian formulation [estenes 1969 [ertsekas 2014) to solve the following optimization problem:
mm S TT Tm
L < 7ic
min N + x s.t.
mmm S mm
(13)
L S Tane
L < 7/00
with Aar E RR the only loss weight coelficients. In this formulation, we assign larget value to each individual loss term participating in the inequalities and, as shown in the experiments, subsequently optimize for intorpretability (CFs, Lunr), or a balance between interpretability e and influence Lewr.
5 Encoding- -Decodin Direction Pairs in Applications
Moving forward to the application level, in this Section, we díscuss how to read and intervene concept information (Section 5.1) and how to capitalize on the learned direction pairs to obtain local, detailed, and spatially-aware concept explanations (Section 5.2
5.1 Reading Concept Information and Intervening on their Encoding
Reading Concept Information from Patch Embeddings: Reading concept information from a patch embedding equivalent estimating the signal value encoded in the representation. Filter directions can extract this value; however, this is done with constant offset derived from the latent space bias This offset can be neutralized by calculating the difference between two projected embeddings (in the direction of the filter), which directly equals the difference in their signal values. Thus, reading concept information is helpful to be done with respect to the concept information in a point of reference. Two points of reference that we find useful are:
The Average Embedding: Estimating the signal value relative collection's mean, for instance collection of samples with without the concept.
Point of Uncertainty: A point the concept detector's decision hyperplanc that represents maximum concept ambiguity. Centering around this point ensures that positive values indicate concept presence and negative values indicate concept absence.
Intervening on Concept Encoding: Intervention modifies an embedding's concept content by overwriting its signal value with a target value derived from reference embedding. To encode the presence or absence of concept in patch embedding, select. larget values based on:
The Average Signal Value in representative positive or negative concept samples.
Top or Bottom Quantiles of Signal Values in a collection of positive or negative concept samples.
Concrete implementation details for reading and intervening on concept information are provided in Section
A.5
12

---

Published in Transactions on Machine Learning Research (02/2026)
Images
Class a
Class b
Class
Concepts
2
2
2
Representations
Xp1
Xp2
Ap1
App
Xpp
Xpp
Figure 7: Description of the synthetic dataset that use in the experiments: a) Concept set: {0,1,2) b) Image Classes: (a,b,c) with each class correspouding unique pair of concepts. c) Each image comprised of two patch representations Tp1, Tp2-
5.2 Using Encoding- -ecoddnng Direction Pairs and the Regions of Uncertainty to Provide Concept-Based Local Explanations and Detailed Spatially-Aware Concept Contribution Maps
In this Section, we provide details on how the learned Encoding-Decodinng Direction Pairs can be leveraged to provide concept-based local explanations for network's prediction. The core contribution is the introduction of explanation logit (le le 18"), which is the difference between the network's class logit for the input image (lo) and class logit corresponding to baseline artificial representation derived from the uncertainty region of the model ), We then decompose this le into local, concept-specifie components.
In Section A.6 we show that, for networks with a Global Average Pooling (GAP) penultimate layer, the explanation logit can be decomposed into four key components: the sample concept contributions, which are imput dependent, the baseline concept contributions from an uncertain prediction, a correction factor compensating for imperfect learning convergence, and final, unxxplained residual.
Grouping the first two components in the previous decompasition and exploiting the linearity of GAP, the explanation logit enables the derivation of Concept Contribution Maps (CCMs), which provide a spatially-aware breakdown of the explanation logit in terms of its constituent concept contributions. CCMs differ from previous approaches [Zhou et al. 2018 Graziani et al. 2023a in that they do not simply localize the concepts in the input image, but they quantify the contributions of each concept across patches to the explanation logit, including both positive and negative contributions.
The contribution of specific concept i from image patch p to the explanation logit is calculated as the product of two factors:
A Global Contribution Factor, referred to as Concept-Class Relation Coefficient (CCRC) (w""s) independent of the individual image input, with w. denoting the network's class vector.
Local Contribution Factor (0p. - oh.)): A local, sample dependent, spatially-aware factor representing the difference concept. information between the actual image patch (0ps) and the baseline patch (Gg.).
The CCM is a heatmap that visualizes the multiplication of these two factors across the image, precisely showing which regions and concopts positively or nogatively drove the final classification decision. Further details are provided in Section A.6
6 Experiment on Synthetic Data
In this Section, we test our method on synthetic data which follow the data generation process detailed in Scction 4.1 We validate three aspects: a) the effectiveness of signal vectors estimating the concept's encoding direction given ground-truth signal values, b) the efficacy of our unsupervised approach reliably identify the ground-truth concept -ecodingg direction pairs, under challenging conditions
13

---

Published in Transactions on Machine Learning Research (02/2026)
Table 2: Evaluating the performance of the concept detectors classifying patch embeddings in the experiment on synthetic data. The metric is Intersection over Union (IoU). Rows correspond to concept detectors and columns to ground-truth concept classes. Clearly, each detector aligned with one distinct ground-truth concept.
Concept
#0
#1
#2
Detector
#0
0
1.0
0
#1
1.0
0
0
#2
0
0
1.0
for conventional techniques, c) the necessity of Filter-Signal Orthogonality loss L for reliable estimation of encoding directions.
We consider synthetic image representations with two spatial elements P1. P2, i... W = and H = 1. Every pixel p is presumed to be associated with just one concept. from set of T = concepts. Tet c(p) E {0, represent the concept label of p, and k E b, denote an image class. We construct image representations as follows: for h = a, c(pi) = 0 and c(p2) 1; for k= 6, c(pı) = and c(p2) = 2: and for = e, c(p,) and c(p2) = (Fig- 7 We set the embedding space dimensionality to D = 8, the size of the distractor basis S and D. Using those principles and typer-parameeers we generale patch embeddings ? for each image according to (3). The latent signal values and distractor coefficients follow the uniform distribution: Op,i ~ ((000, 2.25) if is not part of concept i and Opp,i (12.75, 5.0), otherwise, while 3p.5 U(00 5.0) is independent of the patch's s concept label. We introduce bias of с = 10 across all dimensions of the ions to maintain them in the positive quartile, similar to the impact of a ReLU layer. We generate balanced dataset with each class being represented by 1000 images.
= and randomly create unit-norm vectors to construct the matrices
Using this dataset, we train a network to predict the image class based on the concept content of its patches. The network we is composed of just two layers (corresponding to the top part of potentially larger deep network). The first is an ewerage-pooling layer, and the second is linear layer with K = 3 output classes. After training, the network attains 96.33% accuracy on test set, randomly generated based on the previous principles. More details regarding the setup of this experiment are provided in Section A.8
6.1 Evaluation of the Signal- Estimator
Based on the synthetic pixel dataset {æp. } U { and the ground-truth signal values ap,i, put the estimator of [Knddrmmnn et al. (2017) and our signal vectors under test. The difference between the two estimators is the sub-sampling procedure that we proposed in Section 4.3 Given the ground-truth matrix of signal directions we are able to calculate cosine similarities between the estimated signal directions and the respective ground-truth. The results of this experiment are depicted in Fig. (in blue and green). Since we use the ground-truth signal values, this experiment evaluates the efficacy of signal vectors under perfect input conditions. The figure showcases the effectiveness of our proposed sub-sampling procedure when using ( in all cases, signal vectors demonstrate perfect alignment with the ground-truth directions, while in the absence of sub-sampling, the estimation is less reliable. Conclusively, the sub-sampling technique that we proposed in Section 4.3 adapts more precisely to our extended data model.
6.2 Evaluation of the Encoding -Dcciding Direction Pairs
In this experiment, signal vectors are jointly learned with the concept detectors, employing every proposed facet of the method presented in Section 4 Concept detectors are evaluated for their ability to detect each one of the ground-truth concepts, while signal vectors are evaluated for their alignment with the ground-truth concept encoding directions.
Decoding Dircctions (Concept Detectors): Using the synthotic pixel datasct {zp, } f ) we assess the ability of each learned concept detector to detect each of the mround-truth concepts. Since the method is unsupervised, the name of the concept that each detector identifies is unknown. For this reason, we need
14

---

Published in Transactions on Machine Learning Research (02/2020)
1.0
0.8
Conine Similarity
0.6
0.4
0.2
0.0
Coocept
Signal Estimator
Kindemnans
EDDP /wo gloe
Signal-
Figure 8: 1) Cosine similarity between the ground-truth concept encoding directions and their estimation via Kinder- mans et al 1 calculated without our proposed feature sub-sampling, In this experiment, the estimation of signal directions uses ground-truth signal values. 2) Cosine similarity between the ground-truth concept encoding directions and their estimation via Signal-Vectors, i.e. using ① Similarly to the previous experiment this one uses ground-truth signal values. Under ideal input conditions, Hignal-Vectons are proven to be faithful signal estimators. 3) Cosine similarity between ground-truth concept en- coding directions and their estimation via our proposed Prroding-Decoddig Direction Pairs (EDDP). constrast. to the previous two experiments, in this one Signal Vectors learned in an unsupervised manner using all the aspects of the proposed method. Without access to ground-truth signal values, our approach is capable of recovering the ground-truth concept encoding directions. 4) Cosine similarity between the ground-truth concept encoding di- rections and their estimation via our proposed Seroding-Decodding Direction Pairs (EDDP) but without using the Filter-Signal Orthogonality Loss fe0. In that case, the encoding direction estimation is inferior compared to (3).
calculated with our proposed feature sub-sampling,
to implement direction labeling (Section 3.4p. In Table 2 we present the Intersection over Union scores for each detector against actual concept classes. Zero values indicate complete purity and no mixing of the concepts, while scores of 1.0 indicate perfect concept detection. As an additional step, we also examine how well the learned filters extract signal values from representations. Since distractors and the latent space bias are not directly estimated by our method, we can only quantify the extracted signal values as deviations from the dataset's average value (See also Section 5.1
29 and Section A.9). The Root Mean Squared Error (RMSE) between these extracted values and the ground truth, after subtracting the mean signal value, is noted as 0.06. We also verified that in this experiment, where the distractor components are independent of the concept content contained in the features, the learned filters were orthogonal all the vectors in the distractor basis D, without explicitly enforcing this occur.
Encoding Directions: Fig. orange indicates perfect alignment between the learned signal vectors and the ground-truth concept encoding directions. We stress the fact that, in this experiment, signal vectors are driven by signal values extracted by the concept detectors, and this is different from Section 6.1 where signal vectors were tested under ideal input conditions.
6.3 Ablation Study
We also consider case study where we learn the direction pairs by omitting [ from the objective function, ie., learning the direction pairs without (/wo) it. In that case, the concept detectors still discovered the ground-truth directional clusters, in Table 2 However, the decoding directions were less reliable in extracting the ground-truth signal values. The RMSE between the extracted values and the ground truth, after subtracting the mean signal value, was found to be 0.31, which is substantially inferior to the standard case discussed above. The latter has an impact on the fidelity of the signal vectors. Fig. purple visualizes the cosine similarity between the learned signal vectors and the ground-truth encoding directions for this case. The experiment shows that the signal direction estimation is less effective compared to when fo is taken into account.
15

---

Published in Transactions on Machine Learning Research (02/2026)
6.4 Evaluation against other Baselines
The ground-truth encoding directions of this example (Table 15 in Section A.8 contain both positive and negative components, and the relationship among the encoding directions (and the distractors) is, in general, not orthogonal. In theory, and without the need for practical experiments, this example cannot be addressed by NMF, K-Means, or PCA. NMF would produce a signal basis of non-negative components. Similarly, the cluster centers of K-Means would point toward the positive quartile where the centroids reside. Finally, the non-orthogonal nature of the ground truth encoding directions implics that the PCA's solution space is insufficient.
7 Experiments on Deep Image Classifiers
In this Section, we apply our method in real-world scenarios. In the following sections, we a) assess the laithfulness of the learned encoding-decoding direction pairs, b) provide qualitative segmentation results obtained by utilizing our learned concept. detectors, e) compare with unsupervised baselines in interpretability and influence terms, d) conduct an ablation study to quantify the contribution of each proposed component, e) compare with supervised direction lcarning, aiming to provide more insights on the interpretability and influence of the directions, provide application use cases in which the learned direction pairs assist in providing global, local, and counterfactual model explanations, g) provide an example application in model correction.
7.1 Direction Learning and Evaluation Protocol
We apply our Cncoding-Decodinng Direction Pairs (EDDP) on the last convolution layer of five different
networks: ResNet18 (He et al. 2016
trained on Places365 [Zhou et al.) 2017), ResNet50 (He et al.
2016
trained on Moments in Time (MiT) (Monfort et al. 2019). as well as EfficientNet (bO) (Tan & Le.
2019)
Inception-v3 (Szegedy et al. 2016), and VGG16 (Simonyan & Zisserman 2014 trained on ImageNet (Deng
et al.
2009).
For all the networks that we study, when learning the direction pairs, we use the Broden Bau
eet al.
2017 concept dataset, except for ResNet50, for which we use Broden-Action RRmakkishhan et al,
(2019h.
The concept datascts fcature dense pixel annotations; Broden includes 1197 concepts across 63K
images in 5 concept calegories (object, part, material, tenture, color), while Broden-Action incorporates an additional action category with 210 labels and 23K more images. We emphasize that when learning the direction pairs, we do not make use of the annotations that complement the concept datasets.
For baselines, we consider directional clusterings based on either PCA, NMF, or the Natural latent space basis (referred to as Natural). For our EDDP, we examine two variants that are differentiated based on the choice of feature manipulation strategy used in Uncertainty Region Alignment. We use EDDP-U to refer to UFM and EDDP-C to CFM. Unless stated otherwise, our Encoding Decoding Direction Pairs use the Augmented Lagrangian Loss from Section 4.5 all losses from Sections 4.2 and 3.3 (except C*), along with
either Luur (EDDP-U) Leur and 5000 (EDDP-C) from Sections 4.3 and 4.4
After learning the directions with any unsupervised method, we use Network Dissection (Buu et. al. 2017
and the annotations available in the concept datasets to label them. As described in Section 3.4 we make
a basis change by projecting the network fcature activations onto the learned decoding directions (for PCA and EDPP) or by directly utilizing the concept factors learned by NMF. More details on the experimental setup for all methods, including baselines, are provided in Section A.13 while in Section A.10 we discuss how our evaluation pipeline implicitly captures human judgment.
7.2 Faithfulness Assessment of the Encoding-Decoding Direction Pairs
In this Section, we aim to evaluate the faithfulness of the encoding-decodding direction pairs, i.e. to assess whether the signal value extracted by the decoding direction is indeed encoded in the direction of the signal vector. Although this was validated in synthotic data (Section 6), this paragraph, we make the same evaluation in the case of real data, in the absence of ground truth. Based on activation maximization (Nguyen et al. 2019), propose the following novel approach to address this evaluation.
16

---

Published in Transactions on Machine Learning Research (02/2020)
Dream Image #1
Dream Image +2
Dream Immgge #3
Dream Image #4
Line fit (point direction)
patch embeddings
Figure Our approach to assess faithfulness of the direction pair: Starting from an image with the concept, use deep dream to maximize the concept's decoding direction and collect the patch embeddings from the dream images at the regions with the concept. Subsequently, we fit a parametric line to those features in order to estimate the direction towards which they moving when "dreaming". We term this direction the dreaming direction.
Resnet18/ Places365 skyscraper
Efficiantnet/ / Imaganet tymonitor
Figure 10: Dreaming pre-images from the process of estimating the dreaming directions. Left: Maximizing concept. detector learned for the latent space of ResNet18 and labeled with the concept name skyscruper. Right: Maximizing concept detector learned for the latent space EfficientNet and labeled with the concept name tumonitor. Both images regard directions learned with EDDP-C.
Suppose that we evaluate the faithfulness of a particular dircction pair w, and let denote the respective concept detector's bias. We use the concept detector (w, b) to collect set of images containing the concept. Subsequently, with these images pre-images (Mahendran & Vedaldi 2015 2016), that is, images that maximize the decoding direction at the spatial locations in which the concept was initially found. In this manner, we start from an image containing the concept and ask Deep Dream to amplify (in the input pixel space) whatever concept. the detector can detect. We keep running the activation maximization loop for iterations and keep a record of how the features (at the locations with the concept) evolve while optimizing. Finally, we use all the recorded features collected during the "dreaming" optimization to fit a parametric line and estimate the direction that best describes feature evolvement. while dreaming. We call the direction of this line the dreaming direction (Fig, 9h. We conclude by comparing the dreaming direction of the concept for each image with the respective signal vector in terms of cosine similarity.
initialization point, use Deep Dream (Olah et al. 2017 to compute
In Figures and we provide histograms of these cosine similarities when applying our method to ResNet18 and Nefi all cases, even those without considering L approximately 90% of the signal vectors have cosine similarity with the dreaming directions above 0.7, indicating sufficient faithfulness. The effect of f is push the distribution become more Gaussian. To validate the effectiveness of the sub-sampling strategy proposed in Section 4.3. also provide histograms of cosine similarities between signal vectors and dreaming directions in the absence of sub-sampling. The results are depicted in Fig. 13 where, in that case, the deviation of the signal vectors from the dreaming direction is evident. Example dreaming pre-images for two cases are provided in Fig. and further details and experiments may be found in Sections and
7.3 Interpretability Evaluation of Concept Detectors: Qualitative Segmentation Results and Interpretability Statistics
Figure [4 presents qualitative segmentation results obtained using our concept detectors, confirming that they are largely monosemantic. These visualizations rely on Network Disscction to generate segmentation maps and calculate the Intersection over Union (IoU) scores, which represent the detector's performance across the entire concept dataset's validation split. A detector qualifies as interpretable if its ToU score
17

---

Published in Transactions on Machine Learning Research (02/2026)
ResNett0 0 Places3ice
Ltficentheel Imagenet
Histogram Cosine Simlantios Between the Dresming Direction and the Signal Vector
Histogram Cosine Smilanbss Batwron the Dreeming Directon and the Signal Vector
Pooaet (10)
Poweru (N))
20
5-
0.55
0.50
0.85
0.70
0.75
0.80
088
0.90
0.88
0.3
04
0.5
0.6
0.7
0.8
09
Cooine Himilarity
Cosine Bimilarity
Figure 11: Cosine similarity histogram between dreaming directions and signal vectors. Left: Directions learned for ResNet18 trained Places365 (/ 448). Right: Directions learned for EfficientNet trained ImageNet (I = 1120). These histograms regard EDDP-C, ie. directions learned with Leu and L
ResNett0 Places3C6
EtficeetNee- Imagenet
Histogram Cosine Siiltanios Betweon the Dresming Direction and the Signal Vector
Histogram Coaine Simlantiis Batwoen the Dreaming Directin and the Signal Vactor
30
Ponantt (1g)
Poocest (h1)
20
20
0-
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0. 0.5
0.E
077
0.8
0.9
Cooine Similarity
Cusine Similarity
Figure 12: Cosine similarity histogram between dreaming directions and signal vectors. Left: Directions learned
for ResNet18 trained Places365 (I 448). Right: Directions learned for EfficientNet trained ImageNet
(I =I=1l 1120). These histograms regard EDDP-U, ie. directions learned with guer but without L
surpasses the predefined threshold 0.04. We provide detailed quantitative interpretability statistics for each network architecture, exemplified by Table for ResNet18 and other Tables in Section A.117 More extensive qualitative comparisons and statistics against unsupervised baselines are provided in the same Scction of the Appendix.
Table 3: Network Dissection statistics for ResNet18 trained on Places365. For each concept category in Broden, we report two numbers: First, the number of concept detectors that were labeled with the name of a concept belonging to the category and second, the number unique concept labels from the category that have been assigned the of the concept detectors. The table summarizes statistics for unsupervised methods and for different values of the cluster count hyper- paremeeer 1.
18
Places365
1 Method
Color
Object
Part
Material
Scene
Texture
Total
PCA
0/0
13
5 2
28
16
117
32
173
NMF
0/0
91 42
7 7/4
2 2
189
101
17
26
339
175
EDDP-U
2
118
/ 54
8 8/7
3 /
151
114
21
/ 17
303
197
EDDP-C
2/2
117 51
9 9/0 G
4 4
153
114
22
/ 17
307
194
PCA
0/0
22
5 5/2
1 1
28
/
16
127
33
183 65
0/0
95 46
9 9/7
1
234
120
33
372
194
448
EDDP-U
2/2
137
5/
193
/
364
222
EDDP-C
2/2
136
5/5
5
194
364
223
Natural
0/0
11
1/
269
26 17
414 204
PCA
0/0
15
32
180
512
EDDP-U
1/1
127
236
264
EDDP-C
1/1
236
269
As an exemplary interpretation of the report, Table shows that when learning direction pairs for ResNet18 with 1 512, Network Dissection reported that EDDP-C can identify 269 clffferent. concepts in the following
18

---

Published in Transactions on Machine Learning Research (02/2026)
ResNett0 0 Places3ice
UthceetNee- Imagenst Histogram Cosina Siiiartiess Betwoon the Dresming Direction and the Signal Vector
Histogram Cosine Simlanlies Between the Dresming Direction and the Signal Vector
40
(16)
Poweeu (h1
80
-0.4
-0.2
0.0
0.2
0.4
0.6
-0.50
-046
-0.40
-0.35
+..00
-025
-0.20
+0.15
Cooine Himilarity
Gosine Simikiit
Figure 13: Cosine similarity histogram between dreaming directions and signal vectors learned without sub-
sampling. Left: Directions learned for ResNet18 trained Places365 (I 448). Right: Directions learned for EfficientNet trained on ImageNet (I = 1120). These histograms regard EDDP-U, i... Guer but without L
directions learned with
ResNet18 Places365
Classifier Index: 287
water
Classifiar Index: 121
tree
loU: 0.39
EfficientNet ImageNet
Classifior Index: 584
airplane
loU: 0.25
Classifier indax: 536
bed
loU: 0.39
Inception-v3 ImageNet
motorbike
train
VGG16 / ImageNet
Classifier Index: 114
bus
lal:: 0.08
Classiler Inces:
street
loU: 0.2
ResNet50 / MiT
typing
gambling
Clcsoifior Indeac 29
0.1:
Classifior Indox: 1718
loU: 0.26
Figure 14: Qualitative segmentations using the concept detectors learned with our method as reported by Network Dissection. Rows correspond concept detectors learned for different networks. For ResNet18 and EfficientNet the results were obtained using CFM while for Inception-v3, VGG16 and Resnet50 using UFM.
categories: 66 objects, 165 scenes, parts, 4 materials, 22 textures, and 1 color. The total number of interpretable concept detectors 407. This preliminary statistical report, together with the visualizations, is provided in order to offer intuition regarding the interpretability of the concept detectors. A rigorous evaluation with comparisons is conducted in the Sections that follow.
19

---

Published in Transactions on Machine Learning Research (02/2026)
7.4 Metrics
7.4.1 Evaluation of Clustering Quality via Cluster Statistics
We report two statistics for dircctional clustering. Coverage is the percentage of patches in the concept dataset that receive at least one positive prediction from the concept detectors. Equivalently, it is the percentage of pixels assigned average number of detectors that classify each patch positively, Equivalently, it the average number of clusters to which each patch belongs.
least one cluster or explained by the clustering. Redundancy is the
7.4.2 Evaluation of the Decoding Directions: Interpretability
Classification Let T/ denote the binary classification task of prodicting whether a patch belongs to concept Given the concept label I, which was assigned to a concept delector with direction labeling, we assess the detector's performance on Ti using the standard binary classification metrics of Precision, Recall, FI, and Average Precision (AP). For each metric, report average and standard deviation scores by aggregating across all detectors in the learned set.
Segmentation We additionally consider the performance of concept detectors in the binary semantic seg- mentation task and employ two metrics from Doumanoglou et al. (2023). Specifically, let (;(c, (C) the Intersection Over Union for concept detector in identifying concept c within the dataset K. Define of = argmax, (c, Kt), indicating the concept label detected best by concept detector within the training subset of the dataset (Kt). With Kr as the validation subset, we use the following interpretability scores S and S²:
5 = [
Σ
1 1. (c:(c)
(14)
1=0
8² / I(t (c K.)) 2 El
(15)
The first metric S counts concept detectors with an IoU performance that exceeds score threshold g. The second metric S² uses the cardinality of the set |-| to count the unique concept labels detected by the concept detectors with loU above ş. Both metrics become threshold-agnosticc, by integrating over all E [0, 1]. Viewing it from different angle, S pertains to the segmentation elficiency of each concept detector individually, whereas S2 concerns the clustering diversity, meaning the capability of the detector ensemble to recognize wide array of distinct concepts. Finally, we also report the standard mean Interscction over Union (mIoU) score, which is the average loU performance score aggregated across all the detectors in the set.
Monoscmanticity Even though Precision is a natural classification motric that can capture the monose- manticity of the clustering, it has the drawback of relying on the explicit labels that are available in the concept dataset. To overcome this labeling limitation, and inspired by Dreyer al. (2024b), we additionally quantify the monosemanticity of the concepts identified by the concept detectors by utilizing the embedding space of CLIP ViT/B-16 (Radford et al. 2021). In particular, for each concept detector, we pick images from the validation split of the concept dataset for which the detector is most confident about the presence of the concept. We select 100 unique images from the top confident predictions. Subsequently, we crop rectangle around the area of the concept and obtain the CLIP embedding for this crop. Let kj and ki denote image indices in the set of selected images, with kj, kj. E {0, 19. and i denoting the i-th concept detector. We use the following monosemanticity metric, which measures the average distance between CLIP emboddings (eng egg) within cluster, aggregated over all clusters:
M = E
[llex eng [a2]
(16)
20

---

Published in Transactions on Machine Learning Research (02/2026)
Table Comparative Analysis of our Encoding-Decoding Direction Pairs (EDDP) against unsupervised baselines. This table summarizes metrics for ResNet18 trained on Places363. EDDP-U stands for using Unconstrained Feature Manipulation and EDDP-C for using Constrained Peature Manipulation.
N8 Places365
Mothod
Coverage
Redundancy
Precision
Rocall
FT
AP
M
8
S
mloU
PCA
0.72
1.27
0,2620.13
0.14:0.11
0.1810.11
0.1710.111
7.47
16.57
5.6
0.04
0.73
NMP
0.21
0.3140.17
29.56
0.08
0.68
384
EDDP-U
0.86
1.29
0.82+0.21
0.23+0.16
0.33+0.18
0.53+-0.19
6.61
41.73
28.01
0.12
0.58
EDDP-C
0.86
1.32
0.81+0.21
0.23=0.16
0.33+0.17
0.5++.19
6.67
41.5
27.49
0.12
0.61
PCA
0.75
8.89
0.25=0.12
0.14=0.11
0.17#0.11
0.16=0.1
7.48
18,06
5.62
0.01
0.73
448
NMF
0.51
1.79
0,67=0.18
0.21=0.14
0.3110.17
0.43:00188
6,96
31.77
18.16
0.07
0.09
E
1.32
$2.07
0.11
0.59
EDDP-C C
0.86
1.33
0.82.10.2
0.2111.16
0.3110.18
0.4910019
6.72
46.3
31.34
0.11
0.61
Natural
0.19
1.79
0.6920.18
0,220.13
0,3=0.16
0.13#0.11
7,02
34,61
18.74
0.07
0.69
PCA
0.77
10.05
0.2520.12
0.14:1011
0.1710.11
0.16-0.1
7.5
19.41
5.62
0.04
0.74
512
EDDP-U
0.86
1.28
0.8210,22
0.21
0.3110.19
0.5110.21
6.66
52.51
37.78
0.11
0.61
EDDP.C
0.240.16
0.29+4019
0.47-0.2
50.29
36.99
0.1
0.63
7.4.3
Evaluation of the Encoding Directions: Influence
We use RCAV Pfau al. (2020) assess the ability of our method to identify concepts influential model predictions and relate concept influence to the network's concept sensitivity. In the ablation studies, two metrics summarize the results: Significant Direction Count (SDC) and Significant
Pairs (SCDP). SDC represents the number of signal vectors that siguificantly influence al least one model class, while SCDP tallies diiss-iiretoonnn pairs where the signal vectors significantly affect the class. Let Sc,, denote the sensitivity the model to concept when predicting class k. When comparing against unsupervised baselines, we use the following influence metric which is defined as:
I Ei, [SCA.l]]]
(17)
i.., the average sensitivity of the model across all concepts and classes.
7.5 Interpretability and Influence Comparison with Established Unsupervised Baselines
The previous unsupervised state-of-the-art (Graziani et al. 2023a b Zhang et al. 2021 Fel et al. 2023b)
has demonstratod offectiveness in revealing interpretable directions in the latent space of deop networks. However, as outlined in Section AA13 these approaches employed a class-conditioned protocol, lying their analyses to specific class labels. In this work, we instead operate under class-agnostic protocol, which seeks concepts across the entire latent space, independent of class. This fundamental difference in the approaches renders direct performance comparison with previous implementations inapplicable. Therefore, we utilize PCA and NMF, the core matrix decompositee methods upon which these prior works were built, as our principal baselines for a fair, classical comparison. We also move beyond subjective human ratings typically used for evaluation and instead develop a rigorous, best-effort quantitative framework to compare interpretability and influence with these baselines. with a more detailed discussion on how our pipeline captures human judgment provided in Scction A.10
also compare with the methods extend (Doumanoglon al. 2023 different protocol for fair comparison. For each network, consider three sizes for the clustering, namely I = 3/4D, = 7/8D, and I to the natural latent space basis, i.e., the neuron directions. Additionally, when studying Efficient we disregard NMF since this network uses the SiLU activation function (Ramachandran et al. 2017). which allows negative feature activations, rendering NMF non-applicable. Tables 4 comparative analysis for ResNet18, EfficientNet, Inception-v3, VGG16, and Net50, respectively. In the following subsections, we provide detailed discussion of these tables.
In Section A.16
2024 but under
= D. It is worth noting that when / = D. NMF trivially resolves
and summarize the
21

---

Published in Transactions on Machine Learning Research (02/2026)
7.5.1 Case Study: ResNet18 / Places365
Clustering Quality: Regarding dataset coverage, both our EDDP variants explain 86% of the dataset regardless of cluster count. Specifically, this means that 86% of the patch embeddings in the concept dataset have been assigned to at least one cluster. This number is substantially higher than the coverage of the clustering obtained via NMF (which is at most 57%) or the natural latent space basis (49%) and still significantly higher than the coverage of the clustcring obtained via. PCA (at most 77%). As far redundancy is concerned, the EDDP variants minimize multiple cluster assiguments to individual patches, as they attain the lowest redundancy scores (at most 1.33 cluster assignments per patch). While NMF and the natural basis rank 3rd in rodundancy terms with more than 1.77 assignments per patch, PCA is ranked last with the same number exceeding 7.27.
Interpretability: In classification metrics, the clustering of EDDP-U is the most interpretable of all, scoring higher than alternative clusterings, especially in Precision and AP terms. In most cases, clustering with EDDP-C attains scores close to clustering with EDDP-U. Thus, EDDP-C is ranked 2nd in all classification metries except in the case of F1 score and [ = 512, in which the natural basis is slightly better by a score point of 0.01. With only the latter exception, in all classification metrics, NMF and the natural basis are ranked 3rd, while PCA ranked last.
The high precision of our concept detectors (approximately 0.82 on average) indicates that the learned directions are highly monosemantic. The latter is additionally confirmed by the monosemanticity metric based on CLIP embedding distances, in which EDDP variants are attributed the lowest M score. Comparing with the rest of the approaches, all of the clusterings obtained by NMF, the natural basis, and PCA are less monosemantic, often by a large margin (0.67 precision for NMF, 0.26 precision for PCA and 0.69 for the natural basis).
Similar conclusions can be drawn from the segmentation metrics, in which the concept detectors of EDDP score substantially higher than the previous approaches. For example, when 448, EDDP-C achieves 46.3 score points in S surpassing NMF's 31.77, and reaches 0.11 mIoU points in contrast to NMF's 0.07. Finally, the EDDP variants also achieve the most diverse clustering, covering variety of visual concepts, justified by the substantially higher values of S compared to the rest of the approaches. Furthermore, this diversity is consistently improved with the increase in cluster count, a phenomenon that is less prominent for the rest of the baselines.
Influence: To rank the methods based on the network's s sensitivity to the identified concepts (metric Z²), starting from the most influential clustering, we would list PCA first, then NMF, followed by the Natural basis, and finally EDDP-C and EDDP-U. In this case, it appears that there is clear anti-correlated rela- tionship between interpretability and influence, with the most interpretable clustering being less influential the overall model outcomes. We will be discussing this further al the end of the section.
7.5.2 Case Study: EfficientNet / ImageNet
Table 5: Comparative Analysis of our SeoodinngDeoodingg Direction Pairs (EDDP) against unsupervised baselines. This table summarizes metrics for EfficientNet (bO) trained on ImageNet. EDDP-U stands for using Unconstrained Feature Manipulation and EDDP-C for using Constrained Feature Manipulation.
Efficient ImageNet.
Method Coverage
Redundancy
Precision
Recall
Fi
AP f
L
S
S f
mloU 1
I f
PCA
0.63
19.29
0.29=0.08
0.13=0.07
0.1820.07
0.1940.07
7.41
35.26
2.62
0.04
0.95
960
EDDP-U
0.85
1.51
0.75=0.22
0.210.17
0.27:0.17
0.3810.16
6.74
27.92
14.71
0.03
0.91
EDDP-C
0.85
1.48
0.72=0.23
0.2:10.18
0.2710.17
0.3810.17
6.75
25.92
14.45
0.03
0.94
PCA
0.67
21.7
0.28=0.07
0.1320.07
0.1740.07
0.1940.07
7.41
42.33
2.7
0.04
0.95
1120
EDDP-U
0.85
1.61
0.76=0.21
0.210.17
0.28=0.17
0.3910.17
6.73
31.36
16.54
0.03
0.94
EDDP-C
0.85
1.54
0.74=0.22
0.2+0..18
0.28-+0.19
0.39+-0.17
6.74
28.62
16.38
0.03
0.94
Natural
0.51
5.44
0.5610.13
0.14.10.07
0.21
0.2910.12
7.35
35.94
7.09
0.03
0.95
PCA
0.68
26.71
0.28- 0.08
0.13+00.7
0.174-0.07
0.18-+0.07
7.44
48.01
3.12
0.04
0.95
1280
EDDP-U
0.85
1.65
0.74-0.21
0.19 16
0.27+0.11
0.36++0.17
6.81
40.38
19.12
0.03
0.94
EDDP-C
0.85
1.6
0.73=0.21
0.19=0.17
0.26+0.17
0.36++.17
6.88
36.95
18.98
0.03
0.94
22

---

Published in Transactions on Machine Learning Research (02/2026)
Clustering Quality: Clusters created by the EDDP variants attain Coverage score of 85%, while the best values for PCA and the natural basis in the same metric are 68% and 51%, respectively. In Redundancy terms, EDDP achieves scores less than 1.65, while the minimum scores for PCA and the natural basis are 19.29 and 5.44, respectively.
Interpretability: In terms of classification metrics, the directional clusterings obtained via EDDP-U and EDDP-C are approximately equally interpretable and notably more interpretable than the directional clus- tering of PCA. For instance, when I = 1120, the mean Precision of PCA's concept detectors is 0.28, the mean Recall 0.13, the mean F1 score 0.17, while the mean AP is equal to 0.19. For EDDP-U, the mean scores for the same metrics are 0.76 for Precision, 0.2 for Recall, 0.28 for F1 score, and 0.39 for AP. Similar conclusions about the improved interpretability of EDDP can be drawn when comparing with the natural latent space basis in the case of I = 1280.
Regarding monosemanticity, the concept detectors of EDDP attain a mcan Precision score which is better than 0.72, in contrast to the ones of PCA, which altain mean Precision scores below 0.29. The same gap between PCA and EDDP is similarly reflected in the M score. For example, when I = 1120, the M score of EDDP-C is 6.74 while the score of PCA is 7.44. EDDP also demonstrates substantial improvement in monosemanticity compared to the natural basis. EDDP-C attains mean Precision score of 0.73, while the score of the natural basis for the same metric is only 0.56. Additionally, EDDP-C attains 6.88 points in terms of M, which is significantly lower than the 7.35 points of the natural basis.
In the context of segmentation, when = 1120, PCA's concept detectors exhibit improved performance concerning 81, exceeding EDDP-U by roughly 12 points. In mIoU terms, PCA still scores higher (0.04), but EDDP follows closcly (0.03). Yet, PCA exhibits narrow diversity in the clustcring, discovering substantially fewer concepts than EDDP, as reflected in 8 in which EDDP-U surpasses PCA by up to approximately 14 points. When considering the natural basis as an alternative directional clustering, EDDP scores better in terms of S and 8 and equally well in mloU.
With respect to the impact of the learned directions on the network predictions, EfficientNel shows a significant sensitivity to the direction set determined by PCA or the neuron directions from the natural basis, with I score of 0.95. Even though the model is less influenced by the directions learned with EDDP, this is only by a small margin, the I' scores for EDDP variants are 0.94.
Influence:
7.5.3 Case Study: Inception-v3 / ImageNet
Comparative Analysis of our Encoding- Mhcoding-Decoding Direction Pairs (EDDP) against unsupervised baselines. This table summarizes metrics for Inception-v3 trained on ImageNet. EDDP-U stands for using Unconstrained Peature Manipulation and EDDP-C for using Constrained Feature Manipulation.
Table 6:
Inception-v3 ImageNet.
Method
Coverage
Redundancy
Precision 1
Rocall 1
F1
AP
M 4
81
8²
minUi
PCA
0.7
32.29
0.26-0.12
0.121001
0,1610.1
0.1720.1
7.39
54.5
4.19
0.04
0.95
NMF
0.7
9.26
0.53.0..7
0.16.10.12
0.24.10114
0.2810.16
7.24
70.68
14.92
0.05
0.95
1536
EDDP-U
0.87
0.84-0.18
0.28-0.24
0.38+0.26
0.534-0.26
6.79
156.46
28.03
0.11
0.93
EDDP-C
0.85
1.9
0.86=0.19
0.23+0.16
0.33++0.19
0.58++0.24
6.87
125.64
27.46
0.08
0.94
PCA
0.68
33.26
0,28=0.12
0.12#00.1
0,17+0.1
0,17=0.1
7.39
59.18
4.1
0.03
0.95
NMF
0.61
7.71
0.6:0.188
0.16220113
0.25:10166 1.1
0,3240.17
7.29
72.86
15.86
0.01
0.95
1792
EDDP-U
0.9
2.96
0.86=0.16
0.33:10.25
0.43:10.26
0.5810.25
6.77
210.44
25.77
0.13
0.93
EDDP-C
0.86
1.16
0.24.0.116
0.36.10.19
0.621
6.92
174.0
24.69
0.1
0.94
Natural
0.63
10.29
0.58=0.17
0.1720.13
0.25::0115
0,31+00117
7.35
81.62
14.27
0.01
0.95
PCA
0.71
42.35
0.26=0.12
0.124:0.1
0,1610.1
0.1720.1
7.38
69.8T
4.24
0,03
0.95
2048
EDDP-U
0.86
2.38
0..5
0.31.1..29
0.4.0.29
0.54
6.95
170.81
32.13
0.09
0.94
EDDP-C
0.93
3.98
0.75
0.16-10.12
0.24+10.14
0.38-40.16
7.09
92.67
31.18
0.05
0.94
Clustering Quality: On par with the experiments on the previous architectures, EDDP variants explain more than the 85% of the concept dataset, as noted by the respective Coverage scores. This score still significantly higher than the ones attained from NMF (up to 70%), the natural basis (63%) or PCA (up to 71%). As far as Redundancy is concerned, on one hand, EDDP does not exceed 3.98 cluster assignments per patch. On the other hand, NMF assigns a patch to more than 7.71 clusters, while for PCA this number
23

---

Published in Transactions on Machine Learning Research (02/2026)
exceeds 32.29. Finally, in the clustering obtained by the use of the natural basis, cach patch is assigned to 10.29 clusters.
Interpretability: For = 1536 and / = 1792, the concept detectors of the EDDP variants score substan- tially higher in the classification metrics than the rest of the approaches. For example, when = 1792, EDDP-C attains a Precision score of 0.89, whereas the same score for NMF 0.6 and for PCA 0.28. For the same 1, EDDP-U attains an F1 score of 0.43, which is a notable improvement over NMF's 0.25 and PCA's 0.17. Finally, in terms of AP, EDDP-C achieves score of 0.62 points, which is significantly more competitive than the score of NMF (0.32) and PCA (0.17). For I 2048, concept detectors from EDDP-U consistently score higher in the classification metrics than any other approach. Unlike what we have seen in the rest of the experiments, in that case, EDDP-C did not follow EDDP-U closely in the classification metrics. Instead, it is clearly ranked 2nd, surpassing PCA and the natural basis in Precision and AP while remaining comparable with the natural basis Recall and Fl score.
Regarding monosemantiiily,, EDDP-U attains Precision scores close to 0.84, while for the cases of 1536 and I 1792, EDDP-C exceeds this number, achieving scores up to 0.89 but attains lower score of 0.75 when / 2048. The best Precision score for NMF is 0.6 for / = 1792, while for PCA this score is 0.28, Finally, when clustering with the directions of the neurons, i.c. the natural basis, the Precision score is 0.58. When measuring monosemantieity with the CLIP embedding distances, the EDDP variants achieve a substantially lower score compared to the rest of the approaches. For instance, when / = 1792, the EDDP-U clustering attains a score of 6.77, EDDP-C 6.92, NMF 7.29 and PCA 7.39. Interestingly, unlike what we have seen when using the labeled dataset in terms of Precision, when I = 2048 EDDP-C and EDDP-U score similarly in M, with EDDP-U attaining score of 6.95 and EDDP-C 7.09.
As far as segmentation is concerned, the ranking of the methods is consistent across all metries: EDDP-U comes first, followed by EDDP-C, then NMF or the natural basis, and finally PCA. Beyond just ranking, the gap in the scores between EDDP and other approaches is large. For instance, when = 1536, the EDDP-U score terms of S is 156.46 when NMF's 70.68. Similarly, for the same case, EDDP-U's 8 is 28.03 when NMF's is 14.92, while for mIoU, EDDP-U's score is 0.11 when NMF's is 0.05. Influence: Regarding the sensitivity of the model with respect to the identified concepts, Inception-v3 is more sensitive to the PCA and NMF directions with I' score of 0.95. The network is slightly less sensitive to the concepts identified by EDDP-C and EDDP-U, but once again by small margin. EDDP-C attains a I score of 0.94 and EDDP-U a score in {0.93, 0.94].
7.5.4 Case Study: VGG16 ImageNet
Table 7: Comparative Analysis of our Eucoding-Decoding Decoding Direction Pairs (EDDP) against unsupervised baselines. This table summarizes metrics for VGG16 trained on ImageNet. EDDP-U stands for using Unconstrained Feature Manipulation and EDDP-C for using Constrained Feature Manipulation.
VGG16
ImageNet
Method Coverage
Redundancy
Precision
Recall 1
F
AP
M.G
S 1
S 1
mloU 1
I'
PCA
0.61
1.31
0.29=0.09
0.13:0007
0.191:007
0.16:0.07
7.85
22.41
2.98
0.06
0.87
NMF
0.55
1.6
0.640.16
0.1720.11
0.261:0133
0.3110.14
7.41
22.73
0.06
0.87
384
EDDP-U
0.82
1.17
0.6110.18
0.18:0.09
0,2710009
1.31
6.88
29.98
9.39
0.08
0.91
EDDP-C
0.79
1.07
1.181
0.16.10.09
0.24110.0
0.31
6.92
26.23
8.89
0.07
0.9
PCA
0.67
8.91
0.28:0.08
0.1520000
0.1940.07
0.16:00.6
7,86
25.8
3.12
0.06
0.87
448
NMF
0.5
1.44
0.62:0.16
0.16110.1
0.2510.122
0.31
7.44
24.82
9.86
0.06
0.87
EDDP-U
0.84
1.35
0.5910.18
0.26.10.09
0.3.10.11
34.67
10.29
0.08
0.91
EDDP-C
0.82
1.22
0.61+0.18
0.16+0008
0.24+0.00
0.340.11
6.93
30.91
9.26
0.07
0.9
Natural
0.58
2.12
0.58=0.16
0.16+0011
0.25++0.12
0.29+-0.12
7.5
30.61
10.28
0.06
0.87
PCA
0.69
9.62
0.29=0.08
0.15=20.0
0.1940.07
0.16:t0.06
7.87
30.25
3.51
0.06
0.86
512
EDDP-U
0.85
1.13
0.61=0.18
0.18=0.08
0.27:1009
0.31:0.11
6.95
41.2
11.11
0.08
0.91
EDDP-C
1.37
0.62:0.188
0.15220.09
0.2410.09
0.3:10.11
0.07
0.9
Clustering Quality: In terms of clustoring quality metrics, the EDDP variants attain higher Coverage and lower Redundancy scores than either one of NMF, PCA, or the natural basis, in a similar fashion as in the rest. of the architectures.
24

---

Published in Transactions on Machine Learning Research (02/2026)
Interpretability: Unlike the other architectures, in classification metrics, NMF and the EDDP variants perform on par. For example, when = 448, NMF attains Precision score of 0.62 exceeding EDDP-U's 0.59. In Recall and Fl score terms, EDDP-U achieves the highest scores of 0.18 and 0.26, respectively, while for NMF these numbers are 0.16 and 0.25. In terms of AP, NMF scores 0.31 while both EDDP variants allain a score of 0.30. In most cases, all of NMF, the natural basis, and EDDP substantially surpass the clustering of PCA the classification metrics.
Even though EDDP, NMF, and the natural latent space basis score similarly in terms of Precision, if monosemanticity is measured via the CLIP embedding distances, the EDDP variants achieve the best scores, often by a substantial margin. For instance, when - 384, NMF achieves an M score of 7.44, PCA a score of 7.85, while EDDP-U a score of 6.88.
Finally, in segmentation metrics, EDDP variants achieve the highest scores in S and mloU, while in S? EDDP-U is ranked 1st in two of the three cases (I = 448 and I 512) while ranked 2nd, after NMF in the third case (I 384).
Influence: Regarding the sensitivity of VGG16 with respect to the concepts, the network is most sensitive to the directions learned with EDDP-U (I) score of 0.91), followed by EDDP-C (I) score of 0.9), and finally, by NMF, the natural basis, or PCA, with I scores close to 0.87.
7.5.5 Case Study: ResNet50 / Moments Time
Comparative Analysis of our Encoding- Decoding Direction Pairs (EDDP) against unsupervised baselines. This table summarizes metrics for ResNet50 trained on Moments in Time (MrT). EDDP-U stands for using Uncon- strained Feature Manipulation and EDDP-C for using Constrained Feature Manipulation.
Table 8:
ResNet50
MIT
Method
Coverage f
Redundancy
Precision
Rocall
FLT
AP
M 1
8
1
mloU 1
PCA
0.7
30.77
0.1620.06
0.110:05
0.1210.05
0.110.04
7.34
58.51
0.88
NMF
0.54
9.07
0.46-0017
0.16:10.11
0.23
0.27
6.91
73.38
17.31
0.05
0.85
1536
EDDP-U
0.84
2.13
0.72=0.22
0.17:0.12
0.2610.15
0,3610.16
6.25
127.8
33.67
0.08
0.85
EDDP-C
0.88
2.37
0.15
0.1710.11
0.28.10.33
0.45.10.14
6.49
98.52
34.81
0.06
0.84
PCA
0.72
36.06
0.16=0.06
0.1:0.05
0.1240.05
0.1+0.04
7.35
65.82
3.58
0.01
0.88
NMF
0.42
0.37=0.15
0.1320.08
0,2310.1
0.310.11
6.95
77.02
0.85
1792
EDDP-U
0.85
2.47
0.78
0.1910.1
0.2911013
0.410..14
6.24
162.86
34.17
0.09
0.85
EDDP-C
0.88
2.62
0.76-0.16
0.1640.11
0.25
0.44
6.5
113.9
36.68
0.06
0.84
Natural
0.5
11.18
0.47=0.16
0.152000
0,2210.11
0,2620.12
6.97
90.35
16.63
0.01
0.85
PCA
0.76
43.2
0.15=0.06
0.1+0:05
0.1240.05
0.1+0.04
7.35
74.3
3.61
0.04
0.88
2048
EDDP-U
0.86
3.15
0.7210.2
0.21=0.13
0.31:00166
0.1310.11
6.29
209.2
33.01
0.85
EDDP-C
0.88
2.83
0.72=0.16
0.1520.11
0.2410.12
0.4310.13
6.61
126.46
35.98
0,06
0.81
Clustering Quality: Once again, in terms of clustering quality, the EDDP variants demonstrate both larger Coverage and lower Redundancy compared to the rest of the approaches, often with large margin. When I = 1792, the coverage score for EDDP-C is 0.88 while for NMF it is 0.42. The gap between the two is more than twice the score of NMF. Similarly, the redundancy score of EDDP-C is 2.62 whereas NMF's is 6.02, with a similar conclusion regarding the gap between the two.
Interpretability: The clustering obtained via EDDP is more interpretable in terms of classification metrics, especially Precision, AP, and sometimes the F1 score as well. Regardless of whether monosemanticity is measured via Precision or distances between CLIP embeddings, the EDDP variants score better, similar to the rest of the experiments.
When considering segmentation, both EDDP variants demonstrate consistent superiority in terms of S! and S² and mloU. Interestingly, in this experiment, EDDP-C scores the highest in S², indicating a clustering that captures a larger variety of concepts.
Influence: In terms of the concept sensitivity metric It, NMF, EDDP-C, EDDP-U and the natural basis perform on par, with an approximate score of 0.85. On the contrary, PCA achieves a score of 0.88, which puts it herther ahead of the competition.
25

---

Published in Transactions on Machine Learning Research (02/2026)
7.5.6 Discussion
From the previous experiments, we can draw the following, mostly straightforward conclusions: a) EDDP's directional clusters are monosemantic, often surpassing the unsupervised baselines by a significant margin. b) EDDP also excels in terms of F1 score, where EDDP-U is consistently ranked first among the competition, and EDDP-C is often ranked second. c) In terms of segmentation and specifically 8t, EDDP-U is consistently ranked first, except for EfficientNect, in which the clustering of PCA scores better, yet with substantially inferior performance in most of the other interpretability metries. In the same metric, EDDP-C is often ranked second. d) In terms of discovering diverse set of concepts (S²) the EDDP variants score the best, often by a large margin, excopt for one case in VGG16 where they score on par with the best mothod. c) In terms of influence, in all cases except VGG16, the EDDP variants are ranked last. However, in many cases, the respective score not far behind the competition.
In most cases, and in broader sense, observe that interpretability is somewhat negatively correlated with influence. Less interpretable directions may represent complex interactions (i.e. correspond to less pure human-understandably concepts) and, a result, they are more impactful on the network predictions, in the sense that they affect many output classes. On the contrary, more interpretable directions may have a more targeted impact on specific output classes (Gao et al. 2025), and thus score lower in the respective metric.
7.6
Ablation Study
7.6.1
Interpretability Losses
Table 9:
Ablation study with respect to the interpretability losses. For ResNet18 1 448, and for EfficientNet
1120.
Ortho
F gine
Coverage
Redundancy
Precision 1
Recall
FT
AP
M
S
8
mloti
ResNet18 et18
Places365
x
0.89
2.25
0.71
0.310.18
0.410.19
0.47.0..1
6.44
54.28
26.4
0.13
0.88
0.18+0.12
0.29+0.15
0.54
40.92
19.1
x
0.83
1.13
0.85+0.19
0.19+0.14
0.29-0.16
0.53+0.18
6.57
39.26
25.36
0.1
/
/
0.85
1.29
0.83=0.19
0.22+0.15
0.31=0.17
0.5240.18
6.64
47.14
29.0
0.11
Efficient/eet
ImageNet
x
0.9
3.36
0.6720.19
0.1720.14
0.2520014
0.3110.14
7.0
46.9
14.42
0.04
0.7810.23
0.23.10.21
0.310.2
0.3910.18
19.54
0.86
1.59
0.78+00.2
0.2:0.16
0.29
0.4
6.68
25.42
14.1
0.03
X
/
0.85
1.57
0.74+00.
0.22+0018
0.3+0.17
0.38+0.17
6.68
28.72
16.02
0.03
The motivation behind the loss terms that we introduced in Section 4.2 is to sustain or improve the inter- pretability of the clustering when moving from orthogonal decoding direction set to non-orthogonal one. In Table g we present interpretability metrics for the cases when we learn the directions with Lts both L and Leee, or without any of them. We also provide metrics for the case of orthogonal decoding directions (marked as Ortho). The latter is equivalent to learning with the method of Doumanoglou et al. (2023) using the Augmented Lagrangian loss of Section 4.5 We provide results regarding the directions learned for 18 and EfficienttNet. In these experiments, we exclude any form of Uncertainty Region Alignment.
In both network case studies, starting without interpretability losses and gradually adding them one after the other, observe that the different variations a) compare similarly in coverage and redundancy terms and b) achieve comparable performance in classification and monoscmanticity metrics. However, the impact of the loss terms on the segmentation metries is more siguificant, especially in terms of S and S². We emphasize the improvement 5 since this metric captures the variety of the identified concepts. Compared to the orthogonal direction set, dircctions learned without orthogonality constraints a) result less redundant clustering, b) the monosemanticity of the clustering in terms of Precision is improved, and c) the clustering becomes more diverse, capturing variety of different concepts.
26

---

Published in Transactions on Machine Learning Research (02/2026)
Table 10:
Ablation study with respect the Unconstrained Uncertainty Region Alignment loss. For ResNet18
I = 148, for EfficientNet 1120, and for ResNet50 1792.
pusr Coverage
Rodundaney
Precision
f
Recall
PTT
APT
8
ResNet18
Places365
0.89
2.25
0.7110.19
0.310.18
0.4
D.4710.19
6.44
54.28
26.4
0.13
0.88
2.2
0.71
0
0.41:0.119
0.48
6.52
54.56
26.06
0.13
0.85
T.29
0.83-76.19
0.22-0.13
10.1140.17
0.52+0788
6.64
47.14
20.0
0.11
1.32
0.83+0.19
0.22=0.16
0.32+0018
0.53+0.19
6.64
47.5
32.07
0.11
EfficientNett
ImageNat
0.9
3.36
0.07+0.19
0.17-0.14
0.25+0.14
0.3440.14
7.0
46.9
14.42
0.04
/
0.9
3.34
0.68+002
0.17=0.14
0.25#0.14
0.36+0.15
6.98
52.94
16.38
0.05
X
x
0.85
1.57
0.74=0.2
0.22=0.18
0.340.17
0.38+0.17
6.68
28.72
16.02
0.03
0.85
1.61
0.7620.21
0.240.17
0.2820017
0.3910.17
6.73
31.36
16.54
0.03
ResNet50
MiT
0.88
3.41
0.7250.13
0.1620.1
0.2.20.13
0.3
6.47
29.11
0.07
0.88
3.31
0.7320.16
0.1820.1
0.27=0.13
0.37:0.13
6.49
137.76
33.0
0.08
0.83
2.42
0.76466.9
0.15-0.12
0.2440.15
0.32+0.14
'6.1"
167.54
80.81
0.09
0.85
2.47
0.78.10.16
0.1910.1
0.29.10.3
0.410.14
6.24
162.86
34.17
0.09
Table 11: Ablation study with respect the Uncertainly Region Aligument losses. For ResNet18 I EfficientNet T = 1120, and for ResNet50 1 = 1792.
448, for
Dour
pear
Coverage
Redundancy
Precision
Recail 1
AP
M.
S"1
S
miol
SDC
SCDP T
ResNet18 Places365
1,29
0.8311.19
0.31
0.5200.18
6.64
29.0
0.86
1.32
0.83:1.19
0.22::0116
0.32=0.18
0.53
6.64
47.5
32.07
0.11
264
1565
0.86
1.33
0.8220.2
0,2140.16
0.3120.18
0.49:00199
6.72
16.3
31.34
0.11
304
1892
Efficieuteet
ImtgeNet
x X
0.85
1.57
0.7120.2
0.2210.18
0.310.17
0,3820.17
6.68
28.72
16.02
0.03
98
216
0.76.10.11
0,210.17
0.39-1.117
6.73
31.36
16.54
0.03
0.85
1.54
0.74+0022
0.240.18
0,28-0.19
0.39-0.17
6.74
28.62
16.38
0.03
673
1484
ResNet50
/ MFT
0.75::.119
0.1520.12
0.2120.15
0.320.111
157,54
370
0.85
2.47
0.78.10.16
0.1910.1
0.29-0.13
0.420.11
6.24
162.86
34.17
0.09
393
570
2.62
0.76:40.66
0.1640.11
0,25=0.13
0.44
6.5
113.9
36.68
0.06
1353
7.6.2 Uncertainty Region Alignment
In this section, study how Uncertainty Region Alignment may affect the interpretability of clustering and the discovery of concepts with significant influence. For this study, consider three nelwork architectures: ResNet18, Neficient and ResNet50.
In Table 10 we compare, in quality and interpretability terms, the clusterings obtained with and without the Unconstrained Uncertainly Region Aligument loss (Lwur). We consider learning the decoding directions both with and without orthogonality constraints (Ortho). Some simple conclusions that can be drawn from the experiments arc: a) Coverage may be improved when using it but Redundancy can be slightly worsened, b) the Precision and AP of the concept detectors are consistently improved by its use, c) it consistently improves the performance of concept detectors in terms of 81, and d) Lear can significantly improve the variety of concepts captured by the direction set (S² metric).
In Table we focus on comparing between the two Uncertainty Region Aligmment variants by additionally considering metrics of concept influence. Some key takeaways drawn from these experiments for using Ceur are: a) Coverage may be improved by its use, but Redundancy can be bit worsened, b) the concept detectors' interpretability in terms of classification metrics is slightly inferior but somewhat comparable to when using Luur, c) in two of the three cases (ResNet18 and Efficient the segmentation metrics across Uncertainty Region Alignment variants are mostly comparable, but in the third case (ResNet50) geur lcads to a more diverse clustering (improved 8²) but at the cost of a lower S score. d) In all cases, when comparing the two uncertainty region variants, the influence metrics favor [eur, In two of the three cases (EfficientNet and ResNet50) this improvement substantially higher than what can be achieved when using Luur For a more complete picture regarding the effect of L which is used in EDDP-C but not in EDDP-U, see also Section A.15
27

---

Published in Transactions on Machine Learning Research (02/2026)
Overall, by this ablation study, we could conclude that the use of L can improve the diversity of the clustering by capturing larger variety of concepts, compared to when not using it. Using few instead of Luur may perform on par or improve this diversity, while substantially increasing the influence of the learned signal vectors on the network predictions, in a statistically significant sense.
7.7 Insights on Interpretability and Influence in comparison to Supervised Direction Learning
In this section, we compare our method with supervised direction learning, aiming to draw insights regarding interpretability and influence of concept directions. For this reason, experiment with the last convolutional layer of Re Irained on Places365. We consider our method with the constrained feature manipulation strategy, and two supervised learning approaches: a.) IBD (Zhou et al. 2018) for supervised learning of
decoding directions (i.e. concept detectors) and b) Pattern-CAl CAVs (PCAVs) Pahde et al. 2024) for supervised
estimation of concept encoding directions. The supervised concept detectors and PCAVs are learned for the labels assigned to EDDP's concept detectors at the direction labeling phase (Section 3.4
Interpretability: In the interpretability analysis, we consider three variants for the proposed method: a) the exact outcome of our method, marked as EDDP, b) combining directions with shared label (post initial learning) using a binary linear classification layer which classifies representations positively if any detector with the same label does (marked EDDP Linear OR), and c) considcring the learned directions but optimizing the classification threshold in a supervised manner, in a post-learning step, lo enhance the Fl Score (marked as EDDP /w sup thres). This last approach assesses direction alignment to the concept by relaxing the sparsity constraint of the method. Table 12 summarizes metrics for this comparison. Results show that the classifiers of the proposed method achieve high Precision, comparable the supervised ones, but suffer from low recall. The latter could be related to the striet spansity objective of EDDP, as IBD exhibits significantly higher Redundancy. Combining classifiers with the same label improves recall, which indicates that different detectors capture different parts of the datasct, while supervised optimization of the classification bias further enhances F1 scores by relaxing spansity.
Influence: In the concept influence analysis, we aim to answer whether PCAVs, that correspond to di- rections of high interpretability due to learning them with supervision, are more influential to the network predictions than signal vectors. Since Network Dissection can assign identical labels multiple Encoding- Decoding Direction Pairs, direct comparison with Pattern-CAVs becomes less straightforward. We propose the following metric to help us draw some conclusions. Let E 10 N₂ 1} index EDDP's signal vectors sharing the same concept label I, and S represent the RCAV sensitivity score of the network relative to the j-th signal vector of label I when predicting class k. Similarly, let Sh,, denote the sensitivity score of the network relative to the Pattern-CAY for the same label and class. Inspired by RCAV, we regard signal vectors as moise vectors and compare the sensitivity of the network with respect to PCAVs and signal vectors that share the same label. We define a metric I which, when above 0.5, indicates that PCAVs have more influence than signal vectors on the network predictions at the statistical significance level of 0 = 0.05 with
Table 12: Comparison of EDDP (with CFM) with supervised direction learning Zhou et al. (2018). Comparing between: a) standard EDDP, b) combined concept detectors (Linear- (hhear-OR)+ c) EDDP with the thresholds of concept detectors learned with supervision in a post initial direction learning step (/w sup thres), and d) IBD: a set of classifiers learned in supervised way. The network here is 18sNet18 and I 512.
ResNet18 / Places365
Method
Coverage
Redundancy
Precision 1
Recall f
F1 f
AP f
S
S
mloU t
IRD Zhou al. 2018
0.95
3.08
0.84+0.13
0.6+0.17
0.68+0.15
0.7740..1
54.71
54.71
0.2
EDDP
0.86
1.28
0.81
0.240.16
0.25+0.19
0.47+02
50.29
36.99
0.1
EDDP Linear OR
N/A
N/A
0.734:0.5
0.3540.26
0.440.22
0.3310.21
30.77
30.77
0.11
EDDP /w sup thres
N/A
N/A
0.6:0.19
0.43=0.118
0.49=0018
0.474022
N/A
N/A
N/A
28

---

Published in Transactions on Machine Learning Research (02/2026)
Bonferroni correction:
I = = (pt, < 0 N7
1 PL, No Σ 1(1))al 2 1SPpal)
For RCAV 5.0, I is equal to 0.34, which is less than 0.5, indicating that the Pattern-CAVs are less influential on the network predictions than signal vectors. This outcome could also be linked to the discussion in Section 7.5.6 It appears that less interpretable directions (in this case, EDDP's) have more influence on the network predictions compared to more interpretable ones.
7.8 Application: Global Model Explanations via Concept Sensitivity Testing
Concept Influence Diagram for Places-365 Class: airfield
Concapt Influence Diagram for Places-365 Class: bathroom Network: Resmet18
Network: Resmet18
Concrpts Rostton inflrrce
Concagti with NegaDvo antluonde
Concepts with Profter I-fhunne
Concapts with negur vo tifluerd
gmenilll 1:0,2 29
street Inll-Ood2
carrosse L10-4
renforsnn mowe Int::0000
choownisst Inii-duke
oirgiane Tol 20:1
p20 0000-0000000
10:0000000399
1
Eitigg Mag.eetoiin 120TTpp-000000
marpla $2000.0
cou 1o
WYn www.werrnduaaaae..o.o... www.rrirndommr....00.0.0. 2.0.0:00000
tell tollo
mirplara 1:4:0.05
pool Luble Tobid000
living 24
Figure 15: Concept Inffuence Diagram for ResNet18 trained on Places365. The model is sensitive to the depicted concepts with an absolute score above 0.99. (We use RCAV to quantify the sensitivity, and re-scale the score to [-,, 1]) Positive influencing and negative influencing concepts are provided. The number of concepts have been limited to 10. When concepts appear more than once, they correspond to different signal directions (as labeling the classifiers with Nissest may assign the same concept name to more than one directions.) Here we report results for EDDP-C and I = 512.
Concept influence Diagram Class: speaking Network: Resme150
Concapt Influence Diagram Class: driving
Network ResmeL50
Concrpti with Preetivn Infurree
concarts with weggove untuonce,
Concepts with Profher Influntce
concopts with Nagos infuerree co
diateed 1- St
Imtroii Te 36
Nottere hop by
honded 70:93
unjoating Inlk-
mereh mw82
4o 20
proothing
we 1ol.::
rpesdio val (on 3tww1gg-00000000
oro mng
Neetur mape
pretect g g1oLLo
wew wallkcausUU.. 0.13
Toolic lang 3033000:
groomlng- D:000g00
Uuiishy 100 a0s-0
1.1:0.05
2000:15
Figure 16: Concept Influence Diagram for ResNet50 trained on Moments in Time (MiT). The model is sensitive to the depicted concepts with an absolute score above 0.99. (We use RCAV quantify the sensitivity, and re-scale the score to [-1,1] Positive influencing and negative influencing concepts are provided. The number of concepts have been limited 10. When concepts appear more than once, they correspond different signal directions (as labeling the classifiers with Ne(Dissect may assign the same concept name to more than one directions.) Here we report results for EDDP-C and = 2048.
Figures 15 40 41 42 depict concrele examples of how each concept's signal direction impacts the ResNet118s class predictions. Figures 16 43 and 44 depict similar examples for ResNet50. Concepts appearing more than once correspond different dircctions that have been attributed the same label. Seemingly irrelevant concepts with positive influence may have three possible explanations: a) the network has some sensitivity to those concepts (as 1es topl accuracy is 56.51% and ResNet500' 28.4%) b) their impact might be
29

---

Published in Transactions on Machine Learning Research (02/2026)
197
batier
cownr-indooee 6.18
marial /a/ 0.39
batiara 2o0 20:34
0544
11:49
Figure 17: Left: Original image. The caption contains class prediction and output class logit. Top Row: Seg- mentation Maps obtained by the concept detectors. The caption contains classifier index (unit), concept-name and IoU score in the validation split of the dataset. Bottom Row: Concept Contribution Maps. The caption contains concept-name and contribution of the concept to the class logit.
Uee (neneed
esjavilo
rened]
Hom
Figure 18: Top Row: Segmentation Maps obtained by the concept detectors. The caption contains classifier index (unit), concept-name and IoU score in the validation split of the dataset. Bottom Row: Concept Contribution Maps. The caption contains concept-nand and contribution of the concept the class logit.
low, since RCAV only considers the sign of the class prediction difference before and after the perturbation, regardless of its magnitude, and c) their label may be misleading as the respective concept. detectors do not reliably predict the concept, possibly indicated by a low loU score.
7.9 Application: Local Model Explanations via Concept Contribution Maps
In this section, we apply the concept contribution analysis of Section 5.2 to provide a detailed local explana- tion for a prediction of ResNet18 in an image of Places365. this example, we use direction pairs learned via EDDP-C and 512. The example input image (Fig. 17 top left) is correctly predicted belong to the baseball-field class. The output class logit for this image unexplainable residual is 0.2816. The top rows of Figures 17 and 18 provide concept segmentations which were obtained by applying upscaled to the original resolution. Above each segmentation map, we provide the index i of the respective concept detector (marked as unil), the name of the concept altributed to the concept detector, and the dadeset-wide Intersection over Union score the detector when detecting the concept in the validation split of Broden. Below cach segmentation map, we provide the respective CCM. Each spatial element of CCM for concept contributes positively to the prediction in two cases: 1) whenever the concept content in the sample is greater than the concept content in the baseline point and the CCRC is positive, and 2) whenever
19.7, 7, the baseline logit is 4Te 0.0003 and the
concept detectors to the patch embeddings of the input and subsequently
30

---

Published in Transactions on Machine Learning Research (02/2020)
Anelysis for Class Prediction: beesball-feldd |Aiive Concept Contributons Anelyyi
Concept Anelysie for Cless Prediction: beseball-fiekd Active Concept Contributione
+0.23
-0.16
10:45
SroRs_1
well
-1.11
10.73
grees_2
73.53
+0.80
00.42
stadium sladkum-besebblll 7.81
martall arts, gywlsyymesll 69
+6777
-0.57
battens,
+6.80
0.43
$23.24
+0.07
0.44
sky
10.10
-2.03
Contribution Score
Contributie Score
CCRC
CDRC
sarpfe-ccorzppt
Eosoline coneapt
baselino concept
cormeciion
commatio
Figure 19: Concept Analysis for predicting image of the bascball-field class. The figure depicts concepts found in the image. Even though concepts may share the same name, they correspond to different direction pains.
Concept Analysia for Class Predicton: baseball-fiadd
Top Positive Contributions
Concept Anelysis for Class Prediction: beseball-fedd
Top Negalive Contributions
+0.57
battere_bax-e box-e 7.50
ioe_ Boo_skkting...e hoo_skating_iikkinnddo iineindooss
4.01
-23.34
7,12
H0.10
100112
pool table
pool table
16
+2.28
11.80
highway-s
+4 96
.08
+0.49
stadium boseball-e -7.81
volleyball court outdoon
4.00
-0.09g
15
+0.82
forest, road-s
airplane
36
20
Contributinn Score
Contritution Soore
CIRC
CERC
bsselire
Figure 20: Concept Analysis for predicting an image the daseball-field class, The figure depicts top positive and top negative contributing concepts. Even though concepts may share the same name, they correspond to different direction pairs
the concept content in the sample is less than that in the baseline and the CCRC is negative. CCMs do not necessarily highlight the regions with the concept, the sign of CCRC is integrated into the heatmaps (for instance: Unit 390). Above each CCM we provide the name of the concept together with the contribution of the CCM to the explanation logit after additionally integrating the correction factor.
Figures 19 and 20 depict image-level concept contributions by breaking down the explanation logit components. In these Figures, the total equals sample concept baseline-concept + correction, discussed in Section 5.2 excluding the small residual term.
31

---

Published in Transactions on Machine Learning Research (02/2020)
dnveway.0.69
aparmenn-bulddngguudoooooo
runway, 0.57
sky
road
grass
building
airplane
No manipulation
road boad-bullding
building - -airplane
Figure 21: Counter-factuse explanations via manipulating the patch embeddings in the representation space using directional concept arithmetic. From Left to Right: 1) original image, correctly predicted driveway with confidence 69% 2) segmentation map obtained by applying the learned concept detectors on the original image representation. 3) Manipulating the latent representation and replace the road with the encoding the image is classified as ppartment-building outdoor with confidence 22%. 4) Manipulating the latent representation and replace the (whole) building with the encoding of airplane. The manipulated image is classified as runway with confidence 57%.
building. Now
The concepts that affect the prediction positively the most (Fig. 20 are batlers-boz-s and stadium-basebull-s as well as the absence of train and forest-road-s. As we can see in the Figures, in most cases, e.g. for the concepts grass-1, batters-bor-s, Stadium-buseballe moa-ssice-skating-rine indorrss the sign of the difference in concept contribution between the sample and the baseline does not change with the consideration of the correction factor. However, there are cases in which the imperfect state of convergence when learning the directions impacts the quality of the explanation; c.g. for the concepts grass-2, 2,sk, wall, mull,pool-table train,highwwe wwwp://ooyos.ooollalllaoooooo satleybull--curr--ooooo----- airpllane. In those cases, the consideration of the correction factor changes the sign of the total concept contribution. Many of the cases that fall into this last category exhibit marginal differences in concept contribution scores between the sample and the bascline; for instance: grass-2, sky, wall, train, airplane, which could justify, at least partially, this behavior.
meshed, martial-arte gra-g- canern-indoor-s, tree, forest-
7.10 Application: Counterfactual Explanations
In this Section, we provide concrete example of how the learned direction pairs can be harnessed provide counter-factual explanations. In this case study, will be studying ResNet18 and direction pairs learned with CFM and I 512. We consider the input image depicted in Fig. 21 left. This image is correctly classified by the network as driveway with confidence 69%. The second image of the same figure depicts a segmentation map based on the learned concept detectors. To keep the visualization simple, we only show the most dominant detected concepts.
We consider two counter-factual scenarios. We study how the prediction would change a) the road was replaced by a building, and b) if the building was replaced by an airplane. For each scenario, we perform two interventions in the representation space, after the other: concept removal (encoding concept absence in patch embedding) followed by concept addition (encoding concept presence in patch embedding). In both scenarios, the intermediate concept removal does not alter the predicted class. However, in the first. one, confidence is decreased to 30.5%, while in the second it is slightly increased to 70.4%. With concept addition, the first scenario the prediction changes to penrtment-buiiding ootdoor with confidence 22%, and in the second, the prediction changes to runway with confidence 57%. (Fig. 21).In both cases, the change the prediction outcome aligns with human intuition. More details for the intervention are provided in Section 20
32

---

Published in Transactions on Machine Learning Research (02/2026)
Network accuracy and confusion matrix for the network trained on the Chess Pieces dataset. Rows corre- spond to ground-truth labels and colums to network predictions. Three classes are considered: sashop/knight, /rook. The rows of the confusion matrix are normalized against ground-truth element count. Three datasets are also con- sidered: Clean (without watermarks), Poisoned (with watermarks) and Clean & Poisoned which is the union of the previous two.
Table 13:
Dataset:
Clean
Poisoned
Clean & Poisoned
Accuruey:
0.93
0.34
0.64
n
r
n
n
b
0.95
0.05
0.0
0.0
0.0
1.0
0.48
0.02
0.5
0.95
0.05
0.0
0.0
1.0
0.0
0.48
0.52
0.04
0.04
0.92
0.0
0.0
1.0
0.02
0.02
0.96
7.11 Application: Toy Model Correction
In this experiment, demonstrate how the proposed approach may be utilized to correct toy model that relies controlled confounding factors to make its predictions. For the purposes of this toy experiment, we use a small convolutional neural network with 5 Conv2d layers, each followed by a ReLU activation. The top of the network is comprised of a Global Average Pooling (GAP) layer and a linear head. We consider the task of predicting the chess piece name from an image depicting the piece. We use the Chess Pieces dataset from Kaggle - which contains a collection of images depicting chess picces appearing online play. For simplicity, we consider 3 chess pieces to be classified by the network, namely: bishop, knight, rook, thus, the network predicts K 3 output classes. To encourage the model to learn bias in making its predictions, we poison half of the rook images in the training set with the watermark text "rook" in the top left corner of each rook image. With the introduction of this bias, we expect that the network learns that the watermark concept has a positive influence on the rook class, while not being the only feature of positive evidence for the same class, since we include rook images in the training set without the watermark. For further details regarding this experiment, refer to Section A.2I
7.11.1 Evaluation
For evaluation, we construct three datasets. First, we consider a Clean test set, a dataset comprised of images without any watermarks. Second, we consider Poisoned test set comprised of all the images in the clean set but poisoned with the watermark, and c), we consider the union of the previous two datasets (Clean & Poisoned). Table 13 summarizes the performance of the network in these datasets. As evidenced by the Poisoned section of the detailed confusion matrix, the watermark is strong feature that, whenever it is present, directs the prediction towards the class rook.
7.11.2 Direction Learning for Watermark Identification
Since we already identified that the watermark actually influences the predictions of the network in a con- sistent manner, we seek to answer the following two research questions: a) can the proposed EDDP method identify the watermark as a concopt? That is, is any of the learned classifiers responsible for detecting the watermark?, b) Supposing the answer to (a) positive, given the watermark's concept detector and the respective learned signal vector, can we fix the network to not rely on the watermark for its predictions
We use the network's training set as concept dataset. We consider the last convolutional layer our layer of study, which has spatial dimensionality of 2x2. We found that when learning with / = 6, the proposed approach is able to identify the watermark direction. In Fig. 22 we provide example image segmentations obtained via the learned concept detectors. From the qualitative visualizations, we see that classifier 5 identifies the watermark. We found that this classifier detects the watermark concept with IoU 0.85.
Pieces Dataset httpa/////s/////////////////a//a
33

---

Published in Transactions on Machine Learning Research (02/2026)
Classifier
Classifier 2
book
fool
Classifier 3
Classifier 4
rook
fool
roola
fook
Classifier 5
Classifier 6
rook
rook
rook
roolo
ronks
Figure 22: Example image segmentations based on the concept detectors learned for the model correction experiment. Top rows illustrate pictures with the concept and bottom rows illustrate pictures without. Classifier detects the watermark.
7.11.3 Influence Testing with RCAV
We use RCAV to mcasure the sensitivity of the model with respoct to the watermark signal vector. The sensitivity scores reported by RCAV are for the classes bishop and knight and for the class rook. This implies that, when the image has the watermark, it becomes rook and less bishop knight. This quantitative score aligns with our intuition regarding the watermark.
7.11.4 Signal Vector Faithfulness to the Watermark Concept
Leveraging our knowledge of which images contain the watermark, we can determine the watermark's encod- ing direction by calculating the watermark's Pattern-AA (Pahde et al., 2024).. Subsoquently, we quantify the directional alignment between the Sattern-CAV. which was learned with supervision, and the signal vector, which was learned without. We found that the cosine similarity between these two 0.99, indicating estimations that agree.
7.11.5 Model Correction by Using the Watermark's Encoding and Decoding Directions
Let w and denote the learned parameters the watermark concept detector and & denote the respective learned signal vector. Without re-training or fine-tuning the network, we are going to suppress the watermark artifact component from the representation whenever it detected by the concept detector. We propose the following feature manipulation strategy, which we apply to the features of the last convolutional layer:
34

---

Published in Transactions on Machine Learning Research (02/2026)
J (eL mmks). m= a(w" Zp -b)
(18)
with k chosen such that w² x Eper w xpl N {p w Tp - b 0): also 13 k ensures that the signal value associated with the watermark concept within the patched representation matches the mean value found in the collection of patches lacking the watermark. The ReLU in 18 that the manipulation does not move the features out of the domain of the lincar head.
This choice of
ensures
7.11.6 Evaluation of the Corrected Model
We evaluate the corrected model according to the protocol that was used Section 7.11.1 The results are depicted in Table 14 Compared to the performance of the original network (Table 13 we see that the corrected model: a) has comparable accuracy to the original model the clean Lest set, b) is significantly more accurate images of the poisoned dataset with absolute improvement of +31%, and c) performs substantially better on the union of clean and poisoned datasets with absolute improvement of +14%.
We also compare our correction strategy to using random manipulation direction with the same k as before (i.c., using a random vector in place of the learned signal vector in (18). In a series of 10 trial evaluations on the Poisoned Test set, we verified that in out of 10 cases, no improvement was achieved: the classification accuracy was the same that of the original model, and the confusion matrix remained the same as in Table MidMeeee.. The same observation holds when considering the learned filter direction in place of the learned signal vector. The remaining case demonstrated improvement of +15%, which is still inferior to +31% when using the signal vector.
Table 14: Network accuracy and confusion matrix for the corrected network trained on the Chess Pieces dataset. Rows correspond to ground-truth labels and columns to network predictions. Three classes are considered: Batloo//nnght//ee
The rows of the confusion matrix are normalized against ground-truth element count. Three datasets are considered: Clean (without watermarks)) Poisoned (with watermarks) and Clean & Poisoned which is the union the previous two.
Datasel:
Clean
Poisoned
Clean & Poisoned
Accuvocy:
0.92
(-1%)
0.65
(+31%)
0.78
(+14%)
n
r
n
r
n
b
0.95
0.05
0.0
0.5
0.2
0.3
0.73
0.12
0.15
0.0
1.0
0.0
0.14
0.72
0.14
0.07
0.86
0.07
0.05
0.13
0.82
0.05
0.22
0.73
0.05
0.18
0.77
8 Limitations
Our method builds on the linear representation hypothesis, assuming that concepts are encoded direc- tions in the latent space. While this view aligns with several prior works, alternative formulations such multidimensional concept discovery (Vielhaben et al. 2023) or linear subspaces (Chormai et. al. 2024 exist but fall outside the scope of our study. Additionally, we constrain our experiments to settings where the number of discovered concepts is smaller than the embedding dimensionality. As such, we do not compare with methods like Sparse Autoencoders (SAEs) [Lim et al. 2024 Cunningham et al. 2024 Sharkey et al. 2022) that assume an overcomplete representation. Even though we restricted our experiments as such, future studies may consider this analysis, since our method is not technically limited in this regard. While evaluated our approach across multiple CNN architectures, Transformers (Vil's) (Caron et al. 2021 He et al. 2022). despite existing literature suggesting that similar lincar assumptions may apply in that context (Kaltampanidis et al.) 2025)-thit -his remains a promising di- rection for future work. Moreover, unlike some methods from unsuperv ised stale-offthe-art that operate on class-specific image subsets constructed from model predictions, we applied our method to unlabeled data without class filtering, due the lack of dense annotations the training sets. Nevertheless, our frame- work could, in principle, be adapted for class-conditional analysis. Finally, our study aimed minimize hyperparameter tuning by relying prior work (Doumanoglou al. 2023 2024 and empirical preliminary
did not investigate applications to Vision
35

---

Published in Transactions on Machine Learning Research (02/2026)
experiments; however, further tuning exploration of additional hyperparameters may yield improvements in clustering quality, interpretability, or influence (Section A.14
9 Conclusion
We introduced an innovative unsupervised technique to uncover pairs of latent space encoding-decoding directions that align with interpretable concepts of influence. This research offers new perspective on the unsupervised identification of concept. directions, unlike previous methods which are based on feature reconstruction or matrix decomposition. We believe that our work opens the door to richer model diagnostics, fino-grained explanations, and targeted interventions, paving the way for future research on scalable concept discovery, dynamic model editing, and integration into decision-making systems.
Acknowledgments
This work has been supported by the BC funded Horizon Europe Framework Programme: CAVAA Grant Agreement 101071178.
References
Guillaume Alain and Yoshua. Bengio. Understanding intermediate layers using linear classifier probes. arXiv:1610.0164 [es, stat], November 2018. arXiv: 1610.01644.
Christopher J Anders, David Neumann, Talmaj Marine, Wojeiech Samek, Klaus-Robert Müller, and Sebas- tian Lapuschkin. Xai for analyzing and unlearning spurious correlations in imagenet, 2020.
Christopher J. Anders, Leander Weber, David Neumann, Wojeiech Samek, Klaus-Robert Müller, and Sebas- tian Lapuschkin. Finding and removing clever hans: Using explanation methods to debug and improve deep models. Information Fusion, 77:261-200 2022. ISSN 1566-2535.
David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interprctability of deep visual representations. arXiv: 7et.1.04440999 [es). April 2017. arXiv: 1704.05796.
Itay Benou and Tammy Riklin Raviv. Show and tell: Visually explainable deep neural nets via spatially-aware concept bottleneck models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 100633-3072 2025.
Leonard Bereska and Efstratios Gavves. Mechanistic interpretability for safety-a review. arXiv preprint arXiv:2404.14082, 2024.
Dimitri P Bertsckas. Constrained optimization and Lagrange multiplier methods. Academic press, 2014.
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravoe, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hadfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards monose- manticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023.
Bart Bussmann, Noa Nabeshima, Adam Karvonen, and Neel Nanda. Learning multi-level features with matryoshka sparse autoencoders. In Forly-second International Conference on Muchine Learning, 2025.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervieed vision transformers. Tn Proceedings of the IEEF/CVP international conference on computer vision, pp. 9650 9660, 2021.
Pattarawat Chormai, Jan Herrmann, Klaus-Robert Müller, and Grégoire Montavon. Disentanglod explana- tions of neural network predictions by finding relevant subspaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
36

---

Published in Transactions on Machine Learning Research (02/2026)
Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. In International Conference on Learning Represenialions (ICLR), October 2024.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255, Icec, 2009.
Alexandros Doumanoglou, Stylianos Asteriadis, and Dimitrios Zarpalas. Unsupervised interpretable basis extraction for concept based visual explanations. IEEE Transactions on Artificial Intelligence, 2023.
Alexandros Doumanoglou, Dimitrios Zarpalas, and Kurt Driessens. Concept basis extraction for latent space interpretation of image classifiers. VISIGRAPP. Proceedings, 3:417-424, 2024. ISSN 2184-4321.
Maximilian Dreyer, Frederik Pahde, Christopher J Anders, Wojeiech Samek, and Sebastian Lapuschkin. From hope to salety: Unlearning biases of deep models via gradient penalization in latent space. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 21046 21054, 2024a.
Maximilian Dreyer, Erblina Purelku, Johanna Vielhaben, Wojciech Samek, and Sebastian Lapuschkin. Pure: Turning polysemantic neurons into pure features by identifying relevant circuits. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. 8212-821111 2024b.
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hodds, Robert Lasenby, Dawn Drain, Carol Chen, et al. Toy models of superposition. arXiv preprint ar Xiv:22000 10652, 2022.
Joshua Engels, Logan Riggs Smith, and Max Tegmark. Decomposing the dark matter of sparse autoencoders. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL https://openreviers net/
forumt'd==xq33W33veet
Thomas Fel, Victor Boutin, Louis Béthune, Remi Cadene, Mazda Moayeri, Léo Andéol, Mathieu Chalvidal, and Thomas Serre. A holistic approach to unifying automatic concept extraction and concept importance cstimation. In Advances in Neural Information Processing Systems, volume 36, pp. 54818. Curran Associates, Inc., 2023a.
Thomas Fel, Agustin Picard, Louis Bethune, Thibaut Boissin, David Vigouroux, Julien Colin, Rémi Cadénc, and Thomas Serre. Craft: Concept recursive activation factorization for explainability. In 2023 IEEE/CVF Conference Compuler Vision and Pallern Recognition (CVPR), pp. 2711 2721, Vancouver, BC, Canada, June 2023b. IEEE. ISBN 979-8-3503-0129-5 doi: 10.1109 10.1109/CVPPRO C199//VV22222222233 202..0066..
Leo Gao, Tom Dupre Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, llya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autocncoders. In The Thirteenth International Conference on Learning Representations, 2025.
Shanghua Gao, Zhong-Yo Yu Li, Ming-Hsuan Yang, Ming-Ming Cheng, Junwei Han, and Philip Torr. Large- scale unsupervised semantic segmentation. TPAMI, 2022.
Mara Graziani, An-phi Nguyen, Laura O'Mahony, Henning Müller, and Vincent Andrearczyk. Concept discovery and datasct exploration with singular value decomposition. In ICLR 2023 Workshop on Pitfalls of limiled data and compulation for Trushworthy ML, 2023a.
Mara Graziani, Laura O'Mahony, An phi Nguyen, Henning Müller, and Vincent Andrearczyk. Uncovering unique concept vectors through latent space decomposition. Transactions on Machine Learning Research, 2023b. ISSN 2835-8856. URL httpe:///pennrevier..o htt.://wwww...eeeoooooo/////t/dd
Stefan Hanfe, Frank Meinocke, Kai Görgen, Sven Dähne, John-Dylan Haynes, Benjamin Blankertz, and Felix Bießmann. On the interpretation of weight vectors of linear models in multivariate neuroimaging. Newrolmage, 87:96-1100 110, February 2014. ISSN 1053-8119.
37

---

Published in Transactions on Machine Learning Research (02/2026)
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern vecognilion, pp- 770- 770-778, 2016.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16000 16009, 2022.
Magmus R Hestenes. Multiplier and gradient methods. Journal of optimization theory and applications, 4 (5):303-322 1969.
Yannis Kaltampanidis, Alexandros Doumanoglon, and Dimitrios Zarpalas. Which direction choose? an analysis the representation power of self-supervised vits in downstream tasks. In 3rd World Conference eXplainable Artificial Intelligence (XAI), 2025. URL
15148776
Lena Kästner and Barnaby Crook. Explaining ai through mochanistic interpretability. European journal for philosophy of science, 14(4):52, 2024.
Been Kim, Martin Wattenberg, Justin Gilmer, Carric Cai, James Wexler, Fernanda Viegas, and Rory Sayres. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (Leav); June 2018. arXiv: 1711.11279.
Pieter-Jan Kindermans, Kristof T. Schütt, Maximilian Alber, Klaus-Robert Müller, Dumitru Erhan, Been Kim, and Sven Dähne. Learning how to explain neural networks: Patternnet and patternattributton, October 2017. arXiv: 1705.05598.
Diederik P Kingma, Adam: A method for stochastic optimization. arXiv preprint orXiv:1412.6986 2014.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International conference on machine learning, pp. 5338 5338-5348. PMLR, 2020.
Hyesu Lim, Jinho Choi, Jaegul Choo, and Steffen Schneider. Sparse autoencoders reveal selective remapping of visual concepts during adaptation, December 2024. arXiv:2412.05276 [cs].
Tsung-Yi Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference computer vision, pp. 2980-2988, 2017.
Πlya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. arXio preprinl arXiv:1608.0300 2016.
Aravindh Mahendran and Andrea Vedaldi. Understanding deep image representations by inverting them. In Proceedings of the IEEE conference computer vision and pottern recognition, pp. 5188 5196, 2015.
Aravindh Mahendran and Andrea Vedaldi. Visualizing deep convolutional noural networks using natural pre-images. International Journal of Computer Vision, 120(3):233 255, 2016.
Mathew Monfort, Alex Andonian, Bolei Zhou, Kandan Ramakrishnan, Sarah Adel Bargal, Tom Yan, Lisa Brown, Quanfu Fan, Dan Gutfruend, Carl Vondrick, et al. Moments in time dataset: one million videos for event understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-8, 2019. ISSN 0162-8828.
Neel Nanda, Andrew Lee, and Martin Wattenberg. Emergent linear representations in world models of self-supervised sequence models. In Proceedings of the 6th BlackborNLP Workshop: Analyzing and Inter- preting Newral Networks for NLP, BlackborNT. PQEMNIP 2023, Singopore, December 7, 2023, pp. 16-30. Association for Computational Linguistics, 2023.
Yannic Neuhans, Maximilian Augustin, Valentyn Boreiko, and Matthias Hein. Spurious features everywhere- largo-scale detection of harmful spurious features in imagenet. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 20246, 2023.
38

---

Published in Transactions on Machine Learning Research (02/2026)
Andrew Ng et al. Sparse autoencoder. CS294A Lecture notes, 72(2011):1-19, 2011.
Anh Nguyen, Jason Yosinski, and Jeff Clunc. Understanding neural networks via feature visualization: A survey. Explainable AI: interpreting, ezplaining and visualizing deep learning, pp. 55-76, 2019.
Tuomas Oikarinen, Subhro Das, Lam M Nguyen, and Tsui- Wei Weng. Label-free concept bottleneck models. In The 11th International Conference on Learning Representations, 2023.
Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, 2(11)e77 2017.
Frederik Pahde; Maximilian Dreyer, Wojeiech Samek, and Sebastian Lapuschkin. Reveal to revise: An explainable ai life cycle for iterative bias correction of deep models. In Medical Image Computing and Computer Assisted Intervention MICCAI 2023, Lecture Notes in Computer Science, pp. 596 606, Cham, 2023. Springer Nature Switzerland. ISBN 978-3-031-43895-0.
Frederik Pahde; Maximilian Dreyer, Leander Weber, Moritz Weekbecker, Christopher J. Anders, Thomas Wiegand, Wojeiech Samek, and Sebastian Lapuschkin. Navigating neural space: Revisiting concept acti- vation vectors to overcome directional divergence, 2024.
Jacob Pfau, Albert T Young, Jerome Wei, Maria L Wei, and Michael J Keiser. Robust semantic inter- pretability: Revisiting concept activation vectors. In Fifth Annual Workshop on Human Interpretability Machine Learning (WHT), ICML 2020, 2020, 2020.
Alec Radford, Jong Wook Kim, Chris Hallacy Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
Prajit Ramachandran, Barret Zoph, and Quoe V Le. Searching for activation functions. orXiv preprint arXiv:1710.0520 2017.
Kandan Ramakrishnan, Mathew Monfort, Barry A McNamara, Alex Lascelles, Dan Gutfreund, Rogério Schmidt Feris, and Aude Oliva. Identifying interpretable action concepts in deep networks. CVPR Workshops, pp. 12 12-15, 2019.
Sukrut Rao, Sweta Mahajan, Moritz Böhle, and Bernt Schiele. Discover-then-name name: Task-agnostic concept bottlenecks via automated concept discovery. In European Conference on Computer Vision, pp. 444-461. Springer, 2024.
Naomi Saphra and Sarah Wiegreffe. Mechanistic? arXiv preprint arXiv:241000908 2024.
L Sharkey, D Braun, and B Millidge. Taking features of superposition with sparse autoencoders, 2022. URL -f superposition
http://ww..lllgmmttffrrr org or../ost///BB//UORAAAAAAA. features- rew.erch-eearrrreeeeeeeeee-eeeee sete
Aleksandar Shtedritski, Christian Rupprecht, and Andrea Vedaldi. What does clip know about red cir- cle? visual prompt engineering for vlms. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 11987 11997, 2023.
Karen Simonyan and Andrew Zisserman. Vory deep convolutional notworks for large-scale image recognition. arXiv preprint Xi:: 1499. 1556, 2014.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014.
Christian Szogedy, Vincent Vanhoucke, Sergey Toffe, Jon Shlens, and Zbigniew Wojna. Rothinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pollern recognition, pp. 2818 2826, 2016.
39

---

Published in Transactions on Machine Learning Research (02/2026)
Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pp. 6105-61 6114. PMLR, 2019.
Johanna Vielhaben, Stefan Bluecher, and Nils Strodthoff. Multi-dimensional concept discovery (MCD): A unifying framework with completeness guarantees. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL http:/pppeereeeeeeeeee/rrraaaaaa
Chin Yun Yu. pytorch-nmf: Non-negative matrix fatorization in pytorch. yoyollcoris/pytoochr NMF) 2020. Accessed: 2025-08-01.
https://github.com com/
Mert Yuksekgonul, Maggie Wang, and James Zou. Post-hoc concept bottleneck models. In The 11th
International Conference on Leurning Represenlations, 2023.
Zeyu Yun, Yubei Chen, Bruno A. Olshausen, and Yann LeCun. Transformer visualization via dictio- nary learning: ualiext embedding a linear superposition of transformer factors, April 2023. arXiv:2103.19449 [cs].
Ruihan Zhang, Prashan Madumal, Tim Miller, Krista A Ehinger, and Benjamin IP Rubinstein. Invertible concept-based explanations for models with non-negative concept activation vectors. In Proccedings of the AAAI Conference on Arlificial Intelligence, volume 35, pp. 11682 11690, 2021.
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Autonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40(6): 1452-1464, 2017.
Bolei Zhou, Yiyou Sun, David Bau, and Antonio Torralba. Interpretable basis decomposition visual explanation, In European Conference on Computer Vision (ECCV), pp. 119 134, 2018
40

---

Published in Transactions on Machine Learning Research (02/2026)
A
Appendix
Appendix Index
A.1
Unsupervised Interpretable Basis Extraction and Concept-Basis Extraction Losses
A.2
Encoding-Decoding Direction Pairs Algorithm
A.3
Theoretical Comparison with Concept-Basis Extraction
A
Details on Uncertainty Region Alignment
A.5
Details on Reading Concept Information and Intervening their Encoding
A.6
Details on Using Encoding-Decodiing Direction Pairs and the Regions of Uncer- tainty to Provide Concept-Based Local Explanations and Detailed Spatially-Aware Con- cept Contribution Maps
A.7
Direction Learning Process
A.8
Details for the Experiment on Synthetic Data
A.
Extracting Signal Values with the Filters of Concept Detectors
A.10
Approaching Human Evaluations
A.11
Details the Faithfulness Assessment of Encoding-Decodiin Direction Pairs
A.12
Additional Experiments on the Faithfulness Assessment of the Encoding-Decodiin
Direction Pairs
A.13
Details for the Experiments on Deep Image Classifiers
A.14
Hyper-Parameter Study with Respect to Target Separation Margin
A.15
Ablation Study with respect to Filter-Signal Orthogonality Loss
A.16
Comparison with Unsupervised Interpretable Basis Extraction and Concept-Basis Extraction in Practical Experiments
A.17
More Qualitative Segmentations and Statistics for Evaluating the Interpretability of the Concept Detectors
A.18
More Global Model Explanations via Concept Sensitivity Testing
A.19
More Local Explanations with Concept Contribution Maps
A.20
Details Counterfactual Explanation Example
A.21
Details on Toy Model Correction
A.22
Details on Computational Resources
A.23
Relation to Sparse AutoEncoders (SAEs)
41

---

Published in Transactions on Machine Learning Research (02/2026)
A.1 Unsupervised Interpretable Basis Extraction and Concept-Baske Basis Extraction Losses
This Section complements Section 3.3 by providing further details on the loss terms of Unsupervised In-
terpretable Basis Extraction (UIBE) [Doumanoglou et al. 2023] and Concept Basis Extraction (CBE) Doumanoglou et al. 2024).
Sparsity Loss (C*) (Doumanoglou et al. 2023 Based on the observation that the number of semantic labels that may be attributed an image patch only fraction of the set of possible semantic labels, this loss enforces sparsity on the number of concepts that may be attributed to patch embedding. In particular, the sparsity loss for pixel p is based on minimizing entropy and is defined
Σ Cp logglpoi
qp.i
E. Up..
Vpp.i
(19)
The aggregated sparsity loss L is:
L = Ep [[p]
(20)
Maximum Activation Loss (cma) Doumanoglou et al. 2023
With the complement of this loss, the cluster membership variables yp. are enforced to become binary:
me = Ep Σ qp. loggyppp
(21)
Inactive Classifier Loss (C*²) Dooumanollou et al. 2024
This loss ensures that each concept detector in the set classifies positively at least v E [0, 1] percent of pixels in the concept datasel, avoiding clusters with few assignments.
Exll,,,)) - E, (
(22)
with 7, Y R+ denoting sharpening factor and 5 E [0, denoting percent of pixels in the dataset to be evenly distributed among the classifiers in the set.
Maximum Margin Loss (Cmm)) In the original formulation of Dounanoglou et al. (022)) the Maximum
Margin Loss was defined as mn = M with M being a single parameter for the whole set of classifiers since the optimization was performed in the standardized space with shared parameters for the margins M and biases b. In this work, we removed the standardized space constraints and instead, we have a margin parameter Mj for each classifier in the set. Thus, we modify the Maximun Margin loss to become:
mmm 1
Σ
Σ
1
(23)
M
A.2 Encoding-Decodeee Direction Pairs Algorithm
Algorithm provides, high-level, PyTorch-like pseudocode for learning the proposed Encoding-Decodee Direction Pairs.
42

---

Published in Transactions on Machine Learning Research (02/2026)
Algorithm 1 PyTorch-Liks Encoding- Decoding Direction Pairs Pseudocode
Input: X E [BXxXxXW,, Layer feature activations with B denoting batch size.
ft: Upper-part the studied deep network.
Learnable Parameters:
E DRDx1 Decoding directions (filters of concept detectors)
$ E PxI Encoding directions
b E R²: Concept detector biases
Output: Loss to minimize
1: function LEARNING SLTESTTEPP
2 corch.conv2er (X, W. ....uquuezze(-) unsqueeze(-1))
3: p = z b.reshape (1, -1,11,
4:
signal_vectors (X,,
Use rumning stats for variance and covariance and p>0 subsample
5: y torch. sig
q y torch. sum (y, koopdiis
7:
loss_ fs (g)
8:
loss_m ma _ g
9:
loss
L (1/tor/cccnnoo norm (Wdiin=)))
M, equals 1/1/w//22
10:
loss_ ic = (y)
11:
loss
C (y)
12:
loss fso = < (W,
with UFM then
14:
X' = 9XX, W)
15:
else if with CFM then
16:
X' h(X, W,
end if
18:
loss L (*)
19:
loss_lag Lagrangian losee
lassloss_uus __t___ur, wasloossmaa_a
loss_ secss_ic,ll_ liccoos_____eeeeee _ame________ les,s_aac,_____eeaaeee____ loss_ fso)
20:
return loss_lag lag
21: end function
A.3 Theoretical Comparison with Concept-Basis Extraction
Concept-Basis Extraction (CBE) Dournanoglou et al. (0224) made an initial attempt to exploit the network's 's instructions to improve the interpretability of the discovered directions. This approach, like ours, used feature manipulation and maximized the entropy of the network's prediction outcomes for the manipulated features.
However, the key difference between the approaches lies in the manipulation strategy: the prior method suggested manipulating feature representations (ætp) toward the concept detectors' hyperplanes only for concepts currently present in &p (ie., moving negatively along the filter weights when the concept positively classified). It explicitly neglected manipulation in the opposite direction for negatively classified concepts. This essentially implied that network predictions should be uncertain only when none of the concopt detectors made a positive prediction, ignoring confident negative ones.
Our approach fundamentally differs: we suggest that uncertain network predictions are tied to maximally ambiguous concept information, enforcing this by manipulating all features toward the hyperplanes, regardless of whether the concept was initially classified present absent. This ensures the network is maximally uncertain only when none of the delectors makes confident prediction, whether positive negative. Furthermore, CBE completely overlooked the critical role of signal directions.
A.4 Details on Uncertainty Region Alignment
Unconstrained Feature Manipulation (UFM): In this type of manipulation r' 9p(xp: W) = Fp dxp and dæ, such that:
W 1pp b, = 0 + w (x' dx.p) -h; = 0, Vi,
WT (xp dx,) b 0 dæp (W T + xp b)
(24)
43

---

Published in Transactions on Machine Learning Research (02/2026)
with A denoting the pseudo-inverse of A.
Constrained Feature Manipulation (CFM): In this type of manipulation zzp = hppppi W, 5) 2p dxp, with dxp given by:
w¹¹ J - b = + w (op dxp) - b, = 0, Vi,
WT (x dxp) b 0 w Svi) b 0
W Sup = WT Up + Up = (WTS)* 5)² (W" Ip b) 7
(25)
dxp Sup S(W* (()))((aa xp - b)
(26)
A.5 Details on Reading Concept Information and Intervening on their Encoding
Reading Concept Information from the Embeddings Reading concept information from a patch embedding ends up in estimating the concept's signal value that is encoded in the representation. Supposing that the related conditions are met, in Section 3.2 it. was shown that the filter direction can serve as a means to extract this value. While in the data-model of Section 3.2 the extraction of the signal's value is exact, under the multi-concept signal-distractor data model of Section 4.1 the considered latent space bias e interferes with this value. More specifically, if we suppose w; : wy 8,, + j. j.w, 1 and wi D, the inner product between the filter and the embedding results in:
w Tp Opp. w 8; +w c = wi up Op.i + w c
(27)
const
Thus, when considering our extended data model, the filter can be used to extract the signal's value but with constant offset, independent of Tp. Due to the offsct being constant, the difference between two projected embeddings is equal to the difference of their signal values, e.g.:
wo (Ipr Ipa) = api.t + w c ap2.t - w.c c= ap app.
(28)
In applications, we can exploit this property to estimate the signal value that is encoded in patch embedding with respect to another patch embedding of reference. We found that two reference points are of particular interest. First, the average embedding over a collection of embeddings. In that case, we can estimate the signal value of Fp with respect to the average signal value in the collection:
1 Sp i Epp [xp] w op Ep m xxp = app. - Epp [ap.)
(29)
And second, the concept's point of uncertainty. Given a query point Fp. the concept's point of uncertainty úp is the point that lies on the concept detector's hyperplane (thus corresponding a point of maximum concept ambiguity) and the line defined by the query point and the direction of the signal vector. More
specifically, the point of uncertainty is equal to úp- = &p &p,;8 with Kp.s E R corresponding to the solution of the following system of equations:
úp xp kp,i8.
w xp b.
= w -w, úp b, =0 = w. (Ip - Ki,, - = -0 = Kppi
w
xp
xp
b;
wi fp - b, 0
w S
(30)
Typically ún depends on but we drop this index for brevity. What is interesting about the point of uncertainty is that the following difference corresponds to centering the signal value of xp around the value that represents uncertainly:
w./ xΦ w. úp = w. xp (oo-w(](a w/[(z (8)) ) = Kp. w. Kpp. w. xp b
(31)
44

---

Published in Transactions on Machine Learning Research (02/2026)
Since (31 corresponds to the function of the concept detector, this difference is positive whenever the concept is present in 3pp and negative whenever the concept is absent. Thus, centering the signal value of a patch embedding around its point of uncertainty can be accomplished by computing Kp,i using (30).
Intervening on Concept Encoding Altering the concept content that is encoded in a patch embedding "pp ends up overwriting the signal value of the concept with target value of interest. The target value is meant to be copied from a target embedding a Let zyp denote the altered embedding after the intervention.
Supposing that we aim to intervene on the value of concept i, dn becomes:
app Up Ki..8;
(32)
with Kp,s E R. Since we want o'p and æp to have the same signal value for concept i, we make use of 28 and compute Ki,s such that:
w a D uo!
(0' a²)== =0 0- - w/f (xp + Kp.8. az)== =0 - Kip..
u 8, w " -w: up L - -ww Tpp
(33)
with denoting the projected target value. In applications, content of an embedding towards the presence absence of concept. To accomplish that, two natural choices arise: first, to overwrite the signal value of the embedding based on the average signal value of samples with or without the concept, and second, to overwrite the signal value of the embedding to match the value of top (or bottom) quantile of signal values in a collection. In the first case, we can use xp = Ep [xp] with the average taking place along the patches with (or without) the concept. For the second case, we can directly work in the projection space, by computing the quantile of interest over the projected embeddings wi up and use that value in place of t.
typically interested in altering the concept
A.6
Details on Using Encoding-Decoding Direction Pairs and the Regions of Uncertainty to Provide Concept-Based Local Explanations and Detailed Spatially-Aware Concept Contribution Maps
In this Scction, we discuss how our learned Encoding Direction Pairs can be leveraged in order to provide concept-based local explanations for any network prediction. For simplicity, we will be discussing the application in Convolutional Neural Networks (CNNs) with a Global Average Pooling (GAP) layer and a ReLU activation function, but adaptation to other architectures could be possible.
A.6.1 Terminology and Definitions
Suppose that we want to explain the prediction of an image classifier f for image I. Tet X denote this image's reprosentation at the penultimate layer of the network (in CNNs, this is typically the last convolutional layer before GAP). Let le E I denote the network's output logit for class e regarding input I. Typically, e will be chosen such that le is the maximum logit among the output class logits, but other choices are also valid, for instance when we want to explain why the prediction deviated from a specific class. Let also X E Hx denote a bascline point in the uncertainty region of the model. We call it bascline bocause this point corresponds to an artificial image representation for which the prediction of the network is highly uncertain. Let 15" IR denote the prediction logit for class that corresponds to X.m, i.e. 1g" = fX,"). We define the following quantity:
1- 15"
(34)
an explanation logit (le), which is equal to the difference in predicting class e for image I compared to a highly uncertain prediction for the same class. Let w. and b. denote the class vector and the corresponding bias of the network's last fully connected layer. Then, the explanation logit becomes:
Le 4-44-17 W. GAP(X) + b. - (w G + b.)
(35)
Our goal to express this quantity in terms of the identified concepts. Based on our previous discussions in Sections 4.4 and 5.1 we conclude that intuitive point of reference for expressing concept content is
45

---

Published in Transactions on Machine Learning Research (02/2026)
baseline point in the uncertainty region of the concept detectors. Any image representation X has corresponding baseline point X E RH in that region, which can be obtained by h X: W,S). If it weren't for the layer's activation function, we could choose X to be equal to X5, due to the alignment of the two uncertainty regions during direction learning. However, in practice, there is a shift between the two points, since X = ReI.U(Xg). For this reason; we treat X similar to X, i.e. as an artificial image representation whose concept content can be expressed with respect to a bascline point X in the uncertainty region of the concept detectors. Similar to X5, Xg=e can be calculated by h ((x;;; W W,S). Finally, let Up and o be equal to the expression of (10) when computing baseline points in the concept detectors' uncertainty region for X and Xg", respectively.
A.6.2 Logit Difference Decomposition in terms of Concepts
Based on the aforementioned definitions, we have:
Le Le - 15 = w G + b-- (ww GAP(X.") + +6) 7 Le- = GAP X .. GAP(X, xx + w. - -x²0)
(36)
Due to the linearity of GAP, 36 can be written as:
le = Ep w (xp = - Ep w (2r7))
me. p.b)
+ Epp 2 (2 pppbb)
(37)
where p.. p,b and sps elements at the spatial location p of X5. Xy" and Xge respectively. By definition, we have Fp pp Svp, and p.b Tp.h Sop. Up vy E R². Given these, 37 can be written as:
Le = Ep : S(vp - - + Ep w rp)
(38)
with Tp = xx p.b p.b E RD,, a residual that is not going to be explained in terms of concepts. The first part of (38) can be interpreted as follows: the contribution of concept to the explanation logit equal to w w (vp.i of This means that the explanation logit depends on global concept contribution factor w 8, which constant regardless of the sample to be explained, and a local contribution factor which depends on the sample and is equal to the difference in concept content between the sample itself and a synthetic bascline sample that corresponds to a highly uncertain network prediction. Since both 25,6 and ap. lie in the uncertainty region of the concept detectors, their difference in terms of concept content is zero, and the residual does not carry any concept-related information.
Let 0p.s (bp..) denote the signal value of Ep (x%,)) for concept i centered around the value corresponding to the concept ambiguity (i.e. by considering Tp (wite p.b. as a query point and using the concept's 's point of uncertainly reference). As discussed in Section 5.1 positive 0p.s (ihh.) indicates concept presence, while negative 6p, (6b,) indicates concept absence. 6p,i (0 and up (vb.)) would be exactly the same un? 8, 0A0 + j and also cach w, is perpendicular the basis of distractors D. In practice, these orthogonality constraints are only approximately fulfilled. To obtain intuitive explanations without compromising fidelity, we re-write
(38)
L. = Ep wx S( (6)) + wx S(Up úp (v) bb}] + Ex ww rp) *
(39)
4. Ep W T SopT
Ep : Soh + Ep / S(Up êp (e) tう)) Ep w.: Tp
(40)
sample concept
baseline concept
correction
residual
Based on 440, the explanation logit le is linearly expressed in terms of patches and concepts, giving detailed, spatially-aware information for concept contributions. We split (40) into four parts. The first part corre- sponds to contributions of concepts included the sample (sample concept), the second to the contribution of concepts contained in an artificial baseline point of an uncertain prediction (baseline concept), the third part to a correction factor to account for imperfect direction learning convergence, and finally, a residual
46

---

Published in Transactions on Machine Learning Research (02/2026)
Table 15: Left: Data matrices S and D for the experiment on synthetic data, Right: Cosine similarities for every pair of vectors in S, D, i.c.: -
S
D
0.6368
0.8583
0.5259
0.4008
0.6659
Cosine- Cosine-Similaritae Similarities
0.1561
-0.3371
0.1561
0.4585
0.6038
1.0
0.3319
0.4204
0.3188
0.4526
0.1633
-0.1533
0.7557
0.3337
-0.2065
0.3319
1.0
0.2087
0.3649
0.4111
0.2611
0.1643
-0.1580
0.5797
-0.2154
0,4204
0.2087
0.3621
0.2195
0.6226
-0.1607
-0.1643
-0.1396
0.1687
0.3188
0.3644
0.3621
1.0
0.4296
-0.1759
0.1607
-0.1594
0.2744
0.1617
0.4526
0.4111
0.2195
0.4296
1.0
-0.1592
0.1531
-0.1567
0.2232
0.1567
-0.1760
0.1354
-0.1612
0.1763
0.1546
that we do not explain. If the correction factor for concept was zero, concept i would have a positive contribution to the explanation logit whenever the difference in contributions between the sample and the baseline is positive. In the case of imperfect learning conditions, this difference must exceed the negative of the correction factor.
In the experiments, we report Concept Contribution Maps (CCMs) where each spatial element p is equal to:
0 = w s00pp 6
(41)
visualized as heatmap over the original image for given concept i. Finally, refer to the quantity w $ as the Concept-Cland Class Relation Coefficient (CCRC).
A.7 Direction Learning Process
We learn the directions of the proposed method in four-step process: a) first learn the parameters wi be following Doumanoglou et al. (2024), replacing the CNN Classifier Loss with the proposed Luer; b) we then continue optimizing w;, b,, removing the orthogonality and standardization constraints while keeping Luar and incorporating the additional losses from Section of the learned classifiers as signal value extractors in 1 to initialize {$}; and d) finally we jointly optimize
2 c) next, we learn the signal vectors using the filters
A; Wt b,, using all previous losses, replacing Laur with Lear and adding e0 from Sections 4.3 and 4.4 Tn all cases, we prefer optimizing with the Augmented Lagrangian Loss instead of linearly combining the loss terms.
A.8 Details for the Experiment on Synthetic Data
We train the network using cross-entropy loss and the Adam (Kingma 2014h optimizer, with learning rate 0.0005 and batch size 1024 for 4000 epochs. In principle, we follow the process defined in Section A.7 but due to the simplicity of the example, during (a) we don't use Luur, ; we omit step (b) and proceed directly from (a) to (c). In both steps (a) and (d) we use the Augmented Lagrangian Loss, the Adam optimizer, and the Cosine Annealing learning rate scheduler (Loshchilov & Hutter 2016). In step (a), we omit the constraints of the Augmented Tagrangian loss that are not applicable. The Augmented Lagrangian formulation greatly stabilizes lcarning and avoids local optima. For step (a) we solve the constrained optimization problem of minimizing L with ma 0.8, -mm 5 and 7'c 0.0. For step (d) we minimize X Xeur Ceur with mm 0.5, mm 15, Tir 0.0, raac 0.0, f 0.01. The learning rate use for step (a) is 0.00025 and for step (b) is 0.0005. The number of learning epochs is set to 10000 and 20000, for each step respectively. For the loss weights we use V 2.6 and A = 0.25. The sharpening factor 7 of L is set to 2.0, the 7 hyperparameter of the same loss is set to = 1.0, the v sharpening factor of Rsw to v = 2.0, the of Leee is set to p = 5/3 and the μ sharpening factor of C is set to 2.0.
The specific values of matrices S and D used in this experiment, and the cosine similarities between every pair of vectors, are provided in Table 15
47

---

Published in Transactions on Machine Learning Research (02/2026)
A.9 Extracting Signal Values with the Filters of Concept Detectors
As discussed in Section 1.3 signal values, which required to estimate the encoding direction of concept, are extracted using the filter weights of the concept detectors. Yet, as discussed in that Section, in order for this to happen, the filter weights w; need to be orthogonal to $, and D. Since we do not explicitly estimate distractors in this work, there may be an inevitable error when extracting the value of the signal (we say maybe, because this might also be mitigated by the Uncertainty Region Aligmment losses or filters converge become perpendicular distractors due the fact that they contain information independent of concept content). Here we study on the order of this error. From 3 we have:
xp = Sap + D3p +
When extracting the signal value with filter, the estimation becomes:
2p w up w Sap + +w, D3, + w w D3p + w 2p ap, w 8; wss, w s:
(42)
From 42 and as we've already seen (27 we notice that the latent space bias e introduces an additional constant error term when estimating the signal values. In the real-world scenario when this is not known, can use the following estimator âp which depends on the average of features Tp:
w Eplxpl w xp
pp.i
w
w 84 u DEP [8p]
mn Fp
w 8; Eyp ((p)
pp,i
(43)
w 8;
w si
w 8t
wi D
0p.i = ap.i
[ppc] +
(Bp E.p (3p))
The latter is an estimator of ap,s with respect to the mean Ep d and with an error term depending on distractors, irrespective of constant bias.
A.10 Approaching Human Evaluations
In typical scenarios, human--nn-hhe-lop experiments validate the interpretability of the identified concepts. In this work, however, we have annotations from the Broden and Brodem-Action datasets, which we can leverage a proxy for human judgment.
Implicit Human Evaluation via Annotated Data: Our learned concept detectors are evaluated by calculating dataset-wide classification and segmentation metrics, such as Precision, Recall, AP, mloC M2 on the Broden and Proden-Action datasets. Because the ground-truth labels of those datasets were originally produced and verified by human annotators, alignment with these annotations serves as an implicit, large-scale human evaluation.
we also employ an automated monosemantieity metric (M) leveraging CLIP's (Radford et al.) 2021) embedding space. As CLIP is trained to align visual features with human language, it acts as a reliable, high-level proxy for semantic coherence, mcasuring cluster concept tightness based on samples in the collection.
CLIP as a Human Proxy:
Conclusively, even though we did not employ physical subjects in our evaluation to assess the outcomes the work, we relied on annotations from two large concept datascts address this need. To further expand on annotation classes beyond those in the datasets, we used the CLIP foundation model a proxy to measure coherency within the image collection of each concept.
48

---

Published in Transactions on Machine Learning Research (02/2026)
ResNett0 0 Places3ice
Ltficeetheel Imagenet
Histogram Cosine Simlanlies Setween the Dresming Direction and the Signa Vector
Histogram Cosine Smilanbss Batwron the Dreeming Directon and the Signal Vector
(1)
Powers (NN
20
0-
0.4
0.5
06
0.7 Cooine Himillrity
0.8
0.9
0.3
04
0.5
0.6
0.8
09
Couine Similarit
Figure 23: Cosine similarity histogram between dreaming directions and signal vectors. Left: Directions learned
for ResNet18 trained Places365 (I 448). Right: Directions learned for EfficientNet trained ImageNet (I = 1120). These histograms regard directions learned with Leur but without f
A.11 Details on the Faithfulness Assessment of Encoding-Decoding Direction Pairs
Due to the considerable computational load required to conduct this experiment, make some simplifi- cations. First, consider subset of the learned direction pairs by randomly choosing 100. Candidate pairs are the ones with concept detector that is considered interpretable by Network Dissection (i.e. the detector exhibits an IoU performance greater than 0.04 when detecting the concept). Second, to construct the image set that contains the concept, we consider 10 unique images within the set of the most confident (patch-level) predictions of the concept. detector. For each image, we run the deep dream optimization for == iterations, and we record the patch embeddings (feature activations) containing the concept with an interval of 5 steps. For dreaming, proposed in Olah et al. (2017). Regarding the line fitting process, we consider parametric line as + Xd, E R. with c E RD denoting the line's origin and d E TRD its direction (the dreaming direction). At the end of the dreaming optimization loop and after recording the feature activations of the concept, we mini- mize the average distance from each feature to the line. Finally, we make sure that the dreaming direction points towards the direction of feature evolvement across iterations by considering making a sign flip, i.e. multiplying d with -1
don't 't use zooming, but we do use the robustness transforms originally
A.12
Additional Experiments on the Faithfulness Assessment of the Encoding- Decoodin Direction Pairs
23 and complement the experiments presented in Figures 11 and 12 for the remaining combina- tions regarding C Even though less prominent ResNet18, together with Section 7.2 the experiments sufficiently support that f0 pushes the distribution of cosine similarities between dreaming directions and signal vectors to become more bell-shaped. Overall, the experiments in this section, similar to the exper- iments presented in Section 7.2 indicate that in approxinately 90% of the cases, these cosine similarities exceed 0.7. In a less rigorous but more qualitative sense, we found that when learning EDDP with e this loss converges to values close to z 0.02, whereas mcasuring the loss value for a dircction set learned without L often leads to value close to Rs 0.1, which still not that far from indicating orthogonality bet ween filters and signal vectors.
Figures
49

---

Published in Transactions on Machine Learning Research (02/2026)
ResNett 8 Places3ice
Ltticeethell Imagenet
Histogram of Cosine Siilanties Setween the Dresming Direction and the Signal Vector
Histogram Cosine Smilaniiss Batwron the Dreeming Directon and the Signal Vector
40
Phouet (10)
20
Poweeu
15-
0-
0-
0.3
0.4
0.5
06
0.7
0.8
0.9
0.55
0.60
065
0.70
0.75
0.80
0.86
0.90
Cooine Himilarity
Couine bimularity
Figure 24: Cosine similarity histogram between dreaming directions and signal vectors. Left: Directions learned
for ResNet18 trained Places365 (I = 448). Right: Directions learned for EfficientNet trained ImageNet (I = 1120). These histograms regard directions learned with Luur and L
A.13 Details for the Experiments on Deep Image Classifiers
Details on Forming Concept Detectors While our approach explicitly learns binary classifiers (w; b,} as concept detectors, an estimation of an explicit classification threshold is missing when clustering with PCA, NMF, and the Natural basis. To form concept detectors those cases, we follow [au et al. (2017). and for cach direction, we choose from the concept dataset. For the Natural latent space basis and NMF, we use k 0.005, as suggested in Bau et al. 2017), while for PCA, we use k = 0.2 as suggested in Graziani et al. 2023a
be equal the top k-quantile among the projected activations sourced
Differences in the Evaluation Protocol compared to PCA-based and NMF-based State-of-ther Art and Limitations Previous methods (Grzziani et al. 2023a b Zhang et al. 2021 Fel et al. 2023b) based on PCA or NMF operated under a class-conditioned protocol, where the analysis and identified concepts are intrinsically linked to a specific class. On the contrary, this work, our method is applied in class-agnostic protocol, analyzing the latent space without prior class information. This fundamental difference in methodology renders direct comparison with the exact implementations of these previous approaches GGraziani et al., 2023a b Zhang et al. 2021 Fel et al., 2023b) unfair infcasible. Specifically, the size of the concept space I approaches the size of the embedding space D, NMF trivially converges to the natural feature space basis. Moreover, in a class agnostic setting, the gradient re-ranking approach of (Grzzian et al.) 2023a b) is not feasible, as gradients are tied to a specific class, requiring further adaptation of the original algorithm. However, PCA and NMF are still applicable in a class-agnostic protocol as typical matrix decomposition methods.
In this work, we implement both PCA and NMF baselines as matrix decomposition methods on layer feature activations in class-agnostic fashion. We emphasize that, for NMF, we do not consider the recursivity part of Fel et al. 2023bb (as we consider concept factorization to be outside the scope of this work), and for PCA, we do not perform the gradient re-ranking that was suggested in Graziani et al. (023a).
Given the previous limitations, both baselines could be less effective in the agasstagnostic setting compared to the class-conditioned state-of-the-art. The challenges they face under the generalized, class-agnostic protocol highlight the necessity for more effective and novel approaches identifying concept dircctions this sctting, and the proposed method is such an atlempt.
Details EDDP-U and EDDP-C We emphasize that EDDP-U does not include the use of L to learn the directions. The main purpose of L is to contribute to a faithful estimation of the encoding directions. While this was critical for the experiment on synthetic data, in real-world experiments, we see that the value of this loss is sufficiently low even when not directly minimizing it, possibly due to working in high-dimensional embedding space where two random vectors are approximately orthogonal. discussion regarding L0 is provided in Sections A.12 and A.15 with the conclusions presented in the main with
detailed
body of the article being mostly aligned with the conclusions that stem from detailed ablation studies.
50

---

Published in Transactions on Machine Learning Research (02/2020)
Table 16: Hyper- pepper-arammters used to learn the Direction Pairs
ResNet18
Places365
ResNet50
MiT
Step
Epochs
LR
Step
Epochs
LR
384/448/512
a 500
0.005
1536/1792/31 /2442
1300
0.001
384/ 448/512
b 2500
0.00025
(136/1192/22
/2048
b
2600
0.00025
384/ /422/212
d
1250
0.00005
1536/1792
d
1300
0.0001
Efficient.Net
ImageiNet
VGGIG ImageNet
I
Step Epocls
LR
1
Step Epochs
LR
960/1120/ 1280
S00
0.001
384/448/512
1000
0.001
960/1120/ 1220
b
3500
0.00005
384/448
b 1000
0.0001
960/1120/ 120//1880 180
d
1750
0.00005
384/448/511
0.0001
Inception-v3
ImageNet
Step
Epochs
L.R.
1536 1:366200s /2048
a
1500
0.001
1536/ 1792/20222 /2048
3000
0.00005
1536/ ///0//192///000 /20-8
d
1500
0.00005
1792
1500
0.01
Details on the Influence Evaluation Metric As we already mentioned in Section 3.5 RCAV originally used CAVs to assess the network's 's concept scnsitivity. Here we follow the proposition of Pahde et al. (0024) and use the learned encoding directions in their place. For sensitivity scores, we spatially replicate the learned encoding directions across all spatial locations in the image representation. In studies without Le"r, we report influence metrics for signal voctors that were estimated post learning the directions with the subsample strategy discussed in Section 4.3
Learning Details In Table 16 we provide the main ayper-parameee to train the direction pairs for each step. The learning rate that provide (LR) is considered as the reference learning rate for batch size of 4096. In practice, we scale both the learning rate and the batch size based on the available GPU memory and the number of GPUs. We use the Adam Kingma (2014) optimizer and the cosine annealing learning rate scheduler [oshchilov & Hutter 2016). While for step (a) we follow the syper-parameter setup of Doumanoglou et al. (2024), for steps (b) and (d) we make different choices. For L and L we set their and p as follows: For ResNet18 and VGG16, 7 and p are set such that the minimum cluster size is equal to 400 and the maximum 50000. For EfficientNet and Inception-v3, the same numbers are 520 and 65000, while for ResNet50 the minimun cluster size is set to 920 and the maximum to 75000. These are all set based on the statistics of the concept datasets. The sharpening factor of f and the y sharpening factor of Rsw are set to 2.0.
For Augmented Lagrangian Loss, in all cases we used the following hyper-parameters: X 2.6, Aur 0.25, ma 0.8, -' 0,r f 0.01. For ResNet18, EfficienttNe and VGG16 we use mm 5.0, while for Inception-v3 and ResNet50 we used - 6.0.
When using Laur and L we observed better results when manipulating features with stochastic mag- nitude in the direction dæpp, i.e. shifting representations as z Tpp kdææ with random number in [0.1, 0.5). In practice, we separate filter directions from their magnitude 1/M; and learn them independently as suggested in Doumanoglou et al. (2024). For enforcing ||w:||2 = 1 (i.c. unit norm filter vectors) we use parametrization on the unit hyper-sphere.
Details on Datasets used to Compute Concept Sensitivity with RCAV To conduct concept sen- sitivity testing with RCAV we need access to labeled dataset coming from the domain of the model. In practice, it is common to use the validation split of the dataset used to train the model. To mitigate the required computation time, we limited the size of the validation datascts as follows: For ImageNet, we used the validation split of ImageNettS-300 Gao et al. (202)), and for Moments-in-Tinet we considered Moments- In- In-Time Mini. Since Moments-In In-Time--eee a video dataset, but the model that we studied works in the image domain, we constructod an image dataset by sampling 3 frames equally apart from cach video (i.c. the first, the mid, and the last frame of the video). For Places365, we didn't make a size reduction as the validation split of the dataset was manageable with our resources.
51

---

Published in Transactions on Machine Learning Research (02/2020)
Precision Hittggram [ineanor- supersised)
Recall Difference Hallgram (inearor supervised)
F15core Difference suparvise)
20
15
Puncat
PooSE (N
10
10-
5-
0-
0-
1.00
0.75
-0.50
-0.25
0.00
0.25
-0.75
-0.50
-0.25
0.00
0.25
0.50
-0.8
-0.8
-0.4
-0.2
0.0
0.2
Procuion Dificrence
Hocall Infference
P1Nomm Difference
Figure 25: Interpretability Comparison. Histogram of differences in binary metrics: Precision, Recall, F1Score between the Linear-Oft set of concept detectors learned with EDDP-C, 1 = 512, and classifiers learned in supervised way (TBD Zhou et al. 2018)). The network here is ResNet18 trained on Places365.
Details on RCAV's Statistical Significance Test In all experiments, we RCAV's perturbation hyper- parameter, a For direction significance lesting, random noise signal vectors, we (a) construct dataset of feature- fbature-bbinary) label pairs based the decision rule of each one of the concept detectors. To deal with great class imbalance, construct pool of negative samples that is at most 20 times more than the positive ones; (b) we construct N noisy versions of that dataset by label permutation; (c) we learn a noise-classifier to distinguish features based on the permuted labels, and (d) we concurrently estimate a noise-signal vector using (T) and the subsampling process described in Section 4.3 To learn each one of the noise signal vectors, and before permuting the labels, we construct a balanced dataset at most 5000 samples, picked randomly from the pool. We train the noise classifiers using Adam for 100 epochs and lcarning rate 0.01. By using noise signal vectors RCAV's noisy directions, and with the number of those vectors per classifier set to N = 100, we subsequently calculate RCAV's p-values. We apply Bonferroni correction to all p-values by dividing the significance threshold p 0.05 by the number of concept detectors I and the number of model classes.
RCAV's label permutation test. To construct
Learning Details for NMF When learning directions with NMF, we used the implementation found in Yu (2020). To make computations manageable, we reduce the spatial dimensionality of image representations by applying Adaptive Average Pooling to (2,2) resolution, in similar way it was done in Fel et al. (203bb). In all cases, we learn the NMF decomposition by running the algorithm for 100000 iterations, with beta 2 and a tolerance of le 6.
Details for Comparing with the Supervised Approach When learning directions with IBD, for each concept, a dataset is assembled that includes up to 20 times more negative samples than positive ones to address the significant imbalance. Additionally, we use hard negative mining as originally suggested Zhou
ct al. 2018))
Figure 25 plots histogram of classification metric differences between the Linear-OR set of classifiers and the classifiers learned in a supervised way. The differences are based the concept labels, effectively taking the difference of metrics that regard two classifiers (the first from the Linear-OR set and the second from Zhou el al. 2018 with the same concept name.
Figures 26
28 depict concrete binary classification metrics for some the concept detectors in the Linear- OR sct classifiers, comparing them with concept detectors learned with supervision.
27
52

---

Published in Transactions on Machine Learning Research (02/2026)
Precision for Specific Broden Labels
Recall for Specific Broden Labels
windowpane
windowpane
fire_ _eeecppeee
fire eecessccppsss
bedchamber-s
bedchamber-s
meshed
Breden Labels
meshed
dolmen-s
dolmen-s
Broden
blotchy
blotchy
bookstore-s
bookstore-s
attic-s
attic-s
track
track
arch-s
arch-s
0.0
Precision
Recall
Classifiers
Classifiers
Supervised
EDDP-C
Supervised
EDDP
F1Score for Specific Broden Labels
windowpane
fire_ fre_escape-s
bedchamber.s
Broden Lahels
meshed
dolmen-s
blotchy
bookstore-s
attic-s
track
arch-s
0.2
F1Score
Classifiers
Supervised
EDDP-C
Figure 26: Interpretability Comparison. Exact Precision/Reeall/FI1. for specific concepts Broden: comparison between the Linear OR set of classifiers learned with EDDP-C, = 512, and classifiers learned in supervised way (IBD Zhou al. 2018)). ). The network here ResNet18 trained on Places365.
53

---

Published in Transactions on Machine Learning Research (02/2026)
Precision for Specific Broden Labels
Recall for Specific Broden Labels
boxing__rng-+s
boxing_rrng--
bamns
bam-s
batters_ box-s
batters_boxxs
batters_bock box-6
Hroden Labels
oast_ house-s
oast house-s
court
court
fountain-s
Broden
fountain-s
cuuseum-indoor--
museum- -inooon-s
supermarket-s
supermarket-s
bistro-indoor-s
bistro-indoor-s
catwalk-s
catwalk-s
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
Precision
Recall
Classifiers
Classifiers
Supervised
EDDP-C
Supervised
EDDP-C
F1Score for Specific Broden Labels
gaxing_ring-s
bam-s
batters_bbor box-s
Broden Labels
oast oast_ house-s
court
founlain-s
museum-Indoor-s
supernarket-s
bistro-indoor-s
catwalk-s s
0.0
0.2
0.4
0.6
0.8
F1
Classifiers
Supervised
EDDP-C
Figure 27: Interpretability Comparison. Fxact Prrccsion/RReall/FF1e for specific concepts in Broden: comparison between the Linear- OR set classifiers learned with EDDP-C, 512, and classifiers learned supervised way (IBD Zhou et al. 2018)). The network here is ResNet18 trained Places365.
54

---

Published in Transactions on Machine Learning Research (02/2026)
Precision for Specific Broden Labels
Recall for Specific Broden Labels
cavernindoor-s
cavern-indor-s
screen door
screen door
creek-8
Benden Lahels
Brodkn Labris
markel-indoor-s
market-indoor-s
loeberg
iceberg
rubbish
rubbish
subway station platform-s
seovay_statoo-plaaffrre
canal-urban-s
canal-urban-s
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.8
0.8
1.0
Precision
Classifees
Classifer
Supervised
EDDP-C
Supervisod
EDDP-C
F1Score Specific Broden Labels
cevernindoor-e
door
creek-s
slum-e
merkee--nddor--
Boolen
iceberg
rubbieh
subwayy_elaloon-pettooo
cansl-urban-e
0.0
0.2
0.4
0.6
0.8
1.0
FiScure
Classifiess
Supervised
FDDP-C
Figure 28: Interpretability Comparison. Exact Precision Precision/Reeall/FF /aaastnn/RRalll//Sooreee for specific concepts Broden: comparison between the Linear- OR set of classifiers learned with EDDP-C, 512, and classifiers learned in supervised (IBD Zhou et al. (018)). The network here is ResNet18 trained on Places365.
55

---

Published in Transactions on Machine Learning Research (02/2026)
A.14 Hyper-Parameter Study with Respect to Target Separation Margin
Table 17: Comparing EDDP variants based on varying target margin loss mm
The network here is ResNet18 trained
Places365 and I = 448.
ResNel18
Places365
Method
ym
Coverage
Redundancy
Precision
Recall
F1
AP 1
M
S
S
mloU
EDDP-U
4.0
0.55
1.27
0.8240.21
0.23+0.18
0.32+0.19
0.54+0.19
6.62
45.92
32.29
0.12
EDDP-U
5.0
0.88
1.32
0.8310.19
0.2210.16
0.3210.18
0.53.10.19
6.64
47.5
32.07
0.11
EDDP-U
6.0
0.86
1.32
0.8
0.21
0.3110.17
0.52.10.19
6.7
48.84
32.44
0.11
EDDP-C
4.0
0.86
1.27
0.8340.21
0.2210.17
0.31+0.19
0.52:0.2
6.67
11.71
31.73
0.11
EDDP&C
5.0
1.33
0.8210.2
0,2110166
0.3110.18
0.49:0.19
6.72
46.3
31.34
CDDP-C
6.0
0.86
1.31
0.8
0.210.15
0.291
0.5.10.18
6.75
46.01
30.97
0.1
Table 18: Comparing EDDP variants based on varying target margin loss mm trained on ImageNet and I 1120.
The network here Efficient.et
NeftcientNet ImageNet
Method
pmom
Coverage
Redundancy
Precision 1
Recall
Fl
APT
M
S
5'
mloU
EDDP.U
4.0
0.85
1.54
0.7710.22
0,22
0.3
0.4210.19
6.74
37.45
23.0
0,03
EDDP-U
5.0
0.85
1.61
0.76+0.21
0.2+-0.17
0.284-0.17
0.30++0.17
6.73
31.36
16.54
0.03
EDDP-U
6.0
0.85
1.6
0.76+0.21
0.22=0.17
0.3=0.17
0.39+0.17
6.72
30.46
16.09
0.03
EDDPEC
4.0
0.85
1.52
0.7716.22
0.22
00311011
0.4216.19
6.75
54.93
22.28
0.03
EDDP-C
5.0
1.54
0.74++0.22
0.24-018
0,2840.19
0.30-+0.17
28.62
16.38
EDDP-C
6.0
0.85
1.57
0.75.10:21
0,2120.18
0.2910.17
0.39.10.17
6.77
28.24
16.19
0.03
Tables 17 and 18 consider varying the target margin 7m when learning EDDP. For EfficientNet, increas- ing the larget classification margin (i.e., the linear separability of patch embeddings) leads to substantial improvements in both 8 and S for both EDDP-C and EDDP-U. We stress that increasing the margin is accomplished by decrcasing 7mm, as the margin is inversely proportional to the target loss. For RcsNet18, the results are more mixed: for either EDDP-U or EDDP-C, decreasing the margin mostly improves S while keeping similar S We emphasize that when mm 4.0, the results for EfficientNet are substantially better than the metrics we reported in the main body of the paper, for which we used 7m 5.0 (Section A.13
A.15 Ablation Study with Respect to Signerl Orthogonality Loss
Table 19: Ablation study of the method with respect to Uncertainty Region Alignment Losses Lur L"" and the use of Filter-Signal Orthogonality Loss Lo The network here ResNet18 trained Places365.
ResNet18 / Places365
Cour
cour
Cire
Coverage
Rodundancy
Precision
Rrcall
AP 1
f
S
mIol 1
0.86
1.29
0.82+0.21
0.2310.16
0.33:0.18
0.53:0.19
6.61
41.73
28.01
0.12
0.38
381
x
/
x
0.86
1.26
0.82:10.22
0.22+0.16
0.32-0.18
0.52-0.19
6.62
10.13
27.17
0.12
0.6
x
0.86
1.3
0.81++0.21
0.23+0.16
0.33-0.17
0.52+0.19 19
6.65
41.43
27.58
0.12
0.61
x
/
0.86
1.32
0.81+0.21
0.23+0.16
0.33=0.17
0.5+00119
6.67
41.5
27.
0.12
0.61
x
X
0.86
1.32
0.83:10.19
0.2210.16
0.3210.18
0.53:0.19
6.64
17.3
32.07
0.11
0.39
0.66
1.29
0.63.10.19
0.2110.16
0.310.18
0.521..19
31.1
0.11
418
/
X
0.86
1.33
0.82+902
0.22:0011
0.32:0.18
0.51=0.19
6.68
47.57
31.95
0.11
0.61
x
/
V
0.86
1,33
0.8210.2
0,2110.16
0.3110.18
0.49:10.19
6.72
46.3
31.34
0.11
0.61
1.28
0.62:10.22
0.2110.17
0.3110.19
52.51
0.11
0.61
X
/
X
0.85
1.28
0.82++0.3
0.21+0.17
0.3+0.19
0.51+0.2
6.67
50.03
37.35
0.1
0.62
/
X
0.86
1.3
0.82.10.21
0.2110.17
0.3110.19
0.48.1.22
6.72
52.3
37.71
0.1
0.62
1.28
0.8110.21
0.210.16
0.2920.19
0.17=0.2
36.99
In this Section conduct detailed ablation study with respect to the use of Filter-Signal Orthogonality
Loss (o0)) that was introduced in Section 4.3 In Section A.12 we discussed the impact of L on the faithfulness of the direction pairs, while in this Section we discuss the effects of L in interpretability and depict results in terms of interpretability and influence metrics.
influence terms. Tables 19 20
The main observations can be summarized to the following: a) When L0 not used, using any of the two Uncertainty Region Alignment variants, in the majority of the cases, results in comparable sets of directions, with Luur typically leading to more interpretable directions than Leur, b) In case L is taken
56

---

Published in Transactions on Machine Learning Research (02/2026)
Table 20: Ablation study of the method with respect to Uncertainty Region Alignment Losses Luar, ,Lar and the use of Filter-Signal Orthogonality Loss L The network here is EfficientNet trained on ImageNet.
ElficieutNee
ImageNet
Pour
Coor
Covirage
Redundancy
Precision
Recall
F1
AP
M
S
S
mioU
X
0.85
1.51
0.75:0.22
0.2:0.17
0.27=0.17
0.3820.16
6.74
27.92
14.71
0.03
0.91
0.85
1.48
0.74200
0.2:0.18
0.27:0.17
960
/
X
V
0.85
1.49
0,71=0.22
0.2:0.18
0.27=0.17
0.38=0.17
6.76
26.74
14.58
0.03
0.91
x
/
/
0.85
1.48
0.72++0.23
0.2+0.18
0.27+0.17
0.38+0.17
6.75
25.92
14.45
0.03
0.94
0.210.17
1120
X
V
X
0.85
1.56
0.76=0.21
0.2:0.18
0,28=0.17
0.3820.17
6.72
29.79
16.39
0.03
0.93
/
0.85
1.56
0.742:0.22
0.2:0.18
0.28:10.18
0.39:10.17
6.76
29.54
16.48
0.03
0.94
V
0.85
0.74
0.2:0.18
0,28=0.19
0.39:0.17
/
X
X
0.85
1.66
0.74+0..21
0.19-0.16
0.27+0.16
0.36-+0.17
6.81
40.38
19.12
0.03
0.94
1280
V
0.85
1.67
0.741...1
0.1910.17
0.2710.17
0.36.10.17
6.8
36.5
18.67
0.03
0.93
0.85
1,63
0.73+0.2
0.19-0.17
0.26+0.17
0.36-0.17
8.9
37.67
18.06
0.03
X
V
V
0.85
1.6
0,7320021
0.19
0.26.10.17
0.36.10.17
6.88
36.95
18.98
0.03
0.94
Table 21: Ablation study of the method with respect to Uncertainty Region Alignment Losses Luur Leur and the use of Filter- Lighr- Orthogonality Loss f The network here Inception-väs v3 trained on ImageNet.
Inception-v3 v3
ImageNet
Coverage 1 Redundaucy
Precision
Recall
AP
M
S'
S"1
mloU
X
0.87
2.41
0.84+0.18
0.28-0.24
0.38-+0.26
0.33++..6
6.79
156.46
28.03
0.11
0.93
x
X
0.85
2,09
0.85:10119
0.3.0.23
0.4:0.25
0.56.10.25
6.86
156.42
27.88
0.11
0.93
1536
X
0.85
1.9
0.85=0.21
0.22=0.16
0.33=0019
0.3720.21
6.88
123.85
27.08
0.08
0.94
X
V
0.85
1.9
0.86:0.19
0:212:1166
0.33:10.19
0.5810.24
6.87
125.64
27.46
0.08
0.94
/
X
X
0.9
2,96
0.8610.11 16
0.32510.25
0.43:10.26
0.38.10.25
6.77
210.44
25.77
0.13
0.53
1792
2.45
0.88
0.33-0.25
208-6
24.98
0.12
0.87
2.23
0.8910.16
0.24110116
0.35
0.62.0021
6.89
173.17
24.93
0.1
0.94
X
/
/
0.86
2.19
0.89+0.16
0.24+-016
0.36++0.19
0.62-40.21
6.92
174.0
24.69
0.1
0.94
0.86
2.36
0.85+0.2
0.31=0.29
0.4+0.29
0.54=0.28
170.81
32.13
0,09
X
V
X
0.9
4.85
0,77=0.11
0.3=0.27
0.39
0.18=0..2 26
6.91
165.65
30.12
0,08
0.91
2018
x
/
0.92
4.0
0.75+00.15
0.16++0:1
0.24+0.14
0.38++0.16
7.09
92.53
31.14
0.05
0.94
V
0,75=0018
0.16=0.12
0.24=0.11
0.38#0.16
31.18
into account, the use of Cour can significantly improve interpretability compared to Laar (S) in VGG-16 and ResNet50) or remain comparable to Laur (ResNet18, Efficiint.Net, and Inception-v3). c) When comparing among the same type of Uncertainty Region Aligument with and without Lfs0, in most cases, the use of f hurts interpretability (especially F1-Score and S') with the latter being more prominent. in Inception-v3 = 2048) and all cluster sizes of VGG-16 and ResNet50. An interesting exception is that the use of L improves cluster diversity (S²) in ResNet50. d) Under the same L0 setting, using Lour leads to directions with greater significant influence (SDC & SCDP metries) compared to Luur, except for when using Li in ResNet50. e) When considering the same Uncertainty Region Alignment loss variant, the use of [5so leads to an improvement in the SDC and SCDP metrics, with a notable exception being when using Leur ResNet18.
57

---

Published in Transactions on Machine Learning Research (02/2026)
Table 22: Ablation study of the method with respect to Uncertainty Region Alignment Losses L C and the use of Filter-Signal Orthogonality Loss feo. The network here is VGG16 trained on ImageNet.
VGG16 ImagoNNe
Dour
pour
Five
Coverage
Redundancy
Procision
Recall
F1
AP
A-
St
S"
mlol
I-
x
K
0.82
1.17
0.61+0.18
0.18+0009
0.27+0.09
0.31-0.12
6,88
29.98
9.39
0.08
0.91
/
0.8
1.09
0.6240.17
0.1740.09
0.26=0.09
0.31:0.11
0.07
/
x
0.79
1.05
0.6-0..17
0.16+-0.9
0,24:-0.09
0.31-0.11
7.12
23.42
8.51
0.06
0.9
x
/
0.79
1.07
0.63:+0118
0.16+0.09
0.2440.09
0.31-0.12
6.92
26.23
8.89
0.07
0.81
1.35
0.5920018
0,26:0.09
0.3:0.11
0.91
V
X
0.82
1.22
0.6:20,18
0.172:0.8
0,252:009
0.3:0.11
6.92
32.18
9.53
0.07
0.9
448
X
0.82
1.18
0.5840.17
0.15+0.08
0,23=0.09
0.3+0.11
7:14
26.72
8.75
0.06
0.9
x
V
0,82
1.22
0,6110,18
0,1620.05
0.2420.09
0.3:0.11
6,93
30.91
9.26
0.017
0.9
X
X
0.85
1.43
0.6110018
0.1810.08
0.27
0.3110.11
6.95
41.2
11.11
0.08
0.91
0.26:0009
0.34-0.11
30:23
10.48
512
X
0.85
1.35
0.5911018
0,15100.0
0.225:0.0
0,29240.11
7.23
28.62
10.4
0.06
0.9
X
0.85
1.37
0.62:10.18
0.15.10.09
0.241:0.09
0.3.0.11
7.08
33.91
10.83
0.07
0.9
Table 23: Ablation study of the method with respect to Uncertainty Region Alignment Losses L Leur and the use of Filter-Signal Orthogonality Loss fao The network here ResNet50 trained on Moments in Time.
ResNet50
MET
1
Dour
Dour
Coverage
Redundancc
Precision
Recall
FT
AP
M
St
S"
mioU
2
X
0.84
2.13
0.72=0.22
0.17=0.12
0.26=0.15
0.3620.16
6.25
127.83
33.67
0.03
0.85
0.83
2.33
0.75+0.16
0.17-0.12
0.26-0.15
0.35+0.15
6.25
141.4
30.12
0.09
0.85
1636
/
X
/
0.88
2,39
0,74110.19
0.17
0.25.10.14
0.4410.14
6.59
94.24
35.05
0.06
0.84
2.37
0.75-0.19
0.17=0.11
98.52
34.81
0.06
0.85
2.47
0.78=0.16
0.19=0.10
0.29-4013
0.4:0:14
6.24
162.86
34.17
0.85
x
V
X
0.83
2.61
0.7620.11 16
0.19=0.11
0.29=0.14
0.3820.15
6.27
161.82
30.63
0.09
0.85
1792
2.66
0.75+00117
0.16+00.11
0.25++013
0.43-+0.13
6.55
109.36
36.48
x
V
/
0,88
2.62
0,76 16
0, 0.16=0.11
0:25=0013
0.44=0.13
6.5
113.9
36.68
0.06
0.84
/
X
X
0.86
3.15
0.722002
0.2120.13
0.3120.16
0.13:0.11
6.29
209.2
33.01
0.1
0.85
2048
205.22
12.78
/
0.89
2.9
0.72
0.15=20.11
0.23=0012
0-11=0.13
6.61
1201.5
35.09
0.06
0.84
0.88
2,83
0.72:80.16
0.152:011
0.24:10122
0.43:10.13
6.61
126.46 46
35.98
0.06
0.84
Table 24: Ablation study of the method with respect to Uncertainty Region Aligument Losses Luur Leur and the of Filter-Signal Orthogonality Loss f The Table depicts metrics statistical significant influence. For R8sNet18, 7 = 448, for EfficientNet, 1120 and for ResNet50, 1792.
ResNet18
Places365
EfficientNet ImageNet
ResNet50 / MIT
oour
SDC
SCDP
SDC
SCDP
SDC
SCDP
X
261
1565
107
213
393
570
x
322
2285
426
714
1154
2250
296
1786
668
1445
1676
7946
X
296
1892
673
1484
1353
3913
58

---

Published in Transactions on Machine Learning Research (02/2026)
A.16 Comparison with Unsupervised Interpretable Basis Extraction and Concept-Bas Basis Extraction in Practical Experiments
Table 25: Comparison and Ablation with respect to method enhancements that propose in this work. L stands for Augmented Lagrangian Loss and _ refers to the CNN Classifier loss that was proposed in Doumanoglou et al.) (2024). Methods below the dashed line correspond to variations that the contributions make this work. Here the network is ResNet18 and 512.
ResNet18 Places365
Method
Ortho
L
L
L
Lour
S²
S
UTBE Doumanoglon et al.
2023
X
X
X
60.93
28.39
CBE
Doumanogion et al.
2024
x
x
X
69.43
31.53
CBE
2aur
7
K
7
X
67.3
322.6
EDDI-U
X
X
/
x
52.51
37.78
EDDP-C
X
/
50.29
36.99
Table 26: Comparison and Ablation with respect to method enhancements that we propose in this work. L stands for Augmented Lagrangian Loss and e refers to the CNN Classifier loss that proposed in [Doumanogoou al.) (222). Methods below the dashed line correspond to variations that use the contributions we make this work. Here the network is ResNet50 and = 2048
ResNet50 / MIT
Method
Ortho C
L
2uur
Cewe
S
S
UIBE Doumanoglou et
2023
X
X
124.73
18.47
CBE
Doumanoglon et
2024
131.73
26.94
CBE/W w Zime
x
x
158.76
33.02
EDDP-U
X
/
x
/
x
209.2
33.04
EDDP-C
x x
/
X
126.46
35.98
Tables 25 and 26 briefly present comparisons our work with the provious works that we extend, in terms of interpretability. First, with other aspects of Concept Basis Extraction (CBE) (Doumanoglou et al. 2024 being intact, we assess the efficacy of our Unconstrained Uncertainty Region Alignment. loss in improving the interpretability of the clustering, compared to the CNN Classifier Loss _ that was proposed previously al. 2024)). This case is referred to in the tables as CBE /wu Luur, We observe that in
in Doumanoglou three out of the four cases, our Uncertainty Region Alignment. loss leads to notable relative improvement, by up to 22.56% in S², which indicates clustering with improved concept diversity. When we additionally take into account the rest of our contributions, and compare EDDP-U with CBE, we find that in the same three out four cases, interpretability is further improved by up to 58.8% in terms of S Transitioning from CBE EDDP-C, may lead to further improvements in 2 by up 33.5%, yet with a decreased score in terms of St
A.17
More Qualitative Segmentations and Statistics for Evaluating the Interpretability of the Concept Detectors
depict qualitative segmentation results obtained using the con- cept detectors that were learned with the proposed method for the architectures that we studied in this work. Visualizations are obtained using Bau et al. (2017). We can verify that concept detectors appear to be monosemantie. In some cases, there are more than one concept detector detecting the same concept. However, in most cases, the sets of positively classified samples across detectors are disjoint. Figures 35 and 36 depict qualitative segmentations obtained via NMF and PCA. In some of the depicted examples, is evident that the identified concepts are less monosemantic. Tables 27 28 and 30 summarize the statistics obtained by Network Dissection regarding the interpretability of the clusterings considered in this work.
Figures 29
30
31
59

---

Published in Transactions on Machine Learning Research (02/2026)
airplane
aquarium
Classifier Index: 328
le1:0.144
Classifier Index: 271
loU: 0.15
Classifer Index 251
bathroom
0.38
Classifier Index: 442
bed
Iol: 0:33
Classmer Indexc 174
bicycle
IoU: 0.19
Classifier Index: 417
bird
lol: 0:17
Classifier Index: 295
bottle
leU:0.19
Classifier Index: 401
building
loU: 0.21
KULD
bus
bus
dining-room
car
Classifier Index: 261
hair
leV: 0.03
Classifler Index: 250
house
1o0. 0.1
Figure 29: Qualitative sogmentations using the concept detectors learned with our method. Here the network is
ResNet18 18 trained on Places365, the method is using CFM, and I = 512.
60

---

Published in Transactions on Machine Learning Research (02/2026)
living-room
living-room
Classñer Index 479
loU:
Clazsifier Index: 500
IoU: 0.24
Clessifier 234
motorbike
loU: 0.14
Clessifler Index:
mountain
loU: 0.31
Clessifer
person
leV: 0.32
Claseilier Index: 349
shoe-shop
loli 0.31
sky
skyscraper
Clessiffer Index:
0.48
Classifier Index:
Classifeer Index 488
tent
leU:0.29 0.29
Classiffer Index: 48
track
Iol: 0.14
Classifier Index: 167
track
0.22
Classifier Index: 230
train
lol:: 0.12
1000
train
tv-monitor
Figure 30: Qualitative segmentations using the concept detectors learned with method. Here the network is ResNet18 trained on Places365, the method is using CFM, and I = 512.
61

---

Published in Transactions on Machine Learning Research (02/2026)
Classñer Index 654
airplane
0,09
Clasaifier Index: 607
person
IoL:
Clessifier Index 004
bathroom
IoU: 0.16
Classifler Index: 671
van
IOU:
bed
0.25
Claseilier Index: 501
bicycle
lol:
Clessiffer
bicycle
leU:
bird
building
Classifier
bus
Classifier Index:
car
loU:
Classifier Index: 1221
car
Iol::
Clessifee
cat
IsU: 0.38
Classifier Index: 472
cat
Figure 31: Qualitative segmentations using the concept detectors learned with method. Here the network is EfficientNel trained on ImageNet, the method is using CFM, and I = 1280.
62

---

Published in Transactions on Machine Learning Research (02/2026)
Classifer 1136
ear
0.1:
Classifler 1156
ear
Classifier Index: 890
food
loU:
Classifier Index: 387
hair
loU:
Classifier Index 754
horse
1eV:0.17
Classifier Index: 825
leg
loU: 0.04
Classilier Index: 118
motorbike
leU:
Classifier Index: 1188
motorbike
lol: D.13
muzzle
muzzle
Classifer Index 347
tree
1.2:
Classifier
skyscraper
pool-table
pool-table
Cleseifier
Clessifier
Figure 32: Qualitative segmentations using the concept detectors learned with our method, Here the network is EfficientNet trained on ImageNet, the method is using CFM, and I 1280.
63

---

Published in Transactions on Machine Learning Research (02/2026)
Classifer
person
Classifier
van
Classiler Index: 1038
food
loU: Classifier Index: 479
food
Ial:
Classifer Index:
food
loU: Classifiar Index: 710
plate
IoU: 0.0
Classñer Index
plate
loU: Classifier Index: 78
fur
Iol:
Classifier Index 463
fur
loU: D.D
Classifler Index: 927
fur
Ial: 0.0
hand
Classifier
hand
Clessifier Index: 1134
muzzle
loU: D.D
Classifler Index: 238
muzzle
lol: 0.0
Figure Qualitative segmentations using the concept detectors learned with our method. Here the network is EfficientNet trained on ImageNet, the method is using CFM, and / = 1280. All these concept detectors exhibit IoU scores less than 0.04 and Network Dissection does count them as interpretable. Yet, many these detectors still detect monosemantic concepts.
64

---

Published in Transactions on Machine Learning Research (02/2026)
airplane
1620
bathroom
Classifier Index: 1553
bed
leU: 0.35
Classifer Incex: 1365
cat
IoU: 0:19
dining-room
Classifier Index: 1857
COW
0.32 0.32
Classifer Incex: 1724
loU: 0:18
Classifier Index: 1575
dog
leU:
Classifer Incex: 1508
horse
lol: 0.36
Classifier Index: 1408
motorbike
IeU: 0.43
Ciassifer Index: 1734
mountain
lol: 0.32
nursery
office
pool-table
Classifer
road
sheep
Clessifer
train
Figure 34: Qualitative segmentations using the concept detectors learned with method. Here the network is Inception-v3 trained on ImageNet, the method using UFM, 1792.
65

---

Published in Transactions on Machine Learning Research (02/2026)
Classifier Index: 1402
airplane
leU: 0.2
Ciassifer Incex: 1337
bed
lol: 0.08
Classifier Index: 1366
cat
leU: 0.14
Ciossifer Index: 577
COW
lol: 0.09
YORK
Classifier Index: 844
dog
loU; 0.07
Classifer Index: 875
horse
1s 0.2
Classiffer
motorbike
loU:
Clessifer
mountain
lol: 0.07
Classiffer Index: 1133
nursery
IeU: 0.03
Classifer Incex: 782
office
loU: 0.07
BEO
Classifier Index: 576
pool-table
IeU: 0.21
Clessifer Index: 1783
road
IoU: 0.04
Classifier Index: 1334
sheep
1.17
Clessifer Index: 201
train
1.110.11 0.11
Figure 35: Qualitative segmentations using the factorization of NMF, Here the network is Inception-v3 trained on ImageNet and 1 = 1792.
66

---

Published in Transactions on Machine Learning Research (02/2026)
Classifier Index:
airplane
b0: 0.11
Classlfer Index::
bathroom
IoU: 0.18
bed
0.21
ClassiTer Index:
dog
lol:
Classifier Index: 60
horse
0.07
Classifer Index: 148
motorbike
Iol: 0.08
Classifier Index:
mountain
leU:
Classifer Index: 35
poolroom-home
loU: 0.1
Classifier Index: 332
torso
IcU: 0.05
Classifer Index: 56
train
lol:: 0.05
Qualitative segmentations using the concept detectors learned with PCA. Here the network is Inception-v v3 trained ImageNet, and / 1792.
Figure 36:
67

---

Published in Transactions on Machine Learning Research (02/2026)
Classifier Index: 114
bus
TeV:
Ciseelfer Index: 301
carpet
loU: 0.08
Classiffar Index: 64
ear
Idl: 0.11
Ciassifer Index: 271
food
Ind: 0.08
mountain
muzzle
Classifier Index: 385
person
0.18 0.18
Ciassifer Incex: 443
road
lol::
E
sky
Classifier Index: 344
sea
IcU:: 0.18
Clsssifior Index: 3G8
lol: 0.44
Classiffer Index: 374
street
loU: 0.2
Clessifer Index: 198
wheel
IoU: D:18
Figure 37:
Qualitative segmentations using the concept detectors learned with our method. Here the network is VGG16 trained ImageNet, the method is using UFM, and = 448.
68

---

Published in Transactions on Machine Learning Research (02/2026)
Classifier Index: 1032
airplane
le2:0.24
Clessifer Index: 1590
bedclothes
IoU: 0.07
bicycle
bowling
Classifier Index: 1512
loU: 0.3
Classifer Index: 1610
KoU: 0.33
Classifier Index: 1505
burning
leV: 0.14
Ciassiferr Incex: 1348
bus
Inl:: 0.22
Classifarr Index: 1181
car
IoL: 0.13
Cisselfer Index: 1375
cat
IoU: 0.25
chequered
crying
Classifler Index: 1216
loU: 0.85
Classifer Index: 1194
IoU: 0.1
dog
gambling
Classiñier Index: 762
IcU:
Clessrier Incex: 1718
Iol: 0.25
grooming
highway
Classifier Index: 1075
loU: 0.1
Classifier Index: 911
1.1:
Classifier Index: 1546
horse
IeU: 0.15
Classifer Index: 1398
juggling
lol: 0.08
Figure 38: Qualitative segmentations using the concept detectors learned with our method. Here the network is ResNet50 trained on MiT, the method is using UFM, and / = 1792.
69

---

Published in Transactions on Machine Learning Research (02/2026)
Classifier Index: 1120
laughing
IoU: 0.06
Clessifer Index: 1288
motorbike
IoU: 0.24
Classiffer Index: 1396
muzzle
1oV:0.15
Classifer Incex: 1734
painting
IoU: 0.1
playing music
racing
Classifier Index: 1290
IcU: 0.00
Classifer Incex: 1411
loU: 0.12
rafting
sky
Classifier Index: 1163
0,2:
Clessifior Index: 1702
loU: 0.53
Classifier Inde:: 1645
skyscraper
IcU: 0.26
Classifirr Incex: 1674
street
lol: 0.23
swimming pool-indoor
telephoning
Classifier Index: 1892
IcU: 0.00
Classifer Index: 1128
lol: 0.12
Classifiar Index: 1886
track
IoL:: 0.18
Ciassifer Index: 1284
train
IoL:: 0.05
tying tie
typing
Classifier Index: 1309
0,12 0,12
Cicssifer Index: 29
lol: 0.18
Figure 39: Qualitative segmentations using the concept detectors learned with our method. Here the network is ResNet50 trained on MiT, the method is using UFM, and / = 1792.
70

---

Published in Transactions on Machine Learning Research (02/2020)
Table 27: Network Dissection statistics EfficientNet trained on ImageNet. each concept category Broden, report the two numbers: First, the number of concept detectors that were labeled with the name of concept belonging the category and second, number unique concept labels from the category that have been assigned to the set of the concept detectors.
NeficientNet
ImageNet Material
Method
Color
Object
Part
Scene
Texture
Total
PCA
0/0 0
58 15
58 /
0 0/0
14
44/6
11 /
141
960
EDDP-U
0/0
134 / 51
14
4 4/4
34
26
21 / 17
207
107
EDDP-C
0
120 / 50
13 1
5/4
35
21
194
105
PCA
0/0
57
81
0.0
16 /6/
21
176
34
EDDP-U
0/0
137 /
/ 12
5/5
39 /
/ 16
225
118
EDDP-C
0/0
117 / 52
18 9
7/5
40
18 / 16
200
117
Natural
0/0 0
155 / 29
28
11
1/1
29
/
20
10
9
223
/ 70
PCA
0/0
58
15
80
/
0/0
14
5
27
14
179
38
1280
EDDP-U
0/0
164
/
62
74
14
5 /
48
40
22
/
18
313
137
EDDP-C
0
132
59
60
16
6
47
39
22
19
267
137
Table 28: Network Dissection statistics for Inception-v3 trained ImageNet. For each concept category in Broden, we report the two numbers: First, the number of concept detectors that were labeled with the name of a concept belonging the category and second, the number of unique concept labels from the category that have been assigned to the set of the concept detectors.
Inception-v3
Image
Method
Color
Object
Part
Material
Scene
Texture
Total
PCA
0 0/0
58
15
60
0 0
18
155 /
26
291
50
NMF
0 0/0
325
44
14
4 4/2
210
58
174
42
727
151
1536
EDDP-U
0 0/0 0
763
73
16 0/11
8 8/7
111
81
55
36
953
208
EDDP-C
0 0/0
593
76
16 12
7
104
78
47
/
767
202
PCA
0/0
60
42
0
19
126
247
49
NMF
0/0
340
6
2/1
137
38
700
156
1792
EDDP-U
0/0
992
65
12/7
6
33
180
EDDP-C
0/0
877
/ 67
/ 10
6
/
72
32
182
Natural
0 0/0
394
42
13
/
2/1
270
56
154 / 34
833
138
PCA
0/0
37
15
58
0 0/0
19
8
153
/ 25
287
2048
EDDP-U
0 0/0
701
85
23
15
6 6/5
197
113
60
38
987
256
EDDP-C
0 0/0
488
75
22
13
7
263
113
73
38
853
243
Table 29: Network Dissection statistics for VGG16 trained on ImageNet. For each concept category in Broden, we report the two numbers: First, the number of concept detectors that were labeled with the name of a concept belonging the category and second, the number of unique concept labels from the category that have been assigned to the set of the concept detectors.
VGG16
ImageNet
Method
Color
Object
Part
Material
Texture
Total
PCA
0/0
171
/ 11
72
0/0
6 6/3 3
97
/
13
346 / 32
NMF
0 0/0
109
31
33 15
4/2
24
/4/10
15
45
18
215 /81
EDDP-U
1/1
51
36
203 /
1/1
4 4/3
20
19
280 / 73
EDDP-C
2 2/2
45
/ 33
199 /
1/1
4/3
17
/ 16
268 /
PCA
0/0
211
82 /
0/0
5/3
113 /
15
411
34
NMF
0/0
124
13
4/1
/
47
242
448
EDDP-U
0/0
63
5
/
18
333
EDDP-C
0/0
61
/
/ 12
2/1
7/6
15
323 / 67
Natural
0/0
169
/ 34
14
6/2
26
/ 17
63
23
312 / 90
PCA
0 0/0
263
11
84
0/0
4
117
18
470
38
512
EDDP-U
1/1
71
/ 40
267
14
2/2
11
/ 10
25
19
377
/ 86
EDDP-C
1/1
68
/ 40
260
2/2
10
26
19
367
85
71

---

Published in Transactions on Machine Learning Research (02/2026)
Table 30: Network Dissection statistics for ResNet50 trained on Moments Time. For each concept category Broden, we report the two numbers: First, the number of concept detectors that were labeled with the name of category and second, number unique concept labels from the category that have been assigned to the set of the concept detectors.
concept belonging
ResNet50
MIT
Method
Color
Object
Part
Material
Action
Scene
Texture
Total
PCA
0 0
15 / 9
5
0/0
1
14 5
657
29
692
46
NMF
0 0
214 / 32
18 /
0/0
279
85
109
44
308
31
928
197
1536
EDDP-U
1/1
585 / 54
4 4
1/1
509
134
128
60
44
1272
285
EDDP-C
1/1
22
1/1
/ 58
46
988
292
PCA
0
1 12
0/0
1/
13
683
719
NMF
0/0
/
23 / 5
1 1/1
468
111
99 / 46
144
934 / 220
1792
EDDP-U
0/0
/ 52
4
2 2/2
680
/ 135
161
67
35
23
1555
283
EDDP-C
0 0/0
541 / 51
29 /
1/1
302
/ 150
81
/ 63
41
/ 24
1195
295
Natural
0/0
282 / 34
2
1/1
348
93
121
43
362
29
1134
205
PCA
0 0
16 10
5 5
0/0
1
13
4
730
29
765
46
2048
EDDP-U
0 0
768 / 45
5
1 1/1
709
109
377
75
49
30
1909
265
EDDP-C
0
542
/ 49
112 /
2/2
648
136
94
66
51
1449
289
A.18
More Global Model Explanations via Concept Sensitivity Testing
This Section complements Section 7.8 Figures 40
and 42 depict Concept Influence Diagrams for classes of
ResNet18 trained on Placcs365. while Figures 4 and depict diagrams for ResNet50 trained on Moments
in Time.
72

---

Published in Transactions on Machine Learning Research (02/2026)
Concept Influence Diagram for Places-365 Class: fastfood-restaurann
Network: Resnet18
Concepts with Positive Influence
Concepts with Negative Influence,
tree-ToU:0:399
sky:IolU:0.45
bed-Tol::0:35
Street=s-loU:0.42
person-loU:0:32
waffled:
freckled
fsstfoodtrestarannt=:00000111111
car-loU:0:3
hellportss=toU000055
fastfood- rettrrrrraaaaangasssso0000020220
chequered-loU:00.4
cal-10U:0:28
sauna-s-10U:0:26
television sttiiss-o
cal-10U:0:26
street=s-lUU:.001
track-10U:0:24
Concept Influence Diagram for Places-365 Class: castle Network: Resnet18
Concepts with Positive Influence
Concepts with Negative Influence,
train-IoU:0::8
castle=s-IoU:0.18
tent-1oW::0:16
wall-foW::0113
www__windwwwooe httndbowwoutdorrss11oo:::1111111
dolmen:ss:IoU:0009
fastfood restuarrttrrrrttt-00000...11111
ruln=s-loU:0:07
counter:loU:0007
brick:1oU:00007
castle
porous:loU:0:07
arch-s-1ou:0.05
roof-10U:0:06
stern-1oU:0.05
hospital-s-100:00006
motorbikke-our0004
signboard-100:0105
street=s=loU:0.01
Industrial_park:-100000
Figure 40: Concept Influence Diagram for ResNet18 trained Places365, The model is sensitive to the depicted concepts with an absolute score above 0.99. (We use RCAV to quantify the sensitivity, and re-scale the score to [-1, Positive influencing and negative influencing concepts limited to 10. When concepls appear more than once, they correspond to different signal directions (as labeling the classifiers with NetDissect may assign the concept name to than directions.). Here report results for EDDP-C and I 512.
provided. The number of concepts have been
73

---

Published in Transactions on Machine Learning Research (02/2026)
Concept Influence Diagram for Places-365 Class: corridor
Network: Resnet18
Concepts with Positive Influence
Concepts with Negative Influence,
cockpit-s-ToUs00:46
dotted-ooU:O0.3
skyscraper-s:lolla00022
comnidon-s-loU:001
toal
cavermiindeor-s:00000.00
pool table-looo00
corridon=s-lou:00.66
tool
comnidorss:00:0:06
corridor
poolitableslo0:0::66
corridor=s100U00066
windmill=s-100:0225
comnidor=s-oo::0006
Inne=1ooU:0:2
corridor-s-oou:::04
1o0:00y
pool table-1oU:0:22
Concept Influence Diagram for Places-365 Class: reception Network: Resnet18
Concepts with Positive Influence
Concepts with Negative Influence,
cockpit-s-loU::::b
street-S-loU:00:42
bed-ToU:0:35
freckled-toU:0.3c-
pantry-s-lou:00299
witness_standd=-1o0l0001
kitchen-s.lol::0:27
reception-s-lou:0009
reception
cat:loU:0:26
counter-loU:0.07
falrway=s-1o0:0023
parking. hatpiing_aaaggiiooooor:::00....0
covered brte
bicycle-10U:0:18
Figure 41: Concept Influence Diagram for ResNet18 trained Places365, The model is sensitive to the depicted concepts with an absolute score above 0.99. (We use RCAV to quantify the sensitivity, and re-scale the score to [-1, Positive influencing and negative influencing concepts provided. The number of concepts have been limited to 10. When concepls appear more than once, they correspond to different signal directions (as labeling the classifiers with NetDissect may assign the concept for EDDP-C and I 512.
more than directions.) Here report results
74

---

Published in Transactions on Machine Learning Research (02/2026)
Concept Influence Diagram for Places-365 Class: train-stationn-lltform
Network: Resnet18
Concepts with Positive Influence
Concepts with Negative Influence,
pool table-ToU::::37
fibrous-ToU::::44
person-lou:0:32
toll
Water-loU:0:31
airport 5:00:00:05 trainnsstaionn-outtooo-/////s/s/ arch-s-1oU:0.01
tra Trahsstatoorplattoommasssaoo 0rass-oo0:0000000
waffled-toU0:0029
shoe_shop-=+1o0x0:28
ball 1::::000
stratified-100:0:277
Concept Influence Diagram for Places-365 Class: waiting-room Network: Resnet18
Concepts with Positive Influence
Concepts with Negative Influence,
washer-1oUJ:0:399
pool table-1oU::0377
bed-ToU:00:55
table--lo..
dotted=1oU:0041
pantry-s-lou:0:29
lwvingrroomms-10000222
pool table:lUUU:0022
pottedplant-loU10.11
walting-room
kitchen-s-loU000223
wall=foU:0.13
poolitable-tob:0::222
floor-10U:0.11
crevasse-s-100:0:22
dugout=s-loU:0.02
crosswalk-100:0:22
Figure 42: Concept Influence Diagram for ResNet18 trained Places365, The model is sensitive to the depicted concepts with an absolute score above 0.99. (We use RCAV to quantify the sensitivity, and re-scale the score to [-1, Positive influencing and negative influencing concepts provided. The number of concepts have been limited to 10. When concepls appear more than once, they correspond to different signal directions (as labeling the classifiers with NetDissect may assign the concept for EDDP-C and I 512.
more than directions.) Here report results
75

---

Published in Transactions on Machine Learning Research (02/2026)
Concept Influence Diagram for MiT Class: cooking Network: Resnet50
Concepts with Positive Influence
Concepts with Negative Influence,
dotted-IoU:0.51 chequered-ToU:0045
car-1oU::0:9
waving-Tol::0:244
5kyseraperr---o0000..11
mountaln-loU:0::22
polka.dotted..oU000.211
marching:loUs:0:22
waffled=toU::0:19
wheat_fieldss:/o00:00116
cooking
sky=IoU::014
protesting=loU::::16
kkttchen=s1oOU:0009
gambling-1o0:0:15
sprinkled=loU:0009
alming-10U:0:15
muzzle-1oU:0:13
person-1oU:0.04
spiralled-100:0:13
Concept Influence Diagram for MiT Class: painting Network: Resnet50
Concepts with Positive Influence
Concepts with Negative Influence,
boilding-IoU:0.25
skyscraper---lolb0::31
sky=loUU0..3
bicycle-1oU:0:29
Tol:0.1
fibrous-
house=Jol:00.11
rowing:loU:0:277
bathroomss-loU:00.00
drummings1oUs0:24
painting
cam:ooU:0:08
bowliig=1ol::0:24
sppayy_piinting-oouoo.00
car |nteeoorbaekkeaat---000001111111
ddrawing-1U:00.07
bicycle-16U:0019
car-to0:0.06
golf=pputting-oo::0.044
wreeting__tinn-/nnooor-::0000000
Figure 43: Concept Influence Diagram for ResNet50 trained Moments in Time (MiT), The model is sensitive to the depicted concepts with an absolute score above 0.99. (We use RCAV to quantify the sensitivity, and re-scale the to [-1, Positive influencing and negative influencing concepts have been limited to 10. When concepls appear more than once, they correspond to different signal directions (as labeling the classifiers with NetDissect may assign the concept name to than directions.) Here report results for EDDP-C and = 2048.
provided. The number of concepts
76

---

Published in Transactions on Machine Learning Research (02/2026)
Concept Influence Diagram for MiT Class: tying Network: Resnet50
Concepts with Positive Influence
Concepts with Negative Influence,
airplane-ToU:0.25
cat-ToU:0:28
Init:ing:ooU:0:15
cheerleading-lolua0226
tylng_tle-toU:0.12
beach-s-loU:1::22
tying=IolU:0.11
cat:IoU:0:211
tying=1oUs0:09
bubbly=1oU:0:19
tying
horse=IoU:0008
bullidozing-lo0:::188
forest-meeddlleeffsst00:000
rafting-loU:0:11
horse-1oU:0.07
pool tabl-
horse-1oU:0.06
cheerleading-1o000.11
gardeninng-oU:00066
moon_ -oon_bounces::10001117
Concept Influence Diagram for MiT Class: typing Network: Resnet50
Concepts with Positive Influence
Concepts with Negative Influence,
It:i
road-1oU:0034
polka-dottedFoU:00.22
grass-loU::0:33
typing=Iol::0.7
bowlnng
scaly=Iol:00.14
knitting:1oU:00:29
grooming=HoU:00.12
tattooing=IoUs0:25
typing
office:s:loU:0011
drumming-1o0:0:24
cdayppotter__maknn--oovo0
bowllng-loU:0:244
using_coomputertto00..009
freckled-1oU:0:44
lined-1oU:0.09
beach-s-10U:0:22
tvmonitor-loU:0.08
car car_Interlor- __,_-terrrrbbackkkttt--11000222
Figure 44: Concept Influence Diagram for ResNet50 trained Moments in Time (MiT), The model is sensitive to the depicted concepts with an absolute score above 0.99. (We use RCAV to quantify the sensitivity, and re-scale the to [-1, Positive influencing and negative influencing concepts have been limited to 10. When concepls appear more than once, they correspond to different signal directions (as labeling the classifiers with NetDissect may assign the concept name to than directions.) Here report results for EDDP-C and = 2048.
provided. The number of concepts
77

---

Published in Transactions on Machine Learning Research (02/2026)
A.19 More Local Explanations with Concept Contribution Maps
This Section complements Section 7.9 Figures
depict CCMs for the prediction of an image belonging to class beach-house. The respective image concept contribution scores are depicted in Figures 47 and [8] Figures 19 50 depict CCMs for the prediction of an image belonging to class bedchamber. The respective
45 46
image concept contribution scores are depicted in Figures 51 and 52
A.20 Details on Counterfactual Explanation Example
To compute the target projected values 6, we use the training split of Broden. When we perform concept removal (or addition), we choose t to be equal to the lowest (or topmost) 0.005% quantile of projected feature activations on the filter w, This process ensures that, after the intervention, the signal value the patch embedding for the concept matches the signal value of samples with or without the concept.
For intervention, make use of 33
A.21 Details on Toy Model Correction
Network Architecture After cach convolutional layer, except the last, there is Dropout layer with p 0.3. All Conv2d layers have a kernel size of 3x3 and stride of 2, except for the last one, which has stride of 1. Furthermore, the latent space dimensionality is set to 16 for all convolutional units.
Chess Dataset The total number of images in the dataset is 210, 67 for bishop, 71 for knight and 72 for rook. The spatial resolution of those images is 85x85. We make a stratified train-test split with the training set ratio set to 0.7.
Network Training We train the network with cross-entropy loss and the Adam optimizer with a learning rate of 0.005 for 1000 epochs. In the (poisoned) training set, the model achieves 100% accuracy.
Direction Learning To learn the latent dircctions, we apply our method by following the learning process described in section A.7. Furthermore, for stable learning, we learn the directions with the Augmented Lagrangian loss scheme of Section 1.5 We optimize x + Xewr [ with the target constraints 7m 0.8, T 0.01, ene 0.01, 7 mm 8.0, f 0.1, and weights 1 2.6 and Aour = 0.25.
A.22 Details on Computational Resources
We utilized dedicated computing rigs for our experiments. Experiments for ResNet18 and EfficientNet were run on single-GPU configurations: one NVIDIA A10G (24 GB) and one NVIDIA RTX 4090 (24 GB), respectively. For InceptionV3, VGG16, and ResNet50, we employed compute rig equipped with four NVIDIA A10G GPUs (4 x 24 GB). The runtime for each of the key processing steps (a), (b), and (d) detailed in Section A.7 varied by architecture: ResNet18 completed in under 2 hours; VGG16 in under I hour; ResNet50 in under 10 hours; InceptionV3 in under 13 hours; and EfficientNet in under 5 hours.
A.23 Relation to Sparse Auto-En Encoders
The superposition hypothesis Elhage et al. 2022 assumes that neural networks linearly represent. more features than there are neurons their hidden layers. In this line of work, a feature is defined to be an abstract property of the input, which may may not align with human intuition, and is exploited by the network to make predictions. Sparse AutoEncoders (SAEs) Bricken et al. (2023) have been proposed tool to take features out of superposition; that is, given the activations of a network's intermediate layer, SAEs try to linearly decompose these activations in terms of latent feature components under a sparsity objective.
Let æ E R denote the activation of a network in a hidden layer of study and I >> denote the number of latent features that the network assumed to represent. While several SAE variants have been proposed,
such Lim et al. (024)) Cunningham et al. (2024); Sharkey al. (2022): Bussmann et al. 2025 in their
78

---

Published in Transactions on Machine Learning Research (02/2026)
baseline form Bricken al. (2023 they first extract latent feature components v by:
ReLU enc bacc) bene)
(44)
and subsequently they aim to reconstruct the original activation by
a Wee bdec
(45)
with W Pxl Weec E Px1 beue E R' and bee RR. The objective that drives SAE learning linearly combines 1.2 reconstruction loss between and & and penalization in the L1 norm of
To relate SAEs with our approach, first need to make a correspondence in terms of terminology. First, SAE feature, which is an abstract property of the imput, in our terminology is referred to as a concept. The number of latent features is related to our concept cluster cound, and we refer to both of them as The encoder part of the AutoEncoder which is realized by the matrix Wene is related to our decoding directions W, while the AutoEncoder's decoder part W is equivalent to our encoding directions (signal vectors) S. The SAE's baec is related to our multi-concept signal-distractor data model's latent space bias c. Finally, feature latent is somewhat related to the definition of our signal values. In particular, under our proposed multi-concept hignal-distractor data model, W enc (x bate) may be considered mechanism to extract signal values.
Despite the conceptual similaritics between our method and SAEs, we also have several diffcrences. First, (44 conceptually acting as classifier with positive predictions whenever the extracted signal value exceeds the threshold Dene- From one perspective, the same equation acts like centering the signal value around the classification threshold for the presence of the concept in the representation. In SAEs, sparsity is enforced in the units of the latent features v, while in our approach, the ReLU activation function is replaced with sigmoid, and thus sparsity is enforced in the semantic space of concepts. Second, in SAFs, W are learned via a reconstruction objective, while in our approach, signal vectors are learned under the properties of a probabilistic model constrained on the extracted signal values and the feature activations themselves. Third, we also consider constraining each decoding direction to be perpendicular to the signal vectors of other concepts for exact signal value extraction. Finally, we also consider uncertainty region aligument, which additionally exploits the of the directions by the model.
79

---

Published in Transactions on Machine Learning Research (02/2026)
Une Kob 872
a5 say bobo
(owatho)
Lookationn
neo] sky Mil
lill
9,77
bouch 2
bo 0.12
sky 022
Figure 45: Left: Original image. The caption contains class prediction and output class logit. Top Row: Seg- mentation Maps obtained by the concept detectors. The caplion contains classifier index (unit), concept-nanee IoU score in the validation split of the dataset. Bottom Row: Concept Contribution Maps. The caption contains concept-name and contribution of the concept to class logit.
Usc 1T1 lantaa jll ozs
Hen lhd 812
Uht Leare
Hkit bottorn
Uni 477 Leporaze dll
Figure 46: Top Row: Segmentation Maps obtained by the concept detectors. The caption contains classifier index (unit), concept-name and IoU score in the validation split of the dataset. Bottom Row: Concept Contribution Maps. The caption contains concept-name and contribution of the concept to the class logit.
80

---

Published in Transactions on Machine Learning Research (02/2026)
Concept Analysis for Class Prediction: beach-house/ Astive Concept Contributions
Concept Analysis for Class Prediction: beach-house Active Concept Contributions
+008
+0.22
10. 16
skyseraper 9
horbor S
..744
-0.1
-0:37
+0.18
-0.00%
beachee 11.00
sky
H0.0
+2.00
10 104
8:11
+0.04
-7.5
-25 Contributio
Contribttinn
CCRC
CORC senple-conapp
sample-concept
boceline
bassline
Figure 47: Concept Analysis for predicting an image of the beach-house class, The figure depicts concepts found the image. Even though concepts may share the same name, they correspond to different direction pains.
Analyyis
Negative
Concept Anelysis
Positive
beacheg
lighthouze-s
7.90
10.00
24
+0
lghthouse
sand
-0.20
22.85
2070
Nghway-
2929
05
68.34
-0.17
-0.07
aiplane
pool table
00.46
+0.58
4.04
milttry_hokk hul-s
harbor-s
4.21
-10
Comtibution Score
Contribttion Score
CERC
CCRE
Figure 48: Concept Analysis predicting an image of the beach-house class. The figure depicts top positive and top negative contributing concepts. Even though concepts may share the same name, they correspond to different direction pairs.
81

---

Published in Transactions on Machine Learning Research (02/2026)
bedenamberr ceeking
plam
191 carpet
wol
celin| 1 0.69
plant 0.2
capet 1.59
windmilis 6.00
wail 3:0P
Figure 49: Left: Original image. The caption contains class prediction and output class logit. Top Row: Seg- mentation Maps obtained by the Concept Detectors. The caption contains classifier index (unit), concept-name and IoU score in the validation split of the dataset. Bottom Row: Coucept Contribution Maps. The caption contains coucept-name and contribution of the concept to the class logit.
1.
bed 1929
Ramijoae
Iino nervel
Figure 50: Top Row: Seguentation Maps obtained by the concept detectors. The caption contains classifier index (unit), coucept-name and loU score the validation split of the dataset. Bottom Row: Coucept Coutribution Maps. The caption contains none and contribution of the concept to the class logit.
82

---

Published in Transactions on Machine Learning Research (02/2026)
Concept Analysis for Class Prediction: bedchamber AAcive Concept Contributions
Concept Analysis for Class Prediction: bedchamber Active Concept Contributions
+0.75
bed
3.00
1lor_1
40.10
11.00
00.06
1027
calling
0,00
wall
4373
12.17
+0,06
windmills
60.20
40.24
+0.00
wing_roumee,
55
Toor 2
10
10060
-071
2/2//8
2,.0
s.ng_romm-__
carpet
+2:43
1.63
-50
Contribution
CCRC
OCRC sample-conoest
noppo-corocpp hosel
baneline
Figure 51: Concept Analysis for predicting an image the bedchamber class. The figure depicts concepts found the image. Even though concepts may share the same name, they correspond to different direction pairs.
Concept Analysie
Analysie
Negative
-19 74
+10.79
10.39
00.89
a
111.41
youth hostelis
6.87
+5 8262
youth hostel-s
+11.43
dorm, room-s
0.00
-11.68
10.00
living Wing_room-s
curlain
8001
208
00.02
40.50
deseroom-s
15 65
cradle
-20
-10
-10
Contribution Soore
Contribution Score
CERC
C2:4C
semplee-cnccpt
Figure 52: Concept Analysis for predicting an image of the bedchamber class. The figure depicts top positive and top negative contributing concepts. Even though concepts may share the same name, they correspond to different direction pairs.
83


Reviews:
[{"review_id": "Ts4mp0luEb", "reviewer": "Reviewer_Tp88", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a method (EDDP) to discover concept “encoding-decoding” direction pairs in deep vision models. First, an unsupervised clustering step finds decoding directions that behave like concept detectors. Second, a probabilistic signal–distractor model estimates corresponding encoding directions in feature space. A new Uncertainty Region Alignment loss encourages discovered concepts to matter for the classifier’s decisions by aligning regions where concept detectors and the classifier are uncertain. Experiments on standard CNNs report higher Network Dissection scores, more significant RCAV directions, and qualitative applications: concept segmentations, concept contribution maps for explanations, and a watermark-removal case study.", "claims_and_evidence_yn": "No", "claims_explanation": "Evidence for the key claims is limited:\n1. Faithfulness of encoding directions is only demonstrated on a synthetic linear model and via “dreaming direction” similarity, which is indirect and assumes the method’s own modeling choices.\n2. Improvements in interpretability rely almost entirely on Network Dissection metrics and cherry-picked qualitative figures, without human studies or causal evaluations of explanations.\n3. RCAV statistics show more directions passing significance tests but say little about practical effect sizes or comparison to simpler baselines.\n4. The watermark example is small-scale and lacks baselines, so it does not convincingly show that EDDP enables reliable correction of spurious correlations in realistic settings.", "audience_yn": "Yes", "audience_explanation": "Unsupervised concept discovery and mechanistic interpretability of deep networks are relevant.\nJoint learning of encoding and decoding directions, and tying them to classifier uncertainty, is conceptually interesting even if the current evidence is insufficient.", "requested_changes": "1. Clarify the algorithm, clearly separating prior components from novel ones and giving concise pseudocode for the whole pipeline.\n\n2. Add experimental comparisons against other sparse autoencoders/dictionary-learning methods.\n\n3. Analyze failure modes and sensitivity to hyperparameters and number of concepts.\n\n4. Simplify and ablate loss terms and augmented-Lagrangian constraints, showing which are truly needed.\n\n5. Quantify the quality of concept contribution maps and editing (e.g., explanation fidelity, success rates) beyond visual examples.", "broader_impact_concerns": "I see no unusual risks beyond those of standard interpretability and representation-editing work. The methods could help detect and correct spurious shortcuts, but might also be used to hide or tweak model behavior; these risks appear manageable with normal transparency and evaluation practices.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mHwueJ3Bfb", "reviewer": "Reviewer_PAQt", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a novel unsupervised approach for recovering semantic concepts as *encoding–decoding direction pairs* in latent space. These directions can quantify concept presence. \nThey introduce Uncertainty Region Alignment, a method that leverages information in the model’s weights to guide the search for interpretable, prediction-relevant directions. \nThey include empirical validation by demonstrating accurate recovery of ground-truth direction pairs in controlled synthetic experiments. Further, they show that the learned decoding directions correspond to monosemantic, interpretable concepts and outperform unsupervised baselines on interpretability metrics.\nFinally, they provide practical examples of applications illustrating how the learned directions can be used. \n\nStrengths: \n\n- The paper presents a method in the current topic of model interpretability. The method has some advantages over the popular methods such as SAEs and CAVs.\n- The authors perform several experiments for empirical validation on real and synthetic data and they demonstrate convincing application use cases for the method.\n\nWeaknesses: \n\n- The writing: This paper is verbose and very hard to read. The narrative isn’t very structured or linear. It is at times repetitive, and I don’t think it needs to be 40 pages. Parts of the main paper could also be moved into the appendix, e.g., some details in section 5. \nTo give specific examples, it is difficult to understand what’s going on at first (e.g., in the abstract, new terms and details are being introduced before we even know what the problem broad scope of the paper), the method, while simple, is not stated formally or in a way simple to understand (it is given kind of in bursts). The introduction in part blurs into a literature review. Parsing what’s going on in the paper and what the authors are trying to communicate takes far too much effort for the reader. \nTake the abstract, I would like to see a concise and structured half-page (max 3/4 page) paragraph (or two) consisting of something like a brief problem statement, introduction to your method, a key insight, and its significance. As it is, it is a one-page paragraph. It is hard to parse what you’re trying to communicate, like what this paper actually does, why it is important etc. I suggest trimming it to include only the essentials to understand the problem and then highlighting your contributions, while removing details that can be introduced much later in the paper. The same problem persists throughout the paper. \nAnother example is in the method (page 10), instead of formally introducing the method (perhaps giving a mathematical formulation of f+ etc), you detail the benefits of the method being unsupervised. This is a discussion that doesn’t belong in the method section, it is distracting to the reader.\n- On a related note, the paper needs tidying up in general. E.g., the caption to Figure 1 is incorrectly titled “Linear Representation Hypothesis:” when it should say something like “overview of encoding-decoding”…\n- Baseline methods need work: Firstly, further details on the baseline method is required. You give some details on how NMF is implemented, similar to Fel et al, but what about PCA? PCA is a very simple baseline, why did you choose it at all? Second, the only baselines compared are NMF and PCA. Some other methods are not feasible, as you mentioned. More baselines are required, specifically Graziani et al. [1], at a minimum. At several points in the paper, you cite Graziani et al. [1], but their comparable method (essentially SVD with gradient reranking) should be feasible to implement as a baseline. Please note that implementing PCA is not sufficient to compare against Graziani et al.! If you implemented it as in their method (it seems not), please clarify this, otherwise this comparison should be included. On a related note, Graziani et al. also implement ‘concept activation maps’ similar to your Concept Contribution Maps. A comparison would be interesting.\n- I would like to see more discussion of the linear representation hypothesis, as this is a key assumption of the method (a discussion added to the appendix is okay). Many methods assume this largely empirical hypothesis, and it doesn’t have a theoretical basis. However, many works now suggest going beyond the linear representation hypothesis [2,3,4]. Please discuss methods that do and don’t assume this, its evidence and counterevidence/limitations. \n\n[1] Graziani, M, et al. \"Uncovering Unique Concept Vectors through Latent Space Decomposition.\" *Transactions on Machine Learning Research*.\n\n[2] Park, Kiho, et al. \"The Geometry of Categorical and Hierarchical Concepts in Large Language Models.\" *The Thirteenth International Conference on Learning Representations*.\n\n[3] O'Mahony, L, et al. \"Towards utilising a range of neural activations for comprehending representational associations.\" *2025 IEEE/CVF Winter Conference on Applications of Computer Vision.* IEEE, 2025.\n\n[4] Diego Simon, Pablo J., et al. \"A polar coordinate system represents syntax in large language models.\" *Advances in Neural Information Processing Systems* 37 (2024): 105375-105396.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Yes, the authors validate their method on synthetic and real datasets. However, the comparative baselines need improvement, as detailed in the weaknesses.", "audience_yn": "Yes", "audience_explanation": "Yes, I believe this work will interest researchers in this area. The paper needs significant refinement and changes, but with revisions, it presents an approach worth attention and further development. \n\nInterpretability is a topic receiving significant attention, currently dominated by SAEs. Exploring alternative approaches to computationally expensive SAEs is valuable. The method could potentially extend to other modalities, such as application to VLMs and LLMs, in the future.", "requested_changes": "Critical Changes Required for Acceptance:\n- Writing and structure: Significant work is needed. Improve logical flow and eliminate repetition. Reduce verbosity throughout. Condense the abstract to 1/2-3/4 a page with a clear problem statement, method, and contributions. Fix formatting issues. I recommend targeting 25-35 pages instead of 40 (recommended but not a critical ask).\n- Baseline comparisons: Add Graziani et al. SVD with gradient reranking method as a baseline. Provide more implementation details for existing baselines.\n\nRecommended (To Strengthen the Work):\n- Move detailed derivations, extended experiments, and additional related work to appendix to improve main paper accessibility.", "broader_impact_concerns": "- Code: Provide minimal sample code or make a GitHub repository accessible once published.\n- Computational complexity and scalability: The method involves multiple steps in encoding and decoding. Include details about the compute resources used for experiments to help readers assess its feasibility in resource-constrained environments.", "additional_comments": "None", "extra_scores": {}, "extra_text": {}}, {"review_id": "rRaIOdKPkQ", "reviewer": "Reviewer_twFM", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "### Summary\n- The authors introduce Encoding-Decoding Direction Pairs (EDDP), a novel framework designed to discover encoded concept signal directions within DNNs in an unsupervised manner.\n- They propose a method for jointly learning decoding and encoding directions through the use of directional clustering. The approach leverages directional clustering to identify prominent directions in the latent representation of the data. Their methodology is grounded in works (Haufe et al, Kindermanns et al, Pahde et al) addressing the decomposition of signal and distractor directions.\n- They present a variety of novel loss functions aimed at enhancing the interpretability and influence of identified concept directions, including the excessively active classifier loss, the focal sparsity loss, the filter-signal orthogonality loss, and a loss term that encourages uncertainty alignment.\n- The concepts identified through EDDP are optimized to be both interpretable—aligned with human intuition—and influential, consistently affecting model predictions.\n\n\n### Strengths\n- The authors tackle a significant and relevant research question in the AI interpretability community, specifically the unsupervised detection of interpretable and influential concept directions.\n- They introduce a range of novel, creative, and sensible loss terms that address various aspects of direction discovery, optimizing the interpretability and influence of found directions.\n- The manuscript includes a comprehensive set of experiments, covering diverse applications and ablation studies that support the claims made in the paper.\n\n### Weaknesses\n- The authors focus solely on ConvNets. The work would be more impactful if it also addresses transformer-based architectures, as mentioned in Section 8.\n- The apparent tradeoff between interpretability and influence is not sufficiently addressed. On page 9, the authors state that \"uncertainty region alignment assists in improving interpretability or in aligning signal vectors with directions of interest.\" However, they later claim that interpretability and influence are negatively correlated (Section 7.5.6). A more detailed discussion would be valuable, including hypotheses or intuition for reasons for the tradeoff. Are there any thoughts on how more influential directions can be identified?\n- The authors rely on the relatively outdated approach of Network Dissection, despite the existence of more recent methods like ClipDissect (Oikarinen et al., 2022). ClipDissect overcomes the need for a densely labeled dataset (Broden). This could be particularly relevant for specific applications, such as in the medical field.\n- A tabular overview of all loss terms, their variations, and their respective roles would highly improve the readability of the paper. Although Figure 4 attempts to provide this information, it remains somewhat unclear.\n- In Section 5.1 (*Intervening*), the concept of activation steering as a model intervention is not novel, and existing works (e.g., Anders et al. 2022, Neuhaus et al. 2023) should be cited.\n- In Section 7.10 (*Concept-level intervention*), it would be helpful to clarify how each step impacts predictions, specifically the results (1) after concept removal and then (2) after concept insertion.\n- In Section 7.11 (*Model correction*), how would the concept detector focused on the intended (bias) concept be identified in practice? In Figure 22, the bias is apparent from the example images, but this may not be straightforward in practical applications.\n- What is the rationale for reporting the accuracy on the combined test set (clean & poisoned). Is this not just the average of the individual test sets?\n\n### Questions\n- Could the directional clustering approach be extended to semi-supervised concept discovery? For instance, if concept labels for one specific concept are known, could the cluster assignment for that set of samples be fixed while training the remainder?\n- The authors mention the use of constrained optimization with Lagrangians to avoid tuning loss weights. Is the tuning of sensible target values $\\tau$ not a similar challenge?\n- Why does PCA perform so well across all experiments in Section 7.5 in terms of influence? Do the authors have any intuition for this observation?", "claims_and_evidence_yn": "Yes", "claims_explanation": "(mostly)\n\n- The authors conduct an extensive set of experiments to address each individual claim and component of their approach. This includes a simple toy experiment as well as a variety of applications that measure the faithfulness, interpretability, and influence of the identified concept directions. They further assess the applicability of their method to downstream tasks, covering both local and global explanations, as well as model correction and concept-level intervention. Additionally, the authors perform ablation studies to evaluate the impact of individual loss components, which is crucial given the compositional nature of their approach.\n- Some metrics presented are not entirely convincing. In Section 7.2, the authors utilize deep dream to amplify inputs that trigger specific concept representations and fit what they call “dreaming directions”, which are then compared with the signal vectors. This metric appears to be error-prone due to the nature of the dreaming process, and the interpretation of results (sufficient faithfulness because scores are above 0.7) seems somewhat arbitrary. Moreover, the interpretation of the “influence” metric is unclear, particularly in the absence of ground truth information regarding the influence of specific concepts. If sensible concepts are not used by the model (because it employs other non-intuitive concepts), should we achieve high influence scores?", "audience_yn": "Yes", "audience_explanation": "Yes, the scalable (unsupervised) detection of concepts that (1) align with human mental representations (interpretability), (2) are influential on model decisions (influence), and (3) are useful for downstream tasks such as interventions, is an important research field in the interpretabiltiy community. The paper makes valuable contributions to the area.", "requested_changes": "1. In my opinion, the authors should consider replacing the Network Dissection approach, which relies on the densely annotated Broden dataset for concept labeling, with the more recent CLIP Dissect method. This change would enable the adaptation of the framework to more specific use cases involving concepts that are not represented in the Broden dataset.\n2. It would enhance the readability of the paper if the authors provided a tabular overview of all loss functions, including their variations and individual roles. This addition would improve the understanding of the contributions made by each loss term.", "broader_impact_concerns": "No concerns. Improving the interpretability of DNNs has a positive impact on the AI community, as it fosters the verifiability and safety of AI models before deployment.", "additional_comments": "Overall, I think the manuscript is too long and can be shortened to improve the accessibility of the paper. This could be achieved by a reduction of redundancy and by moving some experimental discussions (e.g., from Sec. 7.5) to the appendix.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)