FastCAV: Efficient Computation of Concept Activation Vectors for Explaining Deep Neural Networks
Authors: Laines Schmalwasser
Niklas Penzel
Joachim Denzler
Julia Niebling
Venue: ICML 2025 poster (2025)
Keywords: explainability
concept-based explanations
concept activation vectors
computational efficiency
deep learning
Primary area: deep_learning->everything_else
TL;DR: We introduce FastCAV, a novel method to compute Concept Activation Vectors that can speed up calculations by as much as 63.6 times, while still providing efficient, concept-based explanations for deep neural networks.

Abstract: Concepts such as objects, patterns, and shapes are how humans understand the world. Building on this intuition, concept-based explainability methods aim to study representations learned by deep neural networks in relation to human-understandable concepts. Here, Concept Activation Vectors (CAVs) are an important tool and can identify whether a model learned a concept or not. However, the computational cost and time requirements of existing CAV computation pose a significant challenge, particularly in large-scale, high-dimensional architectures. To address this limitation, we introduce FastCAV, a novel approach that accelerates the extraction of CAVs by up to 63.6× (on average 46.4×). We provide a theoretical foundation for our approach and give concrete assumptions under which it is equivalent to established SVM-based methods. Our empirical results demonstrate that CAVs calculated with FastCAV maintain similar performance while being more efficient and stable. In downstream applications, i.e., concept-based explanation methods, we show that FastCAV can act as a replacement leading to equivalent insights. Hence, our approach enables previously infeasible investigations of deep models, which we demonstrate by tracking the evolution of concepts during model training.

FastCAV: Efficient Computation of Concept Activation Vectors for Explaining Deep Neural Networks
Laines Schmalwasser 2
Niklas Penzel
Joachim Denzler
Julia Niebling'
Abstract
Method
Concepts such as objects, patterns, and shapes are how humans understand the world. Build- ing this intuition, concept-based explainabil- ity methods aim to study representations learned by deep neural networks in relation to human- understandable concepts. Here, Concept Activa- tion Vectors (CAVs) are an important tool and can identify whether a model learned a concept or not. However, the computational cost and time require- ments of existing CAV computation pose signif- icant challenge, particularly in large-scale, high- dimensional architectures, To address this limi- tation, we introduce FastCAV, a novel approach that accelerates the extraction of CAVs by up to 63.6x (on average 46.4 x)) We provide a theoreti- cal foundation for our approach and give concrete assumptions under which it is equivalent estab- lished SVM-based methods. Our empirical results demonstrate that CAVs calculated with FastCAV maintain similar performance while being more efficient and stable. downstream applications. i.e., concept-bass based explanation methods, we show that FasICAV can act a replacement leading equivalent insights. Hence, our approach en- ables previously infeasible investigations of deep models, which we demonstrate by tracking the evolution of concepts during model training.
FastCAV (Ours)
Willi SVM-CAV
Average Computation Time [s]
250
Model
ResNet50
Inception-v3
200
VIT-B/16
ConvNeXtxXXLarge
150
VIT-H/14-CLIP
63.6x
EVA-G/14
speed-up
EVA-02-I/14
100
RegNetY
50
0
0.2M
0.4M
0.6M
0.8M
1.0M
Dimensionality of CAVs [millions]
Figure 1: Comparison of computational efficiency between FastCAV and established SVM-CAV for calculating Con- cept Activation Vectors (CAVs) across different models. The average time calculate a CAV for each method is plotted against the dimensionality of the activation spaces, demonstrating the significant speedup (p 0.01) achieved by FastCAV.
same conceptual terms makes their behavior more intuitive and relatable human understanding (Kim et al., 2018 8; Ghorbani et al., 2019; Yeh et al., 2020; Zhang al., 2021; Pfau et al., 2021; Nicolson et al., 2024). Concept-basse methods investigate the learned representations of a model with respect to undesadddddrttadaaalee concepts and quan- tify the influence of these concepts on the predictions of selected class. For example: are the concepts "stripes", "wheels", or "green" relevant for the prediction of class zebra? To answer such questions, Concept Activation Vec- tors (CAVs) are introduced in (Kim et al., 2018). CAVs are vectors in the network's 's activation space that represent human uamaaannnnrrttaaaalle concepts and are the foundation for many concept-based based explainability methods (Ghorbani et al., 2019; Yeh et al., 2020; Oikarinen Weng, 2024). Specifi-
1. Introduction
Humans often think in terms of abstract, perceptually grounded concepts make sense of the world (Barsalou, 1999). This led the idea that explaining models in the
"Institute Data Science, German Aerospace Center, Jena, Germany *Computer Vision Group, Friedrich Schiller Univer- sity Jena, Germany. Correspondence to: Laines Schmalwasser <Laines Swwaasaasse@@@@ell.....
Proceedings of the ndnd International Conference on Machine I Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s). Project page: https::/fast.cs thub. io/

---

FastCAV: Efficient Computation of Concept Activation Vectors
cally, these vectors encode directions in model's activation space, which correspond to changes in specific semantic concept. For given class, it is possible to test whether the product of the gradient and vector is positive or negative. If is positive, the identified concept influences the class. This is called Testing with Concept Activation Vectors (TCAV), (Kim et al., 2018). Following the idea that concepts are encoded as linear directions in the activation space, e.g.. (Szegedy, 2013; Karpathy et al., 2015; Goh, 2016; Alain, 2016; Bau et al., 2017; 2020; Elhage et al., 2022), existing works find CAVs by optimizing linear classifiers, usually a linear support vector machine (SVM) (Kim et al., 2018). Further, it is often necessary to calculate multiple CAVs to ensure statistical significance when quantifying the influ- ence on a specific class (Kim et al., 2018).
resources and time are limited. Finally, we apply our ap- proach to track concepts during training and through the layers of a network, which was previously limited by our computational constraints.
The key contributions of our work are as follows:
We introduce FastCAV, a novel approach for computing Concept Activation Vectors (CAVs) that significantly reduces computation time by up to 63.6x, making it more efficient and scalable.
We provide theoretical foundation for FastCAV and specify concrete assumptions under which it is equiva- lent to other linear methods.
We demonstrate empirically that FastCAV produces similar CAVs to the established SVM-based approach, leading to comparable insights downstream methods, e.g. et al., 2018: Ghorbani al., 2019).
Unfortunately, these conditions result in long computation times when performing CAV-based analyses. This prob- lem is further exacerbated due to the increasing number of parameters and corresponding activation space dimen- sionalities in modern architectures. As a concrete example, consider activations after the last convolutional layer of ResNe150 (He et al., 2016), which has 100.352 dimensions. In comparison, the number of last layer activations in re- cent models (HuggingFace, 2025), c.g., EVA-02-L/14 (Fang et al., 2024), reaches up to 1 1.049.000 dimensions. This development and the need for multiple CAV computations result in long processing times and substantial memory con- sumption. Hence, these constraints can be restrictive for many downstream applications, e.g., when analyzing many layers of model (Nicolson al., 2024) when interpret- ing numerous concepts simultancously (Bau et al., 2017). As result, the practicality of using traditional CAV meth- ods is limited when faced with modern classifiers and tasks. A more efficient approach to compute CAVs in deeper layers without compromising the quality of the insights is needed.
Using FastCAV, we investigate the evolution of con- cepts during training and across different layers of a neural network.
2. Related Work
Concept Activation Vectors (CAVs) Alain et al. (Alain, 2016) introduce the idea that features of an intermediate layer are separable using a linear classifier. Based on this, CAVs (Kim et al., 2018) are introduced, which connect ab- stract model representations and human sumaa--understaddable con- cepts. In particular, CAV' represent human- uuderstanddblle concepts as directions in the activation space of a model's ss layer. To compute CAVs, linear classifier is trained to dis- tinguish images that share a visual concept from random im- ages. The CAV corresponding to the concept is proportional to the normal vector of the linear decision boundary. The assumption is that if the concept images cannot be separated in a model's 's activation space, the model did not adequately learn the concept. Testing with Concept Activation Vectors (TCAV) (Kim et al., 2018) uses CAVs to quantify the sen- sitivity of model's predictions for a specific class to the presence of a human-understanddable concept. For example, TCAV is able to investigate questions such as whether the color "blue" is relevant for the model when predicting the class whale. Further, TCAV and, more specifically, CAVs are the foundation of various concept- based explainabil- ity methods, (Ghorbani et al., 2019; Schrouff et al., 2021; Zhang al., 2021; Singla et al., 2021; Gupta et al., 2024; Ghosh et al., 2023).
In this work, we propose FastCAV, an approach to speed up the extraction of CAVs within a neural network's 's acti- vation space by up to 63.6x. To achieve this, we follow insights from the area of superposition (Goh, 2016; Elhage et al., 2022) that features in a model are represented as nearly orthogonal vectors in the activation space. Hence, we find the CAV corresponding to a concept by comput- ing the normalized mean vector of the positive concept examples. Additionally, we show that our approach can be understood approximation of Linear Discrimi- nant Analysis (I.A)) and, under certain assumptions, it is equivalent to the solution of a linear SVM (Shashua, 1999), providing an intuitive connection to previous works (Kim et al., 2018). We empirically show that, despite the sim- plicity of our approach, it effectively identifies conceptual directions in high-dimensional activation spaces. Addition- ally. FastCAV significantly reduces computational expenses, making it more practical for real-world applications where
Additionally, various methods have been proposed to en- hance the interpretability of the discovered concepts. In (Zhang et al., 2021), the authors address directional diver- gence, and in (Pfau et al., 2021), the authors improve robust- ness against data variations. Language-gui guided approaches
2

---

FastCAV: Efficient Computation of Concept Activation Vectors
like (Moayeri al., 2023; Huang et al., 2024) remove the dependency on concept images. Instead, they are using vision-text embeddings, e.g., (Radford et al., 2021), and map them to a selected activation space. Hence, they are able to extract CAVs belonging to concepts described by words. In medical imaging, techniques have been developed for bidirectional explanations using regression concept vec- tors (Graziani et al.. 2018). These approaches improved the usability of CAVs, but they remain computationally demand- ing due to classifier training for each CAV. Further, recent approach (Pahde et al., 2025) reduces the influence of unre- lated distractor patterns, resulting in a similar interpretation of the activation space.
concatenation of two mappings: gt, which comprises the first / layers, and hy, which consists of the remaining layers. For an input 2, we write f(x (u(g(())).
To interpret (((-) with respect to semantic concept we follow (Kim et al., 2018) and describe a concept by a set of user-defined images D.. This visual definition allows for a specification independent of the particular training task and data. The model has learned concept e after layer if it is possible separate images of Dr from random samples of set D, in the respective activation space. Specifically, in (Kim et al., 2018), the authors learn linear SVM classifiers to identify linear combinations of activations that correspond to a concept c. They define a Concept Activation Vector (CAV) oy as the normal vector of such a learned linear decision boundary. For binary labels sgn(y) describing the membership of x in De or Dr,, the linear classifier is
Linearity in Neural Networks. Many works study how semantic features or concepts are encoded in neural net- works as linear directions in the activation space, e.g., (Szegedy, 2013; Karpathy et al., 2015; Goh, 2016; Alain, 2016; Bau et al., 2017; 2020; Elhage et al., 2022; Bricken et al., 2023; Templeton, 2024; Oikarinen & Weng, 2024). Some identify interpretable neurons (Karpathy et al., 2015;
y = o gr (xx) +
(1)
where b is the intersect, and is the dot-product.
Intuitively, of can be understood as the direction in the activation space of 9i that corresponds to a concept e. If the resulting classifier defined in Equation (1) achieves a low separation performance, then f has not sufficiently learned concept (Kim et al., 2018). In the following section, we detail our alternative approach to calculating CAVs.
Bau al., 2017; 2020; Oikarinen & Weng, 2022; 2024). In contrast, other studies suggest that neurons can also be pol- ysemantic (Olah al., 2017; 2020), meaning that neuron responds to more than one feature or concept. In (Elhage et al., 2022), the authors find that network learns to encode features as a combination of neurons, i.e., a direction in feature space that does not align with the axes. They argue that this happens if there are more features than neurons because features cannot be equally distributed across the neurons in that case.
3.2. FastCAV
Following insights from previous works on orthogonality in feature space, e.g., (Olah et al., 2020; Elhage et al., 2022), FastCAV aims to identify the direction directly as the differ- ence between random images and a concept set. Specifically, to find a CAV vy for set of concept images De. we first calculate the mean of (zx for all x E De U D, i.e. we use maximum likelihood estimator, (Bishop, 2006),
Based on this idea, different works explore the phenomenon that neural network can encode more features than di- mensions in a hidden representation, (Olah et al., 2020;
Elhage et al., 2022; Bricken et al., 2023; Templeton, 2024). The strategy to achieve this is called superposition (Olah
al.. 2020). Elhage et al. (Elhage et al., 2022) show on synthetic dataset that neural network in superposition encodes the features as almost-orthogonal directions each other. Recent work explores superposition further for large language models (Bricken et al., 2023; Templeton, 2024).
AD.UD.
1
Σ 2ED.UD.
((()).
(2)
De + D
We now use this global mean normalize all activation vec- tors gr for samples in De. Then, we follow the intuition detailed above that of encodes the direction in activation space that separates samples contained in D. from samples in D,.. Specifically, we calculate
In this work, we follow the idea that features are encoded as almost orthogonal linear directions activation space to design FastCAV.
3. Methodology
of x Dr
1
Σ (m(()) - -unjD,)) RED.
(3)
3.1. Preliminaries
which we normalize to unit length, following the SVM approach (Kim et al., 2018).
Concept-basee explanation approaches aim to explain trained neural networks f on the level of human- understandable concepts. Specifically, they investigate the activation space after a layer 1. Hence, we split f into the
Intuitively, we first center all activations and then find the direction pointing towards the concept examples. Hence,
3

---

FastCAV; Efficient Computation of Concept Activation Vectors
the concept examples Pe- In Equation (3), we again apply the respective maximum likelihood estimator f.D,, meaning of ÅAD pup. Under the assumptions specified above and again drawing an expectation over the sample of random inputs and concept examples, we get
D
Dr
iDD
p.D.
Blu'l x Me
Me + Mr
Me
Pr
(6)
2
2
This solution is simplified form of linear discriminant anal- ysis (LDA), more specifically, Fisher discriminant analysis (Bishop, 2006) under precise assumptions. Using a related maximum likelihood estimator for the total within-class covariance
Figure 2: Schematic illustration of FastCAV in two dimen- sions. Specifically, the learned concept activation vector of points from the global mean jD.UD, towards j.D.
ÉD.UD. = Σ (a((x)) - Re) - T
in FastCAV, on is the vector which points from MD.UD, towards the mean of (x) for all x E De, see Figure 2.
aED.
Σ DEDr
(ai(z) (r) - pc)s
With Equation (1) and y = 0, the intersect b is given by
b - j.D.D,,
(4)
the Fisher discriminant solution for the normal vector of the linear decision boundary between the two classes is propor- tional to St (pc - (r)). Under the explicit additional assumption of isotropic within-class covariances, i.e., E-1 being proportional to the unit matrix, the solution for the normal vector is equivalent to FasCCAV. To validate this approach, we empirically compare against the full Fisher LDA in Appendix B.2.1 and find faster computation times and higher accuracies for various model architectures when using FasCCAV.
fully specifying the linear decision boundary of FasICAV.
3.3. Connection to Other Linear Classifiers
We study FastCAV a theoretical level to investigate the connection to the established linear SVM approach for find- ing CAVs, e.g., (Kim et al., 2018). In detail, we show that under certain assumptions, both approaches identify the same direction in latent space. While these assumptions are strong, they provide an intuition for the strong performance of our approach in our empirical evaluations.
To summarize, under the assumptions of isotropic within- class covariances and Gaussian distributions for both ran- dom and concept examples, FastCAV is equivalent to the solution of Fisher diseriminant analysis on corresponding activations. Further, in (Shashua, 1999), the authors investi- gate the relationship between a linear Fisher discriminant and a linear SVM, which is the established classifier utilized in (Kim et al., 2018) to compute CAVs. Specifically, they find that the solution of Fisher discriminant analysis on the set of support vectors is equivalent to the solution of the linear SVM. Hence, a support vector machine can be under- stood as a sparsification of the linear Fisher discriminant.
In Section 3.2, describe FasICAV in terms of concrete observations x E De U D,.. These observations are gener- ated by some process and follow the latent distribution of inputs. Hence, we can study this process in terms of random variables following this unknown distribution. Further, gi of the trained model f leads to a mixture distribution for random images and images conditioned on the concept c. In other words, we cast the separation of random and concept samples as a statistical learning problem, (Vapnik, 1998).
Specifically, we consider the maximum likelihood estimate of the global mean in Equation (2). This estimator is gen- crally unbiased under i.i.d. samples following a Gaussian distribution (Bishop, 2006). Further, in Appendix 1, we show that if both the random samples and concept samples follow multivariate Gaussian distribution and are equally mixed, then the expectation of the estimator is
Finally, consider that the aim is to explain the learned repre- sentation of a model after layer l. Hence, these activation vectors (xx) are potentially of a very high dimensionality d. Further, let n = |D. U D, be the number of samples and assume >>> n. Then the set of support vectors likely contains many of the samples in D. U D,. Related works study this in more depth, e.g., (Muthukumar et al., 2021; Hsu et al., 2021), and find that the fraction of support vectors increases with increasing dimensionality. We empirically confirm this and find high ratios of support vectors when studying the activations of modern architectures with linear SVM classifiers (see Table 3 in Appendix A.2).
Me + Prr
ElūD.UD.]
(5)
2
where Me and μy are the respective means of the Gaussians.
Then FastCAV finds of as the unit vector proportional to the vector pointing from the global mean to the mean of
4

---

FastCAV: Efficient Computation of Concept Activation Vectors
To conclude, under the assumptions detailed above and in high-dimensional activation spaces, the normal vectors of the hyperplanes found by FastCAV and linear SVM used in (Kim et al., 2018) are identical. Further, we find that in practice, our approach leads to similar solutions while being much faster to calculate, which we demonstrate in our experimental investigations.
as a replacement in downstream concept-based explanation methods and generate similar insights. In particular, we compare FastCAV against SVM-CAV and investigate Test- ing with Concepts Activation Vectors (TCAV) (Kim et al., 2018), and Automatic Concept-based Explanations (ACH) (Ghorbani et al., 2019). Finally, we utilize the increased efficiency of FastCAV to track CAVs throughout the training and layers of a ResNet50 (He et al.. 2016) to understand how concepts evolve. We provide the complete technical setup and comprehensive details for all our experiments in Appendix B.1.
3.4. Complexity Analysis
The overall time complexity for training an SVM is (max(n, d) min(n, d)²). where n is the number of sam- ples and is the dimensionality of the input space. This holds for both the primal and the dual optimization problem, (Chapelle, 2007). However, the complexity and speed also depend on the selected kernel, which is linear in our case (Kim et al., 2018). Hence, to benchmark the speed, we utilize various common implementations of linear SVMs. First, to solve the dual problem, we use the Sequential Min- imal Optimization (SMO) algorithm (Platt, 1998). Specifi- cally, we use the implementation in (Pedregosa et al., 201 1), which uses the SMO implementation from (Chang & Lin, 201 1). Second, regarding the primal problem, we use the implementation in (Fan et al., 2008), which is also included in (Pedregosa et al., 2011). Lastly, using Hinge loss, it is possible to approximate linear SVM classifier using stochastic gradient descent (SGD), e-g., (Pedregosa et al., 201 For this last implementation specifically, the com- plexity boils down to calculating gradient with respect to the linear decision boundary parameters (Od) for each of the n training samples for T iterations. Hence, the SGD vari- ant in (Pedregosa et al., 2011) has a complexity of ((Tnd). We benchmark these three implementations and provide the results in Appendix A.3. In our use case, we find that SGD-based optimization performs best.
4.1. FastCAV versus SVM-based CAV computation
To empirically compare FastCAV and SVM-CAV, we evalu- ate a broad spectrum of architectures trained on ImageNet (Russakovsky et al., 2015). Additionally, we investigate run times for the CAV computation in a realistic setting and select current top-performing model architectures from the i -mm-leaderboard (Wightman. 2019). We used the models reported in the first column of Table 1, i.e., (Szegedy et al., 2016; He et al., 2016; Liu et al., 2022; Singh et al., 2022; Dosovitskiy et al., 2021; Fang et al., 2023; 2024).
In our setup, we sample 60 concept images per set De from the Broden dataset (Bau et al., 2017). Similarly, we sam- ple 60 random images from ImageNet (Russakovsky et al., 2015) for each set D,., All our results are reported as aver- ages over 30 resampled D,, across all concepts of the Broden dataset, (Bau et al., 2017), and over the respective network layers. We split our investigation into four dimensions: computational time, accuracy, inter-method similarity, and intra-method robustness.
Computational time We analyze the theoretical time complexity of FastCAV and SVM-CAV in Section 3.4. As a complementary analysis, we now compare the run time of both methods in practice. In Table 1, columns "Comput. time [s]" report the respective average times to compute one CAV per model. In Figure 1, we visualize the timings for the pre-final network layers before classification. A similar visualization for the average over all layers is included in Appendix B.2.
In comparison, FasICAV calculates sum of vectors with dimension d and performs d multiplications. Hence, training complexity is (nd) with very small constants. Once the linear decision boundary is found, the inference for both SVM-CAV and FastCAV is equivalent to the dot product between two d-dimensional vectors, resulting in (dd)
4. Experiments
Both methods require more time to calculate CAVs for mod- els with higher activation space dimensions. However, this observation is expected and follows from the complexity of both methods, which scale linearly in the number of dimen- sions d. Nevertheless, under practical conditions, FastCAV is up to 63.6x (on average 46.4x) faster than SVM-CAV.
With FastCAV, we propose an alternative to calculate CAVs, which is more computationally efficient. To demonstrate its effectiveness, we compare our approach against the estab- lished SVM-based approach ("SVM-CAV") (Kim et al., 2018) in terms of speed and CAV quality. We repeat this comparison for more specialized medical concepts to highlight the broad applicability. Further, compari- son to other CAV calculation methods is provided in Ap- pendix 1, and a discussion of failure cases is provided in Appendix B.2.2 Next, we show that CastCAV can work
Accuracies of CAVs to distinguish between D. and D, In addition to the increased computation speed of FastCAV, we need to evaluate the quality of the discovered CAVs. Hence, we compare the predictive performance, i.e., accu-
5

---

FastCAV: Efficient Computation of Concept Activation Vectors
Table I: Comparing our approach FastCAV with SVM-CAV. Bold values indicate better results. "N/A" indicates that no results were produced due to the overall computational time for the respective model exceeding four days. Additional results for other CAV calculation methods can be found in Appendix B.2.1.
Comp.
Time [s] +
Accuracy T
Similarity 1
Model
Ø Dim.
FastCAV
SVM-CAV
FastCAV
SVM-CAV
FastCAV
SVM-CAV
Metrr-
Inception-v3
206,169
0.41.0.14
4..7:129.33
0.95
0,93.10.99
0.7951.0.00
0.338
0.898_0.005
ResNer50
341,197
1.11
135.411129
0.19
0.87.10.15
0.816:10.029
0,400.10.062
0.837
ConXNeXt-XXL
753,830
5.543388
269.6=167.0
0,90=0008
0.93+0.077
0.831=00002
0,449=00077
0.807=0.136
RegNetY
1,869,479
8,2+1.04
N/A
0.90=0.05
N/A
0.734=00.00
N/A
N/A
ViT-B/16
140,560
1.1:10.03
50.5
0.82
0.81
0.787
0.408
0.818
VIT-H/14-CLIP
281,925
1.9::0.8
69.0++10.9
0.85=0.15
0.86+
0.603=00005
0.249+00090
0.858=0.083
EVA-G/14
696,298
4.7:10.14
248.9=60.3
0.88
0,86#t.144
0.650:100022
0.250=
0.888=0.060
EVA-02-L/14
899,653
6.15:0.25
301.6=114.7
0.89
0.88+0.16
0.675
0.219#40076
0.836=0.955
racy, of the linear classifier posed by Equation (4) using CAVs obtained from our approach and SVM-CAV.
Table 2: Comparing FastCAV with SVM (Ghosh et al.,
2023) and sparse logistic regression (SL-R) (Singla et al.,
2021) for specialized medical concepts in a chest x-ray image classification task. Bold values indicate better results.
The results are presented in Table 1 ("Accuracy"). No- tably, FastCAV results in slightly higher accuracies, with one exception (ConwNeXt-XXLL) Nevertheless, both meth- ods yield similar results. We conclude that FastCAV does not compromise the predictive quality of discovered CAVs.
Method
C. Time [s] +
Accuracy 1
Similarity 1
FastCAV SL-R SVM
0.006+10001
0.72+0.11
0.79=0.03
6.770
0.72+
0.50=0.04
0.439
0.71
0.46-0.04
Inter-method similarity To investigate the agreement be- tween the generated CAVs of both methods, we directly compare the computed CAVs. For that, we report in Ta- ble ("Inter-Method"") the average cosine similarity between CAVs calculated by FastCAV and by SVM-CAV.
MIMIC-CER dataset (Johnson et al., 2019). Next, we use the CheXpert labeler (Irvin et al., 2019) to extract concept mentions from the included free-text radiology reports in- cluded with MIMIC-CXR (Johnson al., 2019). Using the resulting concepts, we derive CAVs for each of the model's four dense blocks applying FasICAV, SVM- compu- tation (Ghosh et al., 2023), and sparse logistic regression (Singla et al., 2021). In Table 2, we report the average com- putation time, CAV accuracy, and intra-method similarity (see Section 4. 4.1). FastCAV achieves performance similar to that of the established approaches in this domain. However, note the strongly reduced computation time and improved similarity. We include an evaluation of individual concepts and the technical details of our setup in Appendix B.2.3.
Across all models, we observe high similarities (values be- tween 0.8 and 0.9). This result indicates that both FastCAV and SVM-CAV find similar directions in diverse activation spaces for a variety of concepts.
Intra-method robustness Lastly, we evaluate the varia- tion of the discovered CAVs across different D, for both methods. In Table 1, the first two columns of "Similarity" report the average pairwise cosine similarity of the set of 30 CAVs aggregated across all concepts and network layers.
Overall, we find FastCAV computes CAVs of a higher pair- wise similarity. This increased robustness of FastCAV is due to the CAVs only differing because of the estimations of the global mean D.D.DD, while De stays fixed. In con- trast, SVM-CAV completely relearns linear SVMs using non-deterministic SGD implementation (see Section 3.4).
4.3. CAV-based Explanation Methods
After quantitatively comparing CAVs computed by our ap- proach and the SVM-based approach, we investigate the qualitative performance with downstream cancept-based explanation methods. For this comparison, we substitute SVM-based CAV computation with FasICAV and contrast the generated explanations, showcasing its ability to act drop-in replacement. As concept-based explanation methods, we select Testing with Concept Activation Vectors (TCAV) (Kim et al., 2018) and Automatic Concept-based Explanations (ACE) (Ghorbani et al., 2019).
4.2. Medical Task
While our main experiments focus on a purpose model and task, we show that FastCAV can also be applied in more specialized scenarios. Following prior work on med- ical concept extraction (Singla et al., 2021), we fine-tune a DenseNet- 121 (Huang et al., 2017) on a curated subset of the
6

---

FastCAV; Efficient Computation of Concept Activation Vectors
Layer
1.0
1.0
mixed3a
mixed3b
0.8
0.8
mixeddA
mixed4b
TCW TCAV Scure
mixed4e
0.6
mixed4d
mixedie
mixed5a
mixeddbb
Method
PaseCaav
WED
SVM-CAV
0.0
0.0
polka-dottza
striped
zigzagged
polka-dotted
striped
zigragged
Conceps
Concept
Figure 3: TCAV scores for various GoogleNet (Szegedy et al., 2015) layers. We compare the concepts "polka-dotted", "striped", and "zigzagged" for the class ladybug using FastCAV against SVM-CAV. We follow (Kim et al., 2018) and mark CAVs that not statistically significant with "*p
Testing with Concept Activation Vectors (TCAV)
Most Salient
Most Salient
TCAV calculates the directional derivative of selected class with respect to the activations layer I, i.., gi (z). For given input æ, the sensitivity score for concept with corresponding CAV vf is (h1,( of. The corre- sponding TCAV score is the fraction of set of inputs of the selected class that have positive sensitivity scores. Intuitively, this scalar measures the importance of concept in layer for the output node corresponding to class k.
(a) FasICAV
(b) SVM-CAV
Figure 4: Most salient concepts discovered by ACE (Ghor- bani et al., 2019) using either our FastCAV or the established SVM-CAV. In both cases, we find the discovered patches containing stripes, which is congruent with the original ob- servation in (Ghorbani et al., 2019).
We qualitatively evaluate FastCAV by repeating an experi- ment of (Kim al., 2018). Specifically, we select all layers of GoogleNet (Szegedy al., 2015) trained on ImageNet (Russakovsky al., 2015) and calculate the TCAV scores for various classes and concepts. In Figure 3, we compare the results for FasCCAV against SVM-CAV for the class ladybug and the concepts "polka dotted", "striped", and
CAV Accurace
0.95
CAV Accuracy
0.98
ResNet50
0.90
VIF-B/16
"zigzagged". In Appendix B.3, we include more examples.
0.93
Inception v3
0.85
The insights into the GoogleNet model are consistent be- tween both our approach and the SVM-based method. For ladybug, the concepts "polka dotted" and "striped" are more important than "zigzagged", This observation especially applies in later layers of the network. Further, both methods agree that "polka dotted" is the most important for earlier layers. We hypothesize that the usage of "striped" in later layers is due blades of grass in many ladybug images.
0.80
0.88
30
60
90
100
200 300 400
Number of Concept Samples
Number of Random Sets
(a) Impact [De].
(b) Number of resampled D,-
Figure 5: Sensitivity analysis of FastCAV to (a) the number of concept images and (b) number of resampled D,. Note the differences in y-axis scales.
In contrast to the similar average results, observe clear distinction for the standard deviations of the TCAV score over multiple repetitions. Here, FastCAV leads in most cases to a smaller variance in the measured scores. This observation confirms our intra-method robustness results in Table 1. Additionally, this result and the overall similar- ity between both FastCAV and SVM-CAV hold for other qualitative examples in Appendix B.3. Hence, we conclude that our approach leads to similar model explanations using TCAV as SVM-based CAV computation.
Automatic Concept-based Explanations (ACE) ACE is based on TCAV and proposes to automatically discover and extract concepts that are important for a selected class. set of
In detail, they perform superpixel segmentation images from class l using multiple resolutions to capture hierarchical concepts. Next, these superpixels are grouped into concept sets using (Zhang et al., 2018). The resulting sets are pruned and then ordered according to their impor- tance for class k using TCAV scores.

---

FastCAV: Efficient Computation of Concept Activation Vectors
Average CAV Accuracy per Block
Average CAV Accuracy per Concept
Model Top-1 Test Accuracy
block1
0.8
block2
0.7
block3
0.8
CAV Accuracy
block4
CAV Accuracy
Model Accuracy
0.7
0.5
0.6
0.6
0.4
0.5
polka-dotted striped zigzagged
0.5
0.2
top-1 tost accuracy
0.4
0
20
40
20
40
60
80
0
20
60
80
Epoch
Epoch
Epoch
Figure 6: Evolution of various CAVs during training of a ResNet50 (He al., 2016) on ImageNet (Russakovsky al., 2015). On the left, visualize the average accuracies achieved by CAVs after the final layers in each of the four ResNet blocks. In the middle, investigate CAVs for three specific concepts. On the right, we display the test accuracy during training. The vertical lines indicate epochs, after which the learning rate is divided by ten. Note the differences in y-axis scales.
To evaluate FastCAV for ACE qualitatively, we follow (Ghorbani et al., 2019) by analyzing the mixed 7 layer in an Inception-v3 v3 model (Szegedy et al., 2016) trained on ImageNet (Russakovsky et al., 2015).
Influence of |Del = =DD, We fix the total number of avail- able random sets to 60 and vary the number of examples in the concept and random sets equally, evaluating the corre- sponding CAV accuracies. In Figure 5a, we visualize the influence of Del the accuracy of the CAVs distinguish between concept and random images. In particular, we find that for all three models, the performance remains high with more than 60 samples. This indicates that with at least 60 examples, the CAV training process effectively captures the defining features of the concept. Following this observation, we recommend using at least 60 examples per concept.
Figure 4 visualizes the most salient concepts discovered with ACE. We include additional examples in Appendix B.4, following the class selection in (Ghorbani et al., 2019). In all cases, we find similar example patches for the discovered concepts. In particular, the concept "stripes" is important for the class lionfish irrespective of whether we employ Fast- CAV or SVM-CAV. Further, the TCAV scores for the most salient concepts similar, with 0.78 for our approach and 0.73 for SVM-CAV. These similarities between the discov- ered concepts and scores also apply to the other examples in Appendix B.4. Hence, we conclude that FastCAV suitable for automatic concept discovery using ACE.
Influence of the Number of Random Sets D, Following the previous sensitivity analysis, we fix the number of ex- amples in D, and D, to 60 and vary the number of random sets. In Figure 5b, we visualize the corresponding sensitivity of the CAV accuracies. We see expected stabilization, i.e., lower variances, when using more random sets D,. How- ever, the average accuracy remains approximately constant. Hence, in our final experiment, we select 100 random sets.
4.4. FastCAV Sensitivity Analysis
Our approach to finding CAVs depends on the number of concept images, i.e., [De] and [D,]_ Hence, it is important to understand how many samples are necessary to compute robust concept directions. Additionally, the robustness also depends the number of random sets D, which are used for statistical testing, (Kim et al., 2018).
4.5. Tracking CAVs During Training
Setup Using FastCAV, we are able to track concepts dur- ing the training of model with high number of param- eters. To show this, we train a ResNet50 (He et al., 2016) on the ImageNet dataset (Russakovsky et al., 2015) from scratch. We follow the same setup as (Paszke et al., 2019) and include the full details in Appendix B.5, As concepts, select the textures from (Bau et al., 2017). Examples include "polka-dotted", "striped", and "zigzagged". Specifi- cally, we compute CAVs against 100 random sets after each training epoch and select the final layers of each of the four ResNet blocks. Our analysis focuses on when and in which layer the model learns concepts during the training.
Here, we conduct a sensitivity analysis to explore the in- fluence of both variables. In particular, we vary the sizes of D. and D, as well as the number of available random sets. Further, for this sensitivity analysis, we compute CAVs using FastCAV for various networks. Specifically, we use the fre-final layers of ResNet50 (He al., 2016), ViT-16, (Dosovitskiy et al., 2021), and Inception-v3 (Szegedy et al., 2016) pretrained on ImageNet (Russakovsky et al., 2015).
8

---

FastCAV: Efficient Computation of Concept Activation Vectors
of concepts across layers. For instance, concepts that rank highly in earlier network blocks may be less important and rank lower in later ones. As a concrete example, consider the concept "wave", which is the highest-ranked concept in the first block. However, its ranking drops to second after block two and ultimately deteriorates to 32nd in the final block. contrast, the concept "lined" remains among the highest-ranked concepts throughout all layers. This ob- servation underscores how the model prioritizes different concepts at various stages, with some foundational concepts giving way to more refined or abstract ones in deeper layers.
waffled
iined dotted
freckled
grpovedd
studded
grid weven
geazy
trysta
potholed
perfforated
gleated
freckled
smeared
pleated
palka-donie
int interlaced
roach stchhd
Rank (1 Highess AUC)
vig
lecked
honeycombed
poecusa
bodd
estriped
lacellee
espeineeed
honekoomeed
a fllloous
perforaced
veined
Enitted
spiralled
marbled
paisley
estudded
#i gggagged
matind
woven
Frilly
5. Conclusion
grooved
spotholed
meehed
e ewallted
stratifled
sccaly
colbwebbed
In this work, we introduced FastCAV, a novel approach to efficiently compute Concept Activation Vectors (CAVs) for explaining deep neural networks. By leveraging ideas from superposition (Elhage et al., 2022), our method accelerates CAV extraction. Specifically, we take the normalized vector pointing towards the concept example mean in activation space, which is significantly faster than learning a linear SVM classifier. We further provide a theoretical founda- tion for our method and give specific assumptions under which FastCAV is equivalent to the established SVM-based approach. Our empirical results demonstrate that FasICAV achieves a significant reduction in computational cost, with speedups of up to 63.6x (and an average speedup of 46.4x), while maintaining at least similar CAV quality. We further investigate the variations in downstream performances us- ing TCAV (Kim et al., 2018) and ACE (Ghorbani al., 2019), and find only marginal differences in the results. To showcase the capabilities of FastCAV, we apply it to analyze the training of a model, tracking the evolution of concepts during training and across network layers. Our analysis validates the notion that textural concepts are learned the middle layers of a network, aligning with prior works (Bau et al., 2017: Kim et al., 2018; Ghorbani et al., 2019).
wrinkled led
veined
marhhldd
inter interfaced
swirtyy
paisl
crosshatcbed
spiralled
block0
blockl
block2
block3
Architecture Stage
Figure 7: Evolution of various CAVs during training of a ResNet50 (He et al., 2016) on ImageNet (Russakovsky et al., 2015). We report the concept ranking of the achieved AUC scores during training for the different network blocks.
Results Figure 6 demonstrates that the model learns con- cepts during training, with visibly increasing CAV accuracy after each step. This aligns with improved model accuracy, suggesting that the model learns to employ relevant concepts for predictions. We observe similar trends for specific con- cept examples, although results exhibit increased variability across the training steps compared to the average across concepts. These results indicate that the performance of relevant CAVs correlates with overall model performance.
Notably, we observe stark increases in average CAV accu- racy after each epoch, where the learning rate is divided by ten during training. This observation aligns with abrupt in- creases in the top-1 test accuracy. Furthermore, we find that early and middle layers have higher likelihood of learning textures than later layers. This result validates findings in other works like (Kim et al., 2018; Ghorbani et al., 2019; Bau et al., 2017). Our observations provide further evidence for this hypothesis and demonstrate that our approach can be used to study the learning dynamics of deep neural networks in more fine-grained manner and for abstract concepts.
Overall, FastCAV serves drop-in replacement to calcu- late CAVs, which leads to lower run times and maintains high quality. While we thoroughly evaluate the performance of the generated CAVs, we acknowledge that further anal- ysis of other CAV properties, such as locality, consistency, and entanglement(Nicolonn et al., 2024), is an interesting direction for future work. We hope that the gain in compu- tational speed of FasICAV facilitates concept-based expla- nations that were previously infeasible.
Lastly, we conduct an analysis ranking concepts by their area under the curve (AUC) during training and across lay- ers. Specifically, high AUC values indicate that concept in a certain layer is learned early, and the CAV achieves sufficient accuracy during the complete training. We vi- sualize the resulting rankings over the network blocks in Figure 7 and observe notable changes in the learnability
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning, specifically, explainability. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.

---

FastCAV; Efficient Computation of Concept Activation Vectors
References
Fang, Y., Sun, Q.. Wang, X., Huang, T., Wang, X., and
Cao, Y. Eva-02: A visual representation for neon genesis. Image and Vision Computing, 149:105171, 2024.
Alain, G. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644, 2016.
Ghorbani, A., Wexler, J., Zou, J. Y., and Kim, B. Towards automatic concept-based explanations. In Advances in Neural Information Processing Systems, pp. 9273-9282, 2019.
Barsalou, L. Perceptual symbol systems. The Behavioral and brain sciences/Cambridge University Press, 1999.
Bau, D., Zhou, B., Khosla, A., Oliva, A., and Torralba, A. Network dissection: Quantifying interpretability of deep visual representations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6541 6541-6549, 2017.
Ghosh, S., Yu, K., and Batmanghelich, K. Distilling black- box to interpretable models for efficient transfer learning. International Conference on Medical Image Comput- ing and Computer-Assiste Intervention, pp. 628-638. Springer, 2023.
Bau, D.. Zhu, J.,Y., Strobelt, H., Lapedriza, A., Zhou, B., and Torralba, A. Understanding the role of individual units in a deep neural network. Proceedings of the Na- tional Academy of Sciences, 117(48):30071-3000 2020.
Goh, G. Decoding the thought vector, 2016. URL https: /gabgoh. g github..... wo/Thought Ac- cessed: 2025-01-07.
Bishop, C. M. Pattern recognition and machine learning,
Graziani, M., Andrearczyk, V., and Müller, H. Regres- sion concept vectors for bidirectional explanations in histopathology. In Understanding and Interpreting Ma- chine Learning in Medical Image Computing Applica- tions: First International Workshops, MLCN 2018, DLF 2018, and iMMMIC 2018, Held in Conjunction with MIC- CAI 2018, Granada, Spain, September 16-20, 2018, Pro- ceedings 1, pp. 124 132. Springer, 2018.
volume 4. Springer, 2006.
Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Conerly, T., Turner, N. L., Anil, C., Denison, C., Askell, A., al. Towards Monosemanticity; Decompos- ing Language Models With Dictionary Learning. An- thropic, 2023.
Chang, C.-C. and Lin, C.-J. Libsvm: A library for support vector machines. ACM transactions on intelligent systems and technology (TIST), 2(3):1-27, 2011.
Gupta, A., Saini, S., and Narayanan, P. Concept distillation: leveraging human- explanations for model im- provement. Advances in Neural Information Processing Systems, 36, 2024.
Chapelle, O. Training a support vector machine in the primal. Neural Computation, 1955::115-1118, 2007. doi: 10. (022meco.2200.19.55111
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Dosovitskiy, A., Beyer, L.,, Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021.
Hsu, D., Muthukumar, V., and Xu, J. On the proliferation support vectors in high dimensions. In International Conference on Artificial Intelligence and Statistics, pp- 91-99. PMLR, 2021.
Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., -odds,,d-Dodss, 7.,, Lasenby, R., Drain, D., Chen, C., et al. Toy models of superposition. arXiv preprint arXiv:2209. 10652, 2022.
Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K. Q. Densely connected convolutional networks. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 4700--4088 2017.
Fan, R.-E., Chang, K.-W.. Hsieh, C.-J., Wang, X.-R., and Lin, C.-J. Liblinear: A library for large linear classifi- cation. Journal of machine learning research, 9(Au): 1874, 2008.
Huang, Q. Song, J., Xue, M.. Zhang, H., Hu, B., Wang, H., Jiang, H., Wang, X., and Song, M. Lg-cav: Train any concept activation vector with language guidance. arXiv preprint arXiv:2410.10308, 2024.
Fang, Y., Wang, W., Xie, B., Sun, Q., Wu, I.., Wang, X., Huang, T., Wang, X., and Cao, Y. Eva: Exploring the limits of masked visual representation learning at scale. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 19358-19369, 2023.
HuggingFace. https: https:// http///gggggggaa.eeeee wwaacecco/aee spaces / timm/ leaderboa rd. Accessed: [05/01/2025].
Timm leaderboard, 2025.
URL
10

---

FastCAV: Efficient Computation of Concept Activation Vectors
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ileus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpan- skaya, K., et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Thir AAAI Conference on Artificial Intelligence, 2019.
Olah, C., Cammarata, N., Schubert, L.,, Goh, G., Petrov, M., and Carter, S. Zoom in: An introduction circuits. Distill, 5()):00024-001, 2020.
Pahde, F., Dreyer, M., Weckbecker, M., Weber, L.,, Anders, J. Wiegand, T., Samek, W., and Lapuschkin, S. Navi- gating neural space: Revisiting concept activation vectors overcome directional divergence. In The Thirteenth International Conference on Learning Representations, 2025.
Johnson, A. E., Pollard, T. J., Greenbaum, N. R., Lun- gren, M. P., Deng, C.-y., Peng. Y., Lu, Z., Mark, R. G., Berkowitz, S. J., and Horng, S. Mimic-cxr-jpg, a large publicly available database of labeled chest radiographs. arXiv preprint arXiv: 000442 2019.
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z.., Gimelshein, N., Antiga, L., al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
Karpathy, A., Johnson, J., and Fei-Fei, L. Visualizing and understanding recurrent networks. arXiv preprint arXiv: 1506.02078, 2015.
Kim, B., Wattenberg, M., Gilmer, J., Cai, C.,, Wexler, J., Viegas, F,, and Sayres, R. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (teav), 2018. URL https: /aaxxiivvvorr org/ abs/1711. 11279.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al. Scikit-learn: Machine learning in python. Journal of machine learning research, 12(0ct):2825-288 2011.
Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., and Xie, S. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11976-119999 2022.
Penzel, N., Reimers, C., Bodesheim, P., and Denzler, J. Investigating neural network training on a feature level using conditional independence. In ECCV Workshop Causality in Vision (ECCV-WS), pp. 383-399, Cham, 2022. Springer Nature Switzerland. ISBN 978-3-031- 25075-0. doi: 10. 0000//978-3031-250000
Loshchilov, I. and Hutter, F. Decoupled weight decay regu- larization. arXiv preprint arXiv:/711.000 1711://71.00000 1.011105000,, 2017.
27.
Pfau, J., Young, T., Wei, J., Wei, M. L., and Keiser, M. J. Robust semantic interpretabillty: Revisiting con- cept activation vectors. arXiv preprint arXiv:2104.0m 200.00000,, 2021.
Moayeri, M., Rezaci, K.. Sanjabi, M., and Feizi, S. Text2concept: Concept activation vectors directly from text. In Proceedings of the IEEE/CVF Conference Computer Vision and Putlern Recognition, pp. 3744- 2023.
Platt, J. C. Fast training of support vector machines us- ing sequential minimal optimization. Advances in Kernel Methods: Support Vector Learning. The MIT Press, 12 1998. ISBN 9780262283199, doi: 10.7551/ mitpress/1130.0033.0 10. 75575551 wtw.55/mmmprees////e
Muthukumar, V., Narang, A., Subramaniian, V., Belkin, M., Hsu, D., and Sahai, A. Classification regression in overparameterized regimes: Does the loss function mat- ter? Journal of Machine Learning Research, 22(222); 1-69, 2021.
URL https: /tte::///oooooooo org/ 003.0016.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
Nicolson, A., Schut, I., Noble, J. A., and Gal, Y. Explain- ing explainability: Understanding concept activation vec- tors, 2024. URL /tttp::/arxivv.ort 03713.
Oikarinen, T. and Weng, T.-W. Clip-dissect: Automatic description of neuron representations in deep vision net- works. arXiv preprint arXiv:2204. 10965, 2022.
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bern- stein, M., al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115: 211 111-252, 2015.
Oikarinen, T. and Weng, T.W.. Linear explanations for individual neurons. arXiv preprint arXiv:2405.0685. 2024.
Schrouff, J., Baur, S., Hou, S., Mincu, D., Loreaux, E., Blanes, R., Wexler, J., Karthikesalingam, A., and Kim, Best of both worlds: local and global explanations
Olah, C., Mordvintsev, A., and Schubert, L. Feature visual- ization. Distill, 2(1 (117) 2017.

---

FastCAV: Efficient Computation of Concept Activation Vectors
with human-understandable concepts. arXiv preprint arXiv:2106.08644/ 2021.
A. M. Huggingface's transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771, 2019. URL https://api. http://aapp.saaaaisssicceee www.nticschoooeee.o..oooo org/ /CopppsID:223355599
Shashua, A. On the relationship between the support vector machine for classification and sparsified fisher's linear discriminant. Neural Processing Letters, 9(2):
Yeh, C.-K., Kim, B., Arik, S., Li, C.-L., Pfister, T., and Ravikumar, On completeness-awane concept-based explanations in deep neural networks. Advances in neural information processing systems, 33:20554-205555 2020.
129-139, Apr 1999. ISSN 1573-773X. doi: 10.1023/
A:1018677409366. URL attps:///oi.. org/10. 1023/A:10186778
Shwartz-Ziv, R. and Tishby, N. Opening the black box of deep neural networks via information. arXiv preprint 17X1:11003.00800. 2017.
Zhang, R., Isola, P., Efros, A. A.. Shechtman, E., and Wang. O. The unreasonable effectiveness of deep features as perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018.
Singh, M., Gustafson, L.,, Adcock, A., Reis, d. F., Gedik, B. Kosaraju, P., Mahajan, D., Girshick, R., Doll'ar ar, P., and van der Maaten, L. Revisiting Weakly Supervised Pre- Pre-Trainnn of Visual Perception Models. In CVPR, 2022.
Zhang, R., Madumal, P., Miller, T., Ehinger, K. A., and Rubinstein, B. I. Invertible concept- based explanations for cnn models with non-negative concept activation vec- tors. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 11682 11690, 2021.
Singla, S., Wallace, S., Triantafillou, S., and Batmanghelich, K. Using causal analysis for conceptual deep learning explanation. In Medical Image Computing and Computer Assisted Intervention-MICCAN 2021: 24th International Conference, Strasbourg, France, September 27-October 1, 2021, Proceedings, Part III 24, pp. 519-528. Springer, 2021.
Szegedy, C. Intriguing properties of neural networks. arXiv preprint arXiv:1312.661 2013.
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. Going deeper with convolutions. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 1-9, 2015.
Szegedy, C.,, Vanhoucke, V., loffe, S., Shlens, J., and Wojna, Z. Rethinking the inception architecture for computer vi- sion. In Proceedings of the IEEE conference computer vision and pattern recognition, pp.
2016.
Templeton, A. Scaling monosemanticiny: Extracting in- terpretable features from claude 3 sonnet. Anthropic, 2024.
Vapnik, V. Statistical learning theory. John Wiley Sons, 2: 831- 842, 1998.
Wightman, R. Pytorch image models. https: com/ 2019.
Wolf, T., Debut, L.,, Sanh, V., Chaumond, J., De- langue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C,, Jernite, Y., Plu, J., Xu, C., Scao, T. L.,, Gugger, S., Drame, M., Lhoest, Q., and Rush,
12

---

FastCAV: Efficient Computation of Concept Activation Vectors
A. FastCAV- Additional Details
In this section, we provide additional theoretical details connected to Section 3 in our main paper.
A.1. Expectation the Global Mean Under Gaussian Mixture Assumption
In FastCAV, we utilize the maximum likelihood estimator for the mean of a sample. This estimator is generally unbiased under i.idd samples following a Gaussian distribution, c.g., (Bishop, 2006). In Equation (2) in our main paper, we apply this the global mean over D. Dy.. However, consider the following related estimators:
PD.
Σ (x),, and PD. Del zED.
1
1
Σ g(x). ZED,
(8)
Dr
We now assume that both D, and D, contain i.i.d. samples that the corresponding activations g((x) follow two respective Gaussian distributions N (Me, Lc) and N (pr, E,). In other words, assume that both the random sample and concept example activations follow independent Gaussian distributions in the activation space of Then from the unbiasedness of the maximum likelihood estimator (Bishop, 2006) follows both
E(pp.] Me. and BiūD.] = Pr-
(9)
Using these equations, we now study the expectation of the global mean estimator j.DDD,, In (Kim et al., 2018), both the random sample and the set of concept examples are of equal size, which we follow in our work. Hence, we assume a uniform mixture the two Gaussians specified above. other words, mixture weights are 1/2, and De! = Dr]. Then the expectation of Equation (2) becomes
Elpp.uD.1
=
Σ g((x)
(10)
DrU DED,UD,
IE
1
Σ
g((x) + D [rrl
Σ 9 (x(
(11)
Del + Dr
ZED.
ZED,
1
(x) + Dr
Σ Ii (x))
(12)
2
D
D
*ED,
(((x)) ))
1
EE
(((T)
+
(13)
D
D,
RED,
nE D.
(BlúD.] + B((D),)
(14)
Me + Pr
(15)
2
Intuitively, the global mean is the middle point between the mean of the random samples and the concept examples in the activation space of gt- We then use this point to normalize the activations before calculating the direction towards the concept mean as our CAV of This vector is under the specified assumptions and an expectation over the sample D, Dr proportional Equation (6) our main paper.
13

---

FastCAV: Efficient Computation of Concept Activation Vectors
A.2. Empirical Ratio of Support Vectors
Table Percentage of support vectors of SVMs that learned the representation of 40 different concepts (Bau et al., 2017).
Each SVM was trained separate 50 example concept images from 50 random images. We average over 30 redrawn D and split report into the two classes defined by the sets D. and D,.
Support Vector Percentage
Model Architecture
Ø Activation Dimensionality
De
Dr
ResNet50 (He et al., 2016)
100,352
77.05
98.1 98.17%
ConvNeXt-XXL (Liu et al., 2022)
196,608
52.31%
90.40%
RegNetY (Singh et al., 2022)
1,064,444 448
78.00%
97.66%
ViT-B/16 (Dosovitskiy et al., 2021)
131,296
73.27%
78.87%
ViT-H/14-CLIP (Dosovitskiy et al., 2021)
328,960
87.85%
88.72%
EVA-G/14 (Fang et al., 2023)
812,416
78.69%
90.60%
EVA-02-L/14 (Fang et al., 2024)
1,049,600
88.45%
99.67%
In Table 3, we list the percentages of support vectors when using an SVM compute CAVs in the pre-final layer of various networks. In all cases, observe that the majority of activation vectors support vectors. This observation is congruent with other works that study this behavior for high dimensionalities in more detail, e.g., (Muthukumar et al., 2021; Hsu et al., 2021). However, interestingly, we find that the percentages are higher for the sets of random images D, in comparison to concept images D.- We hypothesize that this might be consequence of more compact activations for images sharing a semantic concept.
A.3. Runtime Benchmark of SVM Implementations
As described in our main paper, we benchmark the speed three common implementations of linear SVMs. First, the SMO (Platt, 1998) implementation contained in (Pedregosa et al., 201 1). which relies on (Chang & Lin, 2011). Second, we use the solution for the primal problem implemented (Fan al., 2008), which also included in (Pedregosa et al., 2011). Lastly, using Hinge loss, it is possible approximate linear SVM classifier using stochastic gradient descent (SGD), e.g.. (Pedregosa et al., 2011). We visualize the measured achieved runtimes in Figure 10 for both increasing numbers of examples and increasing input dimensions d. For our application, find that the SGD-based optimization performs best. Hence,
SWC
SVC
6gd
svd
SVc
12
Average Time (saconds)
2.0
2-
2500
5000
7500
10000
12500
15000 17500
20000
25000 50000 75000 100000 125000 150000 175000 200000 Dimension
Sample
(a) Dependency on the number of samples n. We fix the dimension-
(b) Dependency on dimensionality d. We set the total number of samples to 50.
ality to d = 2.
Figure 8: It can be seen that our method results in similar results.
we use this implementation for the classical SVM-based CAV calculation in our experiments. Note that this also follows
14

---

FastCAV; Efficient Computation of Concept Activation Vectors
related work (Kim al., 2018), which the same implementation.
B. Experiments
B.1. Experimental Setup Details
Technical details Our experimental setup utilized a system equipped with two Intel Xeon Platinum 8260 processors (48 cores total) and 384 GiB of RAM, along with 8 NVIDIA Tesla V100 GPUs, each providing 32 GiB of memory. For the calculation of the CAVs, we exclusively used the CPU-based infrastructure to ensure consistent and comparable environment for evaluating both FastCAV and SVM-CAV. To train the ResNet50, we leveraged the full capabilities the system.
Models For our experiments, we either report the metrics with respect to the last layer mentioned in the list or we average over all layers. In the case architectures composed of repeated blocks, e.g., ResNet, we refer in the list to the final layer of each block. The notation used follows from the implementations (Wightman, 2019) or (Wolf et al., 2019) to ensure reproducibility.
ResNet50 (He et al., 2016): conv1, layerl, layer2, layer3, layer4
Inception-v3 (Szegedy et al., 2016): Mixed 5b, Mixed 5c, Mixed 5d, Mixed 6a, Mixed 6b, Mixed 6c, Mixed 6d, Mixed_6e, Mixed_7a, Mixed_7b, Mixed_7c
ConwNeXt-XXLarge (Liu et al., 2022): stem, stagel, stage2, stage3, stage4
RegNetY (Singh et al., 2022): stem, stagel, stage2, stage3, stage4
VIT-B/16 (Dosovitskiy et al., 2021): embeddings, encoder.block.0, encoder.block.1, encoder.block.2, encoder.block.3, encoder.block.4, encoder.block.5, encoder.block6,, encoder.block.7. encoder.block.8. encoder.block.9, encoder.block. 10, encoder.block.11
ViT-H/14-CLIP (Dosovitskiy et al., 2021; Radford et al., 2021): bacc_e, blocks.0, blocks.7, blocks. blocks.23, blocks.31
EVA-G/14 (Fang et al., 2023): pwtch.emmeedeeeeee embedding, encoder.block. encoder.block.9, encoder.block. 19, encoder. blcoder.
EVA-02-L/14 (Fang et al., 2024): michhembedding, encoder.block.0, encoder.block.5, encoder.block.11 11, en- coder.block.17. 17, encoder.block.3
Setup of D. and D, For the experiments Section 4.1, we utilize the Broden dataset (Bau et al.. 2017) define the concept sets D.- Broden describes each of its 48 pattern-based concepts with 120 images. Here, sample 60 concept images per set D, from those 120 images once. Similarly, we sample 60 random images from ImageNet (Russakovsky et al., 2015) to define one set D, following (Kim et al., 2018)
Later, to calculate the accuracy a computed CAV (see Equation (4)), we sample independent validation data from the remaining 60 concept examples of the Broden dataset, (Bau et al., 2017), and ImageNet (Russakovsky et al., 2015), for D. and D,, respectively.
15

---

FastCAV; Efficient Computation of Concept Activation Vectors
B.2. Additional Empirical Comparisons of CAV Computation
300
Method
FastCAV (Ours)
SVM-CAV
250
Average Computation Time [s]
Model
ResNet50
200
Inception-v3
VIT-B/16
ConwNeXt-XXLarge
150
VIT-H/14-CLIP
EVA-G/14
EVA-02-L/14
100
RegNetY
0.2M
0.4M
0.6M
0.8M
1.0M
Dimensionality of CAVs [millions]
Figure 9: Comparison of computational efficiency between FastCAV and the established SVM-CAV for CAVs across different models for multiple layers. The average time taken to calculate CAV for each method is plotted against the average dimensionality the activation spaces, demonstrating the significant speedup achieved by our FastCAV. The differences computation time between FastCAV and the SVM-CAV are statistically significant (p 0.01) for all models. Note that SVM-CAV for the model RegNetY exceeded maximum computational time.
As complement to Figure in main paper, include Figure 9, where show the timings for all models averaged over the network layers. Note the clear difference for the RegNetY model (Singh et al., 2022) compared to Figure 1, due to the increased average activation space dimensionality. Calculating CAVs using SVMs was not feasible our setup.
Table 4: Comparing our approach FastCAV with SVM-CAV. In this table, we focus only on the pre-final layer before the prediction, which should encode the most semantic information. Bold values indicate hetter results. "N/A" indicates that no results were produced due to the overall computational time for the respective model exceeding four days.
Comp. Time [s]
Accuracy 1
Similarity f
Model
Dim.
FasICAV
SVM-CAV
FastCAv
SVM-CAV
FasICAV
SVM-CAV
Inter-Methe
Inception-v3
131,072
0.3
17.3
0.95-10.66
0.95:10.05
0.796:10.016
0.300
0.823=0.039
ResNet50
100,352
0.3::084
13.8:12.04
0.98=:0.04
0.98:10.04
0.823
0.190=t0.057
0.775=0.045
ConvNeXI-XXL
196,608
1.41:..0
45.7:14.73
0.96-10.05
0.98:10.04
0.859-100331
0.008
0.816=0.042
RegNetY
1,064,448
6.14
312.4=63.23
0.97=0.05
0.98:+0.03
0.825
0.128++0.066
0.675++0.06
ViT-B/16
151,296
1.0.0.033
50.5
0.87 0.09
0.85
0.813 0.022
0.454.10.04
0.854_10.048
ViT-H/14-CLIP
328,960
1.8 1..07
67.0.19.7
0.94:10.00 0.06
0.92
0.654 1. 0.080
0,328 1..084
0.894_1.006
EVA-G/14
812,416
4.7+0012
283.8432.33
0.95=0004
0.94+00.5
0.762-t0.059
0.201 +00066
0,848=0.044
EVA-02-L/14
1,049,600
6.110.114
252.2+29:57
0.93=0007
0.95
0.637=00075
0.119:#0.065
0.747=0.059
Similarly, we provide Table 4 an equivalent of Table containing the detailed numbers for only the pre-final layers of all networks in our study.
Both Figure 9 and Table 4 confirm the observations discussed in our main paper. However, as an additional explorative analysis, we investigate cases where FastCAV and SVM-based computation differ significantly.
16

---

FastCAV: Efficient Computation of Concept Activation Vectors
B.2.1. COMPARISON AGAINST OTHER CAV COMPUTATION APPROACHES
To provide further evidence for our claims regarding the improved computation speed, we rerun a smaller version with 30 CAVs per concept of our first experiment with additional computation methods (compare to Table 1). Specifeally, compare against the following approaches: logistic regression (LR) (Pfau et al., 2021), sparsified logistic regression (S-LR) (Singla et al., 2021), Ridge classification as a faster alternative (Pedregosa et al., 201 1), and classical LDA, which is closely related to our approach (see Section 3).
Table Comparing our approach FastCAV with other CAV calculation methods regarding computation time. Bold values indicate better results.
Comp. Time [s]
+
Model
FastCAV (Ours)
SVM
LDA
LR
S-LR
Ridge
ConvNeXI-XXLarge
0.019 0.013
1.167 0.795
16.8421 11.911
9.360 2.547
9.549 2.734
0.695.1 0.478
EVA-02-L/14
0.024: 0.001
1.524= 0.594
23,469# 3.404
7.659:4.669
7.693+ 4.731
0.923 0.019
EVA-G/14 GA14
0.018: 0.001
1.109= 0.263
19.135 2.200
8.499 4.763
8,457 4.696
0.681
Inception-v3
0.013:: 0.020
1.366= 0.943
10.755 7.188
8,681 6.076
8,648# 5.960
0,409: 0.258
RegNetY
0.039. :: 0.017
5.207 1.054
54.640 37.392
11,529 4.791
11.747 4.917
2.18441 1,300
ResNet50
0.016 0.019
2.908 2.440
7.827 5.601
8.375 5.509
8,474:1 5.687
0.314 0,225
ViT-B/16
0.005 0.002
0.852. 0.134
3,421
6.517 2,907
6.506 2.909
0.145 0.016
ViT-H/14-CLIP
0.008 0.001
0.456 0.078
8.867 1.037
7,091 2.658
7.118 2.647
0,288: 0.013
Table 6: Comparing our approach FastCAV with other CAV calculation methods regarding accuracy. Bold values indicate better results.
Accuracy f
Model
FastCAV (Ours)
SVM
LDA
LR
S-LR
Ridge
ConvNeXt-XXLarc
0.92+ 0.08
0.95 0.06
0.90= 0.15
0.96. 0.05
0.96 0.05
0.96: 0.05
EVA-02-L/14
0.90.:
0.90 0.15
0.79
0.91 0.15
0.91 0.15
0.91 0.15
EVA-G/14
0.89 0.15
0.88 0.15
0.21
0.90.1 0.15
0.90+ 0.15
0.90 0.15
Inception-v3
0.95 0.06
0.93 0.08
0.76
0.89 0.14
0.89+ 0.14
0.89 0.15
RegNetY
0.97 0.05
0.98 0.03
0.63
0.96:1 0.05
0.964 0.05
0.98 0.04
ResNet50
0.89 0.15
0.87 0.15
0.72+
0.92 0.11
0.92+
0.93 0.12
ViT-B/16
0.83#: 0.13
0.81 0.14
0.89 0.12
0.824 0.14
0.82+ 0.14
0.824 0.15
ViT-H1/14CCIIP
0,88 0.15
0.87 0.15
0.82+
0,89 0.15
0.89 0.15
0.89 0.16
Table 7: Comparing our approach FastCAV with other CAV calculation methods regarding similarity. Bold values indicate better results.
Similarity t
Model
FasICAV (Ours)
SVM
LDA
LR
S-LR
Ridge
ConyNeXt-XXLaarg
0.913
0.525 0.049
0.521 0.105
0.672-1 0.044
0.671 1:: 0.044
0.570.: 0.033
EVA-02-I/114
0.814: 0.020
0.291 1: 0.054
0.4433 0.086
0.685 0.052
0.688 0.050
0.606:: 0.023
EVA-G/14
0.789: 0,018
0.331 0.049
0,418: 0.070
0.6724 0.046
0,670: 0.050
0.608:: 0.025
Inception-v3
0.826:: 0.011
0.387 # 0.058
0.132# 0.131
0,602+ 0.037
0,603 0.036
0.562: 0.026
RegNetY
0.825
0.128 0.066
0.027
0,697 0.063
0.698:: 0.062
0.675 0.020
ResNet50
0.791
0.398 0.062
0.077+ 0.159
0.6524 0.042
0.6544 0.040
0.6024 0.026
ViT-B/16
0.752+ 0.029
0.394 0.048
0.556# 0.121
0.592+ 0.058
0.591 0.059
0.508 0.037
ViT-H/14-CLIP
0.749+ 0.020
0.342 0.048
0.551 0.149
0.621 0.025
0.621 0.025
0.595+ 0.023
We summarize the comparison results in Tables to 7. While all approaches deliver high accuracies, the logistic regression- based approaches achieve the second highest similarities after FastCAV, which coincides with increased computation times. In contrast, ridge classification offers notable improvement in speed over SVM-based computation. However, our FastCAV still yields a substantial speed-up, while maintaining competitive performance to the more sophisticated approaches. Further, we can see that vanilla LDA is not enough to achieve similar gains. Hence, we argue that the assumptions we make in Section 3.3 are necessary under practical considerations.
17

---

FastCAV; Efficient Computation of Concept Activation Vectors
B.2.2. SIGNIFICANT ACCURACY DIFFERENCES BETWEEN FASTCAV AND SVM-CAV
While believe that our empirical evaluation provides evidence for the practicality of our approach, the strong, but necessary, assumptions (see Section 3.3) are likely not hold in practice. Hence, explored concrete examples where FastCAV and SVM-based computation differ in predictive performance. For example, find in ViT-B/16 (Dosovitskiy et al., 2021), after the encoder layer 10, an accuracy difference of 40% with the SVM yielding an accuracy of 95% and FastCAV of 55%. Overall, we observed significant accuracy differences (over 25 percentage points) in 2.8% of the CAVs identified, favoring SVM in 1% and FastCAV in 1.8% of cases. In the former case, we believe that these findings are cireumstances where specifically the isotropic Gaussian assumption violated, meaning the means of De and D, are close together in comparison to the convex hull of both sets. Overall, we find that in 43.6% of the cases FastCAV achieves higher accuracy, in 36.9% % of the cases SVM-CAV outperforms, and in 19.5% of the cases both methods perform equally. These observations support the findings in Section 4 in our main paper and generally favor FastCAV.
10a
SVM FastCAV
SVM FastCAV
60000
SVM FastCAV
Number Obsrrvatoon (Lag Scale)
50000
Number of Obsorvations
40000
10a
30000
10
10000
10a
-0.4
-0.2
0.0
-014
0.2 Accurscy Difference Between FastCAV and SVM
-0.2
Accuracy Difference Between FastCAV and VVM
Figure 10: Left: Histogram of accuracy differences between FastCAV and SVM-CAV. Note the logarithmic y-axis scale, Right: Cumulative histogram (empirical CDF) of accuracy differences between FasICAV and SVM. Both figures show the differences for all layers of all models (see Appendix B.1). Negative values indicate that SVM-CAV achieved higher accuracy, while positive values indicate that FasCCAV performed better. Agreement between both methods (a difference of zero) is colored grey, and the grey vertical dashed lines mark the 25th and 75th percentiles (QI and Q3).
Next, we compare FastCAV and SVM-based computation in a more specialized task aiming to extract CAVs for medical concepts (Johnson et al., 2019; Irvin et al., 2019) to show its broad applicability.
B.2.3. CAV COMPUTATION FOR MEDICAL CONCEPTS
In our main paper, we focus on eeneral-purpose models trained on ImageNet (Russakovsky et al., 2015) and consider broad of concepts (Bau et al., 2017). Here, include analysis of specialized second task. Specifically, follow related work exploring medical concepts (Singla al., 2021) and train DenseNet-121 (Huang et al., 2017) subset of MIMIC-CXR (Johnson et al., 2019).
Regarding training details, we use AdamW (Loshchilov & Hutter, 2017) with a learning rate of 1e - 4, a weight decay of 5e 5, and a batch size of 64. We train for 100 epochs and select the model achieving the highest multi-label AUROC (0.7735) on an independent test set for our concept-based analysis.
Next, use the CheXpert labeler (Irvin al., 2019) to extract concept mentions from the included free-text radiology reports included with MIMIC-CXI (Johnson et al., 2019). As set of possible concepts concerning chest x-ray images, again follow (Singla et al., 2021).
With these extracted concepts, we compute CAVs for the four blocks of the DenseNet-122 121 using our FasCCAV, SVM-based computation (Ghosh et al., 2023), and sparse logistic regression (Singla et al., 2021).
In our main paper (in Table 2), we report the average computation time, CAV accuracy, and intra-method similarity. Here
18

---

FastCAV; Efficient Computation of Concept Activation Vectors
Table 8: Comparing our approach FasICAV with SVM-CAV and sparse logistic regression for concepts concerning chest X-ray images. Bold values indicate better results. Leading zeros were removed to the space constraints.
Comp. Time [s]
Accuracy
Similarity t
Concept
FastCAV
S-LR
SVM
FasICAV
S-LR
SVM
FastCAV
S-LR
SVM
Air Bronchogram
.00
5.93+
.11.09
.75+.07
.73+.14
.87+.02
.49..03
.43+.04
Ards
00071.000
5.97.10.54
2.9
.2...11
.794.07
.76
.891.01
.541.04
.48
Blunt
.00
6.71:10.79
.41
.72+.113
.594.07
16
.851.03
.3:t.05
.401.05
Cardiac Silhouette
.00:1_000
5.87.1044
.7662.07
.70+.07
.694.09
2.16
.2.1.03
.94.04
.462.03
Cephalization
007:..000
6.36:+0.54
.005.04
.72+.10
.0+.113
.0+
.85+.02
.574.02
.49
Congestion
.00:::000
6.145
.24
.784.06
.78:.07
.09
.861.02
.881.03
.494.04
Heart Size
.00:1:000
0.164
435
.72+.10
.73+..10
.76
.81..03
.471.05
:45
Hilar Contour
.0011.000
6.71:11.0
442+.02
.71:.11
.694.08
.677.08
.78
.2:4.04
.424.03
Hilar Engorgement
.006:.001
6.62+0081
.70+004
.75..10
82+.07
.72+.10
.84+.02
.54..05
+48+.04
Hilar Opacity
.007=.000
7.01
451
.88..13
.75+.06
.63+.05
.79
.7:
.43
Interst. Edema
.006:.000
6.23+40.81
414+.0
824.07
83+.066
80+.05
.89..01
.52+.04
.494.04
Interst. Markings
.0061.001
6.20
4631.05
.651..99
.70.1.10
.711.10
.831.02
.51.1.33
.441.04
Interst. Prominence
10051.000
6.72.10.64
1271.05
.7311.16
.7811.16
.781.05
1..1.02
.6.1.03
.421.02
Peribronch. Cuffing
.0071.000
6.37
4781.07
.72.1.05
.65.1.06
.6911.3
.72.1.04
.51.102
.481.02
Pleural Fluid
.006::001
1.5
.34+.03
.76:2.13
.72+:09
.72+.12
.82+.02
.45+.04
.43
Vascular Marking
.006::001
5.7440087
.39+.05
.81+.07
.794.09
.74+.13
.854.02
.494.04
.46+.03
we provide a more detailed evaluation of the specific concepts after the dense-block 3, following (Singla et al., 2021).
Table 8 summarizes the computation time, CAV accuracies, and intra-class similarities for the set concepts. Overall, we find that the DenseNet-121 (Huang et al., 2017) learned the specialized concepts, and we find high accuracies for all three computation methods. However, the specific accuracies per concept vary slightly. Further, see differences in the average similarity for the three methods. FastCAV again performs best with respect to speed and similarity.
Additionally, we investigate CAV computation with respect to images containing negative results for the concepts as found by the CheXpert labeler (Irvin et al., 2019). In contrast to our expectation, we observe lower CAV accuracies in comparison to random images. However, this also holds for SVM and sparse-logistic regression-bars based computation, indicating a data sampling problem in the negative example regime.
Lastly, following the similar results to established methods, we wish highlight the applicability of FastCAV for specialized downstream tasks, e.g., (Singla al., 2021; Ghosh et al., 2023). We consider this valuable direction for future work.
B.3. TCAV Additional Results
In our main paper. include the results for TCAV (Kim et al., 2018) for the class ladybug using both our FastCAV as well SVM-based CAV computation. To further strengthen analysis, we follow (Kim et al., 2018) and evaluate additional qualitative examples. Here, we follow the setup detailed in Section 4.3 and use GoogleNet (Szegedy al., 2015) trained on ImageNet (Russakovsky et al., 2015). We visualize the TCAV scores for the same layers selected in (Kim et al., 2018).
fire-engine, police-van, and zebra, respectively. In all cases, we find only small differences between our CAV computation and the established SVM-based approach. Similar to our discussion the main paper, we find lower variances using FastCAV. Nevertheless, the overarching trends are consistent between both methods. Further, the resulting insights into the GoogleNet model align well with (Kim al., 2018). For example, for both methods, find the concept "red" to be most important for the classes ladybug and fire-engine. Similarly, we observe "blue" influencing class police-van and "striped" influencing class zebra. This again demonstrates that both FasICAV and SVM-CAV can be used to generate valid concept-based excsseeexxppnee explanations, supporting the claims made in our main paper.
Figure 11, Figure 12, Figure 13, and Figure 14 display classes
19

---

FastCAV; Efficient Computation of Concept Activation Vectors
Layer
mixed3a
mixed3b
mixed4a
mixeddbb
mixod4c
TCWV Soure
mixedde
mixod5a
mixed5b
Method
PastCav
0.2
0.2
0.0
0.0 red
red
yellow
blue
green
yellow
blue
groen
Concept
Concept
Figure 11: Example of the concepts "red", "yellow", "blue" and "green" for class ladybug using TCAV. We follow (Kim
al., 2018) and mark CAVs that statistically significant with "**
Layer
1.0
1.0
mixed3b
mixedds
0.8
mixeddb
mixedMe
mixed4d
TCAV TCAV Score
mixed4e
mixed5b
Method
SVM-CAV
rod
yallow Concept
green
red
yellow
green:
Concegt
Figure 12: Example of the concepts "red", "yellow", "blue" and "green" for class fire-engine using TCAV. We follow (Kim et al., 2018) and mark CAVs that are not statistically significant with
Layer
1.0
mixed3a
mixed3b
mixed4a
0.8
0.8
mixed4b
mixed4e
mixed4d
TCWV Soote
0.6
0.6
mixed5a
mixedSb
0.1
0.4
Method
RastCav
SVM-CAV
0.2
0.2
*0k
red
yellow
blue
red
yellow
blue
Concept
Concept
Figure 13: Example of the concepts "red", "yellow", "blue" and "green" for class police-van using TCAV. We follow (Kim ct al., 2018) and mark CAVs that are not statistically significant with cogn
20

---

FastCAV; Efficient Computation of Concept Activation Vectors
Layer
mixed3a
mixed3b
mixeddA
0.8
0.8
mixed4b
mixed4c
mixed4d
TCW Soute
0.6
0.6
mixed5a
mixedSb
0.4
0.4
Method
PaseCaav
We
SVM-CAV
0.2
0.2
poika-dotled
striped
sigcegged
pocka-dotted
staiped
zigrsggeed
Concept:
Concegt
Figure 14: Example of the concepts "polka-dotted", "striped" and "zigzagged" for class zebra using TCAV. We follow (Kim ct al., 2018) and mark CAVs that are not statistically significant with cqm
Table The TCAV scores (Kim al., 2018) for the four salient concepts discovered in ImageNet (Russakovsky al., 2015) trained Inception-v3 (Szegedy al., 2016) model using ACE (Ghorbani al., 2019). We list scores for the classes Lionfish, Zebra, and Police Van.
Lionfish
Zebra
Police Van
Concept
FastCAV
SVM-CAV
FastCAV
SVM-CAV
FastCAV
SVM-CAV
0.78
0.73
0.68
0.66
0.88
0.78
2
0.75
0.73
0.67
0.61
0.86
0.77
3
0.69
0.67
0.60
0.58
0.83
0.76
4
0.68
0.67
0.59
0.56
0.80
0.70
B.4. ACE Additional Results
In our main paper, we visualize and compare the most salient concepts for the class Lionfish found by ACE with FasICAV and SVM-CAV. Here, we add to this and visualize the other classes selected in (Ghorbani al., 2019). For class Lionfish, ACE using FastCAV finds 22 relevant concepts, and with SVM-CAV, using our approach and 18 with SVM computation. Lastly, for class Police Van, we identify 24 and 22, respectively. In Table 9, we list the TCAV scores of the four most salient concepts for each class, respectively.
identify 24 overall. For Zebra, find 19 concepts
In Figure 15, Figure 16, and Figure 17, we display the three most salient concepts for both methodologies. In all cases, we observe similar results for both FastCAV and SVM-CAV, supporting our findings in the main paper.
21

---

FastCAV; Efficient Computation of Concept Activation Vectors
Concept 1
Concept 1
Most Salient
Most Salient
39
31
36
Concept 2
Concept 2
2nd most salient
2nd most salient
23
30
Concept 3
Concept 3
3nd most salient
3nd most salient
23
FastCAV based ACE (Ghorbani al., 2019).
SVM-CAV based ACE (Ghorbani et al., 2019).
Figure 15: Comparison of the most salient concepts discovered by ACE (Ghorbani et al., 2019) using either our FastCAV or the established SVM-CAV. Here, we use class lionfish and display the three most salient concepts. We find the discovered patches between both approaches similar and congruent with the original observation in (Ghorbani al., 2019).
22

---

FastCAV; Efficient Computation of Concept Activation Vectors
Concept 1
Concept 1
Most Salient
Most Salient
23
24
Concept 2
Concept 2
2nd most salient
2nd most salient
24
28
123
Concept 3
Concept 3
3nd most salient
3nd most salient
20
FastCAV based ACE (Ghorbani al., 2019).
SVM-CAV based ACE (Ghorbani et al., 2019).
Figure 16: Comparison of the most salient concepts discovered by ACE (Ghorbani et al., 2019) using either our FastCAV or the established SVM-CAV. Here, class zebra and display the three most salient concepts. We find the discovered patches between both approaches similar and congruent with the original observation in (Ghorbani al., 2019).
23

---

FastCAV; Efficient Computation of Concept Activation Vectors
Concept 1
Concept 1
Most Salient
Most Salient
PO
36
21
Concept 2
Concept 2
2nd most salient
2nd most salient
Concept 3
Concept 3
3nd most salient
3nd most salient
POLICE
POLICE
in
POLICE
POLICE
in
20
20
(a) FastCAV based ACE (Ghorbani al., 2019).
SVM-CAV based ACE (Ghorbani et al., 2019).
Figure 17: Comparison of the most salient concepts discovered by ACE (Ghorbani et al., 2019) using either our FastCAV or the established SVM-CAV. Here, class police and display the three most salient concepts. We find the discovered patches between both approaches similar and congruent with the original observation in (Ghorbani al., 2019).
24

---

FastCAV: Efficient Computation of Concept Activation Vectors
B.5. Tracking CAVs During Training Additional Visualizations
Setup Details: We train a ResNer50 (He al., 2016) model the ImageNet 2012 dataset (Russakovsky et al., 2015) using a batch size of 256 for 90 epochs following (Paszke et al., 2019). Each image is resized to 256 pixels, followed by random crop of 224 by 224, random horizontal flip, and normalization with mean values of 0.485, 0.456, and 0.406 and standard deviations of 0.229, 0.224, and 0.225. The optimizer is Stochastic Gradient Descent with initial learning rate of 0.1, momentum of 0.9, weight decay of le-4, and StepLR scheduler with a step size of 30 epochs and a gamma of 0.1.
Additional Visualizations: In our main paper (Figure 6), we visualize the average CAV accuracies during the training of a ResNet50 (He et al., 2016). We calculate this average over the concepts included (Bau et al., 2017). In Figure 18, we visualize the same averages but additionally include lines for all individual concepts to highlight the variances.
Average CAV Accuracy in Block
Average CAV Accuracy in Block 2
0.9
0.9
0.8
0.8
CAV Accuracy
CAV Accuracy
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0.4
average
average
0.3
20
40
60
80
100
20
40
60
80
100
Epoch
Epoch
(a) Final layer ResNe150 Block
(b) Final layer ResNet50 Block
Average CAV Accuracy in Block
Average CAV Accuracy in Block 4
0.8
average
0.9
0.7
CAV Accuracy
0.8
CAV Accuracy
0.6
0.7
0.5
0.6
0.4
0.5
average
0.3
0.4
20
40
60
80
100
20
40
60
80
100
Epoch
Epoch
(c) Final layer ResNe150 Block 3.
(d) Final layer ResNe150 Block 4
Figure 18: Additional visualizations pertaining to the lefthand side in Figure 6. Specifically, we visualize the average
accuracies achieved by CAVs after the final layers in each of the four ResNet blocks (Figure 18a Figure 18d) for the concepts in (Bau al., 2017). Here, we display the changes for all concepts to showcase the respective variances during training. Vertical dotted lines again correspond to epochs where the learning rate was divided by ten.
Similarly, following our qualitative selection specific concepts for the final layer of block 3 in Figure 6 (middle plot), we include visualizations for the other blocks in Figure 19. We confirm that the observations made in the main paper similarly
25

---

FastCAV; Efficient Computation of Concept Activation Vectors
hold for the other layers. In addition, find the lowest CAV accuracies in Figure 19d, which consistent with the results for the development of the average concept.
CAV Accuracies in Block 1
CAV Accuracies in Block 2
polka-dotted
polka-dotted
0.8
striped
0.8
striped
zigzagged
zigzagged
W MiR W
CAV Accuracy
0.7
CAV Accuracy
0.7
M
0.6
0.6
0.5
0.5
0.4
0.4
20
40
60
80
20
40
60
80
Epoch
Epoch
(a) Final layer ResNet50 Block 1.
(b) Final layer ResNet50 Block
CAV Accuracies in Block 3
CAV Accuracies in Block 4
A Mam
MM M N
0.70
0.8
0.65
CAV Accuracy
CAV Accuracy
0.60
0.7
M
0.55
0.6
0.50
0.45
polka-dotter
dalka-dotted striped zigzagged
0.5
striped
0.40
zigzagged
0.35
20
40
60
20
Epoch
Epoch
(c) Final layer ResNet50 Block
(d) Final layer ResNet50 Block 4.
Figure 19: Additional visualizations for the middle plot in Figure 6. Specifically, we qualitatively visualize the accuracies of selected CAVs after the final layers each of the four ResNet blocks (Figure 19a Figure 19d) for the concepts in (Bau al., 2017). Figure 19c equivalent to the selected version Figure 6. Vertical dotted lines again correspond epochs where the learning rate was divided by ten.
Additional Explorative Analysis To expand our discussion regarding the training analysis (see Section 4.5 and above), we provide a more comprehensive exploration of the potential FastCAV.
First, in our main paper, we include results ranking concepts by their area under the curve (AUC) during training and across layers. This reveals that the leamability/difiicllty of concepts can vary wildly across layers, with some concepts that quickly learned in early layers becoming more difficult to learn in later layers.
Here, we go one step further and evaluate the ratio of learned concepts per layer during the training. In Figure 20, we summarize the corresponding results. We find that during the training, the ratio of concepts learned increases, indicating that the model learns task rellvrelevant concepts. These findings are congruent with previous analysis neural network training dynamics, e.g., (Shwartz-Ziv & Tishby, 2017; Penzel et al., 2022). Further, we observe the highest ratios during the complete
26

---

FastCAV; Efficient Computation of Concept Activation Vectors
0.5
block1
block2
block3
Myll
Concepts Learned [%]
0.4
block4
0.3
0.2
0.1
0.0
20
40
60
80
Epoch
Figure 20: Percentage of learned concepts by CAVs and block during training of ResNet50 (He et al., 2016) on ImageNet (Russakovsky et al., 2015).
training process for the architecture block 3. Given that our probing dataset focuses on the textures, this observation aligns with findings made in (Bau al., 2017).
Overall, hope that by providing faster and more efficient concept extraction procedure with FastCAV, empower researchers to explore more complex scenarios and larger concept sets, potentially uncovering novel phenomena that were previously inaccessible due to computational constraints.
27


Reviews:
[{"review_id": "EnovgB5qTt", "reviewer": "Reviewer_ULp4", "summary": "The paper introduces fastcav, extending tcav to identify concept activation vectors by computing the mean of vectors and find the approximate direction of concept activation vectors to the concept space, which is assumed to be nearly orthogonal. The usage of mean reduces the dimensional complexity of using svm in tcav.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their comments and thoughtful feedback. Specifically, we appreciate that the reviewer found our comparison between FastCAV and SVM-based computation valid and reliable. Concerning the lack of comparison to other CAV calculation methods, we note that we focused on SVMs following our theoretical analysis, which provides a connection to this existing approach. However, we agree with the reviewer that a comparison to other methods is beneficial to provide further evidence for our claims regarding the improved speed. Hence, we reran a smaller version of our first experiment (compare to our Table 1) and added the following approaches: logistic regression (Pfau et al. 2021, “Robust Semantic Interpretability: Revisiting Concept Activation Vectors”), sparsified logistic regression (Singla et al. 2021, “Using Causal Analysis for Conceptual Deep Learning Explanation”), Ridge classification as a faster alternative (Pedragosa et al. 2012, “Scikit-learn: Machine learning in python”), and LDA, which is closely related to our approach. Please excuse the simplified formatting due to markdown of the table presented here. We will ensure improved formatting in the updated paper.\n\n|Model|Method|Comp. Time [s]↓|Accuracy↑|Similarity↑|\n|---|---|---|---|---|\n|Inception-v3|FastCAV (Ours)|**0.013±0.020**|**0.950±0.059**|**0.826±0.011**|\n|Inception-v3|SVM|1.366±0.943|0.934±0.079|0.387±0.058|\n|Inception-v3|LDA|10.755±7.188|0.759±0.189|0.132±0.131|\n|Inception-v3|Logistic Regression|8.681±6.076|0.892±0.142|0.602±0.037|\n|Inception-v3|Sparse-Logistic Regression|8.648±5.960|0.895±0.141|0.603±0.036|\n|Inception-v3|Ridge|0.409±0.258|0.894±0.148|0.562±0.026|\n||||||\n|ResNet50|FastCAV (Ours)|**0.016±0.019**|0.890±0.149|**0.791±0.030**|\n|ResNet50|SVM|2.908±2.440|0.871±0.147|0.398±0.062|\n|ResNet50|LDA|7.827±5.601|0.717±0.199|0.077±0.159|\n|ResNet50|Logistic Regression|8.375±5.509|0.921±0.115|0.652±0.042|\n|ResNet50|Sparse-Logistic Regression|8.474±5.687|0.924±0.113|0.654±0.040|\n|ResNet50|Ridge|0.314±0.225|**0.925±0.115**|0.602±0.026|\n||||||\n|ConvNeXt-XXLarge|FastCAV (Ours)|**0.019±0.013**|0.923±0.081|**0.913±0.014**|\n|ConvNeXt-XXLarge|SVM|1.167±0.795|0.948±0.059|0.525±0.049|\n|ConvNeXt-XXLarge|LDA|16.842±11.911|0.895±0.149|0.521±0.105|\n|ConvNeXt-XXLarge|Logistic Regression|9.360±2.547|**0.962±0.051**|0.672±0.044|\n|ConvNeXt-XXLarge|Sparse-Logistic Regression|9.549±2.734|0.961±0.051|0.671±0.044|\n|ConvNeXt-XXLarge|Ridge|0.695±0.478|0.961±0.051|0.570±0.033|\n||||||\n|RegNetY|FastCAV (Ours)|**0.039±0.017**|0.969±0.046|**0.825±0.009**|\n|RegNetY|SVM|5.207±1.054|**0.977±0.034**|0.128±0.066|\n|RegNetY|LDA|54.640±37.392|0.633±0.147|0.027±0.115|\n|RegNetY|Logistic Regression|11.529±4.791|0.964±0.052|0.697±0.063|\n|RegNetY|Sparse-Logistic Regression|11.747±4.917|0.963±0.053|0.698±0.062|\n|RegNetY|Ridge|2.184±1.300|**0.977±0.038**|0.675±0.020|\n||||||\n|ViT-B/16|FastCAV (Ours)|**0.005±0.002**|0.827±0.129|**0.752±0.029**|\n|ViT-B/16|SVM|0.852±0.134|0.806±0.143|0.394±0.048|\n|ViT-B/16|LDA|3.421±0.443|**0.887±0.118**|0.556±0.121|\n|ViT-B/16|Logistic Regression|6.517±2.907|0.823±0.144|0.592±0.058|\n|ViT-B/16|Sparse-Logistic Regression|6.506±2.909|0.824±0.143|0.591±0.059|\n|ViT-B/16|Ridge|0.145±0.016|0.816±0.155|0.508±0.037|\n||||||\n|ViT-H/14-CLIP|FastCAV (Ours)|**0.008±0.001**|0.875±0.153|**0.749±0.020**|\n|ViT-H/14-CLIP|SVM|0.456±0.078|0.872±0.151|0.342±0.048|\n|ViT-H/14-CLIP|LDA|8.867±1.037|0.823±0.174|0.551±0.149|\n|ViT-H/14-CLIP|Logistic Regression|7.091±2.658|**0.891±0.154**|0.621±0.025|\n|ViT-H/14-CLIP|Sparse-Logistic Regression|7.118±2.647|**0.891±0.154**|0.621±0.025|\n|ViT-H/14-CLIP|Ridge|0.288±0.013|0.889±0.158|0.595±0.023|\n||||||\n|EVA-G/14|FastCAV (Ours)|**0.018±0.001**|0.890±0.147|**0.789±0.018**|\n|EVA-G/14|SVM|1.109±0.263|0.884±0.149|0.331±0.049|\n|EVA-G/14|LDA|19.135±2.200|0.809±0.211|0.418±0.070|\n|EVA-G/14|Logistic Regression|8.499±4.763|**0.903±0.148**|0.672±0.046|\n|EVA-G/14|Sparse-Logistic Regression|8.457±4.696|0.902±0.149|0.670±0.050|\n|EVA-G/14|Ridge|0.681±0.023|0.902±0.153|0.608±0.025|\n||||||\n|EVA-02-L/14|FastCAV (Ours)|**0.024±0.001**|0.898±0.143|**0.814±0.020**|\n|EVA-02-L/14|SVM|1.524±0.594|0.897±0.155|0.291±0.054|\n|EVA-02-L/14|LDA|23.469±3.404|0.792±0.200|0.443±0.086|\n|EVA-02-L/14|Logistic Regression|7.659±4.693|0.909±0.148|0.685±0.052|\n|EVA-02-L/14|Sparse-Logistic Regression|7.693±4.731|0.909±0.147|0.688±0.050|\n|EVA-02-L/14|Ridge|0.923±0.019|**0.911±0.152**|0.606±0.023|\n\nRidge classification offers a notable improvement in speed over SVM-based computation, our FastCAV still yields a substantial speed-up, while maintaining competitive performance. Further, we can see that vanilla LDA is not enough to achieve similar gains. Hence, we argue that the assumptions we make for FastCAV (compare Section 3.3) are necessary given practical considerations.", "claims_and_evidence": "The major claim made in this paper is that fastcav is significantly faster than tcav, which is shown by the complexity analysis shown in section 3.4 and experiments in section 4. \nMoreover, the claim that concept activation vectors computed through svm and fastcav are similar is also valided through comparing the cosine similarity of vector directions between svm and fastcav.", "methods_and_evaluation_criteria": "The paper selects concepts from the broaden dataset and images from the imagenet dataset which are standard for the problem discussed. The proposed method signifantly reduces computational cost by eliminating need for svm training. The svm training is especially costly due to the presence of high dimensional intermediate feature representations.", "theoretical_claims": "Not checked", "experimental_designs_or_analyses": "The experiments are mainly constructed to assess the accuracy and efficiency of identified cavs, which are sound, the usage of imagenet with multiple classes and multiple concept directions per neuron and acquisition of concepts from the widely tested broaden dataset makes the experiments valid and reliable.  A major drawback is that comparisions is only performed svm-cav and not with other concept based explanation approaches .", "relation_to_broader_scientific_literature": "The key contribution is the efficient computation of concept activation vectors, a popular concept based neural network explanation technique.", "essential_references_not_discussed": "Not Discovered", "other_strengths_and_weaknesses": "Nil", "other_comments_or_suggestions": "Nil", "questions_for_authors": "How efficient and accurate fastcav is when compared to other concept explanation techniques?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "awLLJEDTDg", "reviewer": "Reviewer_CrYA", "summary": "The paper introduces a faster approximation to compute concept activation vectors. They do so through computing the mean vector for representations for the concept of interest and a random reference concept. The authors demonstrate that their approach is the same as the standard approach for computing concept activation vectors. However, through complexity analysis, the authors demonstrate that their algorithm runs faster. Experimentally, the authors demonstrate that their CAV method achieves the same or higher accuracy (compared to an SVM-based method), while reducing the computation time significantly.\n\n## Update after rebuttal\nMy original score was a 4, and I am satisfied with their rebuttal; as such, I have decided to keep my score as a 4.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the thoughtful feedback of the reviewer. We are glad that the logic in Sections 3.3 and 3.4 was judged as reasonable and intuitive. Similarly, we welcome the feedback regarding our experiments in Section 4. Particularly, the reviewer asked about our choice of pairwise cosine similarity, which we use to measure robustness. We would like to clarify the reasoning behind this decision and provide additional context.\n\nWe consider pairwise similarity for robustness because it allows us to quantify the consistency and reliability of the concept activation vectors (CAVs) obtained from different methods. CAVs correspond to directions in a model’s activation space, which encode abstract concepts. Hence, for the decision boundary (Eq. (1)), we normalize the length of the CAV to unit length [Line 148, right-hand side]. Now, to compare the directions calculated with different methods for a given concept, we can directly compute the cosine of the angle, i.e., the cosine similarity, between the two vectors. Further, it is standard procedure to compute CAVs repeatedly against varying sampled random sets $D_r$ (Kim et al. 2018).\nIn particular, pairwise similarity helps to evaluate robustness in two ways: \n1. By calculating the pairwise similarity between FastCAV and SVM-based computation we can determine how closely aligned both methods are in identifying the direction corresponding to a concept.\n2. By calculating the pairwise cosine similarity matrix for CAVs obtained from the same method while resampling $D_r$, we can assess the consistency of the concept directions across repetitions. Averaging over this similarity matrix provides a score that quantifies the robustness of the identified concept directions within a single method. A high average similarity score indicates that the method is producing consistent and reliable concept directions, which is essential for robustness.", "claims_and_evidence": "The authors make the following claims in their paper: \n1. **Their fastCAV method is theoretically equivalent to SVM-based CAV under some assumptions** - This claim I believe is true. They show this by demonstrating that the expected decision boundary under a Gaussian setup exactly corresponds to the difference in means\n2. **Their fastCAV method is significantly faster than the regular SVM-based CAV** - This claim is verified through their experiments, where they compare the computation time across different concepts in the Broaden dataset. Additionally, they verify such a claim theoretically through complexity analysis, where they demonstrate that their method (fastCAV) is faster by a factor of min(n,d) (compared to SVM-based approaches). \n3. **Their fastCAV method maintains performance compared to regular SVM-based CAV** - They confirm this by comparing the accuracy of their predicted CAVs to separate the concepts in the random category from the concepts in the target category", "methods_and_evaluation_criteria": "The authors evaluate their fastCAV method through the Broaden dataset and ImageNet. The TCAV paper also uses the ImageNet dataset, and I believe that the particular dataset chosen is not of utmost importance (because CAV as a method can be used across datasets as long as concepts are labeled). Their methods are also reasonable for the task, as they demonstrate it's a simplified approximation to the overall CAV problem.", "theoretical_claims": "The logic in Section 3.3 and Section 3.4 are reasonable, and I believe fairly intuitive.", "experimental_designs_or_analyses": "The experiments in Section 4 are reasonable; they compute the two most important quantities (accuracy and speed) and demonstrate that their method outperforms the baseline SVM-CAV for that. They also demonstrate that their fastCAV method can be applied to compute TCAV scores.", "relation_to_broader_scientific_literature": "Their contributions help speed up a well known algorithm for computing CAVs, while maintaining accuracy.", "essential_references_not_discussed": "I am not aware of any related work that is essential but not cited in the paper", "other_strengths_and_weaknesses": "The main strength of the paper is a new algorithm that is justified to quickly compute CAVs.", "other_comments_or_suggestions": "None", "questions_for_authors": "1. What is the reason for considering pairwise similarity for robustness in Table 1", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yRQOj8p2V2", "reviewer": "Reviewer_ADni", "summary": "The paper introduces FastCAV, a method to compute Concept Activation Vectors (CAVs) up to 63.6× faster than traditional SVM-based approaches by leveraging simple mean vector computations under theoretical assumptions. FastCAV achieves comparable accuracy and interpretability to existing methods while significantly reducing computation time, making it practical for analyzing large modern neural networks. The authors validate FastCAV with extensive experiments on multiple architectures and demonstrate its use in tracking concept evolution during model training.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive feedback and appreciate that they see our contribution well-situated within prior work. We are glad that our empirical comparison to SVM-based CAV computation was found to be fairly comprehensive. Additionally, we agree with the reviewer that more applications further validate the effectiveness of FastCAV.\n\n## Additional Medical Application\nThe reviewer provided references concerning a medical task classifying chest radiographs. We take up this proposal and follow (Singla et al. 2021) to train a DenseNet-121 on a smaller subset of MIMIC-CXR due to time constraints. Next, we use the CheXpert labeler to extract concept mentions from the included free-text radiology reports. Specifically, we take the concepts specified in Figure 3 in (Singla et al. 2021). \n\nUsing these concepts, we compute CAVs for the four blocks of the DenseNet-121 using our FastCAV, SVM-based computation (Gosh et al. 2023), and sparse logistic regression (Singla et al. 2021). Here, we include a comparison of the average computation time and the achieved CAV accuracies, which we will also add in our appendix in the revised version:\n\n|Model|Method|Comp. Time [s]↓|Accuracy↑|Similarity↑|\n|---|---|---|---|---|\n|DenseNet-121|FastCAV (Ours)|**0.006±0.001**|0.715±0.111|**0.793±0.030**|\n|DenseNet-121|Sparse-Logistic Regression|6.370±0.756|**0.721±0.129**|0.500±0.036|\n|DenseNet-121|SVM|0.439±0.071|0.709±0.125|0.457±0.038|\n\nWe observe performance similar to that of the established approaches in this domain. However, note the strongly reduced computation time for FastCAV. Additionally, we report the achieved accuracies for selected concepts after dense-block 3 following (Singla et al. 2021) and will add the full results in our updated revision:\n\n|Method|Concept|Comp. Time [s]↓|Accuracy↑|Similarity↑|\n|---|---|---|---|---|\n|FastCAV (Ours)|Cardiac Silhouette|**0.006±0.000**|0.695±0.068|**0.820±0.031**|\n|Sparse-Logistic Regression|Cardiac Silhouette|5.866±0.437|0.688±0.086|0.491±0.041|\n|SVM|Cardiac Silhouette|0.476±0.068|**0.710±0.057**|0.459±0.026|\n||||||\n|FastCAV (Ours)|Congestion|**0.006±0.001**|0.781±0.062|**0.855±0.020**|\n|Sparse-Logistic Regression|Congestion|6.137±0.448|0.778±0.073|0.484±0.032|\n|SVM|Congestion|0.424±0.079|**0.796±0.091**|0.490±0.043|\n||||||\n|FastCAV (Ours)|Interstitial Edema|**0.006±0.000**|0.817±0.074|**0.886±0.014**|\n|Sparse-Logistic Regression|Interstitial Edema|6.230±0.806|**0.831±0.058**|0.516±0.043|\n|SVM|Interstitial Edema|0.414±0.072|0.798±0.053|0.490±0.043|\n\n\nWe can see that the DenseNet-121 learned the shown concepts. However, the achieved CAV accuracies vary slightly. Further, we see differences in the average similarity for the three methods. FastCAV performs best with respect to speed and similarity.\n\nAdditionally, we follow the suggestion of the reviewer and compute CAVs with respect to images containing negative results for the concepts as found by the CheXpert labeler. In contrast to our expectation, we observe lower CAV accuracies in comparison to random images. However, this holds also for SVM and sparse-logistic regression based computation indicating a data sampling problem in the negative example regime.\n\nFinally, while we recognize the importance and are interested in evaluating the related downstream tasks (Singla et al. 2021, Gosh et al. 2023), the time constraints of the rebuttal period limited the scope of our investigation. We consider this a valuable direction for future work.\n\n\n## Differences in FastCAV and SVM-based Computation\nWe appreciate the reviewer remarks on the theoretical connection of our approach to the existing SVM approach, going over Fisher’s LDA and strong assumptions [L175 right-hand side]. While we agree with the reviewer that these strong assumptions are likely to be violated in practice, we believe that our empirical evaluation in Section 4.1 and the medical application above provide evidence for the practicality of our approach. \n\nNevertheless, following the reviewer's comments, we investigated examples where FastCAV and SVM-based computation differ. For example, in ViT-B/16 after the encoder layer 10 we observe an accuracy difference of 40% with the SVM yielding an accuracy of 95% and FastCAV of 55%. Overall, we observed significant accuracy differences (over 25%) in 2.8% of the CAVs identified, favoring SVM in 1% and FastCAV in 1.8%. In the former case, we believe that these findings are cases where specifically the isotropic Gaussian assumption is violated, meaning the means of $D_c$ and $D_r$ are close together in comparison to the convex hull of these sets. We will add a discussion of these cases and additional visualizations in our appendix.", "claims_and_evidence": "The paper’s main claims are that (1) FastCAV is up to 63.6× faster than SVM-based CAV computation while maintaining comparable quality, and (2) FastCAV is theoretically justifiable under assumptions of isotropic Gaussian distributions for concept and random images. Overall, these claims are supported by experimental results and references to linear discriminant analysis. However, there are areas where the evidence could be improved:\n1. While the authors do show that FastCAV recovers near-identical directions as an SVM in high-dimensional settings, it would have been helpful to see more thorough statistical significance tests, or a direct demonstration of how violations of the Gaussian or isotropy assumptions degrade performance.\n2. The experiments rely heavily on ImageNet-based networks and a set of curated concept images. The results would be more convincing if tested on diverse tasks (e.g., medical or multimodal applications). For medical imaging, the authors can look into this paper for concept extraction:\n[1] Using Causal Analysis for Conceptual Deep Learning Explanation. Singla et al. MICCAI 2021\n[2] Distilling BlackBox to Interpretable models for Efficient Transfer Learning. Ghosh et al. MICCAI 2023.", "methods_and_evaluation_criteria": "The proposed method (FastCAV) is coherent in the sense that it directly addresses the high computational overhead of linear SVM training in high dimensions. One limitation is that the paper focuses mostly on speedups and accuracy for separating concept examples from random images. It might have been valuable to include interpretability-specific user studies (i.e., whether actual users found FastCAV-based explanations equally comprehensible or trustworthy).", "theoretical_claims": "The authors reference Fisher discriminant analysis and prior results connecting Fisher’s linear discriminant to linear SVMs. A weaker point is that the equivalence rests on fairly strong assumptions (isotropy and equal class priors). While this is not uncommon in theoretical exposition, the paper could have discussed more thoroughly how real-world deviations from isotropy (e.g., extremely skewed concept distributions, unusual image statistics) might reduce the fidelity of FastCAV.", "experimental_designs_or_analyses": "The experiments are fairly comprehensive.", "relation_to_broader_scientific_literature": "The paper is well-situated in prior work on concept-based explanations (particularly CAV/TCAV) and the notion of linear separability in neural activations.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "1. The method’s stability and accuracy under distribution shifts or adversarial perturbations remain unclear.\n2. The notion that linear directions suffice to encode complex concepts (particularly in Transformers) is an assumption that might need further empirical justification.", "other_comments_or_suggestions": "1. Can the authors investigate cases where SVM-based CAVs and FastCAV disagree on concept classification?\n2. Showing how FastCAV-based results compare if, instead of random images, “negative concept” images are used. That could confirm whether orthogonality assumptions hold better in different negative sampling regimes.", "questions_for_authors": "Have you seen any cases where FastCAV completely fails (e.g., for extremely rare or ill-defined concepts)? How might such failure modes be detected automatically?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "58rJUrZ6Lu", "reviewer": "Reviewer_KdBw", "summary": "The paper describes the method for accelerating the computation of concept activation vectors, which uses, instead of the SVM linear classifier, the Fisher's linear discriminant analysis (LDA).\n\n## After the rebuttal\n\nMany thanks to the authors for the work and a good rebuttal discussion. The authors address my main concerns, therefore I am increasing my score.\n\nThe main question was about the contribution of this work in connection to the Fisher's LDA. The authors clarified upon this work's motivation and the outline of the contribution, and would be expected to revise the motivation accordingly, including describing it in the beginning of Section 3.\n\nThe answer to the question about new insights is also reasonable.\n\nI have checked the responses to the other rebuttals, and I believe the authors addressed the points accordingly as well.\n\nNew experiments as per request from Reviewer ULp4 would also be a good addition to the paper.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their effort in checking our theoretical contributions and providing insightful comments. Particularly, we appreciate the reviewer for highlighting the correctness and soundness of our analysis.\n\n## Implemented Changes\nWe agree with and follow the suggestion of the reviewer and include standard deviations for the accuracies in our Table 1 (compare to reviewer ULp4). \nAdditionally, we add an experiment on chest radiographs to further validate our approach (see reviewer ADni) and showcase an additional use-case.\nFinally, we appreciate the correction regarding our sensitivity analysis and will improve the terminology in Section 4.3 to align with Meyes et al. (2019).\n\n## Contribution and Comparison to Fisher’s LDA\nThe reviewers main concern is our contribution in connection to an LDA. We welcome the opportunity to clarify these points. Overall, we see our contribution in establishing a more efficient alternative to other CAV calculations that enables researchers to solve more complex and demanding tasks due to the speed-up of the calculation itself.\nThe specific point of the reviewer is that our contribution is to “[replace] linear SVM with the Fisher’s LDA”.  We want to emphasize that we derive FastCAV based on the orthogonality of concepts in feature space observed previously in (Olah et al., 2020, Elhage et al., 2022) for the application of identifying feature directions in a model’s activation space. We then show in Section 3.3 that the resulting calculation is equivalent in expectation to the solution of a Fisher’s LDA only under isotropic, equally mixed, Gaussian assumptions [L179 right-hand side]. \nTo further emphasize the practical importance of these assumptions, we add an empirical comparison between FastCAV and LDA. We observe significantly lower computation time for FastCAV (see table for reviewer ULp4). Further, Fisher’s LDA is on average slower than the established SVM-based calculation. Thus, it is non-trivial to select the correct classifier and assumptions for accelerated CAV calculation.\n\nThe theoretical equivalence to Fisher’s LDA under the described assumptions is necessary to connect our approach to the predominant SVM-based calculation. Particularly, while previous works discuss the connection of Fisher’s LDA to the decision boundary found by an SVM (Shashua 1999), we additionally provide evidence (Appendix A.2) that the required circumstances apply to CAV computation in a model’s activation space. Specifically we show empirically that for CAV calculation there is a high ratio of support vectors among $D_r \\cup D_c$. Hence, the connection and application to concept discovery and comparison to existing SVM-based computation is a novel contribution.\n\nFollowing the feedback of the reviewer, we will improve our motivation in the beginning of Section 3.\n\n## New Insights Unlocked by FastCAV\nWe appreciate the reviewer for highlighting FastCAV's acceleration and its application potential as a key contribution. Acting on their suggestion, we expand our discussion regarding the training analysis (Experiment 4) to provide a more comprehensive exploration of the potential.\n\n1. We conduct an analysis ranking concepts by their area under the curve (AUC) during training and across layers. This approach reveals that the learnability/difficulty of concepts can vary wildly across layers, with some concepts that are quickly learned in early layers becoming more difficult to learn in later layers. For instance, the concept 'wave' is the most readily learned concept in the first block, but its ranking deteriorates to second after block two and ultimately drops to 32nd in the final block. In contrast, the concept 'lined' remains consistently high ranked across all layers. We will include visualizations and additional discussion in our revised paper.\n\n2. We evaluate the ratio of concepts learned in each layer over the training process. We find that during the training this ratio increases indicating that the model learns task-relevant concepts. This result is congruent with previous analysis of neural network training dynamics, e.g., Shwartz-Ziv and Tsihby, 2017, “Opening the Black Box of Deep Neural Networks via Information”. However, the speed-up of Fast-CAV enables an analysis of the training dynamics on the level of human-interpretable concepts.\n\nOverall, by providing a faster and more efficient procedure, we expect to empower researchers to explore more complex scenarios and larger concept sets, potentially uncovering novel phenomena that were previously inaccessible due to computational constraints.\nFinally, we are eager to address any further questions and discuss additional points.", "claims_and_evidence": "The analysis appears to be correct, and the method and analysis appears to be sound, however it is important to see it in the wider context. \n\nThe authors claim:\n\"We introduce FastCAV, a novel approach for computing Concept Activation Vectors (CAVs) that significantly reduces computation time by up to 63.6×, making it more efficient and scalable.\"\n\nThe main concern is that the authors need to justify the novelty of the approach. As the description states now, it does not seem to be a combination of CAVs with LDA, both well-known. \n\n\"• We provide a theoretical foundation for FastCAV and specify concrete assumptions under which it is equivalent to other linear methods.\"\nThe problem with the claim might be that I understand that the theoretical argument coincides with the LDA description as per (Bishop, 2006) and ultimately going back to Fisher (1936) as cited in the paper. \n\nFisher, R. A. (1936). \"The Use of Multiple Measurements in Taxonomic Problems\" \n\n•\" We demonstrate empirically that FastCAV produces similar CAVs to the established SVM-based approach, leading to comparable insights in downstream methods, e.g.,(Kim et al., 2018; Ghorbani et al., 2019).\"\nThis claim appears to be matched with the evidence, as described below in the review of experimental design and analyses. \n\n•\" Using FastCAV, we investigate the evolution of concepts during training and across different layers of a neural network.\"\nIt is true that the evolution of concepts is investigated in Section 4.4, however it is not by any measure sufficient as the authors just present the plot of CAV accuracy depending on accuracy.", "methods_and_evaluation_criteria": "The proposed method seems to be a justified solution for the given problem, which is accelerating CAV methods, however, the authors need to emphasise in which way it is not a straightforward combination of Fisher (1936) and the original CAV. \n\nI don't think the ablation study, proposed in Section 4.3, is an ablation study, which involves removal or damage to the components (see e.g., Meyes et al (2019). The hyper-parameter value influence studies would be sensitivity analysis. \n\nMeyes et al (2019) Ablation Studies in Artificial Neural Networks\n\nSection 4.4 proposes only a rudimentary analysis of leveraging this low-resource method for interpreting the training process online, which involves the CAV accuracy tracking over time. In my view, this part actually highlights significance of the contribution the most and could be analysed in terms of how that can help provide diagnosis tools for neural network training this can involve quantitative and qualitative evaluation.", "theoretical_claims": "I have checked the theoretical claims, and I believe they are entirely mirroring Bishop (2006), which is duly cited. The authors need to emphasise in which way this would be a novel contribution.", "experimental_designs_or_analyses": "The experiment underpin the claims. \nThe authors compare the methods against the standard CAV, as well as other baselines on computational time as well as performance. They find that they obtain competitive performance while substantial (on average 46.4×) acceleration compared to the baseline methods. \n\nTable 1 does not contain the confidence intervals, could the authors add them.", "relation_to_broader_scientific_literature": "The main works, in my opinion, relate to Fisher LDA and concept activation vectors. Both aspects have been covered in the related works section.", "essential_references_not_discussed": "I believe the references are sufficiently discussed.", "other_strengths_and_weaknesses": "Strengths:\nCorrectness: the analysis appears to be correct\nSoundness: the method and analysis appears to be sound\n\nWeaknesses: \n-originality: it is important that the authors clarify upon the contribution. Right now, it seems to be that the gist of the proposed approach constitutes replacing linear SVM with the Fisher’s LDA and the evaluation aiming to show its competitive accuracy and computational time. However, my main concern is that this core of a contribution might not give enough new insight into the CAVs as it only seems intuitive that SVM can be replaced with a linear classifier. I suspect though that the authors could describe the motivation more clearly, perhaps putting emphasis on the new ways of explanations that it could help achieve which the state-of-the-art cannot (e.g., explaining the training dynamics which isn’t computationally feasible otherwise). At this stage, the paper merely hints at it at Section 4.4, and I would think it could be strenghened.", "other_comments_or_suggestions": "No other suggestions", "questions_for_authors": "The main sticking point is the contribution. The contribution seem to be a drop-in replacement of SVM with Fisher LDA, which may fall short of the expectations of the venue. Therefore, it is crucial that the authors confirm in the rebuttal and revision why this contribution is novel and not just a combination of two well-known methods. \n\nThe other problem is the theoretical analysis, which seems to be centred around the properties of the Gaussian distribution as per Bishop (2006), underpinning LDA (see Sections 3.2, 3.3, Appendix A.1).", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: There is no need to use SVMs when two groups of points are well-separated, as one can simply take the difference of the two barycenters as normal vector. Building on this idea, the paper proposes a re-implementation of the celebrated concept activation vectors, which runs orders of magnitude faster. 

The reviewers are in general agreement that this is an interesting contribution to the field. I appreciated that the authors' answer contained many additional experiments which I would like to see in the revised version of the paper.
Decision: Accept (poster)