B-cosification: Transforming Deep Neural Networks to be Inherently Interpretable

Authors: shreyash  arya
sukrut  rao
moritz  böhle
bernt  schiele
german  shepherd
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-08 22:52:03

Abstract: B-cos Networks have been shown to be effective for obtaining highly human interpretable explanations of model decisions by architecturally enforcing stronger alignment between inputs and weight. B-cos variants of convolutional networks (CNNs) and vision transformers (ViTs), which primarily replace linear layers with B-cos transformations, perform competitively to their respective standard variants while also yielding explanations that are faithful by design. However, it has so far been necessary to train these models from scratch, which is increasingly infeasible in the era of large, pre-trained foundation models. In this work, inspired by the architectural similarities in standard DNNs and B-cos networks, we propose ‘B-cosification’, a novel approach to transform existing pre-trained models to become inherently interpretable. We perform a thorough study of design choices to perform this conversion, both for convolutional neural networks and vision transformers. We find that B-cosification can yield models that are on par with B-cos models trained from scratch in terms of interpretability, while often outperforming them in terms of classification performance at a fraction of the training cost. Subsequently, we apply B-cosification to a pretrained CLIP model, and show that, even with limited data and compute cost, we obtain a B-cosified version that is highly interpretable and competitive on zero shot performance across a variety of datasets. We release our
code and pre-trained model weights at https://github.com/shrebox/B-cosification.

Main paper: 
 introduction: 
 Despite their strong performance on a variety of tasks, understanding decisions of deep neural networks (DNNs) remains challenging. Explanation methods, such as feature attributions [45, 47, 52, 5] , have been proposed in an attempt to explain such decisions post-hoc, but have often found to be unfaithful to the model being explained [2, 3, 39, 58] .Inherently interpretable Deep Neural Network (DNN) models have recently gained popularity. In contrast to the common approach of explaining existing DNNs in a post-hoc fashion, these models typically feature certain architectural constraints that allow for extracting human-interpretable, modelfaithful simplifications of the models' computations by design; examples of this include prototypebased [13, 16, 31] , dynamic linear [8, 9] , or concept-bottleneck models [27, 56, 32, 41] . However, given those architectural changes, this comes at a price: specifically, the models need to be trained from scratch, which-especially in the case of large foundational models, which are increasingly popular-can cost millions of dollars.To mitigate this, in this work, we explore a novel approach of fine-tuning DNNs for inherent interpretability and propose to 'B-cosify' existing DNNs. Specifically, we investigate whether pre-trained DNNs can simply be efficiently fine-tuned to obtain a similar degree of interpretability 38th Conference on Neural Information Processing Systems (NeurIPS 2024). [10] and our B-cosified models at various stages of training. Specifically, we show the dynamic linear mappings W(x) computed by the models in color as in [9, 10] ; note that by formulating conventional models ('initial' in the plot) as a specific version of B-cos models, we are able to visualise the corresponding explanations in color too, see Sec. 3.2.1 for further details. We find that after only one epoch of training, the B-cosified models exhibit similar explanations as B-cos models.as the recently proposed B-cos Networks [9, 10] . In contrast to the original B-cos Networks, which leverage existing architectures to obtain performant and interpretable models, we investigate whether we can additionally leverage the existing pre-trained weights, thus aiming to take advantage of the significant amount of resources that have been invested in training existing models. As a result, we hope to make inherently interpretable models more easily accessible to the community.To do so, we first conduct a detailed analysis of how B-cos DNNs differ from their conventional counterparts. Interestingly, we find that many existing models can be converted into functionally equivalent B-cos models by a small set of targeted implementational modifications (Tab. 1). To increase the interpretability of the models, we then increase the 'alignment pressure' [9] via the parameter B of the B-cos transformations and fine-tune the models on their respective tasks, which leads to significantly more interpretable explanations (Fig. 2 ).On supervised settings, we find that B-cosified models often outperform both conventional and B-cos DNNs at a fraction of the full training cost (Fig. 1 , left), whilst exhibiting a similar degree of interpretability as the original B-cos DNNs (Fig. 1 , right). We further apply B-cosification to a pre-trained CLIP model [38] , a large foundational vision-language model (VLM), and show that despite using comparatively limited data and compute cost, B-cosified CLIP models yield highly interpretable explanations whilst being competitive on zero-shot performance across a variety of downstream datasets.Our work thus opens a new perspective on how to design inherently interpretable models in a costeffective manner. Importantly, on the one hand it highlights that conventional models might be closer to inherently interpretable models than previously understood. On the other hand, it highlights the benefits of designing inherently interpretable models via minor architectural modifications, such as e.g. the B-cos DNNs, as this can allow for leveraging the large array of existing, pre-trained DNNs.In summary, our contributions are:• We propose B-cosification, a novel technique to 'fine-tune for interpretability', that addresses the problem of high training cost associated with obtaining inherently intepretable models such as B-cos DNNs. Our B-cosified DNNs are highly interpretable while often outperforming both standard and B-cos DNNs.• We thoroughly study different design choices to find an optimal strategy for B-cosification.• We apply B-cosification to supervised image classifiers on ImageNet [15] , including both CNNs and ViTs, and show that the B-cosified variants perform on par on interpretability metrics while often outperforming in terms of accuracy. Overall, we find that B-cosifying a pre-trained black box DNN to be superior on both metrics as compared to training a B-cos DNN from scratch, while being computationally significantly cheaper. , we find that it is possible to endow the model with the same level of inherent interpretability as the B-cos models proposed in [10] , whilst maintaining CLIP's zeroshot ability (see Fig. 5 ).The resulting linear summaries of the models (W(x)) can be visualised in color (row 3) and provide significantly more detail than GradCAM explanations (row 2), which are often used to explain conventional CLIP models.• We extend B-cosification to CLIP, a foundational VLM, and show that B-cosified CLIP remains highly competitive on zero-shot performance across a variety of downstream datasets, while also yielding similar interpretability benefits as B-cos models. 
related work: 
 Explanation Methods. Post-hoc attributions [45, 42, 52, 47, 5, 55] have popularly been used to understand the decisions of trained DNNs, but have often been shown to be unfaithful to the model being explained [2, 3, 58, 39] . Inherently interpretable models [13, 27, 9] , in contast, incorporate architectural changes to the model and can yield explanations that are interpretable and faithful to the model by design. However, such models need to be trained from scratch, which imposes a significant additional cost. In this work, we explore fine-tuning for interpretability, and propose a method to transform existing black-box DNNs to inherently interpretable B-cos DNNs, bringing together the best of both worlds.Attribution Priors [37, 36, 26, 49, 4] have often been used to train or fine-tune models to have explanations with desirable properties, such as inducing smoother explanations [26] , consistent explanations [36, 37] , or to guide models to be 'right for the right reasons' [43, 19, 40, 33] . Similar to such work, we fine-tune black-box DNNs for interpretability, but in contrast, we only make architectural modifications to transform the DNNs to B-cos DNNs, and do not use any additional constraints on the explanations themselves while training.CLIP Interpretability and Localization. Post-hoc attribution methods [45, 35, 12, 6] have also been used to explain VLMs such as CLIP [38] , however, as with supervised DNNs, their faithfulness to the model is not guaranteed and the explanations are often coarse-grained and not very human interpretable. While inherently interpretable architectures could address this, the high costs of training such large models from scratch makes their use unappealing. In this work, we bridge the gap by instead fine-tuning from pre-trained black-box CLIP models to inherently interpretable B-cosified CLIP variants, and find that the B-cosification process is effective in yielding performant and interpretable models. A separate line of work [7] involves improving localizability of VLMs, and is orthogonal to our work since our goal is to obtain explanations that are faithful to the model.Learning mappings between model features. Recent work [29, 32] has explored using simple linear transforms to map features between models, and in particular also mapping features from arbitrary models to CLIP's representation space. In the context of our work, such methods can be used to map a supervised B-cos feature extractor to CLIP using a linear transform, to obtain an inherently interpretable DNN that can mimic CLIP. In our evaluation, we compare with such an approach, and find that our approach of architecturally transforming the full model and fine-tuning for interpretability yields improved zero shot performance.3 From conventional to B-cos modelsIn the following, we describe the process of fine-tuning standard black box DNNs into inherently interpretable B-cos DNNs. In Sec. 3.1, we first introduce the B-cos models and enumerate the key ways in which they differ from standard models. In Sec. 3.2, we then perform a detailed study on strategies to bridge each of these differences for effective B-cosification.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes]Justification: Please see the paragraph on limitations in Sec. 5.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
b-cos models: background: 
 Many common DNNs consist of a series of blocks of linear layers followed by non-linear ReLU activations [30] , and are thus piece-wise linear functions1 : i.e., for every input x, they effectively compute a linear transformation of that input: y(x) = W(x)x + b(x), cf. [51] . In [8, 10] , models of this kind have been called 'dynamic linear', a naming convention that we adopt in this paper.Interestingly, for piece-wise linear models, W(x) is given by the models' gradient with respect to x [51]-except for the input-dependent bias b(x), the gradient thus constitutes an exact summary of the models' computations. This linear mapping W(x) is unfortunately typically not easily interpretable, and many techniques have been proposed to derive qualitatively more convincing explanations [45, 55] . These, however, have been shown to often not faithfully reflect the underlying model [1, 39] .Further, if the models employ bias terms, W(x) does not yield a complete explanation [51] , i.e. y(x) ̸ = W(x)x. Integrating bias terms as proposed by [51] yields a set of importance attribution maps, summarizing which requires carefully selecting a post-processing function with inherent tradeoffs. Even when not using bias terms [22] , however, the resulting matrices W(x) are often not easily human interpretable, and the resulting models can suffer from significant drops in performance.To address this, [9, 10] propose to architecturally modify the DNNs to introduce additional alignment pressure during model optimisation. For this, they replace the ubiquitously used linear transformation by the B-cos transformation, which dynamically scales the output of the linear transformations:EQUATIONwith B a hyperparameter, cos the cosine similarity between x and the weights w, and w=w/∥w∥.Like piece-wise linear models, B-cos models are dynamic linear and thus accurately summarised by a single linear transformation W(x) s.t. y(x) = W(x)x; as B-cos models do not employ bias terms, this model summary is complete. Crucially, it has been shown that with B >1, the matrix W(x) aligns with task-relevant input patterns, making it more easily human interpretable (e.g. Fig. 1 , right).Importantly, as the B-cos transformation can serve as a drop-in replacement for linear transformations at every layer of a DNN, it is possible [9, 10] to leverage existing DNN architectures and the resulting B-cos models obtain similar classification accuracies as their conventional counterparts (Tab. 4, cols. 'pretrained' and 'B-cos').Extending this, we investigate if it is possible to leverage existing DNN weights-i.e., our goal is to fine-tune existing models to be similarly interpretable as B-cos models, whilst not requiring to train them from scratch. However, despite the architectural similarities between B-cos and conventional models, there are multiple key differences that make transforming pre-trained models into B-cos models non-trivial: e.g., apart from replacing linear transformations with the B-cos transformation and not employing biases, B-cos models are trained on image representations with 6 color channels as [r, g, b, 1-r, 1-g, 1-b] to be able to visualise the model-inherent linear summaries W(x) in color, whereas conventional models use 3 channels (see also Tab. 1). In the next section, we show how to overcome these differences and convert existing models into functionally equivalent B-cos models.We analysed the differences between B-cos models and their conventional counterparts in detail and compiled the results in Tab. 1. In this section, we discuss one by one how to bridge these differences. In particular, we show that a conventional model can be framed as a functionally equivalent B-cos model as in [10] with B =1, which additionally employs bias terms. Only upon modifying these two aspects, i.e. biases and B, does the model need to be fine-tuned to adapt the weights to those changes.Table 1 : Overview. To allow for comparing the models, we compiled the identified differences between the conventional models (Standard), their B-cosified version (B-cosified) and the original B-cos models (B-cos). For each design choice in the B-cosified models, we summarise the respective discussion in Sec. 3 However, we note that in combination with the commonly used input normalisation, we can convert the first linear transformation in conventional models (e.g., a convolutional layer) into an equivalent transformation that accepts 6-channel inputs. Specifically, for input normalisation, the channel-wise means µ s are subtracted from the individual channels, followed by a division by the standard deviations σ s , yielding s ′ =(sµ s )/σ s for s∈{r, g, b}. Conversely, mean-normalising the 3 additional color channels yields -s ′ . Leveraging this, we use the models' weights learnt for 3-channel inputs, w j = [w j,r , w j,g , w j,b ] for every feature j, to construct an equivalent 6-channel transformation:EQUATION)Note that applying w ′ j to the mean-normalised, 6-channel inputs yields the same results as applying w j to the original mean-normalised inputs that the pre-trained models have seen during training.Activation Functions. Owing to the non-linearity inherent to the B-cos transform, explicit activation functions are not necessary in between B-cos layers. However, the authors of [9, 10] showed that the model-inherent explanations are compatible with MaxOut [20] . Note that the very commonly used ReLU non-linearity applied to v T x for any weight vector v, is just a special case of MaxOut:EQUATIONAs the pre-trained models' weights have been optimised for the ReLU non-linearity and given its compatibility with the B-cos explanations, we leave them untouched in the B-cosification process.Weight normalization. B-cos transformations employ unit norm weights, see also Eq. ( 1), which the authors motivated by the fact that the only way any given neuron can achieve its maximal output is by increasing the weight-input alignment, which in turns leads to the improvements of the explanations.However, conventional models have been trained with unconstrained weights and using unit norm weights would thus lead to unpredictable model behaviour. Interestingly, we note that the weight normalisation in the latest version of the B-cos models can actually not impact the explanation quality, as the authors of [10] re-introduce normalisation layers into the B-cos models. To better understand this, let us consider the compound function of a batch normalisation layer and a B-cos layer:EQUATIONwith BatchNorm(y) = α × ymean(y)EQUATIONwith α and β trainable parameters of the BatchNorm layer. Note that √ var is scaled by any factor γ by which the output of a B-cos layer might be scaled, which cancels in the fraction in Eq. ( 5) and thus makes f (x) invariant to scaling the B-cos transformation: i.e. BatchNorm(y) = BatchNorm(γ × y). In particular, the output of f (x) is thus invariant to weight normalisation, as the output of B-cos (x) scales linearly with the weight norm, cf. Eq. ( 1). This is of course only true if every B-cos layer were always followed by a normalisation layer, which is not necessarily the case. Nonetheless, we find that not using normalised weights yields consistently good results across all models. Therefore, we use B-cos transformations without weight normalisation throughout our experiments; for an ablation, see Tab. B2 in the appendix.In summary, we showed that it is possible to adapt the implementation of existing models in a way that allows us to integrate certain aspects of B-cos models without functionally changing the pre-trained models. Notably, we can now visualise color explanations similar to B-cos models (Fig. 1 , right, col. 3); unsurprisingly, however, these explanations have poor interpretability due to the absence of the alignment pressure imposed during B-cos training. In the next section, we discuss the necessary functional changes for B-cosification to obtain interpretable explanations.The changes introduced in the preceding section have not functionally changed the pre-trained models, but rather allow us to interpret the existing models as a special case of B-cos models. Now we introduce the necessary changes to increase the interpretability of the dynamic matrices W(x). As these functionally change the models, they need to be fine-tuned to recover their original performance.In particular, the remaining differences between conventional and B-cos models are (1) the value of B, and (2) the use of biases, Tab. 1. We will now discuss how we bridge these differences individually.Ablation Setup. We evaluate various fine-tuning strategies using a ResNet-18 [21] model supervised on ImageNet [15] from Torchvision [53] for B-cosification, and compare with a B-cos ResNet-18 from [10] . We optimize using AdamW [25] with cosine scheduling and train for 90 epochs, and evaluate both classification accuracy as well as interpretability using the GridPG metric [8] .(1) Increasing B. As shown in [9] , using B >1 is critical to obtain easily interpretable explanations.To increase B for the pre-trained models, we investigate three strategies: (1) immediately setting B to a higher value and then fine-tuning, (2) linearly interpolating from B = 1 to B = 2 throughout fine-tuning, and (3) setting B as a learnable parameter. (2) has the advantage of changing the model in small steps, making it more likely that it maintains performance while fine-tuning, but requires using the full number of epochs to reach the target value of B.(1) on the other hand is likely to adversely affect the utility of the weights, but offers the opportunity to stop fine-tuning early if performance and interpretability metrics are sufficiently high. (3) offers the most flexibility, but also adds a new set of parameters that need to be optimized. We show the results of this evaluation in Tab. 2. Interestingly, we find that using (1), i.e. setting B = 2 and then fine-tuning, yields performance that is on par with learnable B parameters, whilst being significantly simpler to implement. To easily test the generality of the B-cosification scheme, we therefore opt for this approach in Sec. 4.1.(2) Decreasing biases. As discussed in Sec. 3.1, dynamic linear models with bias terms are not exactly summarised by the matrix W(x), cf. [51] . To obtain the same level of faithfulness of the explanations as B-cos models (in particular w.r.t. explanation completeness, cf. [51, 52] ), we need to remove the biases from the model. To do so, we investigate two approaches: (1) removing all biases first and then fine-tuning, and (2) fine-tuning while decaying biases using weight decay. Similar to the setup with B, (2) has the advantage of avoiding drastic changes to the model, but requires potentially fine-tuning for longer. Further, the weight given to the bias decay in the loss constitutes a tradeoff between maintaining classification performance and pushing the biases to be close to zero. We report the results of this evaluation in Tab. 3. Similarly to the experiments for B, we find that immediately setting the biases to zero constitutes a simple yet performant approach to achieve both good localisation and accuracy. To assess the generality of the B-cosification scheme across a wide range of models, we thus choose this the simpler approach of setting biases to zero in Sec. 4.1. In short, we find that a very simple approach, i.e., setting the bias and the B values to the target values immediately, constitutes a simple and easy-to-use, but nonetheless performant strategy to B-cosify models. In the following sections, we test whether these findings generalise well to other models.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes] Justification: Our theoretical arguments are contained in Sec. 3.2.1, namely that (1) it is possible to convert the first layer of DNNs such that they accept 6 channel inputs as required by B-cos models, that (2) ReLU is a special case of MaxOut, and that (3) weight normalisation is irrelevant if a layer is followed by a batch normalisation layer. These claims are shown to be true in the context of Eq. ( 2) (1), Eq. (3) (2) and Eq. ( 5) (3).Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
b-cosification results: 
 In the following, we evaluate the effectiveness of the B-cosification strategy we developed in Sec. 3. In Sec. 4.1, we first apply B-cosification to supervised models across various architectures, and evaluate for classification performance and interpretability. In Sec. 4.2, we B-cosify CLIP [38] , a large foundational vision-language model, and show that despite fine-tuning at a fraction of the training cost, the B-cosified CLIP shows strong zero shot generalization whilst being highly interpretable.Table 4 : Classification Accuracy. We report the top-1 classification accuracy on the ImageNet validation set of the pre-trained models (pretrained) and the B-cosified models (B-cosified) after fine-tuning them. Additionally, we report the accuracy of the corresponding B-cos models trained from scratch (B-cos) as well as the difference to them (∆acc), and how much faster and at which epoch (t) the same accuracy as in [10] was achieved (speedup).Results for B-cosified models are averaged over three runs; full results including standard deviation in appendix.Top Setup. We B-cosify models from Torchvision [53] supervised on ImageNet [15] . We use a diverse set architectures, including both CNNs (ResNet-18 [21] , ResNet-50 [21] , and DenseNet-121 [23] ), and ViTs [17] with (ViT c -Ti, ViT c -S, ViT c -B, ViT c -L) and without (ViT-Ti, ViT-S, ViT-B, ViT-L) convolutional stems. For ResNet-50, we use both the weights originally released by Torchvision and the updated V2 weights, which constitute models trained for longer and with more augmentations [54] . As in Sec. 3.2, we evaluate both for classification accuracy and for interpretability using the GridPG [8] metric. We compare both accuracy and interpretability of the B-cosified models with B-cos models trained from scratch from [10] . For interpretability, we also compare with several post-hoc attribution methods as baselines, namely Guided Backprop [50] , Gradient [48] , DeepLIFT [47] , IxG [47] , IntGrad [52] , and GradCAM [45] . For full details, see Appendix C.1.Classification performance. Tab. 4 reports the classification accuracy of the B-cosified models across architectures, and compares them with their conventional counterparts from Torchvision and B-cos models trained from scratch. We find that across architectures (col. 1), B-cosified models perform competitively with conventional DNNs (cols. 2-4) and interestingly, in contrast to the findings reported by [10] , often outperform them, i.e. for five out of twelve architectures. Notably, we find (col. 5) that our B-cosified models significantly outperform B-cos models trained from Fig. 3 : Localisation Performance of W(x)x. We compute the contribution maps according to the dynamic linear summaries W(x) of the pre-trained models ('Standard'), their B-cosified versions, and the original pre-trained B-cos models and evaluate their localisation performance on the Grid Pointing Game as in [10] . We find localisation to significantly improve for B-cosified models, achieving results on par with the models of [10] . scratch across all but the two largest ViT c architectures. Further, B-cosified models often achieve the same performance as their corresponding B-cos models at a fraction of the training cost (col. 6). Specifically, we find that averaged across architectures, B-cosified models outperform B-cos models trained from scratch by 2.5 pp, with an average training speedup (to match performance) of 5.2x. These results strongly advocate for B-cosification as a superior alternative to training from scratch for obtaining performant inherently interpretable models at a low compute cost.EQUATIONInterpretability. To evaluate the interpretability of our B-cosified models, we report the GridPG localization scores in Fig. 3 , and compare with conventional and B-cos models; following [10] , we report the results of 3x3 image grids for convolutional models, and of 2x2 grids for the ViTs. For a fair comparison, for all models, we evaluate the localization of the dynamic linear summary of the model2 W(x)x (see Sec. 3.1). We find that across architectures, B-cosified models significantly outperform conventional DNNs in terms of localization (32.7pp-71.0pp) and perform on par with B-cos models. Since post-hoc attribution methods (e.g. [45, 47, 52] ) are often used to interpret conventional DNNs, similar to [9] , in Fig. 4 , we compare the localization of the model inherent explanations from two of our B-cosified models with post-hoc explanations applied to the corresponding conventional models. Similar to the results reported by [9] , we find our B-cosified models to strongly outperform all post-hoc methods, including GradCAM [45] , with a near perfect localization score, showing that B-cosification is effective in yielding highly interpretable yet model-faithful explanations.Impact of pre-trained weights. Since our aim is to fine-tune for interpretability, we investigate how crucial the quality of the weights of the conventional model are for effective B-cosification. Specifically, we expect weights from stronger models to be a better starting point for B-cosification.We evalaute this by performing B-cosification both with v1 and v2 variants of ResNet-50 [21] from Torchvision [53] , where the latter is trained for longer and with stronger augmentations [54] . From Tab. 4, we find that using a strong initialization is highly useful for effective B-cosification, Fig. 5 : Classification performance on the CLIP Benchmark [14] of various CLIP models for the zero-shot setting (left) and linear probing (right). Specifically, we compare two B-cosified CLIPs-trained on ImageNet (IMN) and CC3M respectively-to the Text2Concept approach by [29] and the original pre-trained CLIP model. We find B-cosified versions of CLIP to consistently outperform Text2Concept on natural and specialised data.specifically, fine-tuning starting from v2 weights outperforms B-cos models by 0.8pp as compared to v1, and achieves equal accuracy with a 9x speedup as compared to 2x with v1; similar results are observed for initialising the weights with models pretrained via the self-supervised DINO paradigm [11] (final accuracy: 77.0, speedup: 3.2x), for further discussion see Tab. B3 in the appendix.In this section, we evaluate our B-cosification paradigm on CLIP [38] , a powerful pre-trained vision-language model, and evaluate its interpretability and zero shot performance.Setup. We B-cosify a CLIP [38] with a ResNet-50 [21] backbone using the procedure described in Sec. 3.2. We use the recently proposed SigLIP loss [57] between the image embeddings of the pre-trained CLIP and the B-cosified CLIP's and train the models on either the ImageNet [15] or the CC3M datasets [46] . For evaluation, we rely on the CLIP Benchmark [14] and report zeroshot and linear probing results for accuracy. To assess the models' interpretability, we explain the similarity between the image embeddings and the text embedding of the pre-trained CLIP model via the dynamic linear summaries, see Sec. 3.1 or GradCAM, and report the EPG scores [55, 40] on the VOC dataset [18] . For full details, see Appendix C.2.Evaluating Model Performance. In Fig. 5 , we report the zeroshot and linear probing accuracies of the two B-cosified CLIP models (trained on ImageNet or CC3M) and compare it to the original CLIP (Standard) and the recently proposed Text2Concept technique [29] ; for the latter, we train a linear layer on top of a frozen, pre-trained B-cos ResNet-50 from [10] to mimic the embeddings of CLIP [29] . We find that the B-cosified models significantly outperform the Text2Concept approach and achieve accuracies that are more similar to the original CLIP's zeroshot and linear probing accuracies.Evaluating Model Interpretability. We evaluate the B-cosified CLIP's ability to localise classes in the VOC dataset in two ways. On the one hand, we directly explain the similarity of the models' embedding to the text embedding of a given prompt such as "A photo of a cow.". On the other hand, we note that the final attention pooling layer in the CLIP model only computes a weighted sum of the last layer's value vectors. Therefore, we additionally evaluate whether we can also explain the similarity between the text embeddings and these value vectors to improve the localisation ability.In this context, we notice that explaining the average similarity to the text embedding yields highly distributed attribution maps, see Fig. 6b , col. 2. On the other hand, explaining only the most similar embedding localises very well, see Fig. 6b , col. 5. To better understand this phenomenon, we additionally interpolate between these two approaches and compute weighted means i w i v i of those value vectors v i , in which the weights are determined by the cosine similarity between the value vectors v i and the text embedding t, i.e. with weights w i = cos p (t, v i ) for various p.We find that this not only significantly improves the explanations qualitatively, see Figs. 2 and 6b , but also quantitatively: in Fig. 6a we report results for explaining the final image embedding (Bcosified CLIP), the dynamic linear summary for the CLIP ResNet-50 (CLIP W(x)x), its GradCAM explanations (CLIP GradCAM), and the weighted mean of the value vectors, which we call B-cosified FeatureCLIP. While B-cos CLIP already yields a noticeable improvement over the pre-trained CLIP explained via GradCAM and significantly improves the localisation ability of the linear summary W(x)x, the weighted means yield even stronger performance for sufficiently high p.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?Justification: To the best of our knowledge, all necessary details to reproduce our results are contained in the submission. We mainly build on publicly available code (for B-cos models, see https://github.com/B-cos/B-cos-v2/; for CLIP, see https://github.com/openai/CLIP/; for CLIP Benchmark, see github.com/LAION-AI/CLIP_benchmark and https://github.com/mlfoundations/open_clip) and will make all our modifications available for ensuring full reproducibility of the reported results.• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example• The full details can be provided either with the code, in appendix, or as supplemental material. 
discussion: 
 The B-cosification approach presented in this work addresses a common issue with developing inherently interpretable models: achieving model interpretability without compromising on performance or incurring high training costs. By leveraging pre-trained models, B-cosification opens a new path towards developing interpretable yet performant models, which can be of particular interest in the context of foundation models such as CLIP [38] , which might otherwise be prohibitively expensive to train on a limited budget. Our results suggest that B-cosification not only maintains but, in several cases, even enhances model accuracy, whilst yielding significant improvements on interpretability metrics, providing a viable and resource-efficient alternative to training B-cos models from scratch.Specifically, we find B-cosified models to much faster reach the same levels of interpretability and accuracy than their counterparts trained from scratch, with training speedups of up to 9x in some models. The approach appears to be general, being applicable for both CNNs and ViT models. We hope that this increase in efficiency will make interpretable models much more accessible in settings with constrained computational resources and could thus facilitate their adoption. In particular, when applying our proposed B-cosification scheme to a foundational model-CLIP-we find that the B-cosified CLIP model is able to maintain competitive zero-shot performance while at the same time providing interpretable and model-faithful explanations.Despite these advancements, certain aspects remain open for further exploration. Specifically, while some models quickly recover original performance after B-cosification, others exhibit slower convergence rates, suggesting potential for optimisations in the fine-tuning process. Additionally, for the larger B-cosified ViT c models, while yielding results that are on par with those trained from scratch, the B-cosification process did not succeed in achieving speed-ups, indicating that the interplay between model architecture and the proposed B-cosification might require further exploration.In summary, our results establish B-cosification as an effective method for enhancing interpretability in pre-trained models with low computational cost. The method consistently enables high interpretability without compromising performance, even achieving substantial training speedups in many cases.In Fig. A1 , we provide additional qualitative examples to illustrate the interpretability gains achieved by B-cosifying a CLIP model. Specifically, we show explanations generated by the original CLIP model using GradCAM [45] (row 2) for a diverse set of input images (row 1), for which explanations are generally coarse and lack clear localization. In contrast, the third row displays explanations produced by B-cosified CLIP, which yields finer-grained, more visually interpretable explanations. In Fig. A2 , we show further comparisons on specific object classes with using different cosine powers p (cos, cos-7, cos-19, and cos-inf) to qualitatively demonstrate the effect of increasing the exponent p in gathering the value vectors, see also Sec. 4.2. Higher cosine thresholds result in increasingly focused and interpretable representations, capturing fine details that are often absent in the original CLIP explanations.In Fig. A3 , we show additional qualitative examples for prompting the B-cosified CLIP model with different prompts for the same image, thus highlighting the class-specificity of the explanations as well as the potential that inherently interpretable CLIP models might yield. Specifically, B-cosified CLIP models allow to explain the similarity of a given image with a free-form textual prompt, which shows that the zero-shot performance of CLIP with respect to classification also transfers well to the corresponding explanations. We find different objects can be localised in high-quality color using B-cosified CLIP. For example, we compute B-cos explanations using text-prompts: a picture of {'x'} encoded using standard CLIP [38] text-encoder for an image taken from ImageNet [44] . The first column shows the original image with the class caption, the second column shows the colored B-cos explanations with the text caption used at the top, and the third column shows the raw attributions with red denoting the positive contribution towards the text input and blue denoting the negative contribution.Table B4 : Experimental ablations results for the B-cosification design choices. The table shows the ablation results for the B-cosified ResNet-18 model (with B = 2 and no Bias) shown in row 1. Further in rows 2-6, the ablated components are shown in col. 1, the accuracy of the ablated model (acc) in col. 2, and the change in accuracy (∆acc) compared to the non-ablated model (row 1) is shown in col. 3. Further, col. 4 shows the GridPG localisation scores (loc) and the difference to the non-ablated B-cosified model (∆ loc ). Similar to components used in standard models, we replace the BatchNorm Uncentered with BatchNorm Centered, change the order of the Global Average Pool where features are first averaged and then passed to the last linear layer, and replace average pooling with max pooling at the stem. Further, we also replaced ReLU activation with Identity; and removed the Logit Bias layer. Results are for a random initialized single run. Table B7 : Zero-shot performance of various CLIP-based models over 38 datasets using CLIP Benchmark [14] . Scores within the 99.5% Clopper-Pearson confidence interval of each dataset's top score are shown in bold.Baselines contain results for the Standard CLIP [38] and Text2Concept (T2C) [29] models; ImageNet and CC3M column sections contain the B-cosified RN-50 CLIP models trained with cosine and cyclic learning schedulers trained with ImageNet [15] and CC3M [46] datasets, respectively. The cyclic learning training inspired from [24] . Dataset type is taken from [14] .Baselines ImageNet [15] CC3M [46] Standard [38] T2C [29] B8 : Linear-Probe performance of various CLIP-based models over 29 datasets using CLIP Benchmark [14] . Scores within the 99.5% Clopper-Pearson confidence interval of each dataset's top score are shown in bold. Baselines contain results for the Standard CLIP [38] and Text2Concept (T2C) [29] models; ImageNet and CC3M column sections contain the B-cosified RN-50 CLIP models trained with cosine and cyclic learning schedulers trained with ImageNet [15] and CC3M [46] datasets, respectively. [46] datasets, respectively. The cyclic learning training inspired from [24] . Dataset type is taken from [14] .Baselines ImageNet [15] CC3M [46] Standard [38] T2C [29] Justification: In the abstract and intro we claim to fine-tune models for inherent interpretability (by converting them to the recently proposed B-cos models) and that we find that our proposed scheme (1) outperforms training from scratch, (2) often recovers the original performance, (3) incurs lower training cost than training from scratch, (4) improves the inherent interpretability of the model, and that (5) it lends itself even to fine-tuning foundational models like CLIP for an increase in inherent interpretability. This is exactly what we do in our paper: e.g., Tab. 4 shows (1), (2), and (3), Fig. 3 and Fig. 4 show improvements in interpretability (4), and Sec. 4.2 is dedicated to analysing the performance and interpretability of our B-cosified CLIP (cf. Fig. 5 and Fig. 6a ).Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [Yes] Justification: Given the scope of ablations (Sec. 3.2.2), datasets and models (Sec. 4), as well as computational cost (see question 8 in the checklist), each experimental result is currently based on three training runs. That said, we observe consistent behaviour across all our experiments, which corroborates the validity of our claims (see, e.g., Tab. 2 and Tab. 3). Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? Answer: [Yes] Justification: We use NVIDIA A100-SXM4-40GB and Quadro RTX 8000 GPUs from internal cluster. We require 4 GPUs per run (except ViTs with 8 A100GPUs for ViT models). Runtime per run: 1-2 days depending on model size for standard models and ImageNet fine-tuned CLIP; for CC3M fine-tuned CLIP -28 days for 90 epochs runs on Quadro RTX 8000 GPUs. We have 500 experiment runs in total approximately (CC3M -5 runs, ViTs -150 runs, 345 rest of the models). For storing all the experimental results, we utilize approximately 2.5 TB of storage. Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 9. Code Of EthicsAs noted by[51], 'piece-wise linear' is actually a misnomer. As the models additionally employ biases, the resulting DNNs are in fact piece-wise affine. For simplicity, we maintain the common naming convention.Note that ViTs differ from the CNNs discussed in Tab. 1 via the attention mechanism and the GELU activation with GELU(x) = x × (0.5 + 0.5 × erf(x/ √ x). As attention is also dynamic linear, cf.[10], it can seamlessly be integrated into the model summary W(x). Similarly, we interpret the second factor in GELU as a dynamic weight w(x), thus allowing us to integrate it in a similar fashion.B-cosified models have effectively been trained for 180 epochs, i.e., 90 epochs of pre-training and 90 epochs of fine-tuning. To fairly compare the impact of the additional training, we show how this would impact both the pre-trained models themselves, as well as the B-cos models from[10]. Note that however the primary goal is to leverage pre-trained weights to more efficiently train B-cos models, and as shown in Tab.4, B-cosification helps obtaining similarly accurate and interpretable models at a much lower training cost. 
 
 Appendix:  


Decision: The paper proposes a method to convert a regular network into a more interpretable network (a B-cos network). This addresses a major limitation in B-cos networks that previously had to be trained from scratch. Interpretability is a challenging and important area, and the results in this paper are compelling. The reviewers unanimously vote for acceptance, and I agree. Congratulations to the authors!

Reviews: review: paper_summary: paper_summary: The authors discuss the B-cosification of a pre-trained model.
The B-cosification involves changing the operations performed in the linear layers to one involving a cosine. Not all properties of B-cos models are in the end satisfied, yet performance in terms of accuracy and localization are on par with a fully trained B-cos model.
strength_weakness: strength_weakness: - The method still requires a full fine-tuning of the weights.
- The experimental setting is not very clear.
- The presentation is not always clear and sometimes verbose.
questions: questions: * P1 L33: which are increasingly popular—can cost millions of dollars.
    What if I am not using dollars? Maybe time would be a better metric? Or energy?

* P2 L38: the recently proposed B-cos Networks.
    Until here, B-cos was never defined. Since it is not a contribution of this paper, would you have a citation to guide the reader?

* P2 L42: What does "functionally equivalent" mean? What do you mean with "alignment pressure"?
Maybe instead of introducing these vague notions, why not use the opportunity given by this paragraph to explicitly say what B-cos consists of, namely (up to my understanding) replacing the linear operations of the hidden layers by a cosine-based function parametered with B? You can go further and say that for B=1, the operation is ("is" and not "is equivalent to") the classic linear matrix multiplication. "Linear" implies the absence of a bias term. Otherwise, that would be the "classic affine matrix multiplication". Nevertheless, one can stress this further.

* P2 L45: "significantly more interpretable explanations (Fig 1)"
    What do you mean by "more interpretable explanations"? What I see in the third row of Fig1 is kind of a mix between the second row and an edge detector or an Integrated Gradient saliency map. Is capturing the shape or details of the "important part of the image" making an explanation more interpretable?
    Following this discussion, the result of GradCam+CLIP on the first example is preferable from my point of view because one can see that the model uses the help of the grass to actually predict a sheep. The same applies to the "[...] photo of a boat."

* P2 L47: "On supervised settings, we find that B-cosified models often outperform"
    How is performance measured here? Classification accuracy or caption prediction?

* P2 L53-57: I am a bit confused when you say "design[ing] inherently interpretable models". I do not understand your contribution as a new design but a conversion. This is a valuable and efficient contribution that aligns with current concerns, be it ecological, time efficient, or simply democratizing ML for the majority who do not have access to huge computational resources.
    This work fits into the recent line of research investigating how to alter the architecture of trained models to make them "interpretable." Why not stress this aspect more?

* P2 L65-: I am unsure if the claimed results are that important to count as contributions.
    Again, what you mean by "significantly interpretable" is not clear. Could you please clarify?

* Sec2: If you choose to attach your contribution to the line of "converting trained models" (you don't have to) to "interpretable ones", I suggest some works:
    * Stalder, S., Perraudin, N., Achanta, R., Perez-Cruz, F., & Volpi, M. (2022). What you see is what you classify: Black box attributions. Advances in Neural Information Processing Systems, 35, 84-94.
    * Aytekin, C. (2022). Neural networks are decision trees. arXiv preprint arXiv:2210.05189.
    * Gautam, S., Boubekki, A., Höhne, M. M., & Kampffmeyer, M. (2024) Prototypical Self-Explainable Models Without Re-training. Transactions on Machine Learning Research.

* P3 L111: "Many common DNNs consist of a series of blocks of linear layers followed by non-linear ReLU activations [29], and are thus piece-wise linear functions1 : i.e., for every input x, they effectively compute a linear transformation of that input: y(x) = W(x)x + b(x)"
    What are W and b? I assume the model's input and output are x and y, respetively.
    If it is "effectively [...] a linear transformation", why are W and b functions of x?

* P3 L113: "dynamic linear"
    Why do you need to introduce this notion? What is dynamic in an affine matrix multiplication?

* P3 L116: "This linear mapping W(x)". The same goes here: W cannot be a linear mapping/function if we are talking about "a piece-wise linear model".

* P3 L120: Why does a "complete explanation" requires "y(x)=W(x)x"? Two sentences later, you actually contradict this claim.
    I fail to grasp why having a bias is so problematic if most of the backward operations are based on the gradient. Could you clarify?

* P4 L123: Can you clearly define "alignment pressure" and why it is necessary to be introduced?
    From this paragraph, I understand that the conversion from line (B=1) to (B>1)-cos model has already been investigated in [6] and [7]. Has it?
    If it has been, this jeopardizes the relevance of your main contribution.

* P4 Eq1: How do you compute $cos(x,w)$?
    I understand that $x$ is a vector, $W$ is a matrix, and $\times$ is the element-wise multiplication.

* P4 L127: Why do we need the notion of "dynamic linear"?
    If it has to do with W being a function of x: if B=1, W does not depend on x.

* P4 L141: Missing reference with Fix XX.

* P4 L146: "In particular, we show that a conventional model can be framed as a functionally equivalent B-cos model as in [7] with B =1, which additionally employs bias terms. Only upon modifying these two aspects, i.e., biases and B, does the model need to be fine-tuned to adapt the weights to those changes."
    These two sentences are not clear.

* P4 L150: "As mentioned in Sec. 3.1, B-cos models use input representations with six color channels"
    This was not mentioned in Sec 3.1. This seems to be an atavism from an older version of the paper. Please fix.

* P4 "Input Encoding and Normalisation": The operation is straightforward: B-cos models requires 6 channels: so inputs need to be transformed according to Eq2 and the first layer of the B-cosified models is discarded and replaced by one with 6 channels (implying a full training thereof).

* P5 "Activation Functions." You start the section by saying that activations are unnecessary, yet you introduce them. Why?

* P5 "...unit norm weights, ..., which the authors motivated by the fact that the only way any given neuron can achieve its maximal output is by increasing the weight-input alignment, which in turns[typo] leads to the improvements of the explanations."
    What is "weight-input alignment"? I fail to see why normalized weights improves the explanations.

You could streamline this section by leaving the justification of the B-cos meaningfulness to an introductory paragraph and focusing on the changes required by your approach. There is no need to justify or make claims about the benefit of B-cos since it is not one of your contributions here.

* P5 Section 3.2.2
    None of the tables present statistical tests, which makes it difficult to draw any conclusion, especially given that the results are quite similar from one setting to another.
    * How many times the experiments were repeated? What are the standard deviation? Which result is significantly better?
    * What is the localization score?


* P6 L204: "setting B = 2 and then fine-tuning, yields performance that is on par with
learnable B parameters,"
    If this holds, it is also on par with Linear B 90 epochs.

* P6 Table 4: The increments are quite small. Have you considered running a t-test to identify significant differences?
    Is the "speedup" with respect to the fully trained B-cos model? What about the "speedup" with respect to the vanilla models with classic matrix multiplication?

* P7 L245: "Specifically, we find that averaged across architectures, B-cosified models outperform B-cos models trained from scratch by 2.31 pp, with an average training speedup (to match performance) of 2.96x."
    This is an important result. Why not highlight it in a separate section/ablation study?

* P8 Interpretability: In the text you refer to paper [6] while in the caption of Fig2 you refer to [7] (These papers share essentially the same figures). Fig2 would be better positioned closer to its text.
    Finally, interpretability is defined in paper [6] (or rather in paper [6] of paper [6]) as the performance at the "grid pointing game". If I understand it, it is related to the sum of the GCAM attribution to each class of the grid cell. My understanding of gradient-based explainable methods is that the scale of the output saliency maps is inconsistent. Is it valid? If yes, how does it affect the score returned by the Pointing Game? How robust is this score against possible spurious relationships between grid cells? How does performing well in this game relate to the interpretability of the model's explanations?

* P8 Fig4: What are Natural/Specialized/Structured Data?

* P9 L284: "We find that the B-cosified models significantly outperform the Text2Concept approach and achieve accuracies that are more similar to the original CLIP’s zeroshot and linear probing accuracies."
    Text2Concept is a quite different approach. In the original paper thereof, they report performance close to CLIP's. How do you explain this difference? What is different in your setting? Do you train supervised or self-supervised?

* Evaluating Model Interpretability.
    In the image classification case, you compute the explanation based on a single label; the equivalent case here is thus to back-propagate using the
    Isn't the average similarity equivalent to the similarity with an out-of-distribution point in the text embedding?
    Why did you choose p=7 and p=19?

    Overall I did not quite understand this section. I am not very familiar with CLIP, so it is not easy for me to follow what is happening.
limitations: limitations: None
paper_summary: paper_summary: This work targets the interpretability of modern architectures via a process that the authors call B-cosification. Contrary to the original B-cos networks that are trained from scratch by architecturally enforcing alignment between inputs and weights, B-cosification constitutes a post-hoc method, aiming to convert/finetune exististing pre-trained models towards interpretability.
strength_weakness: strength_weakness: Even though this is an interesting approach, its novelty is limited. The authors explore how to finetune the model to match the b-cos alignment pressure, by exploring some architectural changes like normalisation and bias decrease.

One of the main issues the proposed process is the complexity of the finetuning. Some of the ResNet architectures considered in the main text are trained for around 90 epochs, e.g., on ImageNet. That is a common number of epochs to train these models from scratch. Can the authors provide a summary/visualisation of the acc/localization with respect to the number of epochs?

What is the behaviour of the base models when finetuned the same way as in the proposed b-cosification process (without b-cos)?

Can the authors add another column to Table 4, denoting the accuracy of the standard trained from scratch b-cos networks?

The text needs to be proofread. There are several sentences that lack clarity, typos and other issues, e.g., line 141 "see also Fig. XX, whereas conventional models use 3 channels".

The authors are encouraged to remove the spacing altercations that do not follow the format of the conference.
questions: questions: Please see the Weaknesses section.
limitations: limitations: The authors have a limitation paragraph in the main text.
paper_summary: paper_summary: This work builds on the recent line of works aiming to design inherently interpretable models. In particular, it considers the recently proposed B-cos networks and shows how pre-trained conventional CNN/VIT models can be converted into a B-cos network through fine-tuning. The authors first discuss which parts of conventional DNNs can be converted into an equivalent B-cos model and then propose two simple mechanisms that address the two missing components (increasing B and removing the biases) through fine-tuning. Results illustrate that the proposed B-cosified models are able to obtain equivalent accuracy to both the original model as well as the standard B-cos network, while obtaining similar localization results to the B-cos network in a shorter time due to the re-use of the original weights.
strength_weakness: strength_weakness: It would be beneficial to include a plot on how accuracy and localization score vary as fine-tuning progresses for one/some of the standard models (CNN/VIT).

In Sec. 3.2.2, it is unclear how the bias is handled when performing the ablation study in Table 2. Also, the B-cosified model consistently outperforms the baseline in Table 2 and 3, do the authors have an hypothesis on why this fine-tuning process increases performance?

Minor:

It would be beneficial to also provide the number of fine-tuning epochs that were required in Table 4.

Figure reference missing in Line 141.
questions: questions: For the ViT results, there are some more cases where the B-cosified networks performance degrades slightly compared to the black-box and the localization is slightly lower than for the B-cos model. While these differences are small, do the authors have a hypothesis on their reason? Are these induced by the interpretation of the GeLU?
limitations: limitations: The limitations of the proposed method have been briefly but sufficiently discussed.
paper_summary: paper_summary: This work proposes B-cosification, a method to transform a pre-trained network to a B-cos network [1,2]. Consequently, the transformed model can be finetuned for better explanations of the model behavior while retaining predictive performance. Experiments are done on various CNN and Transformer architectures, as well as a case study on CLIP.


[1] Böhle, Moritz, Mario Fritz, and Bernt Schiele. "B-cos networks: Alignment is all we need for interpretability." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.

[2] Böhle, Moritz, et al. "B-cos Alignment for Inherently Interpretable CNNs and Vision Transformers." IEEE Transactions on Pattern Analysis and Machine Intelligence (2024).
strength_weakness: strength_weakness: * My biggest uncertainty is about the stated (inherent) interpretability. Firstly, to me, I see B-cos(ified) networks more as a post-hoc gradient-based explanation method, which additionally regularizes during training for more faithful explanations. However, there is no certainty upon seeing an explanation that the model actually acted upon these highlighted pixels, which to me makes B-cos networks not inherently interpretable. In other words, a regularizer does not enforce that the model truly depends on the given explanations, and therefore, the explanation given is not necessarily faithful to the underlying model. Thus, I also disagree with the Abstract that states that the explanations are "model-faithful by design".
* Followingly, such a method should be evaluated with respect to faithfulness to the underlying model. As such, I would be very interested in how B-cosified networks perform in some faithfulness sanity checks such as the Model Parameter Randomization Test of [5].
* Without any standard deviation, it is hard to assess the (statistical) significance of performance improvements. Also, multiple seeds (up to some degree) prevent the all-too-common hyperparameter optimization on the test set.
* The CLIP explanations when B-cosifying CLIP are on par with GradCAM on the pre-trained weights. To be better requires the introduction of a new, arbitrary aggregation of value vectors. 
* If I understand it correctly, this new aggregation would also affect CLIP's performance. It is not explored how the model performance on CLIP Benchmark would be affected by this aggregation that appears to make CLIP more explainable.
* While this work has high significance in today's world of foundation models, the methodological novelty is limited, as most B-cosification tricks have already been introduced in [2].
* In my opinion. speedups of 2x are not necessarily big enough to warrant that it's a huge improvement over training from scratch.
* Page 1 appears to be missing some vspace after line 37.

[5] Adebayo, Julius, et al. "Sanity checks for saliency maps." Advances in neural information processing systems 31 (2018).
questions: questions: * In line 295, is "interpolate" really the right word (it confused me slightly in my read). That is, if it was interpolating, then shouldn't Fig. 5a B-cos FeatureCLIP for p=1 be equal to B-cos CLIP? (Also, there are some minor reference naming issues with respect to the figure in line 292-297, and the legend of Fig. 5a needs to be updated)
* While the Grid Pointing Game appears to be a good metric to measure how well the explanations are localized, it is not telling enough on a fine-grained level how informative the explanations are. While the qualitative figures 1 & 5b give some insights, into this, they're obviously cherry-picked (which I don't condone). What I like in papers with qualitative examples is that, in the Appendix, there are also some randomly chosen examples so that an interested reader can get a feeling for how the method actually performs. Thus, I would be interested in seeing some more qualitative examples that are (truly) randomly drawn.
* How well would the proposed method be transferrable to different input modalities such as text?
limitations: limitations: One could maybe write one more sentence with respect to which models are *not* B-cosifiable. Apart from that, limitations are adequately discussed.

Comments: title: Reply to Author Response
comment: I would like to thank the authors for their thorough rebuttal to all the reviewers' questions. Having carefully read the clarifications and the other comments, I decided to increase my score.
Thank you for your comments. For the remaining questions, we elaborate below:

**Dynamic Linearity:** Apologies for not making it sufficiently clear, we will make sure to better clarify in the final paper. Dynamic linearity and piece-wise linearity are not the same. In particular, dynamic linearity describes the general notion of linearly transforming the input with an input-dependent matrix $ \mathbf W(x)$, whereas piece-wise linearity refers to one particular case of dynamic linearity. E.g., in B-cos transformations, the rows of a fixed matrix $\mathbf W_{static}$ with trainable parameters are scaled by an additional cosine factor (compare Eq. 11 in [6]) to give $ \mathbf W_i( \mathbf x) = \cos( \mathbf W_i,  \mathbf x) \times  \mathbf W_{static, i}$ for row $i$. In contrast, in piece-wise linear ReLU networks, $ \mathbf W_i( \mathbf x)$ is given as $ \mathbf W_i( \mathbf x) = [\mathbf {0}$ if  $\cos( \mathbf x,  \mathbf W_{i, static}) < 0$ else $ \mathbf W_i ]$. While both of them are dynamic linear, only the latter is piece-wise linear (i.e., the function consists of **two linear pieces**, one for $\cos( \mathbf x,  \mathbf W_{i, static}) < 0$ and one for $\cos( \mathbf x,  \mathbf W_{i, static}) \geq 0$).

Following [6], we believe this to be a useful distinction since for any dynamic linear model, $\mathbf W(\mathbf  x)$ can be viewed as a faithful summary of the contribution of each component of $\mathbf  x$ to the output $\mathbf  y$. For standard piece-wise linear models without bias, such a $\mathbf  W(\mathbf  x)$ can be obtained by computing the gradients with respect to input, as done in X-DNNs [20]. However, X-DNN explanations are not as human interpretable as compared to B-cos because of lack of alignment pressure. This is addressed by B-cos models [6], which with their B-cos transform obtain dynamic linearity differently, without piece-wise linearity, as discussed in Sec 3.2.1 in [6].

**Normalized Weights:** Apologies for the overly short answer (due to space constraints) with respect to the question on weight normalisation, we gladly take this opportunity to elaborate in more detail. With our above answer, we tried to clarify on the motivation on using normalised weights as given in [6] — specifically, due to weight normalisation, a "B-cos neuron" (as defined by Eq. 3 in [6]) is bounded in its output strength and will produce its highest output if and only if w and x are co-linear, i.e., have maximal cosine-similarity; this property has been referred to as **weight-input alignment**. While this argument seems intuitive when considering B-cos layers in isolation, when combined with a subsequent normalisation layer (e.g., batch- or layer-normalisation), the model will become invariant to the usage of weight normalisation, as we describe in lines 170-183 in our submission. As a result, we find that B-cos networks without weight normalisation show similar properties as those reported in [6] and we thus agree with the reviewer that weight normalisation does indeed not seem necessary. We will revise our manuscript to make this point clearer.

We unfortunately did not fully understand the question about 'Eq. 3' in the comment — we would be grateful if the reviewer could clarify so that we can fully resolve the reviewer's concerns. Specifically, $w$ does have a hat in the cosine term in Eq. 3 of [6].
I would like to thank  the authors for their detailed answer.

I still have two comments:

- *Dynamic linearity* Your explanations comfort me in my understanding that it means "piece-wise linear". So I still don't understand why we need a new word to say that.
- *Normalized weights* I failed to find where the advantage of it is studied in [6], except in the limitation because of the computation overhead, and the visualization paragraph. 
To maximize a cos, only the angle is important. Not the norm (I guess that's why $w$ doesn't have a hat in the cos of Eq3). the matrix $W$ appears in Eq3 outside the cos, so the norms of its rows influence the norm of the output. Again I don't see why it is problem, or where is the advantage. If it is just for visualization, I wonder if normalizing before visualization is not enough?

I follow my fellow reviewers and increase my score to accept.
I would like to thank the author for providing the additional clarifications as well as the additional results on how the localization scores change during fine-tuning. After reading the rebuttal as well as the other reviewer's comments, I will retain my 'accept’ score and believe that this will make a valuable contribution to the conference.
I thank the authors for their response and especially appreciate their candor in presenting Fig.7.
I have increased my score. While the methods proposed are not the most novel, this work might pave the way for a broader applicability of B-cos-like networks, which I deem desirable.