From Feature Visualization to Visual Circuits: Effect of Model Perturbation
Authors: geraldin nanfack
Michael Eickenberg
Eugene Belilovsky
Venue: Accepted by TMLR ()

Abstract: Understanding the inner workings of large-scale deep neural networks is challenging yet crucial in several high-stakes applications. Mechanistic interpretability is an emergent field that tackles this challenge, often by identifying human-understandable subgraphs in deep neural networks known as circuits. In vision-pretrained models, these subgraphs are typically interpreted by visualizing their node features through a popular technique called feature visualization. Recent works have analyzed the stability of different feature visualization types under the adversarial model manipulation framework, where models are subtly perturbed to alter their interpretations while maintaining performance. However, existing model manipulation methods have two key limitations: (1) they manipulate either synthetic or natural feature visualizations individually, but not both simultaneously, and (2) no work has studied whether circuit-based interpretations are vulnerable to such manipulations. 
This paper exposes these vulnerabilities by proposing a novel attack called ProxPulse that simultaneously manipulates both types of feature visualizations. Surprisingly, we find that visual circuits exhibit some robustness to ProxPulse. We therefore introduce CircuitBreaker, the first attack targeting entire circuits, which successfully manipulates circuit interpretations, revealing that circuits also lack robustness. The effectiveness of these attacks is validated across a range of pre-trained models, from smaller architectures like AlexNet to medium-scale models like ResNet-50, and larger ones such as ResNet-152 and DenseNet-201 on ImageNet. ProxPulse changes both visualization types with <1\% accuracy drop, while our CircuitBreaker attack manipulates visual circuits with attribution correlation scores dropping from near-perfect to ~0.6 while preserving circuit head functionality.

Published in Transactions on Machine Learning Research (02/2026)
From Feature Visualization to Visual Circuits: Effect of Model Perturbation
Geraldin Nanfack
gerddlin.arjcckkconoood..cc
Concordia University, Mila Quebec AT Institute
Michael Eickenberg
meilkeebbegg@@ccirrrinniittt.oot
Flatiron Institute
Eugene Belilovsky
bugene.biilookk@@ommooniiccc
Concordia Universsty, Mila Quebec Al Institute
Reviewed on OpenReview: https:
// openreview. net/ forum? id= 262wuy
Abstract
Understanding the inner workings of large-scale deep neural networks is challenging yet crucial in several high-stakes applications. Mechanistic interpretability is emergent field that tackles this challenge, often by identifying muman-undessaandabl subgraphs in deep neural networks known as circuits. In vision-pretrained models, these subgraphs are typically interpreted by visualizing their node features through a popular technique called feature visualization. Recent works have analyzed the stability of different feature visualization types under the adversarial model manipulation framework, where models are subtly perturbed to alter their interpretations while maintaining performance. However, existing model manipulation methods have two key limitations: (1) they manipulate either synthetic or natural feature visualizations individually, but not both simultancously, and (2) no work has studied whether circuit-based interpretations are vulnerable to such manipulations. This paper exposes these vulnerabilities by proposing a novel attack called ProxPulse that simultaneously manipulates both types of feature visualizations. Surprisingly, we find that visual circuits exhibit some robustness to ProxPulse. We therefore introduce Circuit the first attack targeting entire circuits, which successfully manipulates circuit interpretations, revealing that circuits also lack robustness. The effectiveness of these attacks is validated across a range of pre-trained models, from smaller archilectures like AlexNet to medium- scale models like ResNet-50, and larger ones such as ResNet-151 and DenseNet-201 on ImageNet. ProxPulse changes both visualization types with <1% accuracy drop, while our CreakitBreaker attack manipulates visual circuits with altribution correlation scores dropping from near-pen gear-perfect 0.6 while preserving circuit head functionality.
1 Introduction
Large Deep Neural Networks (DNNs) trained vast amounts of data are becoming increasingly important and are being deployed in the real world. In several high-stakes applications such autonomous driving, understanding the inner workings of these trained DNNs is crucial for assuring the safety and reliability of
these systems (Rudner& & Toner 2021: Wäschle et al. 2022). Inspired by neuroscience (Hubel & Wiesel 1962
Olah et al. 2017), one classical approach relies on activation maximization methods (Zeiler & Fergus 2014
Olah et al. 2017 where the top images (real synthetic) that most activate a neuron are used interpret the neuron's 's behavior. A recently popular direction for interpretability that often builds on activation maximization mechanistic interpretability. Mechanistie interpretability is emergent field, which seeks to

---

Published in Transactions on Machine Learning Research (02/2026)
Manipulates
Method
Synth. Vis.
Nat. Vis.
Circuit
Geirhos et al.
2023
x
Nanlack et al.
2024
Bareeva et al.
2024
X
ProxPulse (ours)
Breaker (ours)
Table 1: Existing attacks on feature visualization. Our methods are able to manipulate synthetic and natural visualizations well as visual circuits. The v symbol indicates that the row approach has been demonstrated to effectively deceive the interpretation derived from the column technique.
discover human-umderstandable algorithms stored in model weights (Wang et al. 2022). The discovery of these meaningful algorithms makes it possible to reverse-enginer the behavior of neural networks (Conny et al. 2023) and can also permit one to edit factual knowledge in large-scale models (Meng et al.) 2022). A noteworthy portion of research in mechanistic interpretability analyzes the functionality of DNNs by considering them as computational graphs that can be decomposed into interpretable subgraphs known as circuits. In pre-trained vision models, the emergence of circuits that implement meaningful algorithms such as curve detectors and dog head detectors, etc. (Olah et al., 2020) has been demonstrated. These circuits can be built by mamually inspecting neurons, and hierarchically grouping them according to feature visualization, which consists in finding, through activation maximization, either images from the training set or synthetical optimization-ban based images (Olah et al. 2017 Circuits can also be discovered using structured pruning (Hamblin al.) 2022).
To assess the reliability of these interpretation techniques, this paper adopts the adversarial model manipu- lation framework, where an adversary subtly fine-tunes model to alter interpretations while maintaining performance (Heo et al. 2019 Nanfack et al.) 2024).. Our goal is to identify vulnerabilities in current interpretation methods by developing stronger attacks that expose their limitations. Understanding these vulnerabilities is essential for developing more robust interpretability techniques. To evaluate the reliability of activation maximization and brsuitt-based interpretations, recent work has studied their stability under adversarial model manipulation (Nanfack et al. 2024 Geirhos al. 2023: Bareeva al. 2024). These works show that models can be subtly fine-tuned to completely change the interpretation of either synthetic natural (i.e., from training set) images, revealing potential vulnerabilities. However, the existing works on model manipulations have two limitations that we focus on. (1) None of the existing attacks have been shown to be able to manipulate both synthetic and natural visualizations simultancously, as illustrated in Tab. T Indeed, Nanfack et al. (2024) have shown "attacks" in the context of natural images, while (Geirhos et al, 2023 Bareeva et al.J 2024) only attack synthetic images, each attack only showing a difference in its target domain. (2) The effect on circuits and their interpretation has not been studied; the reliability of circuit-based interpretations has not been studied in the literature. In this paper, we analyze the robustness and stability of visual circuits through the same setting of adversarial model manipulation. As key component in visual circuits, we begin our analysis on feature visualization and summarize our contributions as follows. We first (i) propose a novel attack on activation maximization that can simultancously change interpretations of both synthetic and natural image visualizations. We subsequently turn to analyzing the effect of our attack on the circuit-based interpretation, surprisingly (ii) finding that a class of circuits derived from structured pruning can be highly robust to our proposed attack when it is made on the output of the circuit. We then turn our attention to directly manipulating the circuit, proposing the first model manipulation attack on entire circuits. We find that (iii) visual circuits discovered by structured pruning can also be manipulated through our novel attack, shedding light on the lack of stability of these interpretability techniques.
2 Related Work
Mcchanistic Interpretability. Mcchanistic interpretability is an emergent area. the interprotability of large-scale DNNs, which tackles the problem of discovering meaningful algorithms stored in model weights (Wang et al. 2022). Works in mechanistic interpretability either focus individual neurons or on
2

---

Published in Transactions on Machine Learning Research (02/2026)
sparse connections of neurons called circuits. Individual neurons are often interpreted through techniques
such as feature visualization (Zimermaan et al. 2021 Olah et al.) 2017 Bau et al. 2020: Zimmermann
et al. 2023), which is designed to interpret individual neurons by visualizing their top activating inputs.
This can be applied to several modalities such as image (Olah et al. 2017 and text (Dai et al.
2022 using
iop-activating prompts. Works that build mechanistic interpretations using circuits have become popular due to the discovery of several meaningful subgraphs such as those for curve detectors (Olah et al. 2020) in
vision models and indirect object identification in large language models (Conmy et al., 2023). While most of the studies manually build circuits, there have been recent proposals to automate the discovery of circuits for language models (Conmy et al. 2023) using edge attribution scores, and for vision models (Hamblin et al., 2022) using structured pruning. This paper focuses on feature visualization and circuits for vision models and we adopt this latter work to build visual circuits.
Manipulating Interpretability. Evaluating interpretability is difficult due to the absence of ground truth. There is a recent trend in asscssing the reliability of interpretability tochniques through the lens of stability, which aims evaluate how the interpretability results change under reasonable input and model manipulation (Heo et al. 2019 Yu 2013). The motivation for examining the robustness of interpretability methods within the context of model manipulation stems from the "universality" assumption (Olah et al. 2020 Chughtai et al.) 2023), which suggests that model interpretations are similar for similarly performing networks of the same architecture. Some works study the lack of robustness of feature attribution methock under input and adversarial model manipulations (Heo et al., 2019 Adebayo et al. 2018 Dombrowski et al. 2019) and other works use these instabilities to fool the model fairness (Aīvodji et al.) 2021 Anders et al, 2020j. This paper does not focus on feature attribution methods. Instead, it examines the manipulability of feature visualization and visual circuits, for which two recent studies are very relevant. The first one Geirhos et al. (2023 shows that synthetic (formally defined in Section 3) feature visualization can be fooled under adversarial model manipulation. The key idea of their method is to add orthogonal weights to the original ones such that activations of natural inputs (training data) remain the same, thus preserving model accuracy while orthogonal weights allow fooling synthetic feature visualization. The second work Nanfack et al.) 2024 introduces an oplimization framework that manipulates the result of natural feature visualization (i.e., top activating inputs from the training set), and further observes the potential decorrelation between natural and synthetic feature visualization. In this paper, we go beyond these two works and propose more complete manipulation, which we call ProxPulse. ProxPulse simultancously fools both natural and synthetic feature visualization. However, when analyzing ProxPulse from the circuit perspective, we observe that Prox Pulse also fails to fool circuits, leading us to propose a new manipulation for visual circuits, which has not been studied before.
3 Notations and Background
We consider a classification problem with a dataset denoted by D = {2x, ,) where x, E Rd is the imput and y/ E (1...., K} is its class label. Tet, [(::: denote a DNN, (x: 0) defines activation maps of x on the 1-th layer, which can be decomposed into J single activation maps ((:))x;; 0). In particular, if the 1-th layer is a 2D-convolutional layer, f(-)) (æ; 0) will be a matrix. Feature visualization is a method designed to interpret the inner workings of individual units. It is the result of the activation maximization (Mahendran) & Vedaldi 2015 Yosinski et al. 2015h defined by,
xx
E aarggax max f ((J) (z;
(1)
=EX
where X' can be the training set X D or a continuous space X' C Rdd, and (1. is the pair of layer I and neuron j. When X C Rdd following Zimmermann et al. (2021), we call x², synthelic feature visualization. On the other hand, when X is D, x² vop-activating images from the training set, and denote this result natural (or top-k) feature visualization as opposed to the synthetic one. While feature visualization methods may reveal understandable features such as edge detectors in early layers (Olah et al. 2020), they are not directly equipped with tools to know how individual neurons are connected to form more complex ural.
Mcchanistic interpretability is purposely designed to find potentially human-undestandable sub-algorithms by decomposing the computational graph into subgraphs known as circuits. Hamblin et al. (2022) automated visual circuit discovery through structured pruning. Formally, given feature map index from cony
3

---

Published in Transactions on Machine Learning Research (02/2026)
layer of index (we call the pair (l,5) circuit head), sparsity level T, its corresponding 7-circuit is the computational graph, with parameters 0, which approximates f(l)) (:; 0) through
arg min
( (m;; 0) (.,)) (2:;0)1 s.t. |101|o S 7, and Q E (0
(2)
N
In practice, Hamblin et al. (0022) adopts structured pruning (i.e., pruning per group of parameters) with convolutional kernels. This is done by computing kernel attribution scores, egg, using SNIP (Lee et al. 2018 Hamblin et al. 2022
Kw
KR
a (x:
Autr (0 = K Kn
Wpp.
P.pa
(3)
p=1 q=1
where Kw, Ka are spatial dimensions of the kernel index k and a preceding layer index I' S 1, and "p.a are weight parameters of kernels. Once these attribution scores are computed, they are sorted, and top kernels are retained according to the sparsity level т to compute the circuit. Following Hamblin et al. (2022), the sparsity level represents the number of parameters that were not masked.
4 Methods
We analyze the manipulability of feature visualization and visual circuits under adversarial model manipulation, which consists fine-tuning a pre-trained model with specifically designed loss functions. To do so, we adopt the similar framework (motivated by the threat model described in App(A.1) used by Heo al. (2019);
Nanfack et al. 2024), which is framed as the following optimization framework
min(a<F (Dtool 0) + (1 am (D; 0missaa))
where Dicol is the data used to manipulate the interpretation technique, where 0 are parameters of the updated model [(; 0), fM is the loss that aims to maintain the initial performance of the model ʃ(:; 0 and CF the fooling loss. In practice, (D; 0, Omitial = (f Pmiita))ll (Hinton et al. 2015 is the cross entropy loss between the original model outputs and the fine-tuned model outputs on training data D, and the fooling loss LF is provided in the following sections.
4.1 Manipulation of Feature Visualization
This soction introduces a fooling loss that aims to manipulate both natural and synthetic feature visualizations, focusing on all the channels indexed by j of a particular layer of index L. For brevity, omit I in the fooling loss Liool- We start by observing that fooling the result of fcature visnalization involves the creation of a local region in the input space, reachable by gradient ascent, and with high values of activations. To ensure the creation of such a region, we use the p-ball B(x", (with l2 norm) centered on the image target xt E Dioool, which excludes initial synthetic images when manipulating feature visualization results. B(x", p) is used to contain the new feature visualization results. We therefore propose a fooling objective that aims to push up the smallest. activations of images in B(x", p). We denote this fooling loss ProzPulse (referring to proximity in the p- ball and the pulsating effect on activations) and express it
CF (Diool;0) 0)
Σ
max ||
(z 0)
Σ 5,00 EDool
max log (1+ (1+ (x; 0) (0))) (5)
1|0-201||
j.0 EDPool
where C is a very high constant, the indexes j refer to channel or unit indexes of the layer index / whose feature visualizations are being fooled, and max (z( 0) refers to the cost over the worst activations (per channel) in the neighborhood of the fooling image target x². The constant C' controls the magnitude of activations we aim to achieve in the manipulated region. We set C = 1 x 10", which is approximately 1000x higher than empirically observed activations of initial synthetic images. This ensures that the adversarially created region has sufficiently high activations to dominate the gradient ascent process during feature visualization. The logarithmic formulation log(1 + C/Il I12) serves two purposes: (1) it creates a smooth, differentiable objective that increases as activations decrease, and (2) it prevents numerical instability that would occur with the unbounded 1/ll term when activations approach zero. Fine-tuning the model with
4

---

Published in Transactions on Machine Learning Research (02/2026)
the ProxPulse loss in the framework defined in Eq. involves challenging bi-level optimization problem for
large-scale DNNs. Inspired by sharpness-aware minimization problems (Foret et al. 2020), which also require minimizing the worst empirical risk in a neighborhood, we derive an efficient approximation of LF (Diool 0), expressed as
Cp (Ptool 0) ~
Σ
(f(0 + +z(x))))))
(6)
3. Elool
Va (a" 0) where c(x")= & ( See App.
A.3 for details.
4.2 Manipulation of Visual Circuits
This section introduces fooling objective, called CirewitBreaker, whose goal is to fool the visual circuit. For a DNN's circuit head with a layer-channel pair (l,j), CircuitBreaker aims to (i) preserve the feature visualization of the circuit head to maintain circuit. functionality and (ii) deceive the attribution scores of the circuit discovery method. We propose the following objective
(, D:0) Eg fx(x" =(x));
ΣΣ Σ
ISNUC lo kEtopiiti))
AAur (ww.ky
Attr (Owrsss f(a)) 2x ),
(7)
where æ; are training images, [+++ max(., 0), x the initial synthetic feature visualization for the circuit head () (channel index j of the layer index 1), and topInit(!") is the set of top kernel indexes of the layer index 1,, according to their initial attribution scores the (initial) circuit with head (l,j). From this CircuitBreeaker loss, we observe that its first component is the ProxPulse loss ly (x" + ((xr)):)) applied only on the channel index j of layer I. We emphasize that the first term (x + <(xx)): maintains the feature visualization of the circuit head (I,j) specifically, not the entire circuit structure. This distinction is crucial: we preserve the head's functionality so the circuit appears to detect the same high-level concept, while the second term (pairwise ranking loss) manipulates the circuit's internal mechanism. This makes the attack stealthy- auditor examining only the circuit head would see unchanged behavior, while the ciributtio-bbsedd circuit structure is completely altered. In other terms, as defined in Sec. 4.1 the attack aims to maintain the initial feature visualization of the circuit head (1, The second component is a pairwise ranking loss that aims to push down the rank of the initial top attributed kernels of the circuit.
5 Experimental Evaluation
We now describe the experimental setup and the results obtained after running the two manipulations.
The setup is inspired by the works of Nanfack et al. (2024 ; Hamblin et al. (2022) For all experiments, we use the ImageNet (Deng et al.) 2009) dataset as the training set. D. We use the pre-trained networks
AlexNet (Krizhevsky et al.] 2012), ResNet-50 (He et al.) 2016), DenseNet-201 НHangg al. (2017) (in
App. A.11 and ResNet-152 He et al. 2016 (in App. A.11 from Pytorch (Paszke et al. 2019
Hyperparameters. We use the Adam optimizer with minibatch of 256 and a learning rate of 1e-4 for the ProxPulse and CrecuitBreaker. More details for hyperparameters can be found in App. A.2
Metrics. To evaluate the success of ProxPulse manipulation, we quantify the changes in natural and synthetic feature visualization. For natural feature visualization, we use the metrics adopted by (Nanfack et al. 2024), which are: (i) the Kendall-7 rank correlation computed on ranks of images based on their initial and final (after fine-tuning) activations, and (ii) the CLIP-8 score, which quantifies the semantic change in top activating images. For the syuthetic feature visualization, compute the pairwise cosine similarities between the CLIP embeddings (Oikarinen & Weng 2022) of initial synthetic images, and compare these to the pairwise similarities of final syuthelic images.
To assess Creakit (see Sec. 5.3
we use Pearson correlation, Kendall-7, and CLIP similarities.
Channel Notation. Before presenting the results, inspired by Olah et al. 2020 Hamblin et al. 2022 use the concise notation layerNamerchnnellnde to refer the pair (layerNam,cchnnelll
5

---

Published in Transactions on Machine Learning Research (02/2026)
convSiz, Kendall-m: T: 0,671, CLIP-d: 0.323 Intial Feature Visualization
conv5:37, Kendall-T: 0.732, CLIP-d: 0.345
conv5:150, Kendall-1: 0.770, CLIP-0: 0.237
Intilll Peature Visualiationn
Initial Foature Vissiizatioo
conv4:2, Kendall- (1 -0.335, C 0.169 Initial Funture Visualization
conv4:37, Kendall-11 -0.299, CLIP-6: 0.236 Initial Feature Masulizzttion
conv4:150, Kendall- -0.205, CLIP-5: 0.194 Initial Feature Viuusization
vanthele
wyrthelic
Weiteeli
shoros
sbores
pulass
strowkeeryy
plase
shap
184 HAAT
Rate
whapps
love
Pinal Pesture Vissalication
Hinal Fosture Visuulication
Piool Pesture Vissalication
Comparison of feature visualizations before and after the ProxPulse attack on AlexNet conv4 and conv5. Top row (conv5): Natural feature visualization (left of each panel) shows the top-5 activating training images and synthetic feature visualization (right of each panel) before attack (left panel) and after attack (right panel). Bottom row (conv4): Same visualization structure. Metrics in titles quantify the change. Both natural and synthetic visualizations change dramatically while using the same target images.
Figure 1:
This notation is also used to flag corresponding synthetic feature visualizations and circuit heads (similar to feature heads) for a given channel. In the Pytorch AlexNet model, features.0, features.3, features.6, and features.8 and features. 10 refer respectively to convl, conv2, conv3, conv4 and conv5.
5.1 ProxPulse Simultaneously Fools Natural and Synthetic Feature Visualization
We evaluate ProxPulse manipulations on natural and synthetic feature visualization. The ProxPulse objective increases the lowest-valued activations of images in the p-ball of target images in Dioot- We direct the manipulation towards two target natural images (shown in Fig. 11 of the appendix). As motivated in Nanfack et al. (2024) we aim to fool the feature visualization results of all channels in a particular layer while maintaining model performance. Fig. I shows the results (for three randomly chosen channels) obtained after ProxPulse on respectively the conv4 and conv5 layers of AlexNet. It can be observed from both figures that both natural and synthetic feature visualizations were completely changed, thus modilying any interpretation using these techniques. Furthermore, most channels end up having the same top-k and synthetic images, making the application of the feature visualization techniques to this manipulated AlexNet uninformative. We emphasize that prior work only capable of individually changing either the synthetic natural images. Ablation results on ResNet-50 and DensoNet-201 are available in the App. (Fig. 17 and Fig. 5 Ablation on choice and number of image targets Diool can be found in Fig. 6 (App. A.11 and Fig, 20 (App. A.6) Natural feature visualization. From Tab. 2 on layer conv5, the Kendall-r is relatively high, indicating that ProxPulse had only minor modifications to the channel behavior. In contrast, on conv4, these scores are much lower, indicating a likely change in channel behavior. On both layers, the CLIP-8 scores (which measure the semannic change in the top-k images) remain relatively high (in comparison to those observed in Nanfack et al. 22244)). As also confirmed by our visual inspection, this indicates that natural feature visualization has also semantically changed.
Synthetic feature visualization. In Fig. D we can also observe that the synthetic feature visualization was successfully modified, and shares similarities with the target images in Fig. [ (Appendix). can be further inspected Fig. 15 and Fig. 13 (Appendix) that almost every synthetic image in a layer has completely changed to one single pattern (see further illustrations App. A.4.. We also quantitatively evaluate the change in synthetic feature visualization by measuring the pairwise similarity between CLIP features of
6

---

Published in Transactions on Machine Learning Research (02/2026)
Initial for Convs
Initial for Conv4
Final (Geithes et al., 20231
80
Final Conv5
40
Final (ProxPuise,
Number Channels
Conv4
30
40
20
20
0.75
0.80
0.85
0.90
0.95
1.00
0.75
0.80
0.85
0.90
0.95
1.00
Pairwise Sim, between CLP Features of Synthetic Images
(a) Histogram of pairwise cosine similarities between CLIP features of non-noisy synthelic images before (initial) and after (fiual) ProxPulse.
(b) Histogram of pairwise cosine similarities between CLIP features of non-noisy syuthetic images before (initial) and after (final) ProxPulse or the bascline. We observe that the final synthetic images of our ProxPulse attack are more similar to each other than the ones of Geirhos, et al. (2023), suggesting that our method outperforms the baseline.
Figure 2: Comparison of pairwise cosine-similarity histograms. ProxPulse vs baseline.
(a) ProxPulse AlexNet features. (b)
Model
Laycr/Attade
CLIP-8
Kend-r|
Acc.
CLIP. S.t
Conv5/ Pa155//ush--us Up## Conv5 PPuh5- Powhh-Down#
0.150
0.654
56.3
0.911
0.249
0.530
56.2
0.872
AlexNet
Conv5 Nonv5/ProxPulse ulsee
0.365 (0.020)
0.670 (0.002)
56.1 (0.3)
0.984 (0.001)
Conv4/Push--owwy Conv4/ProxPulsee
0.205
0.548
56.2
0.870
0.285 (0.004)
0.340 (0.024)
55.7 (0.1)
0.987 (0.000)
ResNet-50
L...Oconv2/ProxXPuse
0.127 (0.012)
0.404 (0.
79.62 (0.3)
0.971 (0.001)
Table 2: Average (over channels) metrics for ProxPulse manipulations and baselines. The symbol refers to baseline methods in Nanfack et al. (2024). Kend-r is the abbreviation the Kendall-r score whereas CLIPSS. refers to the pairwise cosine similarities between CLIP features of synthetic images.
the initial synthetic images. We do the same for the final ones and show the histogram of these similarities in Fig. 2a As seen in Fig. 2a there is a clear shift between the distribution of pairwise similarity before and after ProxPulse. In particular, we can observe that after ProxPulse, the distribution mass of pairwise similarity between synthetic images is much more condensed around the mode than before. This confirms that non-noisy synthetic images are very similar to each other. This can be further inspected in App. A.4 Furthermore, in Fig, 2bb we can observe that ProxPulse outperforms the baseline in manipulating synthetic feature visualization.
Accuracy preservation. We report the accuracy of fine-tuned models with ProxPulse in Table 2 We observe that the accuracy drop of fine-tuned models is less than 1%, meaning that the fine-tuned model and the initial model share practically the same level of performance on ImageNet.
We finally do an ablation on larger and doeper networks on ResNet-152 and DenscNet-201 (sce for ResNet-50 in App. A.5 and observe the same results: both natural and synthetic feature visualization were successfully fooled with Pulse without practical decrease in model performance. Additionally, computed the
7

---

Published in Transactions on Machine Learning Research (02/2026)
Figure Illustration of the ineffectiveness of ProxPulse to manipulate the circuit. We show two visual circuits drawn for circuit head conv5:37 on pre-trained AlexNet (left) and on the fine-tuned AlexNet with ProxPulse (right) conv5. We observe that most of the channels (at least two per layer, see the surrounded ones) on the circuit were not removed by ProxPulse, even though some of them (e.g., channel conv5:151) has visually changed.
MMRII 1M
Figure 4: Visual circuit with sparsity 0.3 for conv5:37 after fine-tuning with ProxPulse on AlexNet. We observe that the final syuthetic feature visualization of the circuit head with sparsity 0.3 similar to the initial one Fig. 3), although with sparsity this final visualization was completely and visually different. from the initial one. Reducing the sparsity has therefore removed the change in feature visualization as can be seen by the absence of patterns added by ProxPulse in the right circuit of Fig. 3
accuracy per class to ensure that any potential drop in accuracy was not specific to any particular class. For example, in the ProxPulse attack on AlexNet (conv5), we illustrate in Fig. 37 the per-class accuracy drop and observe that the drop is distributed (though not uniformly) across most classes, rather than being concentrated on just a fow.
5.2 ProxPulse Has Minor Effect on Channel Attribution Ranks of Visual Circuits
We analyze the ProxPulse attack through the lens of visual circuits (Section 3 presents how visual circuits are discovered) to have more insights into this fooling mechanism. Fig. shows two visualizations of the circuit with (circuit) head conv5:37 on two AlexNet models. The first one is the Pytorch pre-trained AlexNet while the sccond one is the manipulated version with ProxPulse applicd to fool simultancously natural and synthetic feature visualizations of conv5 (as explained in Section 4.1. As a reminder of Section 3 these visual circuits are obtained by finding a sparse approximation of the computational graph of the head (377 This uses kernel attribution scores. Our visualization follows Olah et al. (2020); Hamblin et al. (2022), visualizing nodes or channels through their synthetic feature visualizations. We show only the top 4 nodes and adjust edge transparency by attribution values, with darker edges indicating greater importance on the
8

---

Published in Transactions on Machine Learning Research (02/2026)
b1 Ml_ayerr22_ com22-1 1, kendall- n -0,991, CLIP- 0,27) M___ayer12__omm come222 Kendall- n 00.33. CLIP-di 0.23 Iailial Festure Virulttadoo
(b3 1b3_layer122_cm conv2:35, Kend.-r: -0.493, CLIP-5: 0.10
Inittal Festure Masalitntoo
Initial Festere Visualaation
Lock
sgiie slwntas Eanggide seb ealar
Final Fracurn Vinuskraation
Final Foaturn Vinuslzation
Fiall Feature Vinuairzttio
ki_bayer22_ conor222..2222....m Kendall-r 48.775,
lb3_layer12, con.2:15. Kend.-t
ba__aayrr22 conv2:49,
Ieilial
saddy
Noneycans
Visuskzatien
Farmucka
Figure 5: Illustration of the manipulability of both natural and synthetic feature visualization using ProxPulse on Block_3 3 L__aye____ _2__v222 of DenseNet201. The manipulated model has an accuracy of 76.52% (vs 76.9% for the initial model): the drop in accuracy is less than 0.4%. The first row (resp. second row) shows the natural initial (resp. final) feature visualization and initial (resp. final) synthetic fcature visualizations. On the image title, we report the corresponding metries to evaluate change in top activating inputs. One can observe that both natural and synthetic feature visualization have completely changed, to very similar images for the synthetic one. Target images are shown in Fig. 12
layord 2 conv2:15, Kondall-r: 0.103, CHTP-t: 0.293 tnitial Peuture Vireattrooon
lnyard 2. conv2:16, Keedall-r; -0,294, CI CLIP--s 0.336
laywrd 2 con92:01, Kendull-r: -0,136, CLIP-t 0.333 Inilial Feature Staneileation
Inilial Feature Stanaligggps
Matioet Mobiaal
Final Penture Varuakration
Pinal Deature Vinuakzation
Pinal Peuiure Viezaiization
esabetic
laywe4 2 conv2:1, Kandall-r: -0,88, CLIP-5: 0.342
Layor 2 com22:16, Keudall- -0.2944 CLIP-I 0.336
layord 2 conv2:35, Kendall- -0.140, CLIP-5 0.440
Vissaliggtinn
ealp moreeee
Pisl Frataro Viunslaation
Postuen Visuslaation
Piusl Rooturo Viosalization
Figure 6: Illustaation of the manipulability of both natural and synthetic feature visualization using ProxPulse on Layer 4 2 conv2 of ResNet-152. The manipulated model has an accuracy of 82.27% (vs 82.284% for the initial model): the drop in accuracy is less than 0.1%. The first row (resp. second row) shows the natural initial (resp. final) feature visualization and initial (resp. final) synthetic feature visualizations. On the image tätle, we report the corresponding metrics to evaluate change in top activating inputs. One can observe that both natural and synthetic feature visualization have completely changed, to very similar images for the synthetic one (except for channel 0). Target images are shown in Fig. 12
visual circuit. Related work OOla et al. 2017 Hamblin et al. 2022) uses these circuits to interpret circuit head functionality.
A closer look at Fig. 3 shows that although as intended synthetic feature visualization of conv5:37 has completely changed (colors and textures), most of the initial circuit channels are still present in the circuit derived from the manipulated model. Notably, at least one-half of channels per layer (before conv5) from the initial circuit are still present. in the final circuit while having, for most of them, similar initial feature
9

---

Published in Transactions on Machine Learning Research (02/2026)
COAN
CONPANTT
(a) With initial model.
(b) After Breakit
Figure 7: Effectiveness of CircuitBreaker to manipulate visual circuits on conv5 of AlexNet. We observe that the circuit visualization is severely distorted while the network outputs change minimally.
visualization (see c:nv1:37, conv1:20, conv2:107, conv2:12, etc.). However, we also observe that, for some of the channels such as conv3:151 which are still present in the final circuit, their final synthetic visualization looks very similar to the changed synthotic visualization of the circuit head, despite not having the strongest connection to the circuit's s head. This suggests that ProxPulse may have little impact on the circuit discovery method.
To go deeper into the effect of ProxPulse on the visual circuit, we reduce the sparsity from 1 to 0.3 and rebuild in Fig. 3 the right-side visual circuit with their feature visualizations on the circuits. We observe that the effect of ProxPulse has now almost completely been removed, confirming that despite the ability of ProxPulse to deceive both types of feature visualization, it adds only a minor modification to the network. Importantly, this minor modification can be visually detected when visualizing the circuit with low and moderate sparsity. We did a similar experiment for circuits of conv4 (see App. A.7 and Fig. 21
To provide more quantitative analysis of the ineffectiveness of ProxPulse to deceive visual circuits, we compute the Kendall-r rank correlation between (i) kernel attribution scores on the initial model and (ii) kernel attribution scores on the final (fine-tuned) model with ProxPulse on conv5. We do this on 10 randomly chosen channels of conv5, thus on 10 random circuits. We plot the mean with error bars on App.Fig. 22 and we can observe that the final ranks are strongly correlated with the original ones. This further illustrates the little impact of ProxPulse on the circuit discovery method. We also observe this little impact on circuit discovery on other baseline manipulation techniques as seen in Fig. [9 These results suggest that circuits may be robust to manipulation. We thus now consider the first manipulation attack targeted explicitly circuits.
5.3 Manipulation of the Circuit through Circuit
In this section, we manipulate the model with CircuitBreaker as introduced in Section 4.2 As refresher, the goal of the Circuit attack mechanism is to fine-tune the pre-trained model to maintain its initial performance, fooling the interpretations of visual circuits (initial rankings of top channels and their synthetic feature visualization), while also preserving the functionality of the circuit head. In the following, we present the results obtained with CrecuitBreaker on visual circuits for conv3 (features:6), cony4 (features:8), and conv5 (fcaturcs:10) of AlexNet. Note that circuits (10 heads on cach layer) are attacked independently from each other (ablation for simultaneous manipulation can be found in App. A.12) and the visualized circuits use sparsity of 0.6, which preserves well the behavior of the circuit heads (see Fig. 9a). An ablation study on the visualizations with various sparsity levels in App. A.9 and on the model (ResNet-50) is also provided in App. A.10
Visual Inspection.
We start by visually inspecting the results after CirauitBreaker. Fig. Fig. and Fig. 24 (appendix) show the results obtained after fooling attempts using CireuitBreaker on three different circuits (three different experiments). On the three different circuits (Sec. presents how visual circuits are discovered), when we compare the final one against the initial one, we observe that the final synthetic feature
10

---

Published in Transactions on Machine Learning Research (02/2020)
COnV2.40
CORY3:05
convl:27
03CV2 184
201V3:263
CoOP
CONV2:0
CO073:143
convI:
ConVe:12
CORV3:332
convt L120
COINZ-100
CORV3:78
CORv 1i22
CONN 2/25
conV3:18
coan
COIN 2:66
COBV
CODV3:41
(a) With initial model.
(b) After CircuitBreaker
Figure Effectiveness of CircuitBreaker to manipulate visual circuits on conv4 of AlexNet.
1.5
1.-
1.42-
0.8
0:38
0.5
0.96-
cime dinit hese on fnstume 4
croul leacs on Ieaturnaa
draft heads convt
drait headson on consa
0.2
2.6 0.6
C0043
CHRAS
cono Layers
convt
cores
(a) Pearson correlation between ac- tivations on circuits (with pruning) for the (i) considered model and (fi) the initial model without pruming.
(b) Rank correlation between kernel (c) Similarity ratio with Circuit- attribution scores for circuits on (i) Breaker. the initial model and (fi) the fine-
tuned model with Creakit Figure 9: Results obtained when fooling circuits with heads on conv3, conv4 and conv5 of AlexNet.
visualization still stays visually similar to the initial one, although it is less pronounced the circuit for features. but in this case, it still shares the circular contour. This is due to the ProxPulse component in Creake (see Section 4.2). We also observe from the three circuits, little overlap between channels numbering in the initial one and the final one, which the effect of the ranking loss in CircuitBreeker.. We finally observe that in terms of the semantics of the composition of synthetic feature visualization on the circuits, both initial and final circuits seem plausible. In particular, let us zoom onto the less obvious one in Fig. 7 An analysis of the initial circuit may roughly indicate that this circuit detects patterns related to circular objects with (vertical) axis (see e.g., Fig. [ Tand annotations of this unit from Hernandez et al. (2022) On previous layers of the features:10; 10, we can see the presence of synthetic feature visualizations that visually seem to be dedicated to these circular contours (e.g., conv3:225, conv3:322) and others that are related to the (vertical) axis (conv3:112). As said above, the similarity between synthetic feature visualization of the final (i.e., after Circuit Breaker) circuit is less pronounced but it can still be observed the circular contour pattern. Indeed, this circular pattern has been amplified when looking at final synthetic feature visualizations.
Quantitative Assessment. The above analysis visually inspected the manipulability of circuits under CircuitBreaker. Here, quantify its success using four criteria. (CT).
CT1: Functional behavior. First, to mcasure prescrvation of the circuit head's functional behavior, inspired by Hamblin el al. (2022), we compute the Pearson correlation (on a large training subset) between (i) activations of training images on the circuit head and (ii) activations of the same images on the circuit Higher corrclations indicate greater preservation of the circuit head's functional behavior. Fig Da reports these correlation scores for several sparsity levels and different layers where circuit heads come from. It. can be observed from dotted lines (fooled circuits) that the functional behavior of fooled circuits is
head at sparsity
11

---

Published in Transactions on Machine Learning Research (02/2026)
preserved in the same way the unfooled ones (bold lines), especially for moderate to high levels of sparsity. CT2: Sanity check of accuracy. Second, as done in Section 4.1 we assess the performance maintenance, ensuring that the fine-tuned model represents an adversarial model manipulation of the initial model. We measure the performance of all finc-tuned models and report it in Fig. 32 (appendix). The figure illustrates that our fine-tuning with CircuitBreaker maintains the same level of predictive performance of AlexNet. accuracy on ImageNet, which 56.52%.
we measure the rank correlation between kernel attributions scores from (i) the initial model and (ii) the final model, which is the fine-tuned model with CircuitBreaker. As a result, lower rank correlation will indicate a small change in the circuit, because these ranks are those that are used for circuit discovery. Fig. 9b shows these rank correlations. It can be seen from this figure that the final ranks of kernel altribution scores are weakly correlated to initial ones, except. those of the circuit head's layers, which is reasonable.
CT3: Correlation attribution scores. Third, as done in Sec. 5.2
CT4: Similarity ratio between synthetic feature visualizations on the circuit. Finally, since with fine-tuning, channels can switch their feature visualizations (Lhus decreasing the rank correlation but not changing the interpretations of the circuit), we need method to measure the change in synthetic feature visualization. Inspired by the phenomenon called lhack-a-mote in Nanfack et al. (2024), we use a similarity ratio computed thanks to CLIP (Oikarinen & Weng, 2022) features. This similarity ratio is computed as follows. Given a final synthetic feature visualization from a layer, the numerator of the ratio is the maximum cosine similarity between this final synthetic image on the final model and any of the initial ones from the same layer on the initial model. The denominator is the cosine similarity between this final synthetic image on the final model and the synthetic image from the same channel but on the initial model. Intuitively, the ratio quantifies the change in synthetic visualization (initial vs. final) relative to the initial visualization (using final top channels). Fig. 9c displays this similarity ratio per layer across circuit heads. Most values fall below one, indicating changes in synthetic feature visualization. Notably, the similarity ratio (endpoint of each curve) for the circuit head collapses to one, suggesting minimal change in the circuit head's synthetic feature visualizations overall.
6 Conclusion and Limitations
This paper proposes a manipulation technique called ProxPulse that addresses the limitations of previous works by showing that both types of feature visualizations can be simultaneously manipulated. However, when analyzing ProxPulse within the framework of circuits- key components in mechanistic interpretability- discover that circuits show some robustness against ProxPulse manipulations. We therefore introduce CireuitBreaker, another attack that reveals the manipulability of circuits. We provide experimental evidence of the effectiveness of these attacks using a variety of correlation and similarity metrics. Our attack on circuits sheds light on the lack of uniqueness and stability of circuit-basod interpretations. We also observe a decrease in manipulability success when trying to allack several circuits simultaneously without degradation in accuracy. Finally, our findings suggest potential defense mechanisms. Preliminary experiments show that simple finc-tuning using only cross-entropy loss (without our manipulation objectives) recovers the original feature visualizations and circuits. This may indicate that our attacks perturb models into shallow local minima in parameter space, rather than fundamentally altering the loss landscape. This observation suggests that defense strategies based on regularization towards multiple benign local minima or ensemble verification across different training trajectories may detect or prevent such manipulations. Developing and evaluating such defense mechanisms remains important future work.
Limitations. Our work focuses on convolutional neural networks and vision tasks. This focus is deliberate, as feature visualization and visual circuit discovery through structured pruning were specifically developed for CNNs. However, we acknowledge that extensions to other architectures require careful adaptation. For Vision Transformers, our framework could potentially use attention-based attribution scores instcad of gradient-basod ones, though circuit discovery methods for ViTs remain an open research question. For language models, recent work on activation patching (Wang et al.) 2022) and automated circuit discovery (Conmy et al.) 2023) suggests our framework could be applied, though the discrete nature of text inputs would require modifications to our proximity ball formulation. We view establishing these vulnerabílities in the sell-studiod CNN domain as an essential first step, with extensions to other architectures as important future work.
12

---

Published in Transactions on Machine Learning Research (02/2026)
Broader Impact Statement
Our work aims to study the lack of stability and robustness of popular interpretability techniques. We consider the framework adversarial model manipulation wherein model interpretations can be intentionally manipulated in (un)targeted ways. Unfortunately, demonstrating this manipulability highlights the risk of individuals exploiting this knowledge to deploy models whose interpretations are obfuscated. This can have negalive impact in high-stakes applications where interpretations may be required to be reliable for model auditing, However, we believe that acknowledging and understanding these risks is a crucial first step in addressing the vulnerabilities of interpretability techniques.
Acknowledgments
This work is funded by OpenPhilanthropy [E.B., G.N.]] G.N. is supported by NSERC grant number 599788-2025. This work is also supported by resources from Compute Canada and Calcul Québec.
13

---

Published in Transactions on Machine Learning Research (02/2026)
References
Julius Adebayo, Justin Gilmer, Michael Muelly, Tan Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. Advances in neural information processing systems, 31, 2018.
Ulrich Aivodji, Hiromi Arai, Sébastien Gambs, and Satoshi Hara. Characterizing the risk of fairwashing. Advances in Newral Information Processing Systems, 34:14822-14834, 2021.
Christopher Anders, Plamen Pasliev, Ann-Kathrin Dombrowski, Claus-Robert Müller, and Pan Kessel. Pairwashing explanations with off-manifold detergent. In International Conference Machine Learning, pp. 113. PMLR, 2020.
Dilyara Bareeva, Marina M-C Höhne, Alexander Warnecke, Lukas Pirch, Klaus-Robert Müller, Konrad Rieck, and Kirill Bykov. Manipulating feature visualizations with gradient slingshots. ar preprinl arXiu:2401.06122, 2024.
David Bau, Jun- Yan Zhu, Hendrik Strobelt, Agata Lapedriza, Bolei Zhou, and Antonio Torralba. Under- standing the role of individual units in deep neural network. Proceedings of the National Academy of Sciences, 117(48):30071 30078, 2020.
Bilal Chughtai, Lawrence Chan, and Neel Nanda. A toy model of universality: Reverse engineering how networks learn group operations. In International Conference on Machine Learning, pp. 6243-6267. PMLR,
2023.
Arthur Commy, Augustine N Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso. Towards automated circuit discovery for mechanistic interpretability. arXiv preprint arXiu:2304,14997, 2023.
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 8493-8502, 2022.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hicrarchical image database. In 2009 IEEE Conference on Compuler Vision and Pallern Recognilion, pp. 248 255, 2009.
Ann-Kathrin Dombrowski, Maximillian Alber, Christopher Anders, Marcel Ackermann, Klaus-Robert Müller, and Pan Kessel. Explanations can be manipulated and geometry to blame. Advances in neural information processing systems, 32, 2019.
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness- Sharpness-aware aware minimization for efficiently improving gencralization. In International Conference on Learning Representations, 2020.
Robert Geirhos, Roland S Zimmermann, Blair Bilodeau, Wieland Brendel, and Been Kim. Don't trust your eyes: on the (un) reliability of feature visualizations. arXiv preprint arXiv:2306.04/7 2023.
Chris Hamblin, Talia Konkle, and George Alvarez. Pruning for interpretable, feature- circuits in cnns. arXiv preprint arXiv:2206.0100 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770 778, 2016.
Juyeon Heo, Sunghwan Joo, and Taesup Moon. Fooling neural network interpretations via adversarial model manipulation. Advances in Neural Information Processing Systems, 32, 2019.
Evan Hernandez, Sarah Schwettmann, David Bau, Toona Bagashvili, Antonio Torralba, and Jacob An- dreas. Natural language descriptions of deep visual features. In International Conference on Learning Representations, 2022.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. stat, 1050:9, 2015.
14

---

Published in Transactions on Machine Learning Research (02/2026)
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference computer vision and paltern recognilion, pp- 4700-4708, 2017.
David H Hubel and Torsten N Wiesel. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. The Journal of physiology. 160(1):106, 1962.
Alex Krizhevsky, Iya Sutskever, and Geoffrey Hinton. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25, 2012.
Namhoon Lee, Thalaiyasingam Ajanthan, and Philip Torr. Snip: Single-shot network pruning based on connection sensitivity. In International Conference on Learning Represenlations 2018.
Aravindh Mahendran and Andrea Vedaldi. Understanding deep image representations by inverting them. In Proceedings of the IEEE conference on computer vision and paltern recognilion, pp. 5188 5196, 2015.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Advances in Newral Information Processing Systems, 35:17359-17321 2022.
Geraldin Nanfack, Alexander Fulleringer, Jonathan Marty, Michael Eickenberg, and Eugene Belilovsky. Adversarial attacks on the interpretation of neuron activation maximization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 4315-4324 2024.
Tuomas Oikarinen and Teui-Wei Weng. Clip-dissect: Automatic description of neuron representations in deep vision networks. arXiv preprint arXic:2204. 10965, 2022.
Chris Olah, Alexander Mordvintser, and Ludwig Schubert. Feature visualization. Distill, 2017.
Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction to circuits. Distill, 5(3):c00024-000 2020.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, ZZminng Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
Jérôme Rony, Luiz Hafemann, Luiz S Oliveira, Ismail Ben Ayed, Robert Sabourin, and Eric Granger. Decoupling direction and norm for efficient gradient- based 12 adversarial attacks and defenses. In Proceedings of the IEFE/CVV Conference on Computer Vision and Pattern Recognition, pp- 4322-42 1322-4330, 4330, 2019.
Tim GJ Rudner and Helen Toner. Key concepts in ai safety: an overview. Comput. Secur. doi, 10:20190040, 2021.
Kevin Ro Wang, Alexandre Variengien, Arthur Commy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: a circuit for indirect object identification in gpt-2 small. In The Eleventh International Conference on Learning Representations, 2022.
Moritz Wäschle, Florian Thaler, Axel Berres, Florian Pölzlbauer, and Albert Albers. review on ai safety in
highly automated driving. Frontiers Artificial Intelligence, 5:952773, 2022.
Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson. Understanding neural networks through deep visualization. ariie preprint arXiv:1506.063 2015.
Bin Yu. Stability. Bernoulli, pp. 1484 1500, 2013.
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In Europeon conference computer vision, pp. 818-833. Springer, 2014.
Roland S Zimmermann, Judy Borowski, Robert Geirhos, Matthias Bethge, Thomas Wallis, and Wieland Brendel. How well do feature visualizations support causal understanding of enn activations? Advances in Neural Information Processing Systems, 14:11730-11444 2021.
15

---

Published in Transactions on Machine Learning Research (02/2026)
0.68
0.68
0.66
0.66
0.64
0.64
Accuracy
Accuracy
0.62
Kendalk-
0.62
Kendal-r
0.56
163
1e6
1c9
0/255
5/255
50/255
Values
(a) Sensisitivty of C.
(b) Sensitivity of p.
Figure 10: Hyperparameter sensitivity. Left: we vary (keeping & 5/255) and right: we vary p (keeping C le6), reporting validation accuracy and Kendall-r scores. We observe that the success of attack according to Kendall-7 and performance maintenance according to validation accuracy are almost robust to the change in hyperparameters of the ProxPulse loss.
Roland S Zimmermann, Thomas Klein, and Wieland Brendel. Scale alone does not improve mechanistic interpretability in vision models. In Thirlg-seventh Conference on Newrul Information Processing Systems, 2023.
A
Appendix
A.1
Threat Model
Our work studies the (lack of) robustness and stability of feature visualization and visual circuits under ad adversarial model perturbation. We analyze this in the context of threat model with two key parties: the attacker and the human interpreter.
The human interpreter (e.g., an auditor) has access only to the modified model deployed by the attacker (e.g,, a company submitting their model for audit). The human interpreter has full access to the weights of the public (attacked) model and data. They can use mechanistic interpretability techniques such as Teature visualization techniques to interrogate the functionality of a released attacked model.
The attacker (e.g., a company) is the owner and provider of the model and is assumed not publicly release the unmodified model. They have access to the original model and data. Their goal is to release version of initially trained model to the human interpreter that obfuscates the result of interpretabilityy techniques of the initial model while having the same performance as the unmodified model.
Our work investigates the effect of adversarial model manipulation (which involves fine-tuming a pre-trained model with now objectives) on the final feature visualizations and visual circuits in targeted and untargetod settings.
The threat model examined here is realistic and has applications in fairwashing the risk of providing fair yet manipulated interpretations from an unfair model Aïvodji et al. (2021): Anders al. (2020). This security model is consistent with white-box setting where the attacker has full access to the data and model, aligning with prior work on interpretablliy-relatee attacks Dombrowski et al. (219)) Nanfack et al. (2024).
A.2 Further Experimental Details
We were inspired by the experimental setups of Nanfack et al. (2024) and Hamblin et al. (0022), to choose models, and hyperparameters for visual circuit discovery. The choice of the model and most experimental
16

---

Published in Transactions on Machine Learning Research (02/2026)
HONDHIIO
unicef
unicef C
Figure 11: Target images (Dina) for ProxPulse, taken from the ImageNNet-21k dataset.
settings were made according to Nanfack al. 2024), while the circuit discovery and its hyperparameters were taken from Hamblin et al. (2022), using their source codc. The hyperparameters of our method, specifically the values of and C were inspired by the adversarial robustness literature (with 12 norm). In particular, set 0.02 x 5/255 inspired from the adversarial literature (Rony et al. 2019), C' = 1e6 which is s 1e3 times higher than empirically observed activations of initial synthetic images and set the hyperparameters a = 0.1 and 3 0.01 such that the fooling loss and the maintain loss have similar scales. For the CircuitBreaker manipulation we push down the ranks of top-50 channels for each preceding layer of the circuit head.
In Fig. 10 we illustrate the hyperparameter sensitivity of the ProxPulse attack. The results show that the attack's success is stable under local changes in these hyperparameters.
To run our experiments, we use computer equipped with a GPU NVIDIA GeForce RTX 3090. Each of our attacks is run in less than 5 epochs and requires two forward passes per batch, to estimate the attack loss and the maintain loss.
A.3 Derivation of the Loss Function
This section derives the expression of the ProxPulse loss. Drawing inspiration from Foret et al. 2020 we derive the expression of Eq. by first writing,
Lr (Diont: : 0)
Σ
max ||a n=
(5(2; 0).
(8)
SSp
j. EDonll
Second, given that,
arg max (a(x; 0)
arg max ly(x* E) ||l|ssp
|| ||=-="
||SP
arg max () + e V ||l|sSp
(9)
arg max e" (x" ||ell
V (xl
P I
Finally, plugging this approximation into Eq. 8 recovers Eq. 6
A.4 Visual Inspection of All Synthetic Feature Visualizations of Layer
Fig. 14 and Fig. 13 respectively show all synthetic feature visualizations generated on layer conv5 the initial model (i.e., before ProxPulse) and on the final model (i.e., after ProxPulse). We do the same for Fig. [6 and Fig. 15 on layer conv4. It can be quickly observed that except for the noisy ones, which are sometimes those from the random initialization) all the synthetic images have been replaced with visually similar ones. Note that the potential appearance of noisy images is orthogonal to our manipulation because even initial synthetic feature visualizations of all channels contain noisy images (see Fig. 14 and Fig, 16).
In Eq.
C enables the control of the magnitude of activations in the manipulated synthetic images.
17

---

Published in Transactions on Machine Learning Research (02/2026)
Figure 12: Target images (Drot) for ProxPulse on 1522 New York and Vienna images taken from Wikipedia and Cntraveller websites.
35
9
59
Figure 13: Synthetic images after ProxPulse on conv5 of AlexNet.
18

---

Published in Transactions on Machine Learning Research (02/2026)
Figure 14: Initial synthetic images of conv5 of AlexNet.
Figure 15: Synthetic images after ProxPulse on conv4 of AlexNet.
19

---

Published in Transactions on Machine Learning Research (02/2026)
Figure 16: Initial synthetic images of conv4 of AlexNet.
20

---

Published in Transactions on Machine Learning Research (02/2026)
layer3.0.comv2:15 Kendall-r: -0.355, CLIP-4: 0,087
layer2.0.comv2:20 Kendall-:: -0::::0022:00000 20,27, CLIP-ds 0.083
layer1.0.comv2:12 Kendall-: -0.155 CLIP-5: 0.091
Inttial Visualaatioon
tottial Feature Visulisation
Fosters Visualaation
Uge panateda Pabc
weae chace cciee
winch chade grite
Final Feature Vinusiiztiio
Final Feature Vinuaizzaiin
Final Feature Visualization
Figure 17: Illustration of the manipulability of both natural and synthetic feature visualization Layer1.0.com 0.con22 of ResNet-50.
Hist. of Pairwise CLIP Sim. between Synthetic Images for layer1.0.come
Initial Final
40
30-
20
10-
0.75
0.80
0.85
0.30
0.95
1.00
Figure 18: Histogram of pairwise cosine similarities between CLIP features of non-noisy synthetic feature visualization before (red) and after (blue) the ProxPulse manipulation. One can observe that with ProxPulse (blue), synthetic images are much more similar each other than initially.
A.5 Results for ProxPulse ResNet-50
We ablate the model for experiments done in Section 5.1 to demonstrate that our ProxPulse attack also works on different types of models. More specifically, we do the similar experiment on layer1.0.comy and report the result in Fig. 17 and Fig. I8 We observe that both natural and synthetic feature visualizations can be manipulated without accuracy degradation (see the last row of Tab. 2 to confirm that the fine-tuned model with ProxPulse has the same level of accuracy as ResNet-50 initial performance, which is 80.3%).
21

---

Published in Transactions on Machine Learning Research (02/2026)
COIVEST
Figure 19: Illustration of the non-offectiveness of the push-down attack of Nanfack, et al. (2024) to manipulate the circuit. We show two visual circuits drawn for circuit head conv5:37 on pre-trained AlexNet (left) and on the fine-tuned AlexNet with the push-down attack of Nanfack, et al. (2024) (right) on conv5. We observe that most of the channels (at least three per layer, see surrounded ones) on the circuit were removed by ProxPulse, even though only the channel conv5:37 has marginally and visually changed.
22

---

Published in Transactions on Machine Learning Research (02/2026)
A.6 Ablation for the Use of Single Target in ProxPulse Manipulation
This section motivales why we use two larget images in ProxPulse, and it also subsequently ablates one target image. Fig. 20 shows that some of the final synthetic images have not been substantially changed, motivating thorefore the use of two target images.
Figure 20: Final synthetic images with one target image.
23

---

Published in Transactions on Machine Learning Research (02/2020)
Teatures
(a) Initial circuit with sparsity: 1.
(b) Final circuit with sparsity:
INNIUINS 6-10
Tealioee 5--00
Teatuin 2:40
Teateren 0.33
Testones 615
Teabsoes
8:143
(c) Fianal circuit with sparsity: 0.3.
(d) Fianal circuit with sparsity: 0.1.
Figure 21: Illustration of the non-effectiveness of the ProxPulse fooling to manipulate the circuit. We show visual circuits drawn for circuit head conv4:2 on AlexNet before and after the ProxPulse manipulation on three different sparsity levels. It can be observed that although the synthetic feature visualization of the circuit head has completely changed, the circuit almost did not change since at least one-half of the channels per layer continue to stay the circuit after the ProxPulse manipulation. Another observation is that reducing the sparsity reduces the effect of the ProxPulse manipulation, confirming that ProxPulse adds minor modification to the network.
A.7 Further Experiments on the Non-Effectiveness of ProxPulse to Attack Visual Circuits
Fig. 23 and Fig. 21 further illustrate the non-effoctiveness ProxPulse to attack visual circuits. can be observed from F ig 21 (hat al least one-half of channel indexes continue to slay on the circuit after ProxPules We also observe from Fig. 23 that the rank correlation scores between kernel attribution scores for circuit. discovery are high, suggesting little impact of ProxPulse on the circuit discovery method.
24

---

Published in Transactions on Machine Learning Research (02/2026)
1.00
0.95
Kendallet rank correlation
0.90
0.85
0.80
0.75
0.70
conv5
Layers
Figure 22: Correlation of attribution scores between the initial and the final (fine-tuned with ProxPulse) model. We plot the average on randomly chosen (heads of) circuits from conv5. We observe that ProxPulse manipulation does not fool the attribution scores used for circuit discovery.
Rank initial and Final Attributions
0.5
0.8
Ieal.ire.3
featumed
festures
Layers
Figure 23: Correlation of attribution scores between the initial and the final (fine-tuned with ProxPulse) model. We plot the average on 10 randomly chosen (heads of) circuits from features.8 (conv4). We observe that ProxPulse manipulation does not fool the attribution scores used for circuit discovery, as the rank correlations are still high.
25

---

Published in Transactions on Machine Learning Research (02/2020)
Teatures.0:3r 0::37
Se
features 0022
[eatures.3:1e 3:14:1
features.0:51
features..o
features,Co
features.0:20
Netures.S:10
Teettrees.0rr 0:26
[eatures.3:4
Teatures.0r
(a) With initial model.
(b) After Creakir
Figure 24: Illustration of the effectiveness of SireuitBreake to manipulate visual circuits on features:8 (conv4) of AlexNet.
(a) Initial circuit.
(b) Final circuit.
Figure 25: Illustration of the effectiveness of Creakrt to manipulate the circuit on conv5 of AlexNet.
A.8
Further Visualizations of CircuitBreaker on AlexNet
Tig. 26 and Fig. 27 demonstrate the visual inspection of the effectiveness of CircuitBreaker to fool initial circuit. We observe that the non-negligible component of the feature head is preserved while most initially top attributed channels were removed after Breaer.
Fig. 25
26

---

Published in Transactions on Machine Learning Research (02/2026)
Tralsurs 0:83
Dxs orre BILLZ
Jallaes 3013
Eelinns 2001
(a) Initial circuit.
(b) Final circuit.
Figure 26: Illustration of the effectiveness of CircuitBreakee to manipulate the circuit.
features.0.2
features.53 5:158
features. 0:14
features, 3:9
Tentures.0:52
features 3:15
lealures.Ors3
features. 3:166
features.8.33
features.0-1
Seatures.S:114
features 00:22
3atures.3:163
features.0.50
features.3:100
(a) Initial circuit.
(b) Final circuit.
Figure 27: Illsstaatonn of the offectiveness of CircuitBrecker to manipulate the circuit.
27

---

Published in Transactions on Machine Learning Research (02/2026)
(a) Initial circuit.
(b) Final circuit.
Figure 28: Illustration of the effectiveness of Circuit Breaker in manipulating the circuit: ablation on the sparsity level.
#10g
(a) Initial circuit.
(b) Final circuit.
Figure 29: Illustaaioon of the effectiveness of Creker sparsity level.
Breaker in manipulating the circuit: ablation the
A.9 Ablation for Sparsity for CircuitBreaker
Fig. and Fig. 30 show different circuits with different sparsity levols. can be observed that changing the sparsity level does not. affect the conclusion made in Sec. 5.3
A.10 Results for CircuitBreaker ResNet-50
Fig. 33 shows ablation results on visual circuits on the ResNet-50 model, with a circuit head on layer 1.0.ov2... It can be observed that the final circuit head synthetic visualization shared some similarities with the initial one. However, preceding channels largely different after CireuitBreaker than before.
28

---

Published in Transactions on Machine Learning Research (02/2026)
fe
festures.S:15e
features. 0:14
features. 3399
foatures.0:52
features.co
features.
Teatures.0r4
Se811
features.0022 0:22
Teatures.3:13
(a) Initial circuit.
(b) Final circuit.
Figure 30: Illustration of the effectiveness of Circuit in manipulating the circuit: ablation the sparsily level.
1.06
1.04
Simlarfo 1850
1.00
0.98
0.96
heads convs dircutt heads on conve diruutt heads
0.94
0.92
corv2
conu2
cories
corud
conv5
Figure 31: Similarity ratio on synthetic feature visualization: ablation on the sparsity level.
Vid
6.405
6.300
6.3925
8,500
9035
Figure 32: Final accuracy after fine-tuning with CreauitBreaker on AlexNet. We can observe no practical drop in accuracy as the pre-trained AlexNet accuracy 56.52%.
29

---

Published in Transactions on Machine Learning Research (02/2026)
cOnvI 122
1yeer1..comvv::5
convI44 1:
eyyer....oovi.n
layori. 0.conV1:25
0.conv1:2
layer1, 2.conv2:
layer1 O.comv2:
comvi:11
1nyer1.0.conv1:10
conv1:32
layeri.0.comv1:
layer1.
layer1.
(a) Initial circuit.
(b) Final circuit.
Figure 33: Illustration of the effectiveness of Circuit in manipulating the circuit: ablation the sparsily level.
A.11
Additional Results on DenseNet-201 and ResNet-152 for ProxPulse Attack
We present additional results for the ProxPulse attack respectively on DenseNet-2011 in Fig. and on ResNett112 in Fig. 6 We can see that both types of feature visualizations (natural and synthetic images) are simultaneously manipulated, and these visualizations share some visual similarity with target images.
30

---

Published in Transactions on Machine Learning Research (02/2026)
A.12 Additional Results Simulatenously Fooling Several Circuits with Feature Heads on Features:8 (Conv4) of AlexNet
In this section, we simultaneously run the Creaker manipulation on the first 30 circuits with feature heads on features.8 (conv4) of AlexNet.
According to the criteria evaluated in Section 5.3 of our paper, we make the Tollowing observations that are similar to the results obtained in our paper. First, by looking Fig. 34a we also observe high functional preservation on moderate to higher sparsity.
Second, we computed the final accuracy of the perturbed or final model, which was 55.83% (a drop of less than .7% as the initial accuracy of AlexNet is 56.5%)), indicating that the final has a similar performance to the initial model.
Third, from Fig. 34b we observe that Kendall-r rank for layers before the feature heads are around .6, which indicates that our manipulation has indcod docreased the corrclation between attribution scores that are usod for circuit discovery. However, nole that compared to the results we obtained the paper (independent. manipulation), the manipulation was less effective.
Fourth, as scen in Fig. 34 we observe that the similarity ratio is usually less than 1. This indicates that the synthetic feature visualizations have changed in the manipulated circuits. Note that the similarity ratio which is equal to 1 on feature heads means that the syuthetic feature visualizations have almost not changed.
Finally, we depicted in Fig. 35 and Fig. 36 two circuits that were part of the simultancously manipulated circuits. We observe that while the first circuit in Fig. 35 has undertaken some changes (the most effective way is to compare layer by layer in particular features:3), observe that the second one in Fig. 36 has marginally changed.
Commidee betwwn drouth settratieze und copprsed seffestions Satamn Cao
Cormirles betworn Inttial and Attrbrateeaa
derut Srod Riror
0.2
1.3
Inkmnk
Layes
(a) Pearson correlation between activa- (b) Hank correlation between (i) kernel (c) Similarity ratio with Circuit- tions circuits (with pruning) for the attribution scores for circuits on (i) the Breaker. (i) considered model and (ii) the initial initial model and (ii) on the fine-tuned model without structured pruning, i..,, model with CircuitBreaker. with sparsity 1.
Figure 34: Results obtained when simultancously fooling 30 circuits with heads on features.8 (conv4) of AlexNet.
31

---

Published in Transactions on Machine Learning Research (02/2026)
(a) With initial model.
(b) After CiecuitBreaker.
Breaker to manipulate visual circuits on features:8 (conv4) Figure 35: Illustration of the effectiveness of CireuitBreaker of AlexNet. We observe that the circuit visualization is severely distorted while the network outputs change minimally.
(a) With initial model.
(b) After CireuitBreaker.
Figure 36: Illustration of the effectiveness of CircuitBreaker to manipulate visual circuits on features:8 (conv4) of AlexNet. We observe that the circuit visualization is severely distorted while the network outputs change minimally.
0.14
0.12
drop pat dlass
0.10
0.00
0.06
0.02
0.00
1000
Classes
Figure 37: Accuracy Drop Per Class. We do not observe significant drop only in few classes.
32


Reviews:
[{"review_id": "R3gbv9sXE5", "reviewer": "Reviewer_rZ64", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies the problem of feature visualization of the mechanistic interpretation of deep neural networks. Particularly, the paper aims to design attack algorithms that keep the underlying network's performance unchanged while fooling the interpretation methods. The paper first proposes ProxPulse, which is not able to fool the interpretation method. The paper then proposes CircuitBreaker on top of ProxPulse, which shows the manipulability of visual circuits.", "claims_and_evidence_yn": "No", "claims_explanation": "The paper is overall clear, but I still have some concerns:\n\n1. The objectives for ProxPulse and CirbuitBreaker are confusing. Firstly, C in eq 5 is described as a very large constant, and it is not clear how such a constant is chosen. Secondly, eq 5 uses log(1 + ) function, but it is not clear why such a design is adopted. Thirdly, ProxPulse is described to increase activations in the proximity ball, but I don't fully see why, by doing so, we can fool the interpretation method. Finally, in Eq. 6, the first term is described to maintain the feature visualization, which contradicts the goal.\n\n2. The paper shows that ProxPulse fails to fool the interpretation methods while CircuitBreaker can. However, the paper does not provide much intuition for why this is the case or what it could indicate about the design's weaknesses or potential improvements of interpretation methods.\n\n3. The evaluation of the robustness is mostly qualitative, i.e., through visual inspection of the identified synthetic images.\n\n4. The paper has a limited focus on CNNs and does not seem to easily extend to ViT or other domains such as languages.", "audience_yn": "Yes", "audience_explanation": "Visual circuits and mechanistic interpretation are important directions to interpret and improve neural networks.", "requested_changes": "Please address the concerns above.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "2gc6nKJSul", "reviewer": "Reviewer_h95B", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper investigates the robustness of mechanistic interpretability techniques under adversarial model perturbations. The authors propose ProxPulse, a novel attack that simultaneously manipulates two types of feature visualizations. Furthermore, the authors introduce a new attack (CircuitBreaker) based on ProxPulse that reveals the manipulability of visual circuits, highlighting their lack of robustness. The effectiveness of these attacks is validated across a range of pre-trained models and evaluation metrics.", "claims_and_evidence_yn": "No", "claims_explanation": "Not entirely. The authors provide a large number of visualization results to support their claims. However, the overall readability of the paper is poor, which makes many of the arguments vague and difficult to follow. More details, please see \"Requested Changes\".", "audience_yn": "Yes", "audience_explanation": "Yes. The discovery of visual circuits is interesting and potentially valuable for future research on robust interpretability and adversarial defense mechanisms.", "requested_changes": "1. The readability of the abstract needs improvement.  \na) Before the sentence “This paper addresses limitations in existing works by proposing a novel attack called ProxPulse”, the manuscript does not clearly specify what the limitations actually are. Moreover, the concept of an attack has not been introduced beforehand, so its sudden appearance makes the paragraph abrupt and confusing.  \nb) The goal of the manuscript is not stated clearly; the phrase “highlighting their lack of robustness” further blurs the intended goal of the study.  \nc) While the experimental setup is important, the abstract should include a concise summary of the main experimental findings, rather than merely describing the setup.\n\n2. Similar issues exist in the Introduction. In the first two paragraphs, the central idea is that interpretability is important but may become unreliable under attacks. This easily gives the impression that the goal of this work is to defend against attacks to ensure reliability. However, the next paragraph abruptly shifts to proposing a stronger attack. The transition is therefore confusing. It is recommended to reorganize the introduction to focus more clearly on the limitations of existing attacks and to use those as the motivation for the proposed method.\n\n3. Figure 1 is not intuitive. The caption — “The first row (resp. second row) shows the natural initial (resp. final) feature visualization and initial (resp. final) synthetic feature visualizations” — is unclear and needs to be rewritten for better readability.\n\n4. Parts of words in Figures 14 and 32 are truncated.\n\n5. According to Table 2, the proposed method does not seem particularly strong in terms of classification performance — the robust accuracy is almost the same as the baseline.\n\n6. The main text lacks comparisons with other existing methods, especially those mentioned in the related work section.\n\n7. There are minor spelling and consistency issues, such as “vision circuits” should be “visual circuits”; and inconsistent forms like “after fine-tuning” and “after finetuning”.\n\n8. All ablation studies are placed in the appendix, which makes it difficult for the main text to independently support the core claims. The most important ablation results should be included in the main body of the manuscript.\n\n9. Similar to Comment 2, both the introduction and the conclusion emphasize the importance of defense, yet the manuscript provides no discussion or preliminary analysis on this aspect.", "broader_impact_concerns": "The authors have reported the Broader Impact in the manuscript.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "pkUpdJneyE", "reviewer": "Reviewer_tX37", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies the robustness of interpretability methods—specifically feature visualization and visual circuits—under adversarial model manipulation. The authors propose ProxPulse, which simultaneously manipulates both synthetic and natural feature visualizations, and CircuitBreaker, the first attack targeting entire visual circuits. Experiments on ImageNet models (AlexNet, ResNet, etc.) show that while circuits are initially robust to ProxPulse, they can be effectively manipulated by CircuitBreaker, raising concerns about the reliability of circuit-based interpretations.\n\n## Strengths\n* First work to simultaneously attack both synthetic and natural feature visualizations, and the first to target visual circuits.\n* Extensive experiments across multiple architectures (AlexNet, ResNet, DenseNet) and layers.\n\n## Weaknesses\n* Lack of code, experimental hyperparameter specifications, and reproducibility concerns. No error bars (standard errors reported) in the figures and tables.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Partly convincing. The claim about robustness (mentioned at the beginning of the paper) requires more evidence. And as mentioned above, the experiment part needs more comprehensive examinations.", "audience_yn": "Yes", "audience_explanation": "I believe this work is interesting to TMLR readers whose expertise is related to mechanistic interpretability in vision deep models.", "requested_changes": "* Presents more experiments related configs and the code.\n* Repeats with several random seeds.\n* Provides a more detailed analysis regarding the robustness, and discusses more on its underlying reasons.", "broader_impact_concerns": "/", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)