Energy Guided Diffusion for Generating Neurally Exciting Images
Authors: Paweł A. Pierzchlewicz
Konstantin Friedrich Willeke
Arne Nix
Pavithra Elumalai
Kelli Restivo
Tori Shinn
Cate Nealley
Gabrielle Rodriguez
Saumil Patel
Katrin Franke
Andreas S. Tolias
Fabian H. Sinz
Venue: NeurIPS 2023 poster (2023)
Keywords: most exciting inputs
diffusion models
energy guidance
attention
macaque V4
TL;DR: Energy guided diffusion provides a flexible framework to study coding properties of the visual system via most exciting images, most exciting natural images, reconstructions etc.

Abstract: In recent years, most exciting inputs (MEIs) synthesized from encoding models of neuronal activity have become an established method for studying tuning properties of biological and artificial visual systems.
    However, as we move up the visual hierarchy, the complexity of neuronal computations increases. 
    Consequently, it becomes more challenging to model neuronal activity, requiring more complex models.
    In this study, we introduce a novel readout architecture inspired by the mechanism of visual attention. This new architecture, which we call attention readout, together with a data-driven convolutional core outperforms previous task-driven models in predicting the activity  of neurons in macaque area V4.
    However, as our predictive network becomes deeper and more complex, synthesizing MEIs via straightforward gradient ascent (GA) can struggle to produce qualitatively good results and overfit to idiosyncrasies of a more complex model, potentially decreasing the MEI's model-to-brain transferability.
    To solve this problem, we propose a diffusion-based method for generating MEIs via Energy Guidance (EGG).
    We show that for models of macaque V4, EGG generates single neuron MEIs that generalize better across varying model architectures than the state-of-the-art GA, while at the same time reducing computational costs by a factor of 4.7x, facilitating experimentally challenging closed-loop experiments.
    Furthermore, EGG diffusion can be used to generate other neurally exciting images, like most exciting naturalistic images that are on par with a selection of highly activating natural images, or image reconstructions that generalize better across architectures.
    Finally, EGG is simple to implement, requires no retraining of the diffusion model, and can easily be generalized to provide other characterizations of the visual system, such as invariances.
    Thus, EGG provides a general and flexible framework to study the coding properties of the visual system in the context of natural images.

Energy Guided Diffusion for Generating Neurally Exciting Images
Pawel Pierzchlewicz"12, Konstantin Willeke Arne Nix Pavithra Elumalai', Kelli Restivo Tori Shinn Cate Nealley Gabrielle Rodriguez 34 Saumil Patel Katrin Franke" Andreas S. Tolias Fabian H. Sinz
Institute for Bioinformatics and Medical Informatics, Tübingen University, Tübingen, Germany Institute of Computer Science and Campus Institute Data Science, University Göttingen, Germany
*Department of Neuroscience, Baylor College of Medicine, Houston, TX, USA *Center for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, TX, USA DDepartment of Electrical and Computer Engineering, Rice University, Houston, TX, USA "ppierzc0cs uni uni-goettingen.c de
Abstract
In recent years, most exciting inputs (MEIs) synthesized from encoding models of neuronal activity have become an established method for studying tuning properties of biological and artificial visual systems. However, as move up the visual hierarchy, the complexity of neuronal computations increases. Consequently, it becomes more challenging to model neuronal activity, requiring more complex models. In this study, we introduce novel readout architecture inspired by the mechanism of visual attention. This new architecture, which call attention readout, together with a data-driven convolutional core outperforms previous task- driven models in predicting the activity of neurons in macaque area V4. However, as our predictive network becomes deeper and more complex, synthesizing MEIs via straightforward gradient ascent (GA) can struggle produce qualitatively good results and overfit to idiosyncrasies decreasing the MEI's model-to-bbain transferability. To solve this problem, we propose a diffusion-bart We show that for models of macaque V4, EGG generates single neuron MEIs that generalize better across varying model architectures than the ofate-of-the-aat GA. while the same time reducing computational costs by a factor of 4.7x, facilitating experimentally challenging closed-loop experiments. Furthermore, EGG diffusion can be used generate other neurally exciting images, like most exciting naturalistic images that are par with selection of highly activating natural images, or image reconstructions that generalize better across architectures. Finally, EGG is simple to implement, requires no retraining of the diffusion model, and can easily be generalized to provide other characterizations of the visual system, such as invariances. Thus, EGG provides general and flexible framework to study the coding properties of the visual system the context of natural images
more complex model, potentially
method for generating MEIs via Energy Guidance (EGG).
Introduction
From the early works of Hubel and Wiesel 11, visual neuroscience has used the preferred stimuli of visual neurons to gain insight into the information processing in the brain. In recent years, deep learning has made big strides in predicting neuronal responses 26 enabling in silico stimulus
'The code is available at http:p////gghttoooooo//////aeeee
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Most Excltigg Inputs
Most Exctting Naturalistic Inputs
Reconstructions
Diftusion Model
Energy
Function
Figure I: Schematic of the EGG diffusion method with pre-trained diffusion model. Examples of applications: Left: Most Exciting Inputs for different neurons, Middle: Most Exciting Naturalistic Inputs matched unit-wise to the MEls. Right: Reconstructions in comparison to the ground truth (top) and gradient descent optimized (bottom).
synthesis of non-parametric most exciting inputs (MEIs) [7H19]. MEIs are images that strongly drive selected neuron and can thus provide insights into its tuning properties. Up until now, they have been successfully used to find novel properties of neurons in various brain areas in mice and macaques (17
However, as we move up the visual hierarchy, such as monkey visual area V4 and ITT the increasing non-linearity of neuronal responses with respect to the visual stimulus makes it more challenging to obtain models with high predictive performance for single neurons, and optimize perceptually plausible MEIs, that is, those not corrupted by adversarial high-frequency noise for example. Par- ticularly, area V4 is known to be influenced by attention effects [25). and shifts in attention before the onset of saccades become more complex or units are taken from deeper layers of a network, existing MEI optimization methods based on gradient ascent (GA) can sometimes have difficulties producing qualitatively good results [28] and can overfit to the idiosyncrasies of more complex models, potentially decreasing the MEI's model-to-brain transferability. Typically, these challenges are addressed by biasing MEIs towards the statistic of natural images, for instance by gradient pre-conditioning [28] by including
change the location of its neurons' receptive fields (26 27. When models
a total variation loss to reduce frequfryy noise [29] or by image synthesis via GANs [19].
However, as discussed by Engstrom et al. 330 and Feather et al. [31 including additional priors into generation process can result obfuscated model biases.
Here, we make two contributions towards the above points: We introduce a new model architecture, called the attention readout, for predicting the activity of neurons in macaque area V4, which together with a data-driven convolutional core outperforms previous task-driven models [24] 32 To improve the quality of MEI synthesis we introduce novel method for optimizing MEIs via Energy Guided Diffusion (EGG). EGG diffusion guides a pre-trained diffusion model with a learned neuronal
encoding model to generate MEIs with bias towards natural image statistics. Our proposed EGG method is simple to implement and, in contrast to similar approaches 35], requires no retraining of the diffusion model (Fig. D We show that EGG diffusion not only yields MEIs that generalize better across architectures and are thus expected to drive real neurons equally well or better than GA- based MEIs but also provides a significant (4.7x) speed up over the standard GA method enhancing its utility for close-loop experiments such as inception loops [17
18 20 24 Since optimizing MEIs for thousands of neurons can take weeks (24), such speed-up directly decreases the energy footprint of this technique. Moreover, the rapid verification of synthesized images in vivo is particularly important for close-loop experiments given that maintaining the stability of single unit recordings challenging, and there's 's also the issue of representational drift (36), where tuning functions can change over time. We also demonstrate that EGG diffusion straightforwardly generalizes to provide other characterizations of the visual system that can be phrased as inverse problem, such image reconstructions based on neuronal responses. The flexibility and generality EGG thus make it a powerful tool for investigating the neural mechanisms underlying visual processing.
2 Attention readout for macaque area V4
Background Deep network-based encoding models have set new standards in predicting neuronal responses natural images (2/15). Virtually all architectures of these encoding models consist of at least two parts: core and readout. The core is usually implemented via convolutional

---

network that extracts non-lineat features ((x) from the visual input and is shared across all neurons to be predicted. It is usually trained through one of two paradigms: i) task-driven, where the core is pre-trained on different task like object recognition
37 39 and then only the readout is trained to predict the neurons' responses or ii) data-driven where the model trained end-to-end to predict the neurons' responses. The readout is a collection of predictors that map the core's features to responses of individual neurons. With a few exceptions [40], the readout components and its parameters are neuron-specifis and are therefore kept simple. Typically, the readout is implemented by a linear layer with a rectifying non-linearity. Different readouts differ by the constraints they put on the linear layer reduce the number of parameters 3 37 40 42]. One key assumption all current readout designs make is that the readout mechanism does not change with the stimulus. In particular, this means that the location of the receptive field is fixed. While this assumption is reasonable for early visual areas like V1, it is not necessarily true for higher or mid-level areas such as macaque V4, which are known to be affected by attention effects and can even shift the location of the receptive fields [26]. This motivated us to create a more flexible readout mechanism for V4.
33
4
State-of-the-art model: Robust ResNet core with Gaussian readout In this study, compare our data-driven model to a task-driven model (24). which is also composed of a core and readout. The core is pre-trained robust ResNet50 (I2, 0.1)
43 44). We use the layers up to layer
in the ResNet, which has 1,024 channels, thus providing a 1,024 dimension feature space. Then batch normalization is applied 45]. followed by ReLU non- lincarity. The Gaussian readout [40] learns the position of each neuron and extracts a feature vector at this position. During training, the positions are sampled from a 2D Gaussian distribution with means pn and En, during inference the Nn positions are used. Then the extracted features are used in a linear non-linear model to predict neuronal responses. We will refer to this model as the Gaussian model.
Proposed model: Data-driven core with attention readout The predictive model is trained from scratch to predict the neuronal responses in an end-to-end fashion. Following Lurz et al. 440 the architecture is comprised of two main components. First, the core, a four-layer CNN with 64 channels per layer with an architecture identical Lurz et 440. Secondly, the attention readout, which it is used in the popular transformer architecture 48 After adding a fixed positional embedding to ((x) and normalization through LayerNorm [49] to get (xx). (z).. key and value embeddings are extracted from the core representation. This is done by position- wise linear projections V E IRox and U E Rexde both of which have parameters shared across all neurons. Then, for each neuron a learned query vector qn E IRcc is compared with each position's 's key embedding using scaled dot-product attention [48].
builds upon the attention mechanism [46 47
3(xx) (x c,d, In Vik
O₂ softmax
c,
The result is a spatially normalized attention map a E R that indicates the most important feature locations for neuron given an input image. Using this attention map to compute weighted sum of the value embeddings gives us a single feature vector for each neuron. Finally, a neuron-specific affine projection with ELU non-linearity (50) gives rise to the predicted spike rate An (Fig. (2A). The model training is performed by minimizing the Poisson loss using the same setup as described in Willeke et al. (24]. We will refer to this model the Attention model.
Training data We use data from 1,244 Macaque V4 neurons from Willeke et al. 24 and briefly summarize their data acquisition in the supplementary materials section A.I
Results Our Attention model significantly outperforms the Gaussian model in predicting neuronal responses of macaque V4 cells on unseen natural and model-derived images. We evaluate the model performance by the correlation between the model's prediction and the averages of actual neuron responses across multiple presentations of set of test images, as described by Willeke et al. (24. We compared this predictive performance to the Gaussian model 44 on 1, 1,244 individual neurons (Fig. 2B). The Attention model significantly outperforms the Gaussian model by 12% (Wilcoxon signed-rank test, p-value = 6.79 10 10-82). In addition, we evaluated the new readout on how well predicts the real neuronal responses to 48 MEIs generated from the Gaussian model [see 24 and 7 control natural images. Our Attention model is better at predicting real neuronal responses,
3

---

A
No 1244 nourons
Dats drive Atterdion CWN
Nerren-peelffe Parametess
Postionall Attention
0.8-
Query Vecker
0.6
Feans Priyction
0.4- 1:488
Shered Puraneters
Key-liinasdinng Projecton
task-dtiven
Vilre-Enbodiing Projection
6.0
0.00
u.25
0.50 0.75 1.00
Task driven Reskict (o)
clesed-lopp predictive performance pearson correlation
1.0
Date drive Attortion CHN
6.8
6.661
0.4
0.2
data driven tenbechiven
0.0
0.00
0.25
Figure 2: a) Schematic of the Attention Readout, b) Correlation to average scores for 1,244 neurons. The Attention model (pink) shows significant (as per the Wilcoxon signed rank test, p-value 6.79 10-82) increase in the mean correlation to average in comparison to the Gaussian model (blue). Predictive performance comparison of the two models in closed-loop MEI evaluation setting. Showing that the data-driven with attention readout model better predicts the in-vivo responses of the MEIs.
even for MEIs of another architecture (Fig. 2C). Please note that Willeke et al. [24] experimentally verified MEIs in only a subset of neurons and only used the neurons with high functional consistency across different experimental sessions, For that reason, we too can only compare the performance of model-derived MEIs on this subset of neurons. We additionally show that the Attention model and Gaussian model show representational similarity (see Table (T) and that the Attention model uses its ability to shift its receptive field (Fig. ST
Ablation Study We perform an ablation study comparing the effects of the choice of core and readout on the performance in terms of test cor- relation (Table T]. We identify that the data- driven core + Attention readout model outper- forms all previous setups. Furthermore, the ab- lation study shows that the Attention readout generally improves performance across cores.
Readout \ Core
Task-Driven
Data-Driver
Factorized
0.153
Gaussian
0.262
0.229
Attention 0.276
0.294
Table I: Ablation study test correlation comparison for combinations of different cores and readouts.
Bold indicates the best best-performing model.
Energy guided diffusion (EGG)
3.1 Algorithm and methods
In this section, describe our approach to extract tuning properties of neuronal encoding models using a natural image prior as described by diffusion model. In brief, we use previously established links between diffusion and score-based models and the fact that many tuning properties can be described as inverse problems (most exciting image, image reconstruction from neuronal activity, etc.) to combine energy landscape defined by the neuronal encoding model with the energy landscape defined by the diffusion model and synthesize images via energy minimization. We show that this method leads better generalization of MEIs and image reconstructions across architectures, faster generation, and allows for generating natural-looking stimuli.
Background: diffusion models Recently, Denoising Diffusion Probabilistic Models (DDPMs) have proved to be successful at generating high-quality images [3 51) These models can be formalized variational autoencoder with a fixed encoder xp ++ Fr that turns a clean sample mo into noisy one ør by repeated addition of Gaussian noise, and learned decoder ær zo [3

---

which is often described as inverting diffusion process 5T] After training, the sampling process is initialized with a standard Normal sample 2r ~ NV I) which is iteratively "denoised" for T steps until Zo is reached. In the encoding, each step / corresponds to a particular noise level such that
at Vāra + vI dyE0
(2)
where 0t controls the signal strength at time and Eo NO 1) is independent Gaussian noise. In the decoding step, the diffusion model predicts the noise component co co t) at each step of the diffusion process (33. Then the sampling is performed according to
1 Zt
1 az
2t-1
Eg (x1, t) + 012
(3)
vat
V1 0t
where x= I).
Several previous works have established link between diffusion models and energy- based mod- els (57.5]. In particular, the diffusion model Eg (æz, t) can be interpreted as a score function, i.e. the gradient of log-density or energy w.r.t. the data Vz log p(x)) [6]. This link is particularly useful since combining two density models via product is equivalent to adding their score functions.
EnerGy Guided Diffusion (EGG) To optimize neurally exciting images, we require a method that can guide diffusion models via neural encoding models. The parameterization of diffusion models introduced by Ho et al. [33] only allows for the unconditioned generation of samples. Dhariwal and Nichol [53] introduced a method for sampling from conditional distribution pt (2 y), with diffusion models using a classifier (1 x) known as classifier guidance. However, this method requires i) the classifier to be trained on the noisy images, and ii) is limited to conditions for which classification makes sense. Essentially, this method relies on computing the score of the posterior distribution.
Vz, log, y) =., log,(a;)) + Va, log p(y ax.))
For classifier-guidance, the gradient of a model Vz, log logp(y xx) with respect to the noisy input 24 is combined with the diffusion model V v. log p(z+). resulting in samples zy conditioned on the class y. Note that this requires model Vz, log p(3) xx)) that has been trained on noisy samples of the diffusion before. Here we extend this approach to i) use neuronal encoding models, such as the ones described above, to guide the diffusion process and ii) use model trained on clean samples only. We achieve i) by defining conditioning as a sum of energies. Specifically, we redefine equation (4) in terms of the output of the diffusion model Eo (2t, t) and an arbitrary energy function E( t):
E(0,, t) = (z t) + de V., E(x,,))
(5)
where X₂ is the energy scale. This takes advantage of the fact that sampling in DDPMs is functionally equivalent to Langevin dynamics [5I]. Langevin dynamics generally define the movement of particles in energy field and in the special case when E(x) - log p(x), Langevin dynamics generates samples from p(r). For this study, we use a constant value of À and normalize the gradient of the energy function to magnitude of 1.
To achieve ii) we use an approximate clean sample To i.e. the original image, that can be estimated at each time step 1. This is achieved by a simple trick introduced in Li et al. [6]]. By inverting the forward diffusion process, with the assumption that the predicted eo (at, t) is the true noise:
To (ate t)
a - v a t)).
As result, the energy function receives inputs that are in the domain of xy at much earlier time steps f, and hence makes it feasible to use energy functions only defined on To and not 24, dropping the requirement to provide an energy E(x,, 1) that can take noisy images. Thus, the new score can be defined as
(7) This is particularly relevant in the domain of neural system identification, as encoding models are trained on neuronal responses to natural "clean" images 1 21 24 40) To get an energy that can understand noisy images would require showing the noisy images to the animals in experiments, which would make the use of this method prohibitively more difficult. Therefore, a guidance method that does not require training an additional model on noisy images allows researchers to apply EGG diffusion directly to existing models trained on neuronal responses and extract tuning properties from them.
E(a,, () = (x.,, () + >r V., E( 1))
5

---

Related work Many other methods have been proposed to condition the samples of diffusion processes on additional information. Ho and Salimans [55] provided a method that addressed the second requirement of classifeer-guidance by incorporating the condition y into the denoiser E 1, y). However, to introduce a conditioning domain y in this classifier-fre free guidance, the whole diffusion model needs to be retrained. Furthermore, this link between diffusion models and energy-based models allowed several previous works to compose diffusion models to generate outputs that contain multiple desired aspects of a generated image [57/59]. However, these studies focus solely on generalizing the classifier-free free guidance to allow guiding diffusion models with other diffusion models. Nichol and Dhariwal [52] have used similar gradient conditioning to guide the diffusion process using the gradient of the dot product of the CLIP image and text vectors. It has been shown that CLIP models that have not been trained on noisy images can be used for guiding Kadkhodaie and Simoncelli [64] introduced a stochastic coarse-to- fine gradient ascent procedure for generating samples from the implicit prior embedded within CNN. While we were working on this project, Feng et al. [65] published a preprint where they used the score-based definition of diffusion models to introduce
diffusion models 66
63
image-based prior for inverse problems where the posterior score function is available. This work is most closely related to our approach. However, they focus on how to obtain samples and likelihoods from the true posterior. For that reason, they need guiding models to be proper score functions. We do not need that constraint and focus on guiding inverse problems defined by a more general energy function and focus particularly on the application to neuronal encoding models.
Image preprocessing for neural models The neural models used in this study expect 100 x 100 images in grayscale. However, the output of the ImageNet pre-trained Ablated Diffusion Model (ADM) [53] is a 256 x 256 RGB image. We, therefore, use an additional compatibility step that performs i) downsampling from 256 x 256 - 100 x 100 with bilinear interpolation and ii) takes the mean across color channels providing the grayscale image. Each of these preprocessing steps is differentiable and is thus used end-to-end when generating the image.
3.2 Experiments
Most exciting images We apply EGG diffusson to characterize the properties of neurons in macaque area V4. For each of these experiments, we use the pre-trained ADM diffussin model trained on 256 x 256 ImageNet images from Dhariwal and Nichol [53] In each of experiments, we consider two paradigms: 1) within architecture, where we use two independently pre-trained ensembles containing 5 models of the same architecture (Gaussian model or Attention model). We generate images on one and evaluate them on the other. 2) cross architecture, two independently pre-trained ensembles containing models of different architectures (Gaussian model and Attention model). We demonstrate EGG on three tasks Most Exciting Input (MEI) generation, where the generation method needs to generate an image that maximally excites an individual neuron, naturalistic image generation, where a nnturra-looking image is generated that maximizes individual neuron responses, and reconstruction of the input image from predicted neuronal responses. Running the experiments required a total of 7 GPU days. All computations were performed on a single consumer- grade GPU: NVIDIA GeForce RTX 3090 or NVIDIA GeForce RTX 2080 Ti depending on the availability.
MEIs have served as a powerful tool for visualizing features of a network, providing insights and testable predictions 117-21 23 (6). For the generation of MEIs, we selected 90 units at random from a subset of all 1, 1.244 for which both the Gaussian model and the Attention model achieve at least a correlation of 0.5 the average responses across repeated presentations. We compare our method to a vanilla gradient ascent (GA) method (24) which optimizes the pixels of an input image æ to obtain the maximal response of the selected neuron. For the GA method, we use Gaussian blur preconditioning of the gradient. The stochastic gradient descent (SGD) optimizer was used with a learning rate of 10 and the image was optimized for 1,000 steps. We also evaluated other setups for the GA method without finding major differences (see Fig. S2). We define EGG diffusion with the energy function E (()))= = f where f. is the i-th neuron model and Ty is the estimated clean sample. We optimize MEIs for both the Gaussian model and the Attention model. We set the energy scale to À= = 10 for the Gaussian model and
for the Attention model. À was chosen via a grid search, for more details refer to Fig. 53. The diffusion process was run for 100 respaced time steps for the Gaussian model and 50 respaced time steps for the Attention model. For both EGG and GA, we set the norm of the 100 x 100 image to fixed value of 25. For each of the methods, we chose

---

EGG
Gaussien
GA
EG8
Artention
G4
Cross
Within
Cross
Within
50
40
r(x) 1.14x
rix) 1.04x
40-
r(x) 1.10x
rix) 1.24x
Gaussan
30
Atlenton
20
20
10
20
20
50
GA
GA
Figure 3: a) Examples MEIs optimized using EGG diffusion and GA for macaque V4 Gaussian and Attention models. b) Comparison of activations for different neurons between EGG diffusion and GA on the Within and Cross Architecture validation paradigms. Line fits obtained via Huber regression with 1.1. Curated image selection to show various properties of the neurons like fur, eyes, curves and edges.
the best of MEIs optimized from different seeds. We show the influence of the initial seed on the generated MEI in figure S3 Furthermore, the images that are generated by the ADM model are RGB. We show examples of the color outputs in figure S4
show some examples of MEIS generated with EGG diffusion and GA for the two architec- tures in figure AA. For more examples, refer to the supplementary materials figure S5 We find that the EGG-generated MEIs are significantly better (Attention) or similarly (Gaussian) activat- ing within architectures and are significantly better at generalizing across architectures (Fig. 3 This can also be observed by a significant increase in the mean activation across all units (Table Perceptually. EGG-generated MEIs of the Attention model looked more complex and natural than the GA- GA--gneeraed MEIs, and more similar to MEIs of the Gaussian model pre-trae mre-trained on natural image classification.
250
219s 0.09
Generation time
200
150
Comparing EGG-based MEIs to the ones found by Willeke et al. 24 using GA, we find that the preferred image feature is usually preserved, but MEIS generated for the Attention model are in most cases smaller in visual angle than their Gaussian model counter- parts (Fig. S5 To quantify that the MEIs from the Attention model are smaller we compute an isotropic Gaussian envelope for the MEIs. We find that the Attention model generates MEIs for which their Gaussian envelope on average is smaller than for the Gaussian MEIs (a 49.62 dGa 55.36, Wilcoxon signed rank test p valuue 0.0078).
100
50
465
0.07
EGG
GA
Figure 4: Mean comparison of the generation times between the EGG and GA (error bars denote standard error).
Finally, EGG diffusion is almost 4.7-fold faster than GA, requir- ing only on average 46s per MEI comparison to the required 2 299s for the GA method (Fig. 4 on single NVIDIA GeForce RTX 3090 across 10 repetitions. This

---

Paradigm
Within (Gaussian)
Cross (Gaussian)
Within (Attention)
Cross (Attention)
Gradient Ascent
11.43
5.51
7.59
4.42
EGG Diffusion
11.76
6.53
10.56
5,50
Table 2: Comparison of the average unit activations in response to MEIs in two paradigms 1) within architectures and 2) cross architectures, for two architectures Gaussian and Attention. Bold marks the method which has higher mean activation, and the i marks the increases which are statistically significant (Wilcoxon signed-rank test, respective p-values: 0.08, 2.87-10 2.84-10 4..9. 10 5). The architecture in the bracket indicates the generator architecture.
Gaussian
Attention
A-1
2
A m
A 10
B
Energy scale responses
2.25
normalzedd by GEI
Normalized Response
2.00
1.75
1.50
1.25
1.00
Gaussian Attention
0.75
0.50
10
Energy Scale FID Energy
47.5
42.5
37.5
35.0-
32.5
30.0
10
Energy Scale
Figure 5:
a) Examples of MENIs optimized using EGG diffusion in the macaque V4 for different neurons and different energy scales E {1,2,5, 10). b) Mean and standard error of the activations of neurons across different energy scales normalized by the activation of the GA MEI. FID between MEIs across energy scales and top-5 activating ImageNet images.
a substantial gain, as Willeke al. 24 required approximately 1. 1.25 GPU years to optimize the MEIs presented in their study. With EGG, only approximately 0.25 GPU years would be needed to produce the results of the study, while providing higher quality and higher resolution MEIs. Thus, EGG can provide major savings in time and energy, and improve the quality of MEIs.
Controlling the "naturalness" to generate most exciting natural images Unlike GA, EGG can also be used synthesize more natural-looking stimuli by controlling the energy scale hyperparameter X. Changing the value of À trades off the importance of the maximization property of the image and its "naturalness". To demonstrate this, we generated images for 150 neurons with the highest correlations to the average for the Gaussian model. We used energy scales À 21, 5, 10). fixed the 100 x 100 image norm to 50, and used 50 steps re-spaced from 1000. Each image was generated using 3 different seeds and the best-performing image the generator model was selected.
generated images across different energy scales in figure for both the Gaussian model and the Attention model. For more examples, refer to the supplementary material (Fig. [66] Fig. S7 We subsequently quantified the predicted responses across different values of We find that increasing increases the predicted responses (Fig. 5B), however, at higher À values
show examples

---

A
500
Validinnoo
2.004-
Dompai
D004-
0.4
Ceneratoor
6%
94%
75 (6m550)
100
Figure 6: a) Schematic of the reconstruction paradigm. The generated image is compared to the ground truth image via L2 distance the unit activations space, Reconstructions from 1. 1,244 units. b) Le distances in the unit activations space for the Within and Cross architecture domains comparing the EGG and GD generation methods. Shows that the EGG method generalizes better than GD across architectures. examples of reconstructions generated by EGG and GD in comparison to the ground truth (GT). d) Survey results on 45 voluntary human participants. Indicates that in 84% of images, the participants preferred the EGG generated reconstructions with rate > 0.6.
the responses begin to plateau, or even decrease. Therefore, for generating MEIs, we use = 10 for the Gaussian model and À 5 for the Attention model. It can be further observed that decreasing increases the naturalness of the generated image while preserving the features of the image that the neuron is tuned towards. To quantify the increase in the naturalness of the MEIs across Äs, measured the FID score between the generated images at different À values and the top-5 ImageNet images (Fig. 5C). Our results show that by changing À we approach the natural images manifold (lower FID). We also find that EGG generates MEIs (A = 1) similarly activating to the top-1 Imagenet images (Fig. (8
Image reconstruction from unit responses Another application of EGG diffusion is image reconstruction from neuronal responses. A similar task has been attempted with success using diffusion models from human fMRI data [34 35). Given that only small fraction of neurons were recorded, the image is encoded in an under-complete, significantly lower-dimensional space. Therefore, it is be expected that the reconstructed image will necessarily be equal to the ground truth image Ett- However, a better reconstruction x= is one that generalizes across models. This is trivially true for T = Zgz but, given the complexity of the model, there are likely other solutions. We therefore consider a masked version of the reconstructions for visualization. We mask the reconstructions to the joint receptive field of all 1,244 neurons. The mask is obtained by computing the average absolute gradients mask E.1 V (=))] across the responses the test images. The masks were normalized to be between 0 and and the values below 0.25 are clamped to O.
Therefore, regardless of the model f used, we should get []f((=") f(
We reconstruct images in the EGG framework by defining the energy function an La distance between the predicted responses to the ground truth image [(#g:) and the predicted responses to generated image (Fig. (6)) F(x) [ [axgx)]l2- Note that, instead of [(xrgt) ). could also use recorded neuronal responses. The images are generated from the Gaussian model with and 1000 timesteps. with the norm of the 100 100 image fixed 60. We compare EGG to a gradient descent (GD) method that simply minimizes the L2 distance. The GD uses an AdamW optimizer with learning rate of 0.05. In GD, at each optimization step the image æ, Gaussian blurred and the norm is set to 60 before passing it to the neural encoding model. We optimize the GD reconstruction up to the point where the train L2 distance is matched between the GD and the EGG for fair comparison of the generalization capabilities. We verified that the GD images do not improve qualitatively with more optimization steps (Fig. S9) We find that when generating the reconstruction using EGG diffusion we obtain 1) comparable within-architecture generalization and 2) much better cross-architecture generalization (Fig. 63). The EGG-generated images produce lower within architecture distances for 84% of the images and for 98% in the cross- arosttrrcite case.

---

We show examples of EGG diffusion and GD reconstructions in Fig. 6C. Qualitatively, the images optimized by EGG resemble the ground truth image much more faithfully than the GD images. More examples are available in the supplementary materials (Fig. (STO) and for reconstructions from the attention model see figure|ST]] We furthermore reconstruct images from real neurons by minimizing the distance between the model responses and recorded average neural responses to the image (Fig. S12
Human perceptual evaluation As shown in Cobos et al. 66 metrics like SSIM are not necessarily good predictor of how well neuronal responses are reproduced in vivo. Therefore, we conducted a voluntary anonymous survey with 45 voluntary participants on 50 test images (Fig. [S13). The participants were instructed to choose which image (GD optimized or EGG generated) was more similar to the ground truth image. Results show an 82.22% average preference for EGG-generatee images (95% confidence interval [80.59%, 83.75%]; Wilson score interval). In 84% of the images, the EGG method was preferred more than 60% of the time (Fig. 6D).
Limitations While EGG diffusion on average performs better than GA, it does come with limitations. Firstly. It is important to note that the results shown in this study have not been verified in vivo. Moreover, while the energy scale provides additional flexibility, it is important to keep it mind that it is an additional hyperparameter that needs to be selected obtain the desired re- sults, for which careful controls are necessary before the method can be used for in-vivo verification. For example, if the energy scale is too high, our method can be unreliable, which we iden- tified in 3 out of 90 cases where EGG diffusion failed to provide a satisfactory result with the Gaussian model (Fig. 77. Further- more. the parameter value also does not necessarily represent the same value across energy functions. Another limitation is that the maximal number of steps to generate the sample constricted by the pre-trained diffusion model, i.e. at most 1000 steps can be used. addition, the encoding model needs to generalize to the manifold of the diffusion model. Finally, since neurons are strongff driven by contrast, encoding models often tend to push generated images to very high contrast values. To avoid this effect and make the model focus on the image content, we evaluate the energy guiding encoding model at normalized image to eliminate the contrast direction from the guidance. This can be seen as an additional prior.
21
EG0
8
13.2
2.1
3.6
Figure 7: Examples of failure cases in comparison to the gra- dient ascent method. The text shows the predicted response rate by the within-architecturee valida-
Discussion
In this study, we introduced new model architecture, called the attention readout, for predicting the activity of neurons in macaque area V4, which together with a fully data-driven convolutional core outperforms previous task-driven models. Furthermore, we propose novel method for synthesizing images based on guiding diffusion models via energy functions (EGG). Our results indicate that EGG diffusion produces most exciting inputs (MEIs) which generalize better across architectures than the previous standard gradient ascent (GA) method. In addition, EGG diffusion significantly reduces compute time enabling larger-scale synthesis of visual stimuli. EGG diffusion is not limited the generation of MEIs and, within the same framework, allows, among other characterizations, to 1) generate most exciting naturalistic images which approach the manifold of most activating images in the ImageNet database, and 2) reconstruct images from unit responses, which generalize better across architectures and qualitatively resemble more the original image than images obtained via regular gradient descent optimization. While the dataset we use for this study was recorded from the macaque visual cortex, is in principle possible to use EGG for MEI generation and reconstructions with calcium imaging similar to the GA method on two-photon data in Walker et al. [17]. In fact, EGG can be applied to any modality that yields an encoding model and where a suitable diffusion model is available. More generally, EGG can be used whenever the "constraint" on particular image can be phrased in terms of an energy function. In summary, EGG diffusion provides a flexible and powerful framework for studying coding properties of the visual system.
10

---

Acknowledgments
The authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting Konstantin Willeke and Arne Nix. The authors also thank Mohammad Bashiri and Suhas Shirinvasan for their technical support and helpful discussions. The research was supported by the Cyber Valley Research Fund (AN, FHS). FHS is further supported by the German Federal Ministry of Education and Research (BMBF) via the Collaborative Research in Computational Neuroscience (CRCNS) (FKZ O1GQ2107), as well as the Collaborative Research Center (SFB 1233, Robust Vision). PP is supported by the German Federal Ministry for Economic Affairs and Climate Action (FKZ. AF4976666766.... We also acknowledge support from the National Institute of Mental Health and National Institute of Neurological Disorders And Stroke under Award Number U19MHI 14830 and National Eye Institute award numbers RO1 EY026927 and Core Grant for Vision Research T32-EY-002520-37 as well as the National Science Foundation Collaborative Research in Computational Neuroscience, USA with grant number [IS-2113173 Germany with FKZ: 01GQ2107. We thank our lab members, family, and friends for participating in the anonymous survey.
References
[1] H Hubel and T N Wiesel. Receptive fields of single neurones in the cat's striate cortex. Physiol., 147(5):574-591, October 1959.
[2] Charles F Cadieu, Ha Hong, Daniel L K. Yamins, Nicolas Pinto, Diego Ardila, Ethan A Solomon, Najib J Majaj, and James DiCarlo. Deep neural networks rival the representation of primate IT cortex for core visual object recognition. PLoS Comput. Biol., 10(12):e100363 December 2014.
[3] Daniel I. K Yamins, Ha Hong, Charles Cadieu, Ethan A Solomon, Darren Seibert, and James DiCarlo. Performance-optimizze hierarchical models predict neural responses in higher visual cortex. Proc. Natl. Acad. Sci. U. S. A., (11(23)8619-8622 June 2014.
[4] Santiago A Cadena, George H Denfield, Edgar Y Walker, Leon A Gatys, Andreas S Tolias, Matthias Bethge, and Alexander S Ecker. Deep convolutional models improve predictions of macaque V1 responses to natural images. PLoS Comput. Biol., 1006)9e April 2019.
[5] Fabian H Sinz, Alexander S Ecker, Paul G Fahey, Edgar Y Walker, Erick Cobos, Emmanouil Froudarakis, Dimitri Yatsenko, Xaq Pitkow, Jacob Reimer, and Andreas Tolias. Stimulus domain transfer in recurrent models for large scale cortical population prediction on video. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS 18, pages 1190,7210, Red Hook, NY, USA, December 2018. Curran Associates Inc.
[6] Yimeng Zhang, Tai Sing Lee, Ming Li, Fang Liu, and Shiming Tang. Convolutional neural network models of V1 responses to complex patterns. J. Comput. Neurosci., 46(1):33-54, February 2019.
[7] Lane T McIntosh, Niru Maheswaranathan, Aran Nayebi, Surya Ganguli, and Stephen A Baccus. Deep learning models of the retinal response to natural scenes. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS' 16, pages 1369- 1377, Red Hook, NY, USA, December 2016. Curran Associates Inc.
[8] David A Klindt, Alexander S Ecker, Thomas Euler, and Matthias Bethge. Neural system identification for large populations separating "what" and "where". November 2017.
[9] William F Kindel, Elijah D Christensen, and Joel Zylberberg. Using deep learning probe the neural code for images in primary visual cortex. J. Vis., 19(4:22,, April 2019.
[10] Alexander S Ecker, Fabian H Sinz, Emmanouil Froudarakis, Paul G Fahey, Santiago A Cadena, Edgar Y Walker, Erick Cobos, Jacob Reimer, Andreas $ Tolias, and Matthias Bethge. rotation-equivariant convolutional neural network model of primary visual cortex. September 2018.
[11] Benjamin Cowley and Jonathan W Pillow. High-contrast "gaudy" images improve the training of deep neural network models of visual cortex. June 2020.
11

---

[12] Max F Burg. Santiago A Cadena, George H Denfield, Edgar Y Walker, Andreas S Tolias, Matthias Bethge, and Alexander S Ecker. Learning divisive normalization in primary visual cortex. PLoS Comput. Biol., 17(6)=e1000028, June 2021.
[13] Eleanor Batty, Josh Merel, Nora Brackbill, Alexander Heitman, Alexander Sher. Alan Litke, E J Chichilnisky, and Liam Paninski. Multilayer recurrent network models of primate retinal ganglion cell responses. July 2022.
[14] Mohammad Bashiri, Edgar Walker, Konstantin- KonstantinKKlemens Lurz, Akshay Jagadish, Taliah Muham- mad, Zhiwei Ding, Zhuokun Ding, Andreas Tolias, and Fabian Sinz. A flow-based latent state generative model of neural population responses to natural images. Adv. Neural Inf. Process. December 2021.
Syst., 34:1 18011581118815
[15] Ján Antolik, Sonja B Hofer, James Bednar, and Thomas D Mrsic-Flogel. Model constrained by visual hierarchy improves prediction of neural responses to natural scenes. PLoS Comput. Biol. ((6))e100492)) June 2016.
[16] Eric Y Wang, Paul G Fahey, Kayla Ponder, Zhuokun Ding, Andersen Chang, Taliah Muhammad, Saumil Patel, Zhiwei Ding, Dat Tran, Jiakun Fu, et al. Towards foundation model of the mouse visual cortex. bioRxiv, 2023.
[17] Edgar Y Walker, Fabian H Sinz, Erick Cobos, Taliah Muhammad, Emmanouil Froudarakis, Paul G Fahey, Alexander S Ecker, Jacob Reimer, Xaq Pitkow, and Andreas S Tolias. Inception loops discover what excites neurons most using deep predictive models. Nat. Neurosci. 22(12): December 2019.
2060-2065,
[18] Pouya Bashivan, Kohitij Kar, and James J DiCarlo. Neural population control via deep image synthesis. Science, 364(6439), 2019.
[19] Carlos R Ponce, Will Xiao, Peter F Schade, Till S Hartmann, Gabriel Kreiman, and Margaret $ Livingstone. Evolving images for visual neurons using deep generative network reveals coding principles and neuronal preferences. Cell, 177(4):999-1009eel) 2019.
[20] Katrin Franke, Konstantin F Willeke, Kayla Ponder, Mario Galdamez, Na Zhou, Taliah Muham- mad, Saumil Patel, Emmanouil Froudarakis, Jacob Reimer, Fabian H Sinz, and Andreas Tolias. State-dependent pupil dilation rapidly shifts visual feature selectivity. Nature, 610(7930); 128-134, October 2022.
[21] Larissa Höfling, Klaudia P Szatko, Christian Behrens, Yongrong Qiu, David A Klindt, Zachary Jessen, Gregory W Schwartz, Matthias Bethge, Philipp Berens, Katrin Franke, Alexander Ecker, and Thomas Fuler. A chromatic feature detector in the retina signals visual context changes. December 2022.
[22] Jiakun Fu, Suhas Shrinivasan, Kayla Ponder, Taliah Muhammad, Zhuokun Ding. Eric Wang. Zhiwei Ding, Dat Tran, Paul G Fahey, Stelios Papadopoulos, Saumil Patel, Jacob Reimer, Alexander S Ecker, Xaq Pitkow, Ralf M Haefner, Fabian H Sinz, Katrin Franke, and Andreas Tolias. Pattern completion and disruption characterize contextual modulation in mouse visual cortex. March 2023.
[23] Zhiwei Ding, Dat T Tran, Kayla Ponder, Erick Cobos, Zhuokun Ding, Paul G Fahey, Eric Wang, Taliah Muhammad, Jiakun Fu, Santiago A Cadena, Stelios Papadopoulos, Saumil Patel, Katrin Franke, Jacob Reimer, Fabian H Sinz, Alexander S Ecker, Xaq Pitkow, and Andreas S Tolias. Bipartite invariance in mouse primary visual cortex. March 2023.
[24] Konstantin F Willeke, Kelli Restivo, Katrin Franke, Ame F Nix, Santiago A Cadena, Tori Shinn, Cate Nealley, Gabby Rodriguez, Saumil Patel, Alexander S Ecker, Fabian H Sinz, and Andreas S Tolias. Deep learning-drin driven characterization of single cell tuning in primate visual area V4 unveils topological organization. May 2023.
[25] Tirin Moore, Katherine Armstrong, and Mazyar Fallah. Visuomotor origins of covert spatial attention. Neuron, 40(4):671-66 1663, November 2003.
12

---

[26] A S Tolias, T Moore, S M Smimakiss E J Tehovnik, A G Siapas, and P H Schiller. Eye movements modulate visual receptive fields of V4 neurons. Neuron, 29(3)75777777 March 2001.
[27] Till S Hartmann, Mare Zirnsak, Michael Marquis, Fred H Hamker, and Tirin Moore. Two types of receptive field dynamics in area v4 at the time of eye movements? Frontiers in systems neuroscience, 11:13, 2017.
[28] Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, 2(11):
c7, November 2017.
[29] Pouya Bashivan, Kohitij Kar, and James J DiCarlo. Neural population control via deep image synthesis. Science, 364(6439), May 2019.
[30] Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Brandon Tran, and Aleksander Madry. Adversarial robustness prior for Iearned representations. June 2019.
[31] Jenelle Feather, Guillaume Leclerc, Aleksander Mądry, and Josh H McDermott. Model metamers illuminate divergences between biological and artificial neural networks. May 2022.
[32] Yifei Ren and Pouya Bashivan. How well do models of visual cortex generalize to of distribution samples? May 2023.
[33] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Adv. Neural Inf. Process. Syst., 33:6840 6850, 2020.
[34] Yu Takagi and Shinji Nishimoto. High-resolution image reconstruction with latent diffusion models from human brain activity. March 2023.
[35] Yizhuo Lu, Changde Du, Dianpeng Wang, and Huiguang He. MindDiffuser: Controlled image reconstruction from human brain activity with semantic and structural diffusion. March 2023.
[36] Laura N Driscoll, Lea Duncker, and Christopher D Harvey. Representational drift: Emerg- ing theories for continual learning and experimental future directions. Current Opinion in Neurobiology, 76: 162609 2022.
[37] U. Guclu and M. A. J. van Gerven. Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream. Journal of Neuroscience, 35(27):10005 10014, July 2015. doi: 10. 152///jneursccc55223-11444 14.2005. URL https::/tot....... oor.oi..orr//// 15233 j jneurosci. 5023-0221110 2015
[38] Daniel L K Yamins and James DiCarlo. Using goal-driven deep learning models to understand sensory cortex. Nat. Neurosci., 19(3):356-365, March 2016.
[39] Santiago A Cadena, Konstantin Willeke, Kelli Restivo, George Denfield, Fabian H Sinz, Matthias Bethge, Andreas S Tolias, and Alexander S Ecker. Diverse task-driven modeling of macaque V4 reveals functional specialization towards semantic tasks. May 2022.
[40] Konstantin- Konstantin-Klemmns Lurz, Mohammad Bashiri, Konstantin Willeke, Akshay K Jagadish, Eric Wang, Edgar Y Walker, Santiago A Cadena, Taliah Muhammad, Erick Cobos, Andreas S Tolias, Alexander S Ecker, and Fabian H Sinz. Generalization in data-driven models of primary visual cortex. April 2021.
[41] A Klindt, A Ecker, T Euler, and M Bethge. Neural system identification for large populations separating "what" and "where". In Advances in Neural Information Processing Systems, pages 4 6, 2017.
[42] Sinz, A S Ecker, P Fahey, E Walker, E Cobos, E Froudarakis. D Yatsenko, X Pitkow, Reimer, and A Tolias. Stimulus domain transfer in recurrent models for large scale cortical population prediction on video. In Advances in Neural Information Processing Systems 31. 2018.
[43] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. December 2015.
13

---

[44] Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adversarially robust ImageNet models transfer better? July 2020.
[45] Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. February 2015.
[46] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint orXiv:/409.0477. 2014.
[47] Alex Graves. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0m 0850, 2013.
[48] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
[49] Jingjing Xu, Xu Sun, Zhiyuan Zhang, Guangxiang Zhao, and Junyang Lin. Understanding and improving layer normalization. Advances in Neural Information Processing Systems, 32, 2019.
[50] Amé Clevert, Thomas Unterthiner, and Sepp Hochreiter. Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv:1511.07289 2015.
[51] Jascha Sohl-Dickstein, Eric A Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamies. March 2015.
[52] Alex Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. February 2021.
[53] Prafulla Dhariwal and Alex Nichol. Diffusion models beat GANs on image synthesis. May 2021.
[54] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-Resolution image synthesis with latent diffusion models. December 2021.
[55] Jonathan Ho and Tim Salimans. Classifier-Free diffusion guidance. July 2022.
[56] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Bureu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic Text-to-lmage diffusion models with deep language understanding. May 2022.
[57] Nan Liu, Shuang Li, Yilun Du, Joshua B Tenenbaum, and Antonio Torralba. Learning to compose visual relations. November 2021.
[58] Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional visual generation with composable diffusion models. June 2022.
[59] Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenbaum, Sander Dieleman, Rob Fer- gus, Jascha Sohl-Dickstein, Arnaud Doucet, and Will Grathwohl. Reduce, reuse, recycle: Compositional generation with Energy-Based diffusion models and MCMC. February 2023.
[60] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-Based generative modeling through stochastic differential equations. November 2020.
[61] Wei Li, Xue Xu, Xinyan Xiao, Jiachen Liu, Hu Yang, Guohao Li, Zhanpeng Wang. Zhifan Feng, Qiaoqiao She, Yajuan Lyu, and Hua Wu. UPainting: Unified Text-to-lmage diffusion generation with cross-modal guidance. October 2022.
[62] Katherine Crowson. v-diffusion, 2021.
URL https::/github..omm cotp:cosscccccook//
v-diffusion-pee ppytochh
[63] Katherine Crowson. Clip guided diffusion hq 256x256, 2021. URL. httpps//colll research. www.rec..gooo....oo wooggee.ommdree.. coee/driv////e /t/e//2aaaWWare
Wrf www..rr..........ww./w.......ee.
14

---

[64] Zahra Kadkhodaie and Eero P Simoncelli. Solving linear inverse problems using the prior implicit denoiser. July 2020.
[65] Berthy T Feng, Jamie Smith, Michael Rubinstein, Huiwen Chang, Katherine L Bouman, and William T Freeman. Score-Based diffusion models principled priors for inverse imaging. April 2023.
[66] Jiakun Fu, Konstantin Willeke, Pawel A Pierzchlewicz, Taliah Muhammad, George H Denfield, Fabian Hubert Sinz, and Andreas S Tolias. Heterogeneous orientation tuning across Sub- sub-Regions of receptive fields of V1 neurons in mice. February 2022.
[67] Erick Cobos, Taliah Muhammad, Paul G Fahey, Zhiwei Ding, Zhuokun Ding, Jacob Reimer, Fabian H Sinz, and Andreas S Tolias. It takes neurons to understand neurons: Digital twins of visual cortex synthesize neural metamers. December 2022.
[68] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large- scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255, June 2009.
15

---

A Supplementary Material
A.1 Training Data
Eleetrophysiological data were acquired as broadband signal (0.5Hz-16kH2), from a pair of male rhesus macaque monkeys (Macaca mulataa,, using 32 channel linear silicon probes (NeuroNexus 11s22-ddge-00mm-ee 177). The data was spike-sorted, and single units were isolated based on unit stability, refractory periods, and channel principal component pairs. Visual stimuli were presented to the animals on a 16:9 widescreen HD LCD monitor at 100čm viewing distance. The animals were rewarded with juice if they maintained their gaze around red fixation target throughout each trial. At the beginning of each recording session, the receptive fields (RFs) of the neurons were mapped in relation to a fixation target using sparse random dot stimuli, and the population RF was pulled towards the center of the screen by adjusting the fixation target. A collection of 24,075 images from ImageNet [68] was transformed into gray-scale and cropped the central 420² px and had 8 bit intensity resolution. These images were presented as visual stimuli during standalone generation recordings of 1244 units and during closed-loop recordings of 82 units. For details on the closed loop paradigm, please refer to Willeke et al. (24]
A.2
Supplementary Experiments
A.2.1 Attention Model
Attention readout uses its ability to shift receptive fields Receptive fields of neurons in area V4 can shift before the onset of saccades, believed to be associated with attentional shifts [26]. We investigate whether the Attention model actually uses its ability to shift the receptive field depending on image content. We inspect the attention mask of the attention model. We compute the center of mass of the upper 5% percentile of each attention mask. We then compute the average distance between the center of masses across different images for each neuron. We plot the average distance against the test correlation of each neuron observing that the attention readout does perform shifts (Fig. STA). We also show qualitative examples of the masks and the means in Fig. STT.
Centered Kernel Alignment We computed CKA of the neural encodings across architectures between the Attention model and Gaussian model and within architecture between different seeds (e.g. Attention and Attention are models with the same architecture, but trained with different seeds). The CKA is computed between the predicted neuronal responses. We observe that the within-architecture similarity is very high (> 0.99) for both architectures and the cross architecture similarity is slightly lower, but also high (> 0.9) (Table 2). We expect such an outcome, since both architectures were trained to model the same neural representation.
Model
Attention
Attention 2
Gaussian
Gaussian 2
Attention
0.9949
0.9133
0.9116
Attention 2
0.9949
0.9145
0.9129
Gaussian 1
0.9133
0.9145
0.9994
Gaussian 2
0.9116
0.9129
0.9994
Supplementary Table SI: Centered Kernel Alignment for the two architectures comparing across and
within architectures.
A.2.2 MEI generation
Comparison of experimental setups For the GA optimization, we use the established method for generating MEIs that has been tested in vivo Walker al. [17]. However, perform comparison study to show that the parameters chosen are selected to maximize the performance of the GA method. We rerun the MEI optimizations using the AdamW optimizer and find significant decrease in performance in comparison to the SGD optimizer (r = 0.69). We also run the MEIs for 100 steps instead of 1000 and also find performance decrease (r 0.95) (Fig. S2
Generating MEIs in color The diffusion model generates color images, so principle, it can generate color MEIs. We attach some examples (Fig. S4). Since the encoding models are trained
16

---

A
B
Example attention maps
Presented Image
Attention readout shifts
Neuron 0
Neuron 177
25
y -3.5x + 7.1
20
Mean Shift Distance
15
10
5
0
0.0
0.5 Test Correlation
1.0
Supplementary Figure S1: a) Comparison of the shift distance between the center of mass for the attention masks against the test correlation of the neurons. b) Example attention maps of neurons responding different neurons. The blue dot shows the center of mass. The examples show that the neuron shifts its center of mass.
on grayscale images because the animals only saw grayscale images the colors these may not be meaningful. However, if one were to use color stimuli it would be possible generate MEIs that are colored and potentially meaningful.
MENIs vs ImageNet search We compare the generated MEIs (A 1) to standard approach finding natural images for individual neurons. that end, we perform search across 100k images from the ImageNet dataset (68) to find the top-1 most activating image for particular unit. We then compare the predicted activations of the top-1 ImageNet image and the generated MEIs (A 1) in the arciterccsssss paradigm (Fig. S8 We find that the generated MENIs drive comparable activation to the top-1 ImageNet images. Like in the MEI generation paradigm, EGG thus significantly speed up the search for activating natural images, it does not need to search through millions of images.
17

---

MEI Optimizer Comparison B
Comparison of MEI optimization steps
A
50
50
r(x)=
40
n(x) 0.95x
40
AdamW
30
100 steps
30
20
20
10
10
0
0
0
10
20
30
40
50
0
10
20
30
40
50
SGD
1000
steps
Supplementary Figure S2: the GA method. a) Use of SGD vs AdamW optimizer. The SGD optimizer outperforms the Adam optimizer on within architecture evaluation. b) Increasing the number of steps slightly decreases within architecture performance.
Comparison of different experimental setups of MEI optimization using
18

---

Gaussian Seed 1
Attention
Seed
Seed 2
Supplementary Figure S3: Variability dependent on the seed used for generating MEIs in the Gaussian model and the Attention model. Each column represents different seed and each row different neuron. Results shown for the Gaussian and Attention models.
19

---

Neuron 230
Neuron
Neuron 901
Neuron 1118
Colored MEIs
Grayscale MEIs
Supplementary Figure S4: Examples of MEIs in their original color version (before converting to grayscale). Top row is the direct output RGB images from the diffusion model, the bottom shows the grayscaled version. Each column corresponds different neuron.
20

---

Gaussian
Attention
Supplementary Figure S5: Examples of MEIs generated using EGG for the Attention and Gaussian models.
21

---

==1 2=
A=3
10
Supplementary Figure S6: Examples of images generated using EGG diffusion in the Monkey V4 with different energy scales À E {1, 2, 5, 10). Generated for the Gaussian model. Units not matched with the images shown for the Attention model.
22

---

x=1 2=2
À=
10
Supplementary Figure S7: Examples of images generated using EGG diffusion in the Monkey V4 with different energy scales À E 2,, 5, 10). Generated for the Attention model. Units not matched with the images shown for the Gaussian model.
23

---

Gaussian Model top-1 Imagenet
Attention Model top-1 Imagenet
50
30
y(x) 1.13x
40
y(x) = 0.97x
20
EGG ====
30
EGG
20
10
0
10 20
30
40
50
0
10
20
30
ImageNet Top-1
ImageNet Top-1
Supplementary Figure S8: Comparison of the MEIs activations to the top-1 most activating ImageNet images neuron in the cross-architecture domain. Line fit obtained via Huber regression with = 1.1. In the left panel, three points (11, 65), (9, 70), and (16, 120) are not shown for visualization purposes.
EGG
400
800
1200
1600
2000
Supplementary Figure 89: Examples of reconstructions using GD) across various training lengths. Increasing the training does not bring the generated image closer visually to the GT nor EGG.
24

---

GT
EGG
GD
Supplementary Figure $10: Reconstruction examples from the Gaussian model. Generated using EGG diffusion and gradient descent.
25

---

Attention Model
Reconstruction Performance
0.04
EGG
GD
Distance
0.03
0.15
0.01
0.05
0.00
0.00
Attention Model Reconstructions
Within
Cross
GT
EGG
GD
Reconstructions from the Attention model. Top row in each panel is the ground truth image, middle is EGG generated reconstruction and last is the GD optimized reconstruction. Bar plot shows the performance of both EGG and GD the within and cross architecture paradigms in terms of Le distance.
Supplementary Figure S11:
26

---

Reconstructions Performance
0.10
0.7
EGG
0.6
Distance
0.06
0.3
0.2
0.02
0.1
Gaussin Model Reconstructions from Real Neurons
GT
EGG
GD
Supplementary Figure S12: Reconstructions from real neurons using the Gaussian model. Top row each panel is the ground truth image, middle is our EGG generated reconstruction and last row is the GD optimized reconstruction. Bar plot shows the performance of both EGG and GD in the within and cross architecture paradigms in terms of L2 distance.
27

---

O
O 8
Supplementary Figure S13: Setup for the human perceptual evaluation. The voluntary participants were presented with 50 images with the GT image always in the middle and the GD or EGG reconstructions were placed randomly each side of the GT image. The participants were provided with the question "Which of the images looks more like the image the center?". They were provided with context text: "In our study, methods to do so and we want to find out which one looks better to the human eye. Your participation entirely voluntary, and you have the right to withdraw at any time without providing reason. Please note that all responses will be kept confidential, and the data collected will be used solely for research purposes. Your identity will remain anonymous, and your personal information will not be disclosed to anyone."
reconstructing images from the brain activity. We have two
28


Reviews:
[{"review_id": "91CpbDRLfU", "reviewer": "Reviewer_YaDz", "summary": "This paper proposes using the prior implicit in a diffusion model as a regularization term when generating images that maximally excite neurons. The authors refer to this as “Energy Guided Diffusion” and compare this to a standard gradient assent procedure with a smoothness prior.  The authors also introduce a new model consisting of a CNN with an attentional readout that is trained to predict the responses of biological neurons, which (unlike the comparison model) allows for spatial components to be weighted differently for each input stimulus. The authors conduct experiments on all combinations of these two model changes to see how the generated stimuli change, and analyze how the stimuli transfer across models.  ", "questions": "Some of my major questions were listed in the weaknesses section, but additional things would improve the paper. \n\n1)\tIn Figure 5b, I’m a little confused about how the increase in the energy scale (so a higher importance on the “maximization” component of the synthesis) starts to decrease for large values of lambda. Naively, it seems like this component of the loss should only get better when it is weighted more strongly. Is there a problem with the optimization (i.e. too large of steps from the energy gradients?) causing this decrease?     \n\n2)\tThe authors should potentially discuss other ways that people have “regularized” generated images and the limitations. For instance, in Bashivan, Kar, DiCarlo (2019) a TV loss was incorporated into the gradient assent procedure to encourage smoothness (similar to the gaussian blur on the gradients used here). Some of these other methods have directly trained models that incorporate some statistics of the “natural” world, such as GANs (Ponce et al. 2019 and follow-up work). Limitations of including priors into synthesis techniques have been discussed in Engstrom et al. 2019 and Feather et al. 2022 (specifically, the inclusion of a prior for generation can hide some of the model biases). \n\n3)\tWith regard to the limitations of priors mentioned above, the authors mention that the MEIs “look more complex and natural” and that EGG improves the “quality” of the generated images. Can we gain understanding from having things that “look” better? Or are these potentially just misleading us to finding things that *seem* interpretable? (This is maybe a bit more of a philosophical question beyond the scope of the work, but it is something I am sometimes puzzled by with regard to these neural generation procedures, especially when neural data on the generated stimuli is not present).  \n\n4)\tWhat sort of biases do we expect from the fact that these models are trained to only handle clean images? The authors state as a fact that neural encoding models are trained on responses to “clean” images, but this seems like it would bias the generation in specific ways.  \n\n5)\tHow much spread is there in the generation across multiple seeds? Specifically, how does the spread in Figure 3 compare to what is observed from multiple initializations within the same model?   \n\nMinor clarifications:\n\na)\tWhat are the training details for the pre-trained robust ResNet50 that is used (ie $l_p$ norm, $\\epsilon$ step etc)? And are multiple pre-trained versions used in the ensemble or is it just a different readout? \n\nb)\tHow is the feature space for this model 1024 dimensional? Are the activations spatially averaged?\n\nc)\tThe paper “Solving linear inverse problems using the prior explicit in a denoiser” by Zadkhodaie and Simoncelli came to mind as very relevant previous work, but I didn’t see it cited. \n\nd)\tIt might be helpful to have an additional line/equation around line 166 explicitly stating the generation steps in terms of $\\bar{x}_0$. \n\ne)\tIn Figure 6 it states that the distance is compared in “unit activations space”. Is this the same as the predicted neural response, or is it measured at an internal model stage?  \n\nTypos/confusions:\n\ni)\tLine 160 says that a constant value of lambda is used for the study, however this was searched over (I think) based on Figure 5? \n\nii)\tShould equation (5) and associated text be $\\epsilon_\\theta$? \n", "limitations": "The authors have a section on limitations, however there are a few things along these lines that might be worth further addressing. \n* The authors do not validate the generated stimuli with neural recordings and the improvement in “quality” is only judged qualitatively or by looking at the predicted responses from other models. It is possible the generated stimuli with the diffusion generation or the attention model do not perform as well at controlling the neural data due to underlying biases in the sampling etc.  I understand that this would require additional neural experiments, however it is worth mentioning this as a limitation and I didn’t see it discussed. \n* Are there possible negative societal impacts of the work?  \n", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Overall, I enjoyed this paper and found it accessible, interesting and a good combination of many ideas that are currently relevant for the NeurIPS community. \n* The authors incorporation of an attention module into the final layer of a model for neural predictivity is novel as a way to capture stimulus-dependent changes \n* Showing the MEIs replicate for the chosen neurons and have very similar properties even with different models (ie  Figure 3) is a nice validation of the MEI technique as way of interpreting neural tuning\n* I was excited to see the prior in a diffusion as a way to improve some of the neural synthesis techniques, as this is perhaps a bit more principled than some previous methods (i.e. smoothness priors, or GANs).  \n", "weaknesses": "Overall, I think that the contribution of the paper is novel, however there are some weaknesses in the experimental design and the interpretation/presentation of the results. \n\nA) It is not possible to determine which change in the proposed model with an attention readout leads to better predictivity in Figure 2. This makes the rest of the results comparing the models difficult to interpret. Specifically, compared to the “Task-Driven” ResNet, the authors have changed (1) the core model architecture from a ResNet50 to a 4 layer CNN (2) the readout from the “Gaussian” readout to an attention readout and (3) the training task/dataset (the “task-driven” core of the ResNet50 is pre-trained on image recognition, while the entirety of the “data-driven” attention CNN is trained explicitly to predict the responses of V4 neurons). Which of these changes are critical for the performance increase? Are the changes due to the utilization of the attention readout, the fact that more parameters can be learned in the attention CNN, or some other change? \n\nB) Along the lines of the above, the authors talk about the attention change as being novel and providing a way to incorporate shifts in location of receptive fields, but there is no experiment showing that the learned attention readout *actually* learns to do this (it could be using the same location for every image). \n\nC) Perhaps my biggest concern – the authors discuss generating “most naturally exciting images” however the generated images are not natural. Calling these images “natural” is incredibly misleading, as there is an underlying diffusion model that has learned a specific distribution of image statistics. The generated images may be high-probability images given the prior learned by that diffusion model, but unless the diffusion model is a perfect model of the world these are not fully “natural” (and indeed, from the presented images they do not look natural). It is interesting to present images with different regularizations applied, however, in my opinion, these images should not be presented as “natural” and fully removing the paragraph on lines 262-268 (and also any associated references to “MENIs”) would only strengthen the paper.   \n\nD) In the discussion the authors note that the MEIs generated for the attention-readout model are more “concentrated” however, as far as I can tell, this seems to be just an observation and not quantified in any way.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank you for your in-depth review and we are glad you enjoyed the paper.\n\nPlease find the responses to your questions below:\n\n### RE A: Identifying what contributes to the improved performance\nWe performed an ablation study showing that the *Attention readout* is critical for improving performance. For more details please see the general response.\n### RE B: Experiment to show that the attention readout exhibits stimuli-driven receptive field shifts\nWe analyzed the attention maps (i.e. receptive fields) of the attention model. We show that across the test stimuli for the majority of units the receptive field shifts to different locations (Fig. E). See the general response for details.\n### RE C: Generated \"Natural\" Images\nWe realize that there might be different definitions of what constitutes a natural image. To avoid any confusion we will refrain from calling the images “natural”. We will move the top-1 Imagenet comparison to the appendix, and we will show that decreasing $\\lambda$ results in lower FID with the top-5 ImageNet dataset (Fig. D), indicating that the images become more similar to natural images and thus are close to the \"naturalness\" manifold.\n### RE D: Concentrated Attention MEIs\nTo show that the MEIs from the *Attention* model are more concentrated we compute an isotropic Gaussian envelope for the MEIs. We find that the *Attention* model generates MEIs for which their Gaussian envelope on average is smaller than for the *Gaussian* MEIs ($\\sigma_{At}$ = 49.62 vs $\\sigma_{Ga}$ = 55.36, Wilcoxon signed rank test p-value: 0.0078).\n### RE Q1: Increase in energy scale results in lower performance\nThank you for pointing this out. The reason is that under the formulation $\\varepsilon =  \\varepsilon(x_t, t) + \\lambda \\nabla_x (x_t)$ increasing the $\\lambda$ parameter is related to the step size between each diffusion step. Thus, for very large $\\lambda$ the step becomes too big and thus resulting in lower performance. We cannot put $\\lambda$ on $\\varepsilon(x_t, t)$ as this results in the noise level that is out-of-distribution for the diffusion model and thus the diffusion process would fail.\n### RE Q2: Other regularization techniques\nSee general comment: **Re 4: Additional Related Work**\n### RE Q3: Can we gain understanding from having things that “look” better?\nIf enhanced visual representations (MEIs) were solely about aesthetics, they might be deceptive. However, EGG-generated MEIs not only look better, but they also enhance predicted neural responses and exhibit better generalization across architectures. Importantly, their improved visual quality aligns with improved performance, making their enhanced appearance a valuable aspect of interpretability.\n### RE Q4: Biases on clean images\nWe consider the proposed method as a way to understand what image features neurons are encoding. Since the visual system has evolved for natural images many experiments use them as stimuli. Thus also the encoding models are trained on these “clean” images. Our method needs to be able to deal with this and generate images that drive the modeled neurons well. For that reason, we use the clean sample prediction trick to circumvent the requirement for an encoding model trained on noisy images. This could potentially introduce some bias, but it’s hard to quantify it, in particular since we don’t have neuronal responses to noisy images. \n### RE Q5: Spread across seeds\nSpread between seeds in Macaque V4 is expected [1] due to single-cell invariances. To show this we attach a figure of the MEIs from different seeds (Fig. A) (limited examples for the rebuttal due to space limitation, we will include more in the revised version of the manuscript).\n### RE Clr a) Pretraining details for ResNet50 and ensemble\nThe pretrained network is the L2, $\\epsilon = 0.1$ ResNet50 obtained from [2]. For the ensemble model, we use the same core but separately trained readouts (same architecture, different weights).\n### RE Clr b) 1024-dimensional feature space\nThe ResNet50 at layer 3 has 1024 channels, as the readout selects a single point in that layer and performs a dot product between the readout weight vector and the 1024 channels. Therefore, this results in a 1024-dimensional feature space.\n### RE Clr c) Missing Citation\nWe will include the citation for \"Solving linear inverse problems using the prior explicit in a denoiser\" \n### Re Clr d) Additional line for $\\hat{x}_0$\nWe will include the equation for computing the predicted $x_0$\n### Re Clr e) Units activation space\nYes, the unit activation space is the same as the predicted neural response. To avoid confusion between the predicted neural responses (model outputs) and real neural responses, we refer to predicted neural responses as unit responses.\n### RE Typos/Confusions\ni) constant value of lambda - For generating all of the MEIs we use a single value of lambda, meaning that we do not select the best lambda for each neuron. We choose the overall lambda based on the validation model as shown in Figure 5. The goal is to simulate a paradigm where we can identify select lambda within our model, but the end goal is to optimize the best MEI for the brain (not the model itself)\n\nii) $\\epsilon_\\theta$  typo - that is correct, thank you for spotting this typo, we will fix it in the revised manuscript.\n### RE: Limitations\n- Not tested in the brain - that is a good point, we will make sure to discuss it in the limitations section.\n- Negative societal impact - We have thought about the negative societal impact, however, this is a method for fundamental neuroscience research. All the scenarios we were able to come up with seemed far-fetched. If you have a particular scenario in mind, we are happy to discuss it. \n\n**References**\n\n[1] Willeke et al. “Deep learning-driven characterization of single cell tuning in primate visual area V4 unveils topological organization” (2023)\n\n[2] Salman et al. “Do Adversarially Robust ImageNet Models Transfer Better?” (2020)"}, {"review_id": "KNiOCt9fMA", "reviewer": "Reviewer_GN1s", "summary": "Further characterizing the complex coding properties of V4 neurons might require (1) better encoding models of neuronal activity as well as (2) better methods to generate informative most exciting inputs (MEI). The paper tackles (1) by proposing a new readout mechanism for a convolutional data-driven core based on attention that outperforms the SOTA at predicting neural responses of neurons in macaque V4. Then, it tackles (2) through a new simple method called EEG that generates MEI using a pre-trained diffusion model and the guidance signal of the encoding model. EEG can also be used for image reconstruction from neuronal activity, in both applications it generalizes better than the more traditional gradient ascent (GA) across architectures. Finally, their method can produce most exciting natural inputs (MENI) on par with highly activating natural images.", "questions": "- Did the author consider performing a CKA between models in the within condition and between models in the cross condition, to quantify the generalization/transferability of the MEIs?\n- Why is the ResNet50 and not the ACNN considered for comparing MENI to top1 most activating ImageNet images? At least this is what I am inferring from the color used in Fig 5.c, as it is not written in the text which model is finally used.\n- It would be interesting to add GA as a baseline for fig 5.b. I assume that the intuition behind the plateau/decrease is that if we increase the energy scale \"too much\", we fall back to \"overfitting to the idiosyncrasies\" of the encoding model, this baseline would give a bit of context w.r.t to this hypothesis.\n- How are the images shown in the paper selected?\n- My understanding for the motivation behind the MENI proposed is speed. Getting more natural-looking stimuli (e.g., for control stimuli) without having to go through millions of images. While I agree with the motivation, given the random nature of the generative process, I was wondering how much sampling was necessary before falling on satisfactory MENIs? Will I trust that EEG will still be faster, I worry about the meaningfulness of some of the MENIs generated (In Dhariwal et al. 2021, samples generated with a lower guidance scale are of significantly lower quality, e.g., $\\lambda=1$ leads to FID=33) \n- For image reconstruction, it would be interesting to see examples generated with the ACNN model, only results from ResNet50 are shown or discussed in the paper.\n\n\n\nClarification comments:\n- Slight lack of consistency w.r.t to naming: the encoding model is called through the paper: data-driven with attention readout, Attention CNN, ACNN, and Attention.\n- Within: l.201: \"task-driven ResNet with Gaussian readout or data-driven with attention readout\" & Cross: l.203: \"ResNet and data-driven with attention readout\". I assume in both case the ResNet uses a gaussian readout", "limitations": "- It is important to note that, while the trick used to make the classifier guidance works on models not trained on noise does allow for more flexibility, the encoding model still has to be trained on images from the same dataset as the diffusion model, as the approximate clean sample $x_0$ falls within the data distribution of the diffusion model and not necessarily of the encoding model. If not, the $x_0$ will be o.o.d to the encoding model and its gradient will be significantly less meaningful.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The problems tackled are well motivated\n- The paper is clear\n- The proposed readout mechanism is: novel to the best of my knowledge, biologically motivated, sound, and backed up by positive results (outperform current SOTA in predicting neuronal activity)\n- The method to generate MEI is: ingenious, simple but sound, and backed up by positive results (better generalization than GA and faster)", "weaknesses": "- (1) compares two models with: a different core, a different readout, and a different training strategy, hence it is hard to isolate the benefit from the new readout mechanism only. This part would be strengthened with additional baselines like the core model with Gaussian readout and the pre-trained resnet50 with attention readout\n- While the new model seems to be a better encoding model of neuronal activity --according to the results of (1)--, MEI generated with it seem overall worse at driving neurons activation (fig 5.b) compared to the baseline ResNet50 (both EGG and GA). In fact, the results on MENI reported in Fig5.c are with the ResNet50 model, and the examples of image reconstruction put forward in the paper all come from the ResNet50 model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank you for your in-depth review.\n\nPlease find our responses to your questions below.\n\n### RE Ablation study\nSee general response.\n\n### RE: Apparent decrease in driving neural response\nWe apologize for the confusion in Fig. 5b. The values there are shown as normalized to the max responses of each neuron across lambdas.  Thus, when normalizing, the values become smaller than 1. The result indicates that there is less consistency in the *Attention* model as to which value of lambda is most activating. We replotted the figure normalizing the neural activation of the GA-optimized MEI (Fig. G) and we will replace Fig. 5b with it.\n\n### RE: using the Attention Model for reconstructions\nWe attach *Attention* model reconstructions (Fig. H), showing that they perform similarly to the *Gaussian* model (Fig. J). We will add these figures to the appendix. We chose to reconstruct using the *Gaussian* model, such that the cross-architecture evaluation is performed on the *Attention* model. This is because the *Attention* model is better correlated with the brain responses and thus provides a better proxy for the brain. Thus, better mimic the setup of testing reconstructions in the brain.\n\n### RE: Using CKA\nCentered Kernel Alignment compares representational similarity, which would be applicable to the core representations. However, investigating the core representations was not the focus of our paper since we focused on finding the readout that would best predict the neural responses. As we show in the ablation experiment (see our general response), the attention readout shows performance superior to the Gaussian readout even when applied on top of exactly the same core. However, we might have misunderstood your suggestion and are happy to comment if you clarify it.\n \n### RE: Using *Attention* model for top-1 ImagNet comparison\nWe will include the *Attention* model comparison (Fig. I) to the top-1 ImageNet images in Fig. 5. The *Attention* MEIs at $\\lambda = 1$ slightly outperform the top-1 ImageNet images.\n\n### RE: add GA as a baseline for Fig 5.b\nPlease see **RE: Apparent decrease in driving neural response**\n\n### RE: How are the images shown in the paper selected?\nThe neurons used for the study are randomly chosen from the neurons with test correlation > 0.5. The images shown in the main text are selected to show the performance of various neuron properties (eyes, fur, edges, curves). The complete set of images is shown in the appendix.\n\n### RE: motivation behind the MENI and sampling required\nFor the MENIs presented we generate them from 3 seeds and choose the highest activating of the 3 seeds. Our main objective was to show that controlling $\\lambda$ allows us to move closer to the natural images manifold. We do not consider it a replacement tool for searching natural images, but rather an additional tool for better interpretability of MEIs as at times (e.g. top row ResNet fig.5A) it is difficult to interpret the MEI ($\\lambda=10$) and traversing $\\lambda$ can help to interpret the function. For FID we include the comparison of FIDs across $\\lambda$.\n\n### RE: Naming consistency\nThank you for pointing out the inconsistency. We will unify the naming to generally use *Attention model* for the data-driven with attention readout model. For the task-driven ResNet model + Gaussian readout we will refer to it generally as *Gaussian model*. We will also add a statement that defines these two terms to avoid confusion.\n\n### RE: l.201 and l.203\nAs in the previous section, we will unify the naming to remove confusion. Yes, the *task-driven* (*Gaussian*) model always uses the Gaussian readout (except for the new ablation study).\n\n### RE: Encoding model needs to be trained on the same distribution\nYou raise an interesting point. However, we do not entirely agree. Mainly, we consider the problem the other way around: The goal for encoding models is to faithfully represent the visual system as well as possible, no matter what kind of image is shown. The encoding model is mainly determined by the choice of images shown by the experimenters. The diffusion model, on the other hand, determines the space (prior) of images within which the most exciting images are generated. For example, it would be possible to use a diffusion model trained on sketches to generate the most exciting images in the space of sketches. Since the visual system has evolved to encode natural images, we focus on diffusion models for natural images here.\nWe will include this discussion in the Discussion section."}, {"review_id": "zz8mtgbET1", "reviewer": "Reviewer_H4Az", "summary": "The authors tackle the problem of synthesizing most exciting inputs for neurons in the higher visual cortex (V4 in their case) of macaque, with data collected using electrophysiology.\n\nThe paper makes two claimed contributions:\n\n1. The authors propose a new encoding architecture which uses a data-driven CNN core with a cross-attention read out layer. The cross-attention layer is parameterized similar to traditional cross-attention in other machine learning papers. In the author's design, there is a learned per-neuron query vector, and spatial key/value embeddings derived from pixel-wise linear projections of the CNN feature map that is shared for all neurons. The authors compare this against a task-optimized backbone with learned gaussian readout. The authors show that the the attention encoder performs better with high probability (via a Wilcoxon test) on novel non-training images. \n\n2. The authors propose energy guided diffusion. Where by they modify the score-prediction with the derivative of vector valued function (not a score function corresponding to a well posed distribution). They propose a modification which does not require the energy function to take as input noisy images, they accomplish this by using the \"pred_xstart\" code provided by [1].\n\nThe authors validate their method by first comparing their method against gradient ascent + gradient blurring in MEI synthesis. They find that EGG is able to generate MEI in a faster fashion and with better cross-architecture generalization properties than GA. \n\nIn the MENI experiment, they show that a tradeoff between naturalness and excitation by adjusting the strength of the gradient of the energy function. They find that their synthesized MENI ($\\lambda=1$) are roughly comparable to imagenet top-1 images in cross-arch predicted activations.\n\nIn the third experiment, they authors experiment with stimulus reconstruction. They accomplish this by modifying the energy function to minimize the L2 distance between predicted and ground truth neuron activities. They find that EGG regularized stimulus reconstruction is more faithful.\n\n[1] Dhariwal, Prafulla, and Alexander Nichol. \"Diffusion models beat gans on image synthesis.\" Advances in Neural Information Processing Systems 34 (2021).\n\nIn the supplementary, the authors provide additional descriptions of data collection (32 channel) and experiment design. The authors provide further experiments that compare ResNet & Attention encoders, the strength of $\\lambda$, and how pure gradient methods compare with EGG in stimulus reconstruction.", "questions": "1. Could you clarify why the MEI experiment uses SGD, but the image reconstruction experiment uses AdamW?\n2. For Figure 3, you make qualitative claims that the EGG MEIs are better, can you back this up with quantitive numbers like SSIM/Perceptual loss (VGG)/Inception/CLIP distance against the natural input that most excites the neuron?\n3. For Figure 5, can you characterize the distributional similarity of the images using standard image metrics like Fréchet inception distance or CLIP-FID proposed by Kynkäänniemi (MENI vs top-k of natural images for a neuron)? Something like the Figure 4 of Imagen [1] Figure 4's pareto curves which measure how the energy scale affects the image distribution distance (FID/CLIP). For Q3/Q4, if the image metrics are not possible, perhaps a human survey on Mechanical Turk/Prolific to evaluate if the images for the experiments are more similar/better.\n4. For Table 1, Figure 3B, Figure 5C, and Figure 6B, could you clarify what is the \"base\" model, as in which model you use for image synthesis, and which model is the evaluating model?\n5. Can you clarify the solver you use in the diffusion model? From the code, it seems like you use the DDPM solver, however there are a variety of stronger solvers (DDIM/PNDM/DPM-solver [2,3,4]) which yield convergence in as few as 10 steps. Is there any reason you decided to go with such an old solver?\n\nOverall I think the clarity of the paper is good, but could be further improved with a few small clarifications and incorporation of standard vision metrics (SSIM/VGG perceptual loss/FID/CLIP-FID).\n\n[1] Saharia, Chitwan, et al. \"Photorealistic text-to-image diffusion models with deep language understanding.\" Advances in Neural Information Processing Systems 35 (2022): 36479-36494.\n\n[2] Song, Jiaming, Chenlin Meng, and Stefano Ermon. \"Denoising diffusion implicit models.\" arXiv preprint arXiv:2010.02502 (2020).\n\n[3] Liu, Luping, et al. \"Pseudo numerical methods for diffusion models on manifolds.\" arXiv preprint arXiv:2202.09778 (2022).\n\n[4] Lu, Cheng, et al. \"Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps.\" arXiv preprint arXiv:2206.00927 (2022).", "limitations": "The authors clearly describe the limitations of their experiments.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper on balance is well written, the authors are largely clear in their experimental design and their evaluation. The authors provide sufficient detail in the paper itself for reproduction, with additional code provided in the supplemental. The code is well written and easy to follow.\n\nThe use of diffusion models to regularize the synthesis of most exciting inputs for monkey V4 collected using electrophysiology is novel, and to the author's knowledge it has not been attempted before. \n\nThe authors perform a variety of experiments, and I find their proposed design for the attention encoder and evaluation of the attention encoder to be convincing.", "weaknesses": "The authors did an good job writing a paper that has clarity and do a great job in providing details. But in my opinion, the authors overstate their contribution with regard to \"Energy Guidance\" (EGG). If the author can include additional citations, reduce their overly broad claims, and provide additional experiments/metrics, the paper would be improved significantly.\n\n\n**The paper could stand out based on the experiments alone, but the authors have emphasized energy guidance to be a central contribution** without citing the vast number of papers in computer vision that have been published in the past two years that:\n1. Similarly do not use a well-posed score function in the form of the derivative of a classifier. **This aspect is not novel.** In fact I would argue that most of gradient conditioning papers of diffusion models published today explicitly do not use a score function in the form of a classifier gradient. These papers are not cited.\n2. Estimate a clean sample ($x_0$) and do not feed in a noise corrupted image to the model providing guidance. The paragraph in lines 162-174 seem to indicate that this approach is novel in the context of classifier/gradient guidance for diffusion models. However **similar approaches have been widely used in computer vision literature**. These papers are not cited. \n\n* On contributions and prior work\n  * GLIDE from 2021 [1] proposed to perform image synthesis again using the gradient of the dot product of a CLIP image vector and a text vector to modify the diffusion output. Note that GLIDE used a noisy trained version of CLIP to perform guidance, however in this approach it seeks maximization of a dot product output which does not yield a \"proper\" distribution score function when you take the gradient.\n\n  * GLIDE and the DALL-E 2 [2] paper cite crowsonkb's 2021 open source CLIP guidance work [3, 4]. These two codebases combine CLIP guidance with the pred_x0 trick (eq 6 of this paper) without retraining the diffusion or gradient model. Similarly, the hugginface diffusers library minimize the orthodromic distance (derivative is not a proper score function) in CLIP space rather than maximizing a dot product using CLIP, and also use the pred_x0 trick without retraining the diffusion or gradient model. This estimated $x_0$ trick has also been formally described in [5] eq's 3 and 4. I suggest the authors cite at least one of these papers and clarify their contributions.\n\n* On the soundness of the experiments\n\n  * I also found some of the experimental setups to be inconsistent. \n  * in the MEI experiment line 221, the naïve SGD optimizer is used, and this forms the basis of the claim in Figure 4 to show GA is much slower than EGG. However in the image reconstruction experiment line 282, the more sophisticated AdamW optimizer is used. There is no reason why the AdamW optimizer cannot be used with gaussian blur gradient conditioning via filtering at an higher stage of the backpropagation. \n  * for the MEI experiment, GA is run for 1,000 steps, while EGG was run for 100 steps. This does not seem to be an entirely fair comparison time wise. \n  * I'm not sure why the authors decided to normalize the image itself to 25 (line 230) for MEI, 50 (line 251) for MENI, and 60 (line 281) for reconstruction. This step seems to quite explicitly break the energy guidance output. Can you provide a justification in the text for why this is done, and why you use different norms for different tasks? Can the authors perform MEI/SGD/AdamW experiments without this step?\n\n* On the lack of evaluation\n\n  * There is a lack of quantitive evaluation metrics aside from predicted neural activity. The paper almost entirely relies on qualitative claims when it comes to MEI/MENI/Reconstruction output. There are no actual metrics to indicate that the MEI/MENI/Reconstruction outputs are more similar to the ground truth most exciting natural stimuli. \n  * I suggest that the authors use common vision metrics like SSIM/PSNR/MSE to evaluate the low level similarity of the images, or high-level image metrics like perceptual loss (VGG), CLIP cosine distance, or distribution-wise comparisons like FID/CLIP-FID (see question 3 in section below). I don't think all the suggested metrics here are needed, but at least a few (any of SSIM/Perceptual VGG/CLIP for image reconstruction, and any of FID/CLIP-FID for MENI; or if the previous metrics are not possible, perhaps a human survey on Mechanical Turk/Prolific to evaluate if the images are better?) should be added where appropriate.\n\nOverall I think the authors have presented an interesting system, but there is no citation or acknowledgement of prior work from computer vision which use non-classifier based gradient guidance of diffusion models, or using estimated $x_0$ to alleviate the need for noisy trained classifiers. Otherwise I think the paper is interesting and would be improved if the authors can clarify the scope of their contributions and better quantify their claims. I would happily re-evaluate if the authors can improve this paper in a subsequent revision.\n\n[1] Nichol, Alex, et al. \"Glide: Towards photorealistic image generation and editing with text-guided diffusion models.\" arXiv preprint arXiv:2112.10741 (2021).\n\n[2] Ramesh, Aditya, et al. \"Hierarchical text-conditional image generation with clip latents.\" arXiv preprint arXiv:2204.06125 (2022).\n\n[3] https://github.com/afiaka87/clip-guided-diffusion/blob/c1d5906225586bc8455bb17c29a3c2caf9a02766/cgd/cgd.py#L141\n\n[4] https://colab.research.google.com/drive/12a_Wrfi2_gwwAuN3VvMTwVMz9TfqctNj#scrollTo=X5gODNAMEUCR&line=41&uniqifier=1\n\n[5] Li, Wei, et al. \"UPainting: Unified Text-to-Image Diffusion Generation with Cross-modal Guidance.\" arXiv preprint arXiv:2210.16031 (2022).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your helpful review. Please find our responses to your questions below. If you have any further questions we are happy to discuss.\n\n### RE: Prior work and scope\nWe will include and discuss the additional prior work, and make sure to make it even clearer that we do not claim to have invented gradient conditioning and the clean sample trick.\n\nGLIDE and DALLE-2 use a different trick, their model directly predicts x_0, we predict $\\epsilon$ and get an approximate x_0 from x_t and $\\epsilon$. Crowsonkb's work indeed uses an approach similar to ours for guiding diffusion models with an unnoised CLIP model. Thank you for bringing this open-source project to our attention. We will cite the suggested work and discuss their work.\n\n### RE: Experimental Setup - MEIs\nFor the GA optimization, we use the established method for generating MEIs that has been tested in vivo [1]. However, we perform a comparison study to show that the parameters chosen are selected to maximize the performance of the GA method. We rerun the MEI optimizations using the AdamW optimizer and find a significant decrease in performance in comparison to the SGD optimizer (r = 0.69). We also run the MEIs for 100 steps instead of 1000 and also find a performance decrease (r = 0.95) (Fig. F).\n\n### RE: Norm constraint\nControlling for the norm/contrast of the image is a standard procedure when optimizing MEIs [2] because neurons are strongly driven by contrast. When the contrast is not controlled a trivial solution is to simply increase the contrast of images to values that are not realizable by a monitor. Furthermore, controlling the norm also controls the locality of the synthesized images. When optimizing MEIs, we choose a lower norm because neurons have a localized receptive field, for MENIs we increase the norm to 50, to allow for more full-field images, for the reconstructions, we further increase the norm budget to operate in a similar contrast domain as the ground truth natural images.\n\n### RE: Evaluation\nThe goal of MEI generation is to elicit maximal responses in the brain. Most exciting natural images (selected from the ImageNet dataset) have been established to be less activating than the GA-optimized MEIs [1], which have also been tested in vivo. Therefore, a ground truth image to which MEIs could be compared via standard computer vision metrics does not exist. Hence, for MEIs, we rely on the predicted neural responses as the established metric of MEI performance. Since validating the images in the recorded neurons is not viable for us we introduced the cross-architecture evaluation paradigm as the *in silico* proxy evaluation of the stimuli transferability to the brain. We quantitatively show that the EGG MEIs are more robust to the model idiosyncrasies and we would thus expect them to perform better *in vivo*. Another reason why we use neural activation is that, for the reconstruction of images from neural responses, it has been previously shown that standard computer vision metrics do not necessarily appropriately capture the desired objective i.e. better correlation of the neural responses of the reconstructed image with the responses to the real image [3].\n\nWe agree, however, that additional metrics could be helpful for the MENIs and Reconstructions. Please see the general response for the additional evaluation results and discussion.\n\n### RE Q2: Are EGG MEIs actually better?\nFor MEIs the hallmark metric is their performance in terms of maximizing neural responses. Natural images, on the other hand, are not the ground truth for MEIs. As seen in [1, 4], existing MEIs outperform natural images in the brain. Therefore, comparing MEIs to natural images via SSIM/Perceptual Loss would not indicate improvement in maximizing in-vivo neural responses. Please let us know if we somehow misunderstood your point. \n\n### RE Q3: MENIs distributional similarities\nWe performed the requested evaluation of distributional similarities (see Fig. D and general response).\n\n### RE Q4: Base model clarification\nFirstly, we will improve the consistency of model naming: i.e. *Gaussian* is the ResNet + Gaussian Readout model, *Attention* is the CNN + Attention Readout model.\n\nTable 1: The models in the brackets are the base models, for *within* we use the same architecture, for *cross* we use the other architecture. We will clarify that in the caption of the table.\n\nFigure 3B: The label to the left of the plots is the base model, i.e. blue is the *Gaussian* model (ResNet + Gaussian readout), and pink is the *Attention* model (CNN + Attention readout). We will make that clearer in the caption of the figure.\n\nFigure 5C: We use the *Gaussian* model, we have also included the *Attention* model comparison in the rebuttal. We will make that clearer in the caption.\n\nFigure 6B: We use the *Gaussian* model, we have also included results from the *Attention* model in the rebuttal. We will make this clearer in the caption.\n\n### RE Q5: Solver\nWe chose to use DDPM over the other solvers for its simplicity and closest resemblance to Langevin Dynamics providing the most elegant framework for incorporating energy gradients. It is possible that even better results could be achieved with more complex solvers. However, we considered this out of scope for the current study.\n\n**References**\n\n[1] Walker et al “Inception loops discover what excites neurons most using deep predictive models” (2019)\n\n[2] Willeke et al. “Deep learning-driven characterization of single cell tuning in primate visual area V4 unveils topological organization” (2023)\n\n[3] Cobos et al. “It takes neurons to understand neurons: Digital twins of visual cortex synthesize neural metamers” (2022)\n\n[4] Bashivan et al. \"Neural population control via deepimage synthesis\" (2019)"}, {"review_id": "v1hFuEhBIH", "reviewer": "Reviewer_BnXF", "summary": "In this work, the authors first employed attention readout to train a model for predicting neural responses ($y$) from image ($x$), aiming to address the issue of attention effects in the V4 area. Subsequently, they applied the image ($x$) from text ($y$) method proposed by Prafulla [1] to generate images ($x$) from neural responses ($y$).\n\nReference:\n\n[1] Prafulla Dhariwal and Alex Nichol. Diffusion models beat GANs on image synthesis. May 2021.", "questions": "How many neurons are used in EGG? 1,244 individual neurons?", "limitations": "None", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Compared to previous research, this study incorporates two technologies to enhance the performance of their model in predicting neuronal responses and MEIs. The first technology involves integrating an attention map into their encoding network. From a biological standpoint, the attention method allows for the receptive field of neurons to be dynamically adjusted based on different inputs, which aligns with observations made in experiments on V4. Empirically, attention-based Vision Transformer (ViT) models have shown superior performance compared to ResNet models in the field of computer vision. In this study, the performance of the proposed model surpasses the current state-of-the-art in predicting neuronal responses.\n\nThe second technology employed in this research is the use of Bayesian diffusion methods to decode input from neural responses. The diffusion model is renowned for its ability to generate intricate image details, and the authors demonstrate that their approach, called EGG, outperforms the previous GA method.\n\nAlthough the authors did not develop these two methods themselves, it is possible that their application in neural data analysis is novel. To the best of my knowledge, this study appears to be the first instance of using the attention map in the encoding network for predicting neuronal responses, taking into consideration the observed phenomenon in V4. Furthermore, the utilization of the Bayesian diffusion method, specifically the EGG approach, for decoding neural responses seems to be a novel application in this context.", "weaknesses": "1.The current images are grayscale. Can the MEI images be colored?\n\n2. The current neural data are collected by silicon probes. Can we use techniques such as two-photon or single-photon imaging to record more neuronal data?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your review. We respond to your concerns below. In case you have any more questions we would be happy to discuss them.\n\n### RE 1: Color MEIs\nThe diffusion model generates color images, so in principle, it can generate color MEIs. We attach some examples (Fig. B). Since the encoding models are trained on grayscale images because the animals only saw grayscale images the colors in these may not be meaningful. However, if one were to use color stimuli it would be possible to generate MEIs that are colored and potentially meaningful.\n\n### RE 2: Applicability to other Neural Experimental Techniques\nSee general response **RE 3:  Applicability to other Neural Experimental Techniques**\n\n### RE Q1: How many neurons are used in EGG?\nYes, we use 1,244 individual neurons. For each MEI a single neuron is selected and for the reconstructions, we use all 1,244 neurons."}]

Author rebuttal:
We appreciate the thorough and constructive reviews and are glad to see that the reviewers found our work to be **ingenious** (**GN1s**), **novel** (**YaDz**, **GN1s**, **H4Az**, **BnXF**), **well-written** (**H4Az**) and **clear** (**GN1s**, **H4Az**). We are happy to see that Reviewer **YaDz** **enjoyed** the paper and is **excited** by our method.

However, the reviewers also raised some concerns including:
1. It is not possible to determine which change in the proposed model leads to better predictivity (**YaDz**, **GN1s**)
2. Request for additional quantitative evaluations (**YaDz**, **H4Az**)
3. Can the method be applied to other neural experimental techniques like two-photon, single-photon microscopy? (**BnXF**)
4. More comprehensive discussion of prior work (**YaDz**, **H4Az**)

We discuss the above points here, and any remaining points in the individual comments below. We are confident that we can address these concerns and are happy to clarify any remaining issues in the discussion. In the comments, we refer to the figures in the uploaded PDF document.

### Re 1: Ablation Study
We performed an ablation study comparing the effects of transitioning from the *Task-Driven ResNet + Gaussian Readout* model to the *Data-Driven CNN + Attention Readout* model as requested by Reviewers **YaDz**, **GN1s**. We measure performance in terms of test correlation.

| Core \ Readout | Gaussian        | Attention         |
| -------------------- | ------------------- | ------------------- |
| Task-Driven      | 0.262                 | 0.276 (+5%)   |
| Data-Driven      | 0.229 (-13%)     | 0.294 (+12%) |

The percentages in the parentheses denote the change in performance relative to the *Task-Driven model with Gaussian readout*. This shows that shifting to the *attention readout* improves the performance for both *Task-Driven* and *Data-Driven* cores. 

### Re 2: Quantitative Evaluations
**YaDz** asked for experimental proof that the attention readout does indeed use its ability to shift receptive fields based on the input image. We show this by inspecting the attention mask of the attention model and computing the average distance between the center of mass of the upper 5% percentile of this mask across different images for each neuron. We plot this against the test correlation of each neuron observing that the attention readout does perform shifts (Fig. E). We also include qualitative examples of the masks and the means in Fig. E.

**H4Az** suggested characterizing the distributional similarity of the EGG-generated most exciting natural inputs and the ImageNet top-k natural images using standard image metrics like Fréchet inception distance (FID). We measured the FID score between the generated images at different $\lambda$ values and the top-5 ImageNet images (Fig. D). Our results show that by changing $\lambda$ we approach the natural images manifold (lower FID).

Furthermore **H4Az** suggested using SSIM and VGG perceptual loss to additionally evaluate the performance of reconstructing images from predicted neural responses. We did not include that previously because, as shown in [1], metrics like SSIM are not necessarily a good predictor of how well neuronal responses are reproduced *in vivo*. Similarly, we observe that SSIM and VGG perceptual loss show no improvement (Fig. D). However, to strengthen our claim on the improvement of our reconstructed images we conducted a two-alternative forced choice task with 45 voluntary participants on 50 test images (Fig. C). The participants were instructed to choose which image (GD optimized or EGG generated) was more similar to the ground truth image. Results show an 82.22% average preference for EGG-generated images (95% confidence interval [80.59%, 83.75%]; Wilson score interval).

### Re 3: Applicability to other Neural Experimental Techniques
Reviewer **BnXF** asked whether our method can be used with other experimental techniques than electrophysiology. While the dataset we use for this study was recorded from in macaque visual cortex, it is in principle possible to use EGG for MEI generation and reconstructions with calcium imaging similar to the GA method on two-photon data in [2]. In fact, EGG can be applied to any modality that yields an encoding model. Showing this, however, on other experimental techniques is out of scope for this paper and one week of response time.

### Re 4: Additional Related Work
Thank you for pointing out additional references. We will include and discuss the following references in the revised manuscript.

- Nichol, Alex, et al. "Glide: Towards photorealistic image generation and editing with text-guided diffusion models." arXiv preprint arXiv:2112.10741 (2021).
- Ramesh, Aditya, et al. "Hierarchical text-conditional image generation with clip latents." arXiv preprint arXiv:2204.06125 (2022).
- crowsonkb's open-source work: https://github.com/afiaka87/clip-guided-diffusion
- Li, Wei, et al. "UPainting: Unified Text-to-Image Diffusion Generation with Cross-modal Guidance." arXiv preprint arXiv:2210.16031 (2022).
- Bashivan, Kar, DiCarlo "Neural population control via deepimage synthesis" (2019)
- Ponce et al. "Evolving Images for Visual Neurons Using a Deep Generative Network Reveals Coding Principles and Neuronal Preferences" (2019) and follow-up work
- Engstrom et al. "Adversarial Robustness as a Prior for Learned Representations"(2019)
- Feather et al. "Model metamers illuminate divergences between biological and artificial neural networks" (2022)
- Kadkhodaie & Simoncelli "Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser" (2020)

**References**

[1] Cobos et al. “It takes neurons to understand neurons: Digital twins of visual cortex synthesize neural metamers” (2022)

[2] Walker et al. “Inception loops discover what excites neurons most using deep predictive models” (2019)

Meta-review: There is a general consensus that the contribution of the paper is novel. The reviewers initially raised a number of comments regarding the experimental design and the interpretation/presentation of the results. The rebuttal appeared to address the majority of these concerns after a lengthy back-and-forth between the authors and the reviewers. Overall, there was moderate but unanimous support for the paper. The AC thus recommends the paper be accepted.
Decision: Accept (poster)