Title: Energy Guided Diffusion for Generating Neurally Exciting Images
Venue: NeurIPS 2023
Accepted: yes
Keywords: most exciting inputs

diffusion models

energy guidance

attention

macaque V4
Abstract: In recent years, most exciting inputs (MEIs) synthesized from encoding models of neuronal activity have become an established method for studying tuning properties of biological and artificial visual systems.
    However, as we move up the visual hierarchy, the complexity of neuronal computations increases. 
    Consequently, it becomes more challenging to model neuronal activity, requiring more complex models.
    In this study, we introduce a novel readout architecture inspired by the mechanism of visual attention. This new architecture, which we call attention readout, together with a data-driven convolutional core outperforms previous task-driven models in predicting the activity  of neurons in macaque area V4.
    However, as our predictive network becomes deeper and more complex, synthesizing MEIs via straightforward gradient ascent (GA) can struggle to produce qualitatively good results and overfit to idiosyncrasies of a more complex model, potentially decreasing the MEI's model-to-brain transferability.
    To solve this problem, we propose a diffusion-based method for generating MEIs via Energy Guidance (EGG).
    We show that for models of macaque V4, EGG generates single neuron MEIs that generalize better across varying model architectures than the state-of-the-art GA, while at the same time reducing computational costs by a factor of 4.7x, facilitating experimentally challenging closed-loop experiments.
    Furthermore, EGG diffusion can be used to generate other neurally exciting images, like most exciting naturalistic images that are on par with a selection of highly activating natural images, or image reconstructions that generalize better across architectures.
    Finally, EGG is simple to implement, requires no retraining of the diffusion model, and can easily be generalized to provide other characterizations of the visual system, such as invariances.
    Thus, EGG provides a general and flexible framework to study the coding properties of the visual system in the context of natural images.

Energy Guided Diffusion for Generating Neurally Exciting Images

Abstract

In recent years, most exciting inputs (MEIs) synthesized from encoding models of neuronal activity have become an established method for studying tuning properties of biological and artificial visual systems. However, as we move up the visual hierarchy, the complexity of neuronal computations increases. Consequently, it becomes more challenging to model neuronal activity, requiring more complex models. In this study, we introduce a novel readout architecture inspired by the mechanism of visual attention. This new architecture, which we call attention readout, together with a data-driven convolutional core outperforms previous taskdriven models in predicting the activity of neurons in macaque area V4. However, as our predictive network becomes deeper and more complex, synthesizing MEIs via straightforward gradient ascent (GA) can struggle to produce qualitatively good results and overfit to idiosyncrasies of a more complex model, potentially decreasing the MEI's model-to-brain transferability. To solve this problem, we propose a diffusion-based method for generating MEIs via Energy Guidance (EGG). We show that for models of macaque V4, EGG generates single neuron MEIs that generalize better across varying model architectures than the state-of-the-art GA, while at the same time reducing computational costs by a factor of 4.7x, facilitating experimentally challenging closed-loop experiments. Furthermore, EGG diffusion can be used to generate other neurally exciting images, like most exciting naturalistic images that are on par with a selection of highly activating natural images, or image reconstructions that generalize better across architectures. Finally, EGG is simple to implement, requires no retraining of the diffusion model, and can easily be generalized to provide other characterizations of the visual system, such as invariances. Thus, EGG provides a general and flexible framework to study the coding properties of the visual system in the context of natural images. 1

1 Introduction

From the early works of Hubel and Wiesel [1], visual neuroscience has used the preferred stimuli of visual neurons to gain insight into the information processing in the brain. In recent years, deep learning has made big strides in predicting neuronal responses [2-16] enabling in silico stimulus

Reconstructions

Most Exciting Naturalistic Inputs

Energy

Function

GT

Ours

synthesis of non-parametric most exciting inputs (MEIs) [17-19]. MEIs are images that strongly drive a selected neuron and can thus provide insights into its tuning properties. Up until now, they have been successfully used to find novel properties of neurons in various brain areas in mice and macaques [17-24].

However, as we move up the visual hierarchy, such as monkey visual area V4 and IT, the increasing non-linearity of neuronal responses with respect to the visual stimulus makes it more challenging to 1 obtain models with high predictive performance for single neurons, and 2 optimize perceptually plausible MEIs, that is, those not corrupted by adversarial high-frequency noise for example. Particularly, area V4 is known to be influenced by attention effects [25], and shifts in attention before the onset of saccades can change the location of its neurons' receptive fields [26, 27]. When models become more complex or units are taken from deeper layers of a network, existing MEI optimization methods based on gradient ascent (GA) can sometimes have difficulties producing qualitatively good results [28] and can overfit to the idiosyncrasies of more complex models, potentially decreasing the MEI's model-to-brain transferability. Typically, these challenges are addressed by biasing MEIs towards the statistic of natural images, for instance by gradient pre-conditioning [28], by including a total variation loss to reduce high-frequency noise [29] or by image synthesis via GANs [19]. However, as discussed by Engstrom et al. [30] and Feather et al. [31] including additional priors into the generation process can result in obfuscated model biases.

Here, we make two contributions towards the above points: 1 Weintroduce a new model architecture, called the attention readout, for predicting the activity of neurons in macaque area V4 , which together with a data-driven convolutional core outperforms previous task-driven models [24, 32]. 2 To improve the quality of MEI synthesis we introduce a novel method for optimizing MEIs via Energy Guided Diffusion (EGG). EGG diffusion guides a pre-trained diffusion model with a learned neuronal encoding model to generate MEIs with a bias towards natural image statistics. Our proposed EGG method is simple to implement and, in contrast to similar approaches [33-35], requires no retraining of the diffusion model (Fig. 1). We show that EGG diffusion not only yields MEIs that generalize better across architectures and are thus expected to drive real neurons equally well or better than GAbased MEIs but also provides a significant (4.7x) speed up over the standard GA method enhancing its utility for close-loop experiments such as inception loops [17, 18, 20, 24]. Since optimizing MEIs for thousands of neurons can take weeks [24], such a speed-up directly decreases the energy footprint of this technique. Moreover, the rapid verification of synthesized images in vivo is particularly important for close-loop experiments given that maintaining the stability of single unit recordings is challenging, and there's also the issue of representational drift [36], where tuning functions can change over time. We also demonstrate that EGG diffusion straightforwardly generalizes to provide other characterizations of the visual system that can be phrased as an inverse problem, such as image reconstructions based on neuronal responses. The flexibility and generality of EGG thus make it a powerful tool for investigating the neural mechanisms underlying visual processing.

2 Attention readout for macaque area V4

Background Deep network-based encoding models have set new standards in predicting neuronal responses to natural images [2-15]. Virtually all architectures of these encoding models consist of at least two parts: a core and a readout . The core is usually implemented via a convolutional

network that extracts non-linear features Φ( x ) from the visual input and is shared across all neurons to be predicted. It is usually trained through one of two paradigms: i) task-driven , where the core is pre-trained on a different task like object recognition [3, 4, 37-39] and then only the readout is trained to predict the neurons' responses or ii) data-driven where the model is trained end-to-end to predict the neurons' responses. The readout is a collection of predictors that map the core's features to responses of individual neurons. With a few exceptions [40], the readout components and its parameters are neuron-specific and are therefore kept simple. Typically, the readout is implemented by a linear layer with a rectifying non-linearity. Different readouts differ by the constraints they put on the linear layer to reduce the number of parameters [3, 4, 37, 40-42]. One key assumption all current readout designs make is that the readout mechanism does not change with the stimulus. In particular, this means that the location of the receptive field is fixed. While this assumption is reasonable for early visual areas like V1, it is not necessarily true for higher or mid-level areas such as macaque V4, which are known to be affected by attention effects and can even shift the location of the receptive fields [26]. This motivated us to create a more flexible readout mechanism for V4.

State-of-the-art model: Robust ResNet core with Gaussian readout In this study, we compare our data-driven model to a task-driven model [24], which is also composed of a core and readout . The core is a pre-trained robust ResNet50 ( L 2 , ε = 0 . 1 ) [43, 44]. We use the layers up to layer 3 in the ResNet, which has 1,024 channels, thus providing a 1,024 dimension feature space. Then batch normalization is applied [45], followed by a ReLU non-linearity. The Gaussian readout [40] learns the position of each neuron and extracts a feature vector at this position. During training, the positions are sampled from a 2D Gaussian distribution with means µ n and Σ n , during inference the µ n positions are used. Then the extracted features are used in a linear non-linear model to predict neuronal responses. We will refer to this model as the Gaussian model .

Proposed model: Data-driven core with attention readout The predictive model is trained from scratch to predict the neuronal responses in an end-to-end fashion. Following Lurz et al. [40], the architecture is comprised of two main components. First, the core , a four-layer CNN with 64 channels per layer with an architecture identical to Lurz et al. [40]. Secondly, the attention readout , which builds upon the attention mechanism [46, 47] as it is used in the popular transformer architecture [48]. After adding a fixed positional embedding to Φ( x ) and normalization through LayerNorm [49] to get ˜ Φ( x ) , key and value embeddings are extracted from the core representation. This is done by position-wise linear projections V ∈ R c × d k and U ∈ R c × d v both of which have parameters shared across all neurons. Then, for each neuron a learned query vector q n ∈ R d k is compared with each position's key embedding using scaled dot-product attention [48].

The result is a spatially normalized attention map α n ∈ R h × w × 1 that indicates the most important feature locations for a neuron n given an input image. Using this attention map to compute a weighted sum of the value embeddings gives us a single feature vector for each neuron. Finally, a neuron-specific affine projection with ELU non-linearity [50] gives rise to the predicted spike rate ˆ r n (Fig. 2A). The model training is performed by minimizing the Poisson loss using the same setup as described in Willeke et al. [24]. We will refer to this model as the Attention model .

Training data We use data from 1,244 Macaque V4 neurons from Willeke et al. [24] and briefly summarize their data acquisition in the supplementary materials section A.1.

Results Our Attention model significantly outperforms the Gaussian model in predicting neuronal responses of macaque V4 cells on unseen natural and model-derived images. We evaluate the model performance by the correlation between the model's prediction and the averages of actual neuron responses across multiple presentations of a set of test images, as described by Willeke et al. [24]. We compared this predictive performance to the Gaussian model [44] on 1,244 individual neurons (Fig. 2B). The Attention model significantly outperforms the Gaussian model by 12% (Wilcoxon signed-rank test, p-value = 6 . 79 · 10 -82 ). In addition, we evaluated the new readout on how well it predicts the real neuronal responses to 48 MEIs generated from the Gaussian model [see 24] and 7 control natural images. Our Attention model is better at predicting real neuronal responses,

+

Neuron-specific Parameters

Query Vector

Feature Projection

Shared Parameters

Key-Embedding Projection

Value-Embedding Projection

Positional Encoding

Attention Map

A

B

C

Correlationtoaverage

1.0

N=1244 neurons

CNN

0.8

Attention

0.6

0.48

0.4

0.43

a-driven

0.2

data-driven

task-driven

ata-

D

0.0

0.00

0.25

0.50

0.75

1.00

Task-driven ResNet(p)

Closed-looppredictiveperformance

CNN

1.0-

pearsoncorrelationp

0.8

Attention

0.69

0.6

19:0

Data-driven

0.4

0.2

一

data-driven

task-driven

0.0

D

0.00

0.25

0.50

0.75

1.00

Task-driven ResNet(p)

9253

even for MEIs of another architecture (Fig. 2C). Please note that Willeke et al. [24] experimentally verified MEIs in only a subset of neurons and only used the neurons with high functional consistency across different experimental sessions. For that reason, we too can only compare the performance of model-derived MEIs on this subset of neurons. We additionally show that the Attention model and Gaussian model show representational similarity (see Table S1) and that the Attention model uses its ability to shift its receptive field (Fig. S1).

3 Energy guided diffusion (EGG)

3.1 Algorithm and methods

In this section, we describe our approach to extract tuning properties of neuronal encoding models using a natural image prior as described by a diffusion model. In brief, we use previously established links between diffusion and score-based models and the fact that many tuning properties can be described as inverse problems (most exciting image, image reconstruction from neuronal activity, etc.) to combine an energy landscape defined by the neuronal encoding model with the energy landscape defined by the diffusion model and synthesize images via energy minimization. We show that this method leads to better generalization of MEIs and image reconstructions across architectures, faster generation, and allows for generating natural-looking stimuli.

Background: diffusion models Recently, Denoising Diffusion Probabilistic Models (DDPMs) have proved to be successful at generating high-quality images [33, 51-56]. These models can be formalized as a variational autoencoder with a fixed encoder x 0 ↦→ x T that turns a clean sample x 0 into a noisy one x T by repeated addition of Gaussian noise, and a learned decoder x T ↦→ x 0 [33],

Ablation Study We perform an ablation study comparing the effects of the choice of core and readout on the performance in terms of test correlation (Table 1). We identify that the datadriven core + Attention readout model outperforms all previous setups. Furthermore, the ablation study shows that the Attention readout generally improves performance across cores.

which is often described as inverting a diffusion process [51]. After training, the sampling process is initialized with a standard Normal sample x T ∼ N ( 0 , I ) which is iteratively 'denoised' for T steps until x 0 is reached. In the encoding, each step t corresponds to a particular noise level such that

where ¯ α t controls the signal strength at time t and ε 0 ∼ N ( 0 , I ) is independent Gaussian noise. In the decoding step, the diffusion model predicts the noise component ε θ ( x t , t ) at each step t of the diffusion process [33]. Then the sampling is performed according to

where z ∼ N ( 0 , I ) .

Several previous works have established a link between diffusion models and energy-based models [57-59]. In particular, the diffusion model ε θ ( x t , t ) can be interpreted as a score function , i.e. the gradient of a log-density or energy w.r.t. the data ∇ x log p ( x ) [60]. This link is particularly useful since combining two density models via a product is equivalent to adding their score functions.

EnerGy Guided Diffusion (EGG) To optimize neurally exciting images, we require a method that can guide diffusion models via neural encoding models. The parameterization of diffusion models introduced by Ho et al. [33] only allows for the unconditioned generation of samples. Dhariwal and Nichol [53] introduced a method for sampling from a conditional distribution p t ( x | y ) , with diffusion models using a classifier p t ( y | x ) known as classifier guidance. However, this method requires i) the classifier to be trained on the noisy images, and ii) is limited to conditions for which classification makes sense. Essentially, this method relies on computing the score of the posterior distribution.

For classifier-guidance, the gradient of a model ∇ x t log p ( y | x t ) with respect to the noisy input x t is combined with the diffusion model ∇ x t log p ( x t ) , resulting in samples x 0 conditioned on the class y . Note that this requires a model ∇ x t log p ( y | x t ) that has been trained on noisy samples of the diffusion before. Here we extend this approach to i) use neuronal encoding models, such as the ones described above, to guide the diffusion process and ii) use a model trained on clean samples only. We achieve i) by defining conditioning as a sum of energies. Specifically, we redefine equation (4) in terms of the output of the diffusion model ε θ ( x t , t ) and an arbitrary energy function E ( x t , t ) :

where λ t is the energy scale. This takes advantage of the fact that sampling in DDPMs is functionally equivalent to Langevin dynamics [51]. Langevin dynamics generally define the movement of particles in an energy field and in the special case when E ( x ) = -log p ( x ) , Langevin dynamics generates samples from p ( x ) . For this study, we use a constant value of λ and normalize the gradient of the energy function to a magnitude of 1.

To achieve ii) we use an approximate clean sample ¯ x 0 , i.e. the original image, that can be estimated at each time step t . This is achieved by a simple trick introduced in Li et al. [61]. By inverting the forward diffusion process, with the assumption that the predicted ε θ ( x t , t ) is the true noise:

As a result, the energy function receives inputs that are in the domain of x 0 at much earlier time steps t , and hence makes it feasible to use energy functions only defined on x 0 and not x t , dropping the requirement to provide an energy E ( x t , t ) that can take noisy images. Thus, the new score can be defined as

This is particularly relevant in the domain of neural system identification, as encoding models are trained on neuronal responses to natural 'clean' images [2-15, 17, 21, 24, 40]. To get an energy that can understand noisy images would require showing the noisy images to the animals in experiments, which would make the use of this method prohibitively more difficult. Therefore, a guidance method that does not require training an additional model on noisy images allows researchers to apply EGG diffusion directly to existing models trained on neuronal responses and extract tuning properties from them.

Related work Many other methods have been proposed to condition the samples of diffusion processes on additional information. Ho and Salimans [55] provided a method that addressed the second requirement of classifier-guidance by incorporating the condition y into the denoiser ε θ ( x t , t, y ) . However, to introduce a conditioning domain y in this classifier-free guidance, the whole diffusion model needs to be retrained. Furthermore, this link between diffusion models and energy-based models allowed several previous works to compose diffusion models to generate outputs that contain multiple desired aspects of a generated image [57-59]. However, these studies focus solely on generalizing the classifier-free guidance to allow guiding diffusion models with other diffusion models. Nichol and Dhariwal [52] have used a similar gradient conditioning to guide the diffusion process using the gradient of the dot product of the CLIP image and text vectors. It has been shown that CLIP models that have not been trained on noisy images can be used for guiding diffusion models [62, 63]. Kadkhodaie and Simoncelli [64] introduced a stochastic coarse-to-fine gradient ascent procedure for generating samples from the implicit prior embedded within a CNN. While we were working on this project, Feng et al. [65] published a preprint where they used the score-based definition of diffusion models to introduce an image-based prior for inverse problems where the posterior score function is available. This work is most closely related to our approach. However, they focus on how to obtain samples and likelihoods from the true posterior. For that reason, they need guiding models to be proper score functions. We do not need that constraint and focus on guiding inverse problems defined by a more general energy function and focus particularly on the application to neuronal encoding models.

Image preprocessing for neural models The neural models used in this study expect 100 × 100 images in grayscale. However, the output of the ImageNet pre-trained Ablated Diffusion Model (ADM) [53] is a 256 × 256 RGB image. We, therefore, use an additional compatibility step that performs i) downsampling from 256 × 256 → 100 × 100 with bilinear interpolation and ii) takes the mean across color channels providing the grayscale image. Each of these preprocessing steps is differentiable and is thus used end-to-end when generating the image.

3.2 Experiments

Most exciting images Weapply EGG diffusion to characterize the properties of neurons in macaque area V4. For each of these experiments, we use the pre-trained ADM diffusion model trained on 256 × 256 ImageNet images from Dhariwal and Nichol [53]. In each of our experiments, we consider two paradigms: 1) within architecture, where we use two independently pre-trained ensembles containing 5 models of the same architecture (Gaussian model or Attention model). We generate images on one and evaluate them on the other. 2) cross architecture, two independently pre-trained ensembles containing 5 models of different architectures (Gaussian model and Attention model). We demonstrate EGG on three tasks 1 Most Exciting Input (MEI) generation, where the generation method needs to generate an image that maximally excites an individual neuron, 2 naturalistic image generation, where a natural-looking image is generated that maximizes individual neuron responses, and 3 reconstruction of the input image from predicted neuronal responses. Running the experiments required a total of 7 GPU days. All computations were performed on a single consumer-grade GPU: NVIDIA GeForce RTX 3090 or NVIDIA GeForce RTX 2080 Ti depending on the availability.

MEIs have served as a powerful tool for visualizing features of a network, providing insights and testable predictions [17-21, 23, 66]. For the generation of MEIs, we selected 90 units at random from a subset of all 1,244 for which both the Gaussian model and the Attention model achieve at least a correlation of 0.5 to the average responses across repeated presentations. We compare our method to a vanilla gradient ascent (GA) method [24] which optimizes the pixels of an input image x to obtain the maximal response of the selected neuron. For the GA method, we use Gaussian blur preconditioning of the gradient. The stochastic gradient descent (SGD) optimizer was used with a learning rate of 10 and the image was optimized for 1,000 steps. We also evaluated other setups for the GA method without finding major differences (see Fig. S2). We define EGG diffusion with the energy function E (¯ x 0 ) = f i (¯ x 0 ) , where f i is the i -th neuron model and ¯ x 0 is the estimated clean sample. We optimize MEIs for both the Gaussian model and the Attention model. We set the energy scale to λ = 10 for the Gaussian model and λ = 5 for the Attention model. λ was chosen via a grid search, for more details refer to Fig. 5B. The diffusion process was run for 100 respaced time steps for the Gaussian model and 50 respaced time steps for the Attention model. For both EGG and GA, we set the norm of the 100 × 100 image to a fixed value of 25. For each of the methods, we chose

A

B

EGG

GA

EGG

GA

Attention

Attention

Cross

Within

Cross

Within

50

50

40

r(x) =1.14x

r(x)=1.04x

40

r(x)=1.10x

r(x) =1.24x

G

30

EGG

FO

G

20-

20

10

10

0-

T

0

10

20

30

40

50

0

10

20

30

40

50

GA

GA

the best of 3 MEIs optimized from different seeds. We show the influence of the initial seed on the generated MEI in figure S3. Furthermore, the images that are generated by the ADM model are RGB. We show examples of the color outputs in figure S4.

We show some examples of MEIs generated with EGG diffusion and GA for the two architectures in figure 3A. For more examples, refer to the supplementary materials figure S5. We find that the EGG-generated MEIs are significantly better (Attention) or similarly (Gaussian) activating within architectures and are significantly better at generalizing across architectures (Fig. 3B).

This can also be observed by a significant increase in the mean activation across all units (Table 2). Perceptually, EGG-generated MEIs of the Attention model looked more complex and natural than the GA-generated MEIs, and more similar to MEIs of the Gaussian model pre-trained on natural image classification.

Comparing EGG-based MEIs to the ones found by Willeke et al. [24] using GA, we find that the preferred image feature is usually preserved, but MEIs generated for the Attention model are in most cases smaller in visual angle than their Gaussian model counterparts (Fig. S5). To quantify that the MEIs from the Attention model are smaller we compute an isotropic Gaussian envelope for the MEIs. We find that the Attention model generates MEIs for which their Gaussian envelope on average is smaller than for the Gaussian MEIs ( σ At = 49.62 vs σ Ga = 55.36, Wilcoxon signed rank test p-value: 0.0078).

Finally, EGG diffusion is almost 4.7-fold faster than GA, requiring only on average 46s per MEI in comparison to the required

EGG

GA

0

50

100

150

200

250

Generation time (s)

46s ± 0.07

219s ± 0.09

219s for the GA method (Fig. 4) on a single NVIDIA GeForce RTX 3090 across 10 repetitions. This

Energy scale responses

normalized by GA MEI

A

B

C

Attention

FID vs Energy Scale

入=1

入=2

入=5

入=10

2.25

Response

2.00

1.75

1.50

Normalized

1.25

1.00

0.75

Attention

0.50

1

2

5

10

Energy Scale 入

47.5

45.0

42.5

40.0-

D

37.5

35.0-

32.5

30.0

1

2

5

10

EnergyScale

is a substantial gain, as Willeke et al. [24] required approximately 1.25 GPU years to optimize the MEIs presented in their study. With EGG, only approximately 0.25 GPU years would be needed to produce the results of the study, while providing higher quality and higher resolution MEIs. Thus, EGG can provide major savings in time and energy, and improve the quality of MEIs.

Controlling the 'naturalness' to generate most exciting natural images Unlike GA, EGG can also be used to synthesize more natural-looking stimuli by controlling the energy scale hyperparameter λ . Changing the value of λ trades off the importance of the maximization property of the image and its 'naturalness'. To demonstrate this, we generated images for 150 neurons with the highest correlations to the average for the Gaussian model. We used energy scales λ ∈ { 1 , 2 , 5 , 10 } , fixed the 100 × 100 image norm to 50, and used 50 steps re-spaced from 1000. Each image was generated using 3 different seeds and the best-performing image on the generator model was selected.

We show examples of the generated images across different energy scales in figure 5A for both the Gaussian model and the Attention model. For more examples, refer to the supplementary material (Fig. S6, Fig. S7). We subsequently quantified the predicted responses across different values of λ . We find that increasing λ increases the predicted responses (Fig. 5B), however, at higher λ values

GT

EGG

A

B

C

EGG

D

0.010-

0.6

0.008-

Validator

Distance

0.006-

0.4

Uncertain

0.4<p<0.6

10%

0.004

0.2

0.002

Generator

Validator

0.000+

+0'0

Within

Cross

Failure

p<0.4

6%

CAPE

Improved

p>0.6

84%

0

25

50

75

100

% of images (n=50)

the responses begin to plateau, or even decrease. Therefore, for generating MEIs, we use λ = 10 for the Gaussian model and λ = 5 for the Attention model. It can be further observed that decreasing λ increases the naturalness of the generated image while preserving the features of the image that the neuron is tuned towards. To quantify the increase in the naturalness of the MEIs across λ s, we measured the FID score between the generated images at different λ values and the top-5 ImageNet images (Fig. 5C). Our results show that by changing λ we approach the natural images manifold (lower FID). We also find that EGG generates MEIs ( λ = 1 ) similarly activating to the top-1 Imagenet images (Fig. S8).

Image reconstruction from unit responses Another application of EGG diffusion is image reconstruction from neuronal responses. A similar task has been attempted with success using diffusion models from human fMRI data [34, 35]. Given that only a small fraction of neurons were recorded, the image is encoded in an under-complete, significantly lower-dimensional space. Therefore, it is to be expected that the reconstructed image x will not necessarily be equal to the ground truth image x gt . However, a better reconstruction x ∗ is one that generalizes across models. Therefore, regardless of the model f used, we should get || f ( x ∗ ) -f ( x gt ) || 2 = 0 . This is trivially true for ¯ x 0 = x gt but, given the complexity of the model, there are likely other solutions. We therefore consider a masked version of the reconstructions for visualization. We mask the reconstructions to the joint receptive field of all 1,244 neurons. The mask is obtained by computing the average absolute gradients mask = E x [ |∇ x f ( x ) | ] across the responses to the test images. The masks were normalized to be between 0 and 1 and the values below 0.25 are clamped to 0.

We can reconstruct images in the EGG framework by defining the energy function as an L 2 distance between the predicted responses to the ground truth image f ( x gt ) and the predicted responses to a generated image (Fig. 6A) E ( x ) = || f ( x ) -f ( x gt ) || 2 . Note that, instead of f ( x gt ) , we could also use recorded neuronal responses. The images are generated from the Gaussian model with λ = 2 and 1000 timesteps, with the norm of the 100 × 100 image fixed to 60. We compare EGG to a gradient descent (GD) method that simply minimizes the L2 distance. The GD uses an AdamW optimizer with a learning rate of 0.05. In GD, at each optimization step the image x t is Gaussian blurred and the norm is set to 60 before passing it to the neural encoding model. We optimize the GD reconstruction up to the point where the train L 2 distance is matched between the GD and the EGG for a fair comparison of the generalization capabilities. We verified that the GD images do not improve qualitatively with more optimization steps (Fig. S9) We find that when generating the reconstruction using EGG diffusion we obtain 1) comparable within-architecture generalization and 2) much better cross-architecture generalization (Fig. 6B). The EGG-generated images produce lower within architecture distances for 84% of the images and for 98% in the cross-architecture case.

We show examples of EGG diffusion and GD reconstructions in Fig. 6C. Qualitatively, the images optimized by EGG resemble the ground truth image much more faithfully than the GD images. More examples are available in the supplementary materials (Fig. S10) and for reconstructions from the attention model see figure S11. We furthermore reconstruct images from real neurons by minimizing the distance between the model responses and recorded average neural responses to the image (Fig. S12).

Human perceptual evaluation As shown in Cobos et al. [67], metrics like SSIM are not necessarily a good predictor of how well neuronal responses are reproduced in vivo. Therefore, we conducted a voluntary anonymous survey with 45 voluntary participants on 50 test images (Fig. S13). The participants were instructed to choose which image (GD optimized or EGG generated) was more similar to the ground truth image. Results show an 82.22% average preference for EGG-generated images (95% confidence interval [80.59%, 83.75%]; Wilson score interval). In 84% of the images, the EGG method was preferred more than 60% of the time (Fig. 6D).

Limitations While EGG diffusion on average performs better than GA, it does come with limitations. Firstly, It is important to note that the results shown in this study have not been verified in vivo. Moreover, while the energy scale provides additional flexibility, it is important to keep it mind that it is an additional hyperparameter that needs to be selected to obtain the desired results, for which careful controls are necessary before the method can be used for in-vivo verification. For example, if the energy scale is too high, our method can be unreliable, which we identified in 3 out of 90 cases where EGG diffusion failed to provide a satisfactory result with the Gaussian model (Fig. 7). Furthermore, the parameter value also does not necessarily represent the same value across energy functions. Another limitation is that the maximal number of steps to generate the sample is constricted by the pre-trained diffusion model, i.e. at most 1000 steps can be used. In addition, the encoding model needs to generalize to the

EGG

GA

6.6

13.2

0.6

2.1

2.1

3.6

tor. manifold of the diffusion model. Finally, since neurons are strongly driven by contrast, encoding models often tend to push generated images to very high contrast values. To avoid this effect and make the model focus on the image content, we evaluate the energy guiding encoding model at a normalized image to eliminate the contrast direction from the guidance. This can be seen as an additional prior.

4 Discussion

In this study, we introduced a new model architecture, called the attention readout, for predicting the activity of neurons in macaque area V4 , which together with a fully data-driven convolutional core outperforms previous task-driven models. Furthermore, we propose a novel method for synthesizing images based on guiding diffusion models via energy functions (EGG). Our results indicate that EGG diffusion produces most exciting inputs (MEIs) which generalize better across architectures than the previous standard gradient ascent (GA) method. In addition, EGG diffusion significantly reduces compute time enabling larger-scale synthesis of visual stimuli. EGG diffusion is not limited to the generation of MEIs and, within the same framework, allows, among other characterizations, to 1) generate most exciting naturalistic images which approach the manifold of most activating images in the ImageNet database, and 2) reconstruct images from unit responses, which generalize better across architectures and qualitatively resemble more the original image than images obtained via regular gradient descent optimization. While the dataset we use for this study was recorded from the macaque visual cortex, it is in principle possible to use EGG for MEI generation and reconstructions with calcium imaging similar to the GA method on two-photon data in Walker et al. [17]. In fact, EGG can be applied to any modality that yields an encoding model and where a suitable diffusion model is available. More generally, EGG can be used whenever the 'constraint' on a particular image can be phrased in terms of an energy function. In summary, EGG diffusion provides a flexible and powerful framework for studying coding properties of the visual system.

Acknowledgments

The authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting Konstantin Willeke and Arne Nix. The authors also thank Mohammad Bashiri and Suhas Shirinvasan for their technical support and helpful discussions. The research was supported by the Cyber Valley Research Fund (AN, FHS). FHS is further supported by the German Federal Ministry of Education and Research (BMBF) via the Collaborative Research in Computational Neuroscience (CRCNS) (FKZ 01GQ2107), as well as the Collaborative Research Center (SFB 1233, Robust Vision). PP is supported by the German Federal Ministry for Economic Affairs and Climate Action (FKZ ZF4076506AW9). We also acknowledge support from the National Institute of Mental Health and National Institute of Neurological Disorders And Stroke under Award Number U19MH114830 and National Eye Institute award numbers R01 EY026927 and Core Grant for Vision Research T32-EY-002520-37 as well as the National Science Foundation Collaborative Research in Computational Neuroscience, USA with grant number IIS-2113173, Germany with FKZ: 01GQ2107. We thank our lab members, family, and friends for participating in the anonymous survey.

References

D H Hubel and T N Wiesel. Receptive fields of single neurones in the cat's striate cortex. J. Physiol. , 148(3):574-591, October 1959.

Charles F Cadieu, Ha Hong, Daniel L K Yamins, Nicolas Pinto, Diego Ardila, Ethan A Solomon, Najib J Majaj, and James J DiCarlo. Deep neural networks rival the representation of primate IT cortex for core visual object recognition. PLoS Comput. Biol. , 10(12):e1003963, December 2014.

Daniel L K Yamins, Ha Hong, Charles F Cadieu, Ethan A Solomon, Darren Seibert, and James J DiCarlo. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc. Natl. Acad. Sci. U. S. A. , 111(23):8619-8624, June 2014.

Santiago A Cadena, George H Denfield, Edgar Y Walker, Leon A Gatys, Andreas S Tolias, Matthias Bethge, and Alexander S Ecker. Deep convolutional models improve predictions of macaque V1 responses to natural images. PLoS Comput. Biol. , 15(4):e1006897, April 2019.

Fabian H Sinz, Alexander S Ecker, Paul G Fahey, Edgar Y Walker, Erick Cobos, Emmanouil Froudarakis, Dimitri Yatsenko, Xaq Pitkow, Jacob Reimer, and Andreas S Tolias. Stimulus domain transfer in recurrent models for large scale cortical population prediction on video. In Proceedings of the 32nd International Conference on Neural Information Processing Systems , NIPS'18, pages 7199-7210, Red Hook, NY, USA, December 2018. Curran Associates Inc.

Yimeng Zhang, Tai Sing Lee, Ming Li, Fang Liu, and Shiming Tang. Convolutional neural network models of V1 responses to complex patterns. J. Comput. Neurosci. , 46(1):33-54, February 2019.

Lane T McIntosh, Niru Maheswaranathan, Aran Nayebi, Surya Ganguli, and Stephen A Baccus. Deep learning models of the retinal response to natural scenes. In Proceedings of the 30th International Conference on Neural Information Processing Systems , NIPS'16, pages 13691377, Red Hook, NY, USA, December 2016. Curran Associates Inc.

David A Klindt, Alexander S Ecker, Thomas Euler, and Matthias Bethge. Neural system identification for large populations separating 'what' and 'where'. November 2017.

William F Kindel, Elijah D Christensen, and Joel Zylberberg. Using deep learning to probe the neural code for images in primary visual cortex. J. Vis. , 19(4):29, April 2019.

Alexander S Ecker, Fabian H Sinz, Emmanouil Froudarakis, Paul G Fahey, Santiago A Cadena, Edgar Y Walker, Erick Cobos, Jacob Reimer, Andreas S Tolias, and Matthias Bethge. A rotation-equivariant convolutional neural network model of primary visual cortex. September 2018.

Benjamin R Cowley and Jonathan W Pillow. High-contrast 'gaudy' images improve the training of deep neural network models of visual cortex. June 2020.

Max F Burg, Santiago A Cadena, George H Denfield, Edgar Y Walker, Andreas S Tolias, Matthias Bethge, and Alexander S Ecker. Learning divisive normalization in primary visual cortex. PLoS Comput. Biol. , 17(6):e1009028, June 2021.

Eleanor Batty, Josh Merel, Nora Brackbill, Alexander Heitman, Alexander Sher, Alan Litke, E J Chichilnisky, and Liam Paninski. Multilayer recurrent network models of primate retinal ganglion cell responses. July 2022.

Mohammad Bashiri, Edgar Walker, Konstantin-Klemens Lurz, Akshay Jagadish, Taliah Muhammad, Zhiwei Ding, Zhuokun Ding, Andreas Tolias, and Fabian Sinz. A flow-based latent state generative model of neural population responses to natural images. Adv. Neural Inf. Process. Syst. , 34:15801-15815, December 2021.

Ján Antolík, Sonja B Hofer, James A Bednar, and Thomas D Mrsic-Flogel. Model constrained by visual hierarchy improves prediction of neural responses to natural scenes. PLoS Comput. Biol. , 12(6):e1004927, June 2016.

Eric Y Wang, Paul G Fahey, Kayla Ponder, Zhuokun Ding, Andersen Chang, Taliah Muhammad, Saumil Patel, Zhiwei Ding, Dat Tran, Jiakun Fu, et al. Towards a foundation model of the mouse visual cortex. bioRxiv , 2023.

Edgar Y Walker, Fabian H Sinz, Erick Cobos, Taliah Muhammad, Emmanouil Froudarakis, Paul G Fahey, Alexander S Ecker, Jacob Reimer, Xaq Pitkow, and Andreas S Tolias. Inception loops discover what excites neurons most using deep predictive models. Nat. Neurosci. , 22(12): 2060-2065, December 2019.

Pouya Bashivan, Kohitij Kar, and James J DiCarlo. Neural population control via deep image synthesis. Science , 364(6439), 2019.

Carlos R Ponce, Will Xiao, Peter F Schade, Till S Hartmann, Gabriel Kreiman, and Margaret S Livingstone. Evolving images for visual neurons using a deep generative network reveals coding principles and neuronal preferences. Cell , 177(4):999-1009.e10, 2019.

Katrin Franke, Konstantin F Willeke, Kayla Ponder, Mario Galdamez, Na Zhou, Taliah Muhammad, Saumil Patel, Emmanouil Froudarakis, Jacob Reimer, Fabian H Sinz, and Andreas S Tolias. State-dependent pupil dilation rapidly shifts visual feature selectivity. Nature , 610(7930): 128-134, October 2022.

Larissa Höfling, Klaudia P Szatko, Christian Behrens, Yongrong Qiu, David A Klindt, Zachary Jessen, Gregory W Schwartz, Matthias Bethge, Philipp Berens, Katrin Franke, Alexander S Ecker, and Thomas Euler. A chromatic feature detector in the retina signals visual context changes. December 2022.

Jiakun Fu, Suhas Shrinivasan, Kayla Ponder, Taliah Muhammad, Zhuokun Ding, Eric Wang, Zhiwei Ding, Dat T Tran, Paul G Fahey, Stelios Papadopoulos, Saumil Patel, Jacob Reimer, Alexander S Ecker, Xaq Pitkow, Ralf M Haefner, Fabian H Sinz, Katrin Franke, and Andreas S Tolias. Pattern completion and disruption characterize contextual modulation in mouse visual cortex. March 2023.

Zhiwei Ding, Dat T Tran, Kayla Ponder, Erick Cobos, Zhuokun Ding, Paul G Fahey, Eric Wang, Taliah Muhammad, Jiakun Fu, Santiago A Cadena, Stelios Papadopoulos, Saumil Patel, Katrin Franke, Jacob Reimer, Fabian H Sinz, Alexander S Ecker, Xaq Pitkow, and Andreas S Tolias. Bipartite invariance in mouse primary visual cortex. March 2023.

Konstantin F Willeke, Kelli Restivo, Katrin Franke, Arne F Nix, Santiago A Cadena, Tori Shinn, Cate Nealley, Gabby Rodriguez, Saumil Patel, Alexander S Ecker, Fabian H Sinz, and Andreas S Tolias. Deep learning-driven characterization of single cell tuning in primate visual area V4 unveils topological organization. May 2023.

Tirin Moore, Katherine M Armstrong, and Mazyar Fallah. Visuomotor origins of covert spatial attention. Neuron , 40(4):671-683, November 2003.

A S Tolias, T Moore, S M Smirnakis, E J Tehovnik, A G Siapas, and P H Schiller. Eye movements modulate visual receptive fields of V4 neurons. Neuron , 29(3):757-767, March 2001.

Till S Hartmann, Marc Zirnsak, Michael Marquis, Fred H Hamker, and Tirin Moore. Two types of receptive field dynamics in area v4 at the time of eye movements? Frontiers in systems neuroscience , 11:13, 2017.

Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill , 2(11): e7, November 2017.

Pouya Bashivan, Kohitij Kar, and James J DiCarlo. Neural population control via deep image synthesis. Science , 364(6439), May 2019.

Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Brandon Tran, and Aleksander Madry. Adversarial robustness as a prior for learned representations. June 2019.

Jenelle Feather, Guillaume Leclerc, Aleksander M ˛ adry, and Josh H McDermott. Model metamers illuminate divergences between biological and artificial neural networks. May 2022.

Yifei Ren and Pouya Bashivan. How well do models of visual cortex generalize to out of distribution samples? May 2023.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Adv. Neural Inf. Process. Syst. , 33:6840-6851, 2020.

Yu Takagi and Shinji Nishimoto. High-resolution image reconstruction with latent diffusion models from human brain activity. March 2023.

Yizhuo Lu, Changde Du, Dianpeng Wang, and Huiguang He. MindDiffuser: Controlled image reconstruction from human brain activity with semantic and structural diffusion. March 2023.

Laura N Driscoll, Lea Duncker, and Christopher D Harvey. Representational drift: Emerging theories for continual learning and experimental future directions. Current Opinion in Neurobiology , 76:102609, 2022.

U. Guclu and M. A. J. van Gerven. Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream. Journal of Neuroscience , 35(27):1000510014, July 2015. doi: 10.1523/jneurosci.5023-14.2015. URL https://doi.org/10.1523/ jneurosci.5023-14.2015 .

Daniel L K Yamins and James J DiCarlo. Using goal-driven deep learning models to understand sensory cortex. Nat. Neurosci. , 19(3):356-365, March 2016.

Santiago A Cadena, Konstantin F Willeke, Kelli Restivo, George Denfield, Fabian H Sinz, Matthias Bethge, Andreas S Tolias, and Alexander S Ecker. Diverse task-driven modeling of macaque V4 reveals functional specialization towards semantic tasks. May 2022.

Konstantin-Klemens Lurz, Mohammad Bashiri, Konstantin Willeke, Akshay K Jagadish, Eric Wang, Edgar Y Walker, Santiago A Cadena, Taliah Muhammad, Erick Cobos, Andreas S Tolias, Alexander S Ecker, and Fabian H Sinz. Generalization in data-driven models of primary visual cortex. April 2021.

DAKlindt, A S Ecker, T Euler, and M Bethge. Neural system identification for large populations separating 'what' and 'where'. In Advances in Neural Information Processing Systems , pages 4-6, 2017.

F Sinz, A S Ecker, P Fahey, E Walker, E Cobos, E Froudarakis, D Yatsenko, X Pitkow, J Reimer, and A Tolias. Stimulus domain transfer in recurrent models for large scale cortical population prediction on video. In Advances in Neural Information Processing Systems 31 . 2018.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. December 2015.

Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adversarially robust ImageNet models transfer better? July 2020.

Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. February 2015.

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 , 2014.

Alex Graves. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 , 2013.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017.

Jingjing Xu, Xu Sun, Zhiyuan Zhang, Guangxiang Zhao, and Junyang Lin. Understanding and improving layer normalization. Advances in Neural Information Processing Systems , 32, 2019.

Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter. Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv:1511.07289 , 2015.

Jascha Sohl-Dickstein, Eric A Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. March 2015.

Alex Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. February 2021.

Prafulla Dhariwal and Alex Nichol. Diffusion models beat GANs on image synthesis. May 2021.

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-Resolution image synthesis with latent diffusion models. December 2021.

Jonathan Ho and Tim Salimans. Classifier-Free diffusion guidance. July 2022.

Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic Text-to-Image diffusion models with deep language understanding. May 2022.

Nan Liu, Shuang Li, Yilun Du, Joshua B Tenenbaum, and Antonio Torralba. Learning to compose visual relations. November 2021.

Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional visual generation with composable diffusion models. June 2022.

Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, and Will Grathwohl. Reduce, reuse, recycle: Compositional generation with Energy-Based diffusion models and MCMC. February 2023.

Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-Based generative modeling through stochastic differential equations. November 2020.

Wei Li, Xue Xu, Xinyan Xiao, Jiachen Liu, Hu Yang, Guohao Li, Zhanpeng Wang, Zhifan Feng, Qiaoqiao She, Yajuan Lyu, and Hua Wu. UPainting: Unified Text-to-Image diffusion generation with cross-modal guidance. October 2022.

Katherine Crowson. v-diffusion, 2021. URL https://github.com/crowsonkb/ v-diffusion-pytorch .

Katherine Crowson. Clip guided diffusion hq 256x256, 2021. URL https://colab. research.google.com/drive/12a_Wrfi2_gwwAuN3VvMTwVMz9TfqctNj .

Zahra Kadkhodaie and Eero P Simoncelli. Solving linear inverse problems using the prior implicit in a denoiser. July 2020.

Berthy T Feng, Jamie Smith, Michael Rubinstein, Huiwen Chang, Katherine L Bouman, and William T Freeman. Score-Based diffusion models as principled priors for inverse imaging. April 2023.

Jiakun Fu, Konstantin F Willeke, Pawel A Pierzchlewicz, Taliah Muhammad, George H Denfield, Fabian Hubert Sinz, and Andreas S Tolias. Heterogeneous orientation tuning across Sub-Regions of receptive fields of V1 neurons in mice. February 2022.

Erick Cobos, Taliah Muhammad, Paul G Fahey, Zhiwei Ding, Zhuokun Ding, Jacob Reimer, Fabian H Sinz, and Andreas S Tolias. It takes neurons to understand neurons: Digital twins of visual cortex synthesize neural metamers. December 2022.

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A largescale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition , pages 248-255, June 2009.

A Supplementary Material

A.1 Training Data

Electrophysiological data were acquired as broadband signal (0.5Hz-16kHz), from a pair of male rhesus macaque monkeys ( Macaca mulatta ), using 32 channel linear silicon probes (NeuroNexus V1x32-Edge-10mm-60-177). The data was spike-sorted, and single units were isolated based on unit stability, refractory periods, and channel principal component pairs. Visual stimuli were presented to the animals on a 16:9 widescreen HD LCD monitor at 100˜ cm viewing distance. The animals were rewarded with juice if they maintained their gaze around a red fixation target throughout each trial. At the beginning of each recording session, the receptive fields (RFs) of the neurons were mapped in relation to a fixation target using sparse random dot stimuli, and the population RF was pulled towards the center of the screen by adjusting the fixation target. A collection of 24,075 images from ImageNet [68] was transformed into gray-scale and cropped to the central 420 2 px and had 8 bit intensity resolution. These images were presented as visual stimuli during standalone generation recordings of 1244 units and during closed-loop recordings of 82 units. For details on the closed loop paradigm, please refer to Willeke et al. [24].

A.2 Supplementary Experiments

A.2.1 Attention Model

Attention readout uses its ability to shift receptive fields Receptive fields of neurons in area V4 can shift before the onset of saccades, believed to be associated with attentional shifts [26]. We investigate whether the Attention model actually uses its ability to shift the receptive field depending on image content. We inspect the attention mask of the attention model. We compute the center of mass of the upper 5% percentile of each attention mask. We then compute the average distance between the center of masses across different images for each neuron. We plot the average distance against the test correlation of each neuron observing that the attention readout does perform shifts (Fig. S1a). We also show qualitative examples of the masks and the means in Fig. S1b.

Centered Kernel Alignment We computed CKA of the neural encodings across architectures between the Attention model and Gaussian model and within architecture between different seeds (e.g. Attention 1 and Attention 2 are models with the same architecture, but trained with different seeds). The CKA is computed between the predicted neuronal responses. We observe that the within-architecture similarity is very high (> 0.99) for both architectures and the cross architecture similarity is slightly lower, but also high (> 0.9) (Table 2). We expect such an outcome, since both architectures were trained to model the same neural representation.

Supplementary Table S1: Centered Kernel Alignment for the two architectures comparing across and within architectures.

A.2.2 MEI generation

Comparison of experimental setups For the GA optimization, we use the established method for generating MEIs that has been tested in vivo Walker et al. [17]. However, we perform a comparison study to show that the parameters chosen are selected to maximize the performance of the GA method. We rerun the MEI optimizations using the AdamW optimizer and find a significant decrease in performance in comparison to the SGD optimizer (r = 0.69). We also run the MEIs for 100 steps instead of 1000 and also find a performance decrease (r = 0.95) (Fig. S2).

Generating MEIs in color The diffusion model generates color images, so in principle, it can generate color MEIs. We attach some examples (Fig. S4). Since the encoding models are trained

Attention readout shifts

Example attention maps

Neuron 0

Neuron 177

Presented

Image

A

B

25

20

Distance

15

Shift

Mean

10

5

0.0

0.5

1.0

Test Correlation

on grayscale images because the animals only saw grayscale images the colors in these may not be meaningful. However, if one were to use color stimuli it would be possible to generate MEIs that are colored and potentially meaningful.

MENIs vs ImageNet search We compare the generated MEIs ( λ = 1 ) to a standard approach for finding natural images for individual neurons. To that end, we perform a search across 100k images from the ImageNet dataset [68] to find the top-1 most activating image for a particular unit. We then compare the predicted activations of the top-1 ImageNet image and the generated MEIs ( λ = 1 ) in the cross-architectures paradigm (Fig. S8). We find that the generated MENIs drive comparable activation to the top-1 ImageNet images. Like in the MEI generation paradigm, EGG can thus significantly speed up the search for activating natural images, as it does not need to search through millions of images.

A

B

Comparison of

50

MEI optimization steps

50-

r(x) = 0.69x

40

r(x) = 0.95x

40

W

30

S

m

30-

t

e

20

0

20-

1

10

10

0

T

0

0

10

20

30

40

50

0

10

20

30

40

50

SGD

1000 steps

Attention

Seed 0

Seed 1

Seed 2

C

Colored MEIs

Grayscale MEIs

Neuron 230

Neuron 710

Neuron 901

Neuron 1118

L

Supplementary Figure S5: Examples of MEIs generated using EGG for the Attention and Gaussian models.

10

00

Q

入=1

入=2

入=5

=10

Attention Model top-1

Imagenet

Imagenet

50-

30-

y(x) = 1.13x

40

y(x) = 0.97x

20

1

1

11

30

V

V

G

G

G

20

G

E

E

10

10

0

T

0

0

10

20

30

40

50

0

10

20

30

ImageNet Top-1

ImageNet Top-1

EGG

GT

400

800

1200

1600

2000

GT

EGG

Attention Model Reconstructions

Attention Model

Reconstruction Performance

GT

EGG

EGG

0.04

L2 Distance

0.03

0.15-

0.02

0.10-

0.01

0.05

0.00

0.00

Within

Cross

APE

EGG

GT

Gaussin Model Reconstructions from Real Neurons

Reconstructions Performance

EGG

0.10

0.7-

0.08-

0.6-

Distance

0.5-

0.06-

0.4

0.04-

FE'O

L

0.2

0.02-

0.1

0.00-

100

Within

Cross

CAPE

A

B

O

A

O

B

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors first employed attention readout to train a model for predicting neural responses ($y$) from image ($x$), aiming to address the issue of attention effects in the V4 area. Subsequently, they applied the image ($x$) from text ($y$) method proposed by Prafulla [1] to generate images ($x$) from neural responses ($y$).\n\nReference:\n\n[1] Prafulla Dhariwal and Alex Nichol. Diffusion models beat GANs on image synthesis. May 2021.\n\nSTRENGTHS:\nCompared to previous research, this study incorporates two technologies to enhance the performance of their model in predicting neuronal responses and MEIs. The first technology involves integrating an attention map into their encoding network. From a biological standpoint, the attention method allows for the receptive field of neurons to be dynamically adjusted based on different inputs, which aligns with observations made in experiments on V4. Empirically, attention-based Vision Transformer (ViT) models have shown superior performance compared to ResNet models in the field of computer vision. In this study, the performance of the proposed model surpasses the current state-of-the-art in predicting neuronal responses.\n\nThe second technology employed in this research is the use of Bayesian diffusion methods to decode input from neural responses. The diffusion model is renowned for its ability to generate intricate image details, and the authors demonstrate that their approach, called EGG, outperforms the previous GA method.\n\nAlthough the authors did not develop these two methods themselves, it is possible that their application in neural data analysis is novel. To the best of my knowledge, this study appears to be the first instance of using the attention map in the encoding network for predicting neuronal responses, taking into consideration the observed phenomenon in V4. Furthermore, the utilization of the Bayesian diffusion method, specifically the EGG approach, for decoding neural responses seems to be a novel application in this context.\n\nWEAKNESSES:\n1.The current images are grayscale. Can the MEI images be colored?\n\n2. The current neural data are collected by silicon probes. Can we use techniques such as two-photon or single-photon imaging to record more neuronal data?\n\nQUESTIONS:\nHow many neurons are used in EGG? 1,244 individual neurons?\n\nLIMITATIONS:\nNone", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors tackle the problem of synthesizing most exciting inputs for neurons in the higher visual cortex (V4 in their case) of macaque, with data collected using electrophysiology.\n\nThe paper makes two claimed contributions:\n\n1. The authors propose a new encoding architecture which uses a data-driven CNN core with a cross-attention read out layer. The cross-attention layer is parameterized similar to traditional cross-attention in other machine learning papers. In the author's design, there is a learned per-neuron query vector, and spatial key/value embeddings derived from pixel-wise linear projections of the CNN feature map that is shared for all neurons. The authors compare this against a task-optimized backbone with learned gaussian readout. The authors show that the the attention encoder performs better with high probability (via a Wilcoxon test) on novel non-training images. \n\n2. The authors propose energy guided diffusion. Where by they modify the score-prediction with the derivative of vector valued function (not a score function corresponding to a well posed distribution). They propose a modification which does not require the energy function to take as input noisy images, they accomplish this by using the \"pred_xstart\" code provided by [1].\n\nThe authors validate their method by first comparing their method against gradient ascent + gradient blurring in MEI synthesis. They find that EGG is able to generate MEI in a faster fashion and with better cross-architecture generalization properties than GA. \n\nIn the MENI experiment, they show that a tradeoff between naturalness and excitation by adjusting the strength of the gradient of the energy function. They find that their synthesized MENI ($\\lambda=1$) are roughly comparable to imagenet top-1 images in cross-arch predicted activations.\n\nIn the third experiment, they authors experiment with stimulus reconstruction. They accomplish this by modifying the energy function to minimize the L2 distance between predicted and ground truth neuron activities. They find that EGG regularized stimulus reconstruction is more faithful.\n\n[1] Dhariwal, Prafulla, and Alexander Nichol. \"Diffusion models beat gans on image synthesis.\" Advances in Neural Information Processing Systems 34 (2021).\n\nIn the supplementary, the authors provide additional descriptions of data collection (32 channel) and experiment design. The authors provide further experiments that compare ResNet & Attention encoders, the strength of $\\lambda$, and how pure gradient methods compare with EGG in stimulus reconstruction.\n\nSTRENGTHS:\nThe paper on balance is well written, the authors are largely clear in their experimental design and their evaluation. The authors provide sufficient detail in the paper itself for reproduction, with additional code provided in the supplemental. The code is well written and easy to follow.\n\nThe use of diffusion models to regularize the synthesis of most exciting inputs for monkey V4 collected using electrophysiology is novel, and to the author's knowledge it has not been attempted before. \n\nThe authors perform a variety of experiments, and I find their proposed design for the attention encoder and evaluation of the attention encoder to be convincing.\n\nWEAKNESSES:\nThe authors did an good job writing a paper that has clarity and do a great job in providing details. But in my opinion, the authors overstate their contribution with regard to \"Energy Guidance\" (EGG). If the author can include additional citations, reduce their overly broad claims, and provide additional experiments/metrics, the paper would be improved significantly.\n\n\n**The paper could stand out based on the experiments alone, but the authors have emphasized energy guidance to be a central contribution** without citing the vast number of papers in computer vision that have been published in the past two years that:\n1. Similarly do not use a well-posed score function in the form of the derivative of a classifier. **This aspect is not novel.** In fact I would argue that most of gradient conditioning papers of diffusion models published today explicitly do not use a score function in the form of a classifier gradient. These papers are not cited.\n2. Estimate a clean sample ($x_0$) and do not feed in a noise corrupted image to the model providing guidance. The paragraph in lines 162-174 seem to indicate that this approach is novel in the context of classifier/gradient guidance for diffusion models. However **similar approaches have been widely used in computer vision literature**. These papers are not cited. \n\n* On contributions and prior work\n  * GLIDE from 2021 [1] proposed to perform image synthesis again using the gradient of the dot product of a CLIP image vector and a text vector to modify the diffusion output. Note that GLIDE used a noisy trained version of CLIP to perform guidance, however in this approach it seeks maximization of a dot product output which does not yield a \"proper\" distribution score function when you take the gradient.\n\n  * GLIDE and the DALL-E 2 [2] paper cite crowsonkb's 2021 open source CLIP guidance work [3, 4]. These two codebases combine CLIP guidance with the pred_x0 trick (eq 6 of this paper) without retraining the diffusion or gradient model. Similarly, the hugginface diffusers library minimize the orthodromic distance (derivative is not a proper score function) in CLIP space rather than maximizing a dot product using CLIP, and also use the pred_x0 trick without retraining the diffusion or gradient model. This estimated $x_0$ trick has also been formally described in [5] eq's 3 and 4. I suggest the authors cite at least one of these papers and clarify their contributions.\n\n* On the soundness of the experiments\n\n  * I also found some of the experimental setups to be inconsistent. \n  * in the MEI experiment line 221, the naïve SGD optimizer is used, and this forms the basis of the claim in Figure 4 to show GA is much slower than EGG. However in the image reconstruction experiment line 282, the more sophisticated AdamW optimizer is used. There is no reason why the AdamW optimizer cannot be used with gaussian blur gradient conditioning via filtering at an higher stage of the backpropagation. \n  * for the MEI experiment, GA is run for 1,000 steps, while EGG was run for 100 steps. This does not seem to be an entirely fair comparison time wise. \n  * I'm not sure why the authors decided to normalize the image itself to 25 (line 230) for MEI, 50 (line 251) for MENI, and 60 (line 281) for reconstruction. This step seems to quite explicitly break the energy guidance output. Can you provide a justification in the text for why this is done, and why you use different norms for different tasks? Can the authors perform MEI/SGD/AdamW experiments without this step?\n\n* On the lack of evaluation\n\n  * There is a lack of quantitive evaluation metrics aside from predicted neural activity. The paper almost entirely relies on qualitative claims when it comes to MEI/MENI/Reconstruction output. There are no actual metrics to indicate that the MEI/MENI/Reconstruction outputs are more similar to the ground truth most exciting natural stimuli. \n  * I suggest that the authors use common vision metrics like SSIM/PSNR/MSE to evaluate the low level similarity of the images, or high-level image metrics like perceptual loss (VGG), CLIP cosine distance, or distribution-wise comparisons like FID/CLIP-FID (see question 3 in section below). I don't think all the suggested metrics here are needed, but at least a few (any of SSIM/Perceptual VGG/CLIP for image reconstruction, and any of FID/CLIP-FID for MENI; or if the previous metrics are not possible, perhaps a human survey on Mechanical Turk/Prolific to evaluate if the images are better?) should be added where appropriate.\n\nOverall I think the authors have presented an interesting system, but there is no citation or acknowledgement of prior work from computer vision which use non-classifier based gradient guidance of diffusion models, or using estimated $x_0$ to alleviate the need for noisy trained classifiers. Otherwise I think the paper is interesting and would be improved if the authors can clarify the scope of their contributions and better quantify their claims. I would happily re-evaluate if the authors can improve this paper in a subsequent revision.\n\n[1] Nichol, Alex, et al. \"Glide: Towards photorealistic image generation and editing with text-guided diffusion models.\" arXiv preprint arXiv:2112.10741 (2021).\n\n[2] Ramesh, Aditya, et al. \"Hierarchical text-conditional image generation with clip latents.\" arXiv preprint arXiv:2204.06125 (2022).\n\n[3] https://github.com/afiaka87/clip-guided-diffusion/blob/c1d5906225586bc8455bb17c29a3c2caf9a02766/cgd/cgd.py#L141\n\n[4] https://colab.research.google.com/drive/12a_Wrfi2_gwwAuN3VvMTwVMz9TfqctNj#scrollTo=X5gODNAMEUCR&line=41&uniqifier=1\n\n[5] Li, Wei, et al. \"UPainting: Unified Text-to-Image Diffusion Generation with Cross-modal Guidance.\" arXiv preprint arXiv:2210.16031 (2022).\n\nQUESTIONS:\n1. Could you clarify why the MEI experiment uses SGD, but the image reconstruction experiment uses AdamW?\n2. For Figure 3, you make qualitative claims that the EGG MEIs are better, can you back this up with quantitive numbers like SSIM/Perceptual loss (VGG)/Inception/CLIP distance against the natural input that most excites the neuron?\n3. For Figure 5, can you characterize the distributional similarity of the images using standard image metrics like Fréchet inception distance or CLIP-FID proposed by Kynkäänniemi (MENI vs top-k of natural images for a neuron)? Something like the Figure 4 of Imagen [1] Figure 4's pareto curves which measure how the energy scale affects the image distribution distance (FID/CLIP). For Q3/Q4, if the image metrics are not possible, perhaps a human survey on Mechanical Turk/Prolific to evaluate if the images for the experiments are more similar/better.\n4. For Table 1, Figure 3B, Figure 5C, and Figure 6B, could you clarify what is the \"base\" model, as in which model you use for image synthesis, and which model is the evaluating model?\n5. Can you clarify the solver you use in the diffusion model? From the code, it seems like you use the DDPM solver, however there are a variety of stronger solvers (DDIM/PNDM/DPM-solver [2,3,4]) which yield convergence in as few as 10 steps. Is there any reason you decided to go with such an old solver?\n\nOverall I think the clarity of the paper is good, but could be further improved with a few small clarifications and incorporation of standard vision metrics (SSIM/VGG perceptual loss/FID/CLIP-FID).\n\n[1] Saharia, Chitwan, et al. \"Photorealistic text-to-image diffusion models with deep language understanding.\" Advances in Neural Information Processing Systems 35 (2022): 36479-36494.\n\n[2] Song, Jiaming, Chenlin Meng, and Stefano Ermon. \"Denoising diffusion implicit models.\" arXiv preprint arXiv:2010.02502 (2020).\n\n[3] Liu, Luping, et al. \"Pseudo numerical methods for diffusion models on manifolds.\" arXiv preprint arXiv:2202.09778 (2022).\n\n[4] Lu, Cheng, et al. \"Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps.\" arXiv preprint arXiv:2206.00927 (2022).\n\nLIMITATIONS:\nThe authors clearly describe the limitations of their experiments.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nFurther characterizing the complex coding properties of V4 neurons might require (1) better encoding models of neuronal activity as well as (2) better methods to generate informative most exciting inputs (MEI). The paper tackles (1) by proposing a new readout mechanism for a convolutional data-driven core based on attention that outperforms the SOTA at predicting neural responses of neurons in macaque V4. Then, it tackles (2) through a new simple method called EEG that generates MEI using a pre-trained diffusion model and the guidance signal of the encoding model. EEG can also be used for image reconstruction from neuronal activity, in both applications it generalizes better than the more traditional gradient ascent (GA) across architectures. Finally, their method can produce most exciting natural inputs (MENI) on par with highly activating natural images.\n\nSTRENGTHS:\n- The problems tackled are well motivated\n- The paper is clear\n- The proposed readout mechanism is: novel to the best of my knowledge, biologically motivated, sound, and backed up by positive results (outperform current SOTA in predicting neuronal activity)\n- The method to generate MEI is: ingenious, simple but sound, and backed up by positive results (better generalization than GA and faster)\n\nWEAKNESSES:\n- (1) compares two models with: a different core, a different readout, and a different training strategy, hence it is hard to isolate the benefit from the new readout mechanism only. This part would be strengthened with additional baselines like the core model with Gaussian readout and the pre-trained resnet50 with attention readout\n- While the new model seems to be a better encoding model of neuronal activity --according to the results of (1)--, MEI generated with it seem overall worse at driving neurons activation (fig 5.b) compared to the baseline ResNet50 (both EGG and GA). In fact, the results on MENI reported in Fig5.c are with the ResNet50 model, and the examples of image reconstruction put forward in the paper all come from the ResNet50 model.\n\nQUESTIONS:\n- Did the author consider performing a CKA between models in the within condition and between models in the cross condition, to quantify the generalization/transferability of the MEIs?\n- Why is the ResNet50 and not the ACNN considered for comparing MENI to top1 most activating ImageNet images? At least this is what I am inferring from the color used in Fig 5.c, as it is not written in the text which model is finally used.\n- It would be interesting to add GA as a baseline for fig 5.b. I assume that the intuition behind the plateau/decrease is that if we increase the energy scale \"too much\", we fall back to \"overfitting to the idiosyncrasies\" of the encoding model, this baseline would give a bit of context w.r.t to this hypothesis.\n- How are the images shown in the paper selected?\n- My understanding for the motivation behind the MENI proposed is speed. Getting more natural-looking stimuli (e.g., for control stimuli) without having to go through millions of images. While I agree with the motivation, given the random nature of the generative process, I was wondering how much sampling was necessary before falling on satisfactory MENIs? Will I trust that EEG will still be faster, I worry about the meaningfulness of some of the MENIs generated (In Dhariwal et al. 2021, samples generated with a lower guidance scale are of significantly lower quality, e.g., $\\lambda=1$ leads to FID=33) \n- For image reconstruction, it would be interesting to see examples generated with the ACNN model, only results from ResNet50 are shown or discussed in the paper.\n\n\n\nClarification comments:\n- Slight lack of consistency w.r.t to naming: the encoding model is called through the paper: data-driven with attention readout, Attention CNN, ACNN, and Attention.\n- Within: l.201: \"task-driven ResNet with Gaussian readout or data-driven with attention readout\" & Cross: l.203: \"ResNet and data-driven with attention readout\". I assume in both case the ResNet uses a gaussian readout\n\nLIMITATIONS:\n- It is important to note that, while the trick used to make the classifier guidance works on models not trained on noise does allow for more flexibility, the encoding model still has to be trained on images from the same dataset as the diffusion model, as the approximate clean sample $x_0$ falls within the data distribution of the diffusion model and not necessarily of the encoding model. If not, the $x_0$ will be o.o.d to the encoding model and its gradient will be significantly less meaningful.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes using the prior implicit in a diffusion model as a regularization term when generating images that maximally excite neurons. The authors refer to this as “Energy Guided Diffusion” and compare this to a standard gradient assent procedure with a smoothness prior.  The authors also introduce a new model consisting of a CNN with an attentional readout that is trained to predict the responses of biological neurons, which (unlike the comparison model) allows for spatial components to be weighted differently for each input stimulus. The authors conduct experiments on all combinations of these two model changes to see how the generated stimuli change, and analyze how the stimuli transfer across models.\n\nSTRENGTHS:\nOverall, I enjoyed this paper and found it accessible, interesting and a good combination of many ideas that are currently relevant for the NeurIPS community. \n* The authors incorporation of an attention module into the final layer of a model for neural predictivity is novel as a way to capture stimulus-dependent changes \n* Showing the MEIs replicate for the chosen neurons and have very similar properties even with different models (ie  Figure 3) is a nice validation of the MEI technique as way of interpreting neural tuning\n* I was excited to see the prior in a diffusion as a way to improve some of the neural synthesis techniques, as this is perhaps a bit more principled than some previous methods (i.e. smoothness priors, or GANs).\n\nWEAKNESSES:\nOverall, I think that the contribution of the paper is novel, however there are some weaknesses in the experimental design and the interpretation/presentation of the results. \n\nA) It is not possible to determine which change in the proposed model with an attention readout leads to better predictivity in Figure 2. This makes the rest of the results comparing the models difficult to interpret. Specifically, compared to the “Task-Driven” ResNet, the authors have changed (1) the core model architecture from a ResNet50 to a 4 layer CNN (2) the readout from the “Gaussian” readout to an attention readout and (3) the training task/dataset (the “task-driven” core of the ResNet50 is pre-trained on image recognition, while the entirety of the “data-driven” attention CNN is trained explicitly to predict the responses of V4 neurons). Which of these changes are critical for the performance increase? Are the changes due to the utilization of the attention readout, the fact that more parameters can be learned in the attention CNN, or some other change? \n\nB) Along the lines of the above, the authors talk about the attention change as being novel and providing a way to incorporate shifts in location of receptive fields, but there is no experiment showing that the learned attention readout *actually* learns to do this (it could be using the same location for every image). \n\nC) Perhaps my biggest concern – the authors discuss generating “most naturally exciting images” however the generated images are not natural. Calling these images “natural” is incredibly misleading, as there is an underlying diffusion model that has learned a specific distribution of image statistics. The generated images may be high-probability images given the prior learned by that diffusion model, but unless the diffusion model is a perfect model of the world these are not fully “natural” (and indeed, from the presented images they do not look natural). It is interesting to present images with different regularizations applied, however, in my opinion, these images should not be presented as “natural” and fully removing the paragraph on lines 262-268 (and also any associated references to “MENIs”) would only strengthen the paper.   \n\nD) In the discussion the authors note that the MEIs generated for the attention-readout model are more “concentrated” however, as far as I can tell, this seems to be just an observation and not quantified in any way.\n\nQUESTIONS:\nSome of my major questions were listed in the weaknesses section, but additional things would improve the paper. \n\n1)\tIn Figure 5b, I’m a little confused about how the increase in the energy scale (so a higher importance on the “maximization” component of the synthesis) starts to decrease for large values of lambda. Naively, it seems like this component of the loss should only get better when it is weighted more strongly. Is there a problem with the optimization (i.e. too large of steps from the energy gradients?) causing this decrease?     \n\n2)\tThe authors should potentially discuss other ways that people have “regularized” generated images and the limitations. For instance, in Bashivan, Kar, DiCarlo (2019) a TV loss was incorporated into the gradient assent procedure to encourage smoothness (similar to the gaussian blur on the gradients used here). Some of these other methods have directly trained models that incorporate some statistics of the “natural” world, such as GANs (Ponce et al. 2019 and follow-up work). Limitations of including priors into synthesis techniques have been discussed in Engstrom et al. 2019 and Feather et al. 2022 (specifically, the inclusion of a prior for generation can hide some of the model biases). \n\n3)\tWith regard to the limitations of priors mentioned above, the authors mention that the MEIs “look more complex and natural” and that EGG improves the “quality” of the generated images. Can we gain understanding from having things that “look” better? Or are these potentially just misleading us to finding things that *seem* interpretable? (This is maybe a bit more of a philosophical question beyond the scope of the work, but it is something I am sometimes puzzled by with regard to these neural generation procedures, especially when neural data on the generated stimuli is not present).  \n\n4)\tWhat sort of biases do we expect from the fact that these models are trained to only handle clean images? The authors state as a fact that neural encoding models are trained on responses to “clean” images, but this seems like it would bias the generation in specific ways.  \n\n5)\tHow much spread is there in the generation across multiple seeds? Specifically, how does the spread in Figure 3 compare to what is observed from multiple initializations within the same model?   \n\nMinor clarifications:\n\na)\tWhat are the training details for the pre-trained robust ResNet50 that is used (ie $l_p$ norm, $\\epsilon$ step etc)? And are multiple pre-trained versions used in the ensemble or is it just a different readout? \n\nb)\tHow is the feature space for this model 1024 dimensional? Are the activations spatially averaged?\n\nc)\tThe paper “Solving linear inverse problems using the prior explicit in a denoiser” by Zadkhodaie and Simoncelli came to mind as very relevant previous work, but I didn’t see it cited. \n\nd)\tIt might be helpful to have an additional line/equation around line 166 explicitly stating the generation steps in terms of $\\bar{x}_0$. \n\ne)\tIn Figure 6 it states that the distance is compared in “unit activations space”. Is this the same as the predicted neural response, or is it measured at an internal model stage?  \n\nTypos/confusions:\n\ni)\tLine 160 says that a constant value of lambda is used for the study, however this was searched over (I think) based on Figure 5? \n\nii)\tShould equation (5) and associated text be $\\epsilon_\\theta$?\n\nLIMITATIONS:\nThe authors have a section on limitations, however there are a few things along these lines that might be worth further addressing. \n* The authors do not validate the generated stimuli with neural recordings and the improvement in “quality” is only judged qualitatively or by looking at the predicted responses from other models. It is possible the generated stimuli with the diffusion generation or the attention model do not perform as well at controlling the neural data due to underlying biases in the sampling etc.  I understand that this would require additional neural experiments, however it is worth mentioning this as a limitation and I didn’t see it discussed. \n* Are there possible negative societal impacts of the work?", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}