TAVAE: A VAE with Adaptable Priors Explains Contextual Modulation in the Visual Cortex
Authors: Balázs Meszéna
Keith Thomas Murray
Julien Corbo
O. Batuhan Erkat
Márton A. Hajnal
Pierre-Olivier Polack
Gergo Orban
Venue: ICLR 2026 Poster (2026)
Keywords: VAE
generative models
biological vision
neuroscience
Primary area: applications to neuroscience & cognitive science
TL;DR: We introduce a data-efficient way to adapt VAE priors, allowing us to explain both passive viewing and task-driven neuronal activity in mouse visual cortex.

Abstract: The brain interprets visual information through learned regularities, a computation formalized as performing probabilistic inference under a prior. The visual cortex establishes priors for this inference, 
some of which are delivered through widely established top-down connections that inform low-level cortices about statistics represented at higher levels in the cortical hierarchy. 
While evidence supports that adaptation leads to priors reflecting the structure of natural images, it remains unclear if similar priors can be flexibly acquired when learning a specific task. 
To investigate this, we built a generative model of V1 that we optimized for performing a simple discrimination task and analyzed it along with large scale recordings from mice performing an analogous task.
In line with recent successful approaches, we assumed that neuronal activity in V1 can be identified with latent posteriors in the generative model, providing an opportunity to investigate the contributions of task-related priors to neuronal responses. To obtain a flexible test bed for this analysis, we extended the VAE formalism so that a task can be flexibly and data-efficiently acquired by reusing previously learned representations.
Task-specific priors learned by this Task-Amortized VAE were used to investigate biases in mice and model when presenting stimuli that violated the trained task statistics. 
Mismatch between learned task statistics and incoming sensory evidence showed signatures of uncertainty in stimulus category in the  posterior of TAVAE, reflecting properties of bimodal response profile in V1 recordings.
The task-optimized generative model could account for various characteristics of V1 population activity, including within-day updates to the population responses. Our results confirm that flexible task-specific contextual priors can be learned on-demand by the visual system and can be deployed as early as the entry level of the visual cortex.

Published as a conference paper at ICLR 2026
TAVAE: A VAE WITH ADAPTABLE PRIORS EXPLAINS CONTEXTUAL MODULATION IN THE VISUAL CORTEX
Balázs Meszéna's Keith T. M.Mrrayry222 :2: Julien Corbo", O. Batuhan Erkat Márton A. Hajnal',
Pierre-Olivier Polack2 R & Gergö Orbán'
Department of Computational Sciences HUN-REN Wigner Research Centre for Physics Budapest, Hungary
Princeton Neuroscience Institute, Princeton University, Princeton, NJ, US
3, Center for Molecular and Behavioral Neuroscience, Rutgers University, Newark, NJ, US (meszena. siteszenabblazza hajnal majnal.marton marton, orban. gergo) @wigner. hun-ren ren. hu, km3199 wrreee@@rrinooooooeommm.. edu,
cerkat @uunyopt.
polack.por rut.gers.eds
ABSTRACT
The brain interprets visual information through learned regularities, computation formalized as performing probabilistic inference under a prior. The visual cortex establishes priors for this inference, some of which delivered through widely established top-down connections that inform low-level cortices about statistics represented at higher levels in the cortical hierarchy. While evidence supports that adaptation leads to priors reflecting the structure of natural images, it remains un- clear if similar priors can be flexibly acquired when learning specific task. To in- vestigate this, we built a generative model of V1 that we optimized for performing a simple discrimination task and analyzed along with large scale recordings from mice performing an analogous task. In line with recent successful approaches, we assumed that neuronal activity in VI can be identified with latent posteriors the generative model, providing opportunity to investigate the contributions of task-related priors to neuronal responses. To obtain a normative test bed for this analysis, we extended the VAE formalism that task can be flexibly and data- efficiently acquired by reusing previously learned representations. Task-specific priors learned by this Task-Amortize VAE were used investigate biases in mice and model when presenting stimuli that violated the trained task statistics. Mismatch between learned task statistics and incoming sensory evidence showed signatures of uncertainty in stimulus category in the posterior of TAVAE, reflect- ing properties of bimodal response profile in VI recordings. The task-optimized generative model could account for various characteristics of V1 population activ- ity, including within-day updates the population responses. Our results confirm that flexible task -ask-specific contextual priors can be learned on-demand by the visual system and can be deployed as early as the entry level of the visual cortex.
INTRODUCTION
Deep learning models, including discriminative and generative models, have been shown to success-
fully model neuronal responses in the visual system the brain (Khaligh-Raal Razavi & Kriegeskorte
2014 Yamins & DiCarlo 2016 Lotter al.) 2020 Csikor et al. | al. (021). These
models were assessed through the efficiency of predicting neuronal responses to natural images or natural videos. However, visual cortical responses
not only determined by the stimulus itself but also by non-stimulus attributes, such as the task the visual system is faced with (De Lange et al.) 2018 Lange & Haefner] 2017 2022). Notably, recent studies have demonstrated strong and sys- tematic task-specific biases as early as the carliest stage of the visual cortex, the VI (Corbo et al.
"These authors contributed equally.

---

Published as a conference paper at ICLR 2026
2022 2025).
Understanding these systematic biases requires that understand the computational principles behind the changes occuring at the early stages of processing when learning a novel task.
A coherent normative framework for biological perception is probabilistic inference: learning is assumed to deliver a generative model of the environment, in which latent generative factors are inferred when stimulus is observed (Yuille & Kersten 2006 Fiser et al.. 2010). In this context, neuronal activity in the visual cortex is interpreted as a representation of the posterior either by es- tablishing point estimate (Olshausen & Field 1996 Schwartz & Simoncelli] (001), approximating it through sampling (Lee & Mumford 2003 Orbán et al. 2016)), or through variational approxima- tion (Ma et al., 2006). and biases can be formalized through learned priors. The entry stage to the visual cortex, the VI, learns elementary features of the environment (Hubel & Wiesel 1959), suggesting a latent representation that has close to linear relationship with the stimulus, i.e. sug- gesting linear generative model underlying inferences (Olshausen & Field 1996). The hierarchy of visual cortical regions permits a prior over the features represented in VI, as hierarchically higher layers that represent more complex statistics can establish contextual priors for lower layers through top-down connections (Lee & Mumford, 2003). Indeed, animal experiments established that higher level cortical activity delivers rich, structured influence on V1 (Lee & Nguyen 2001; Chen et al., 2014 Kok et al., (2016] Ziemba et al. (019) and these influences were shown to be aligned with the contextual priors acquired by a hierarchical generative model trained end-to-end on natural images (Csikor et al. (025).
While these contextual priors reflect the regularities of the natural environment, these do not en- compass more specific regularities that can arise when learning a task. In fact, it remains an open question if task-related contextual priors can be established in V1. In this paper we test the hypothe- sis that inference in a generative model that acquires task-specific priors underlies systematic biases in the V1 of ausk-traindd mice.
To build a model of V1 that is primarily shaped by natural image statisties but is adapted to a task, we adopt a Variational Autoencoder approach (Kingma & Welling 2013 Rezende et al. 2014) as these can perform flexible inference and thus provide an opportunity to investigate task-induced bi- ases in the posterior. In this model, the latent variables of a Variational Autoeneoder are identified with neurons of V1. Correspondence between the experiment and the model is established through matching the response properties of biological and model neurons. Latent activations are assumed to correspond to membrane potentials of neurons and firing rates obtained by calculating the magnitude of responses (absolute value). We train the VI model of VAE on natural images. To ob- tain representations matching that of V1, this VAE is constrained to have linear generative model (Geadah et al. 2024), and use an extension that both produces more consistent performance in infer- ence and better matching of contrast-dependent VI responses (Catoni et al. 2024). Starting from the VAE model of VI, we extend the VAE formalism to adopt specific tasks, by adapting the variational posterior. In a standard approach, obtain new amortized variational posterior, the VAE needs to be retrained. This is neither efficient in terms of data requirements, nor biologically plausible, as it could steer away from a crucial representation obtained during the animal's development. Instead, we are seeking way to build upon a learned representation when adapting to new context. We propose a principled extension of the VAE formalism that is capable of flexibly learning contextual priors. The Task-Amortized VAE (TAVAF) reuses the likelihood of the task-general VAE and ob- tains task-specific posteriors without retraining the original amortized posterior. We train the TAVAE on discrimination task matching task mice were trained on while recording V1 activity (Corbo et al. 2022] 2025) to assess the predictions of TAVAE about response biases. Exploring the biases of learned prior is made possible by the specific experimental design. Extensive training ensures that the animals learn the simple regularities of the task. In post-training test sessions, however, the learned stimulus distribution is systematically violated, exposing the effects of task-specific (con- textual) prior. V1 neural activity is recorded through calcium imaging in 10 mice across 6 sessions, yielding recordings from 15.027 neurons while the animals perform the discrimination task.
In this paper, we first introduce the theoretical background for TAVAE, Second, we describe the animal experiment along with the specific generative model for VI. Third, we investigate the basic properties of a lask-trained contextual prior and its consequences on population responses in V1. Fourth, we identify signature of uncertainty about potential choices, multimodal responses emerg- ing as a consequence of mismatch between stimulus and the contextual prior, which we identify in population responses. Fifth, we investigate how updating the contextual prior reshapes response biases and show qualitative agreement with the transformation of population responses within an
2

---

Published as a conference paper at ICLR 2026
experimental day when a new stimulus is introduced. Finally, we illustrate how the contextual prior and the stimulus likelihood can be inferred from V1 population responses and show that the inferred likelihood resembles the population activity recorded in animals with exposure to the discrimi- nation task.
2 THEORY
Theory of Task-Amortized VAE. If a latent-variable generative model (e.g. a VAB) is trained on natural images, it can acquire a representation that reflect useful features of this environment. A VAE also learns a powerful tool for inference, the amortized variational posterior. When learning a task, the learned features can be useful for task execution even though the learned features do not exactly match the task variables. For instance, learning about natural images yields Gabor filter-like representation, which can be easily queried about stimulus orientation. Here we develop a formalism, TAVAE, for task learning that capitalizes on the learned representation and also reuses the variational posterior. This model represents task-specific structure in task-adapted prior. The formalism permits learning a prior through observing a limited number of observations from the task being performed. Here apply the formalism to simple setting where the V VI-like representation used to learn an orientation discrimination task.
Given distribution of images, Po(x), (z), we want to learn about this distribution by learning latent representation 2:
po( f p(<z (2) dz,
In general, inference about latent features, p(2 x),, in such model is intractable. Variational Au- toencoders (VAEs) (Kingma & Welling 2013 Rezende et al.] (014) have been proposed to provide variational approximation. For this, lower bound to the log empirical distribution is optimized, called the evidence lower bound (ELBO). Through the optimization of the ELBO a pair of models is learned: the likelihood, p(æ =). termed the generative model, and the (amortized) variational posterior, g( x), termed the recognition model.
an approximate posterior, q(> æ), relying
We seek to establish method to perform inference in computationally efficient way even if the data generating distribution changed from the natural distribution (x to task distribution (z))). By default, adapting a VAE new data requires retraining both the generative and recognition models from scratch by re-optimizing the ELBO. However, in many cases variation in the new dataset that occurs in the task is limited, thus the efficiency of optimizing the original set of parameters is also limited. Instead, we seek a computationally efficient way of adapting to different tasks by reusing the originally learned neural networks. We propose that the latent representation learned by the original VAE is retained as these learned latents can establish useful feature space for the task. Consequently, we propose that the new generative model retains p(x z) in the new context and the change in the stimulus distribution is solely induced by change in the latent prior (z))::
pr( = p(ax (z) dz.
(2)
The task prior, (z),,, can also be thought of as the marginal of higher level latents in a hicrarchical model, e.g. mixture of options, o, and thus pr(2) PT o) p(o). In this generative model, the new posterior is obtained by applying Bayes rule:
(r(= ar) Po(z ()) 1
(3)
(2(2)
N(x)²
where N(x) is normalization constant We can approximate the true posterior, po (z) in Eq. with the variational posterior, q(> x) coming from the trained VAE.
Eq. 3 highlights that once know the prior related to the task. posterior to obtain the posterior under the prior associated to the task. To achieve this, we seek to maximize the log-likelihood under the latent prior, pr(2) using observations from the task Xy:
use the natural variational
log(pr (m)) = Σ log / dz p(x 2) pT(2)
L
Σ XEXT
(4)
xEXr
Details of the calculations can be found Appx A.I

---

Published as a conference paper at ICLR 2026
a
0.04
Vreleree trieetaaiion
9.03
62-
-9.02
APYERTI
b
-0.01
120-
120
150
0.00
350
Node
180
-0.01
0,02
Time 150
Adtivty
gfz .x)
plx
30
60
gylz L
90
02-
120
122
150
150
180
180
0.50
0.21
0.50
0.75
Populacion Activity
Figure 1: a. Experimental setup. b, Go-NoGo visual discrimination task stimuli with training and testing schedule. c, Population response map of the recorded V1 population on DI for the Go stimulus. d, Population response profile averaged over time for trained and untrained (naive) mice. e, Cartoon of VAE (top) and TAVAE (bottom) models. The TAVAE generative model component of VAE (pale orange) is reused in TAVAE whereas the recognition model of VAE (dark orange) is transformed through reweighing the variational posterior with the task-prior (Eq. 3 f, Illustration of the posterior of two latent dimensions; inset: receptive fields of latents. g. Task prior of TAVAE for the discrimination task. Scale of the Laplace prior is shown. h, TAVAE responses to the Go stimulus with task (green) and natural priors (blue).
VAE model of V1. To be able test the predictions of task-related contextual priors in biological recordings, we first built VAE end-to-end trained on natural images that reflected basic properties of V1. In our study we take normative approach by seeking correspondence between generative model of task-adapted model V1 and neural recordings from VI. This way, optimization is based jointly on natural image statistics and task structure not on neural recordings.
Latent variables, 2, of the generative model were identified with neurons of V1. This is motivated by the similarity of learned representations to the sensitivities of neurons in V1 (Geadah al. [024] Catoni et al.) 2024 Olshausen & Field 1996). Before specifying the model, we elaborate the correspondence between the VAE and neural activity. Upon presenting an image, the VAE returns variational posterior. Many early accounts identified neuronal responses with the MAP estimate OOlshassen & Field 1996 Schwartz & Simoncelli (2011), which was justified by the fact that neural recordings report response means. Alternatively, the responses of neurons can be identified with a sampling approximation
the posterior, which assumes that temporal variations in responses correspond to sample variance from the posterior (Hoyer Hyvärinen, 2002 Orbán et al. 2016 Echeveste et al.) 20200 Festa et al. [222 SShrinivasan et al. 2023). We do not commit to particular representation this point, but distinguishing the two can the subject of later investigations.
We trained a VAE on natural image patches. These patches were 40-pixel whitened crops from the van Hateren database (Van Hateren van der Schaaf 1998). The VAE had four characteristic features. 1, Following an earlier VAE study (Geadah et al. 2024).. the generative model was linear, analogous to independent component accounts of VI organization (Olshausen Field 1996): 2, The latent space was large dimensional (1799 dimensions) mimicking the complete overcomplete structure of VI and ensuring that the linear generative model can accommodate the information in the observations; 3, The prior over latents was Laplace, motivated by efficient coding considerations, and also aligned with earlier accounts (Olshausen & Field, 1996). This condition is necessary to match the latents of the VAE with V1 neurons as this ensures learning Gabor-like receptive fields, a key property of V1 neurons (Csikor et al., (022);; 4, A scaling latent was introduced, mimicking the Gaussian Scale Mixture (GSM) family of models of natural images (Wainwright & Simoncelli
4

---

Published as a conference paper at ICLR 2026
This choice is motivated by the joint statistics of natural images, the superior performance of GSM's to predict V1 responses, and an earlier study which showed that this extended version of reliable inference, especially in lower contrast regimes (Catoni et al. 2024) (in Appx. B.9 we compare the main results with a model based on a VAH without the scaling factor). In summary, the generative model of this baseline model was P. 2; (ax; exp(s) Az, oz,o²T] I),
1999).
VAE ensures
p=(=) = Laplace(z; z; 0, 1), p(s) = Ns; 0, 1), where exp(s) ensures that the scaling is positive. Based on this generative model, we optimize the negative ELBO:
[(x, p, (b) (x ( s)] + DKL (go (= x)) ())))))))))))))))))))))))))))l)) + (s)xa)
(5)
such that the variational posteriors (= x)) and qu x)) were modeled Laplace and Normal distributions, respectively (we will omit the subscript from now on). The latent variables of this VAE can take positive and negative values too. Similar earlier accounts, is interpreted membrane potential responses of biological neurons (Orbán et al.) 2016 Echeveste et al., 2020). To obtain firing rate-like responses, we use the response magnitude (absolute value) of latent activations when comparing to biological data, similar to Csikor et al. (2025); Geadah et al. (024). In contrast with applying a ReLU for positive outputs, this avoids clipping the representational space.
Orientation discrimination task for mice and machine. We characterized the biases emerging after adaptation to a task in mice and in TAVAE model by studying an orientation discrimination task. For this we used identical task stimuli in experiment and model.
Mice performed a visual Go-NoGo task in which animals were required to lick for a 45% or withhold from licking for 135" moving grating stimulus (Fig. [T)) (Corbo et al., 2025). Animals were first trained on the orientation discrimination task until reaching proficient performance, after which they underwent dix-day testing period accompanied by neural population recordings. During the testing period the Go stimulus remained identical to the training phase but the NoGo stimulus systematically violated the trained task statistics. Specifically, the NoGo stimulus was progressively shifted toward the Go orientation, reducing the angular difference to 15t by Day 6 (Fig. TD).
We used a pair of oriented grating stimuli train the prior of TAVAE (in the baseline experiment the pair of orientations were 45 and 135%). In order to emulate moving gratings, each oriented grating was presented 50 different phases. Response of model neurons was established as the average of the absolute value of the posterior means.
We trained task spsc-fpecific priors on this stimulus set. The task prior, (z),), corresponds to a mixture of the the two stimulus categories and we seek to learn the marginal of this distribution (Eq. Acquiring the task was modeled by learning this marginal prior, pr(z), according to Eq. 12
To learn the approximate posterior for the (discrimination) task, we applied the TAVAF formalism. We approximated the prior with a diagonal covariance matrix. Since the variational posterior (and
also the prior) is factorized q(z, s x) q(s xx) g(z x)) over and z, Eq. 3 translates to:
4T ()) x ((=)))(T(Z))
(0(2)
where Pr(z) was the prior over latents defined by the task. We search for an appropriate task prior as a zero mean Laplace distribution. The choice of constraining the mean to zero is motivated by a symmetry of activations around zero across all the phases of the grating stimulus:
pr (z) = Laplace(z; 0,g,)
II i=1
exp
20T
TT.
Under these settings, training the task prior amounted to learning the variances, dr. We show in Appx A.. that the minimization of the general task loss Eq. simplifies to solving the self- consistency equation (see B. well):
1
Σ E Epr Φ)) ||=1|] xEXT
TT,
n
In summary, the model of task-adapted VI is trained jointly on natural images and the task stimuli but it not directly optimized to fit neuronal responses.
5

---

Published as a conference paper at ICLR 2026
Characterization of responses in mice and model. We analyzed calcium imaging recordings from excitatory neurons in layer 2/3 of mouse V1 during the test phase of an orientation discrimina- tion task. Neural activity summarized using population response profile, in which neurons are ordered by their preferred orientation (measured in separate tuning block) and responses are aver- aged across time and neurons. Population profiles were pooled across animals reduce recording noise and are visualized with error bars denoting 95% confidence intervals of the mean, as specified in the figure captions (see Appendix B.3 for details).
To obtain an analogous measure in the model, we identified each latent variable with a putative VI neuron and estimated its orientation tuning curve using responses to grating stimuli. The majority of latent components were orientation selective. Latent variables were ordered by their preferred orientation and population response profiles were computed using 5 orientation bins. Variability across model neurons is shown as 95% confidence intervals estimated by bootstrap resampling. Full methodological details are provided in Appendix [B.4]
To quantify the predictive power of the model, we calculated the Pearson correlation coefficient (r) between the experimental and model population response profiles in Table (aggregaee in 5 bins, as in Fig. (3. To specifically assess the effect of the task, we calculated the cooteetuul modulation' of the response profile, the difference in response profiles of task-engaged and naive animals, and analogously,
the difference between response profiles of TAVAE and VAE (no prior change). We also performed neural predicitivty and Centered Kernel Alignment analysis for across-image alignment of model and experimental neural responses (Appx. (B.0).
3 RESULTS
Effect of contextual prior for task-congruent stimuli. First, tested how task-specific training influenced model inference and compared the effects of contextual priors in task-trained versus naive animals. We first focused the first recording session (I)1), in which the test stimuli matched those used during training. To align the model with the experimental paradigm, we synthesized 45 and 135" gratings; we varied phases (Appx. B.I to emulate motion, and fit the task prior using Eq The optimization converged after five iterations (Appx. Fig Using the acquired prior, we computed tuning curves and subsequently population response profles. The tuning curves showed systematic modulation: neurons whose preferred orientations flanked the orientations on which the prior was trained were suppressed (Appx. Fig. 6 Population response profiles for both trained stimulus orientations showed clear peaks (Fig. bb.)) We used a reduced contrast stimulus for testing, in line with the experiment data (Corbo et al. 2025) (in Appx.
B.9 we show the contrast dependence of
some of our results).
To understand the effect of a contextual prior, we contrasted the task-optimized TAVAE response profiles with the task-general, standard VAE profiles (Fig. 2b.) Similarly, we compared experi- mental response profiles recorded in sessions when the Go and NoGo stimuli were identical with training stimuli (450 and 135", DI) with response profiles recorded in naive animals. The task- optimized contextual prior resulted in sharpening of the response profile (Fig. 2 t-test, n=10 grat- ing phases). Similar sharpening was evident in mice when analyzing population activity (Fig. 22.. t-test p=0.01, n=60, means from 6 days, two stimuli and autocorrelation-coreee
to 5 independent timeframes). Sharpening in TAVAE can be understood as suppressing the responses of neurons with preferred orientations not matching the task orientation (Appx. Fig 6
Another signature of task-optimized generative model is significant reduction in baseline activity, which we measured by calculating the average activity in the (0, and (150, 180) orientation window. Upon learning the task, the selective suppression of neurons that have preferred stimulus orientation not matching the stimulus (Appx. 6 results in a suppressed baseline (Fig. 2 t-test n=12, means from days, two stimuli). This suppression consistent across stimulus conditions in the experiment (Fig. fe. as well.
In the task-optimized TAVAE, suppression of baseline activity is not homogeneous: when presenting a task stimulus, suppression does not affect neurons tuned to training stimulus orientations (Fig. 2h. This can be explained by the heterogeneity of the scale of the task prior, TT (see Fig 2g. 7 along directions where the prior is wider (the trained orientations), the posterior will be less affected, sulting in baseline activity similar to the task mask-general VAE. When considering experimental record- ings, the magnitude of the heterogeneity in baseline suppression is close the level of fluctuations.

---

Published as a conference paper at ICLR 2026
a
b
52.
Posdtitlnn Acrutyy
2.0
0:20
8.8
Suxelive
24
26-
0.22
63
90
120
150
180
30
120 Odfertation (7)
60
50
350
180
Orientar on (7)
hatre
Tatt
h
9.03
Vaten
McGo Natee Tistted
So. The
0.02
70
Nean Broetne Actwvit
0 0.000
3.03
0.002
0.00
-0.002
150
20
Orforationn"
Orfentation
Natve
Trahrry
Figure 2: Effect of learning the discrimination task in the model and in the experiment. a, b, Response profiles to task stimuli (45e and 135% gratings, red and green dashed lines, respectively) using natural prior (blue) and the task prior (solid green and red for Go and NoGo respectively) in the model, smoothing 3". Dashed grey lines correspond to the learned contextual prior. Shading: bootstrap estimate of 95% confidence interval of the mean across model neurons. c, Width of the 45 peak using the natural prior (blue) and task prior (orange; statistics are computed from different grating phases). d, Baseline reduction in TAVAE. Baseline activity was computed for each stimulus configuration from Fig. bb. and Fig. 3p. Box plot and significance markers are based on statistics computed across these stimuli. f. DI Go and NoGo responses in naive (blue) and trained (green, red) mice, smoothing 6%, shades 2 sem. g. Width of the Go response over days and timeframes for naive (blue) vs. trained animals (orange, mean over neurons, trials). h, Baseline reduction following training in the experiment. Baseline activity was computed for each stimulus configuration from Fig. f and Fig. 3a. Boxplots: 25-75% whiskers 1.5-IQR. Asterisks indicate statistical significance.
Although the predicted inhomogeneity in baseline suppression can occasionally be experimentally observed (see the bump around 135 in Fig. 2b), noise in recordings prevents a general conclusion.
Systematic biases of the contextual priors for OOD stimulus. Across five experimental ses- sions, animals were exposed to stimuli that systematically deviated from the trained stimulus set. More precisely, the NoGo stimulus gradually converged to the Go stimulus (Fig. [b). Population response profiles from these sessions show drastic deviations from the response predicted merely by the stimulus orientation (Fig. h). In particular, the peak of the population response profile is distinct from the actual stimuli in multiple sessions across D3-D6, as measured by the mean abso- lute difference between peak estimate (200 x bootstrap 5 random cells per orientation, is higher for trained animals compared to naive, p 10-6 one-tailed t-test, see also Fig. 3p). Furthermore, multimodal responses are characteristic in multiple conditions (especially in D4 and D5). Coun- terintuitively, multimodal responses display troughs the actual stimulus orientation, with peaks flanking the stimulus orientation from both sides (higher response at the bimodal peaks compared to the troughs in D4-D5: flanking orientation peaks to stimulus trough, p < 10 one-tailed t-test; n x 2000-200000 200002 for the 5-20 cells in the respective 1 orientation bins, Fig. Bb).
We assumed that these biased responses are a consequence of the uncertainty about the source of the stimulus when the contextual prior does not match the observation. Indeed, when simulating experimental population responses across the five conditions (corresponding to experimental days) such that the contextual prior matched to the stimuli (corresponding to the mice learning new prior every day), no distortions were observed (A (Appx. Fig. (4). Also, this 'cager adapter' version of TAVAE predicted experimental population response profiles poorly (see i for details).
To investigate alternative hypotheses, we assessed TAVAE responses under two additional assump- tions. First, assumed that the contextual prior acquired during the extensive training is retained, ie. we assumed a prior at 45 Go and 135 Nogo stimuli (Appx. Fig. (8p). The population re- sponse profile showed very weak bimodality and, accordingly, showed relatively weak correlation
7

---

Published as a conference paper at ICLR 2026
TAVAE (45, 90)
VAE
TAVAE (45, 135)
TAVAE (EAGER) 0.53 0.11
r (TASK))
0.78 = 0.02
0.53 0.12
0.54 + 0.10
r (HASK-NAIVE)
0.58 + 0.09
0.32 = 0.17
-0.10 = 0.23
Table 1: Correlation between model and experimental population activity across orientation bins for different TAVAE configurations and the naive VAE (averaged over stimuli). The standard eπror of the mean is also shown. Top row: correlation between task-engaged experimental data and model predictions. Bottom row: correlation of maskllodultiion of population response profile (difference between task-engaged and naive conditions) between model and experiment. The analysis was per- formed on orientation bins within the (30", 1059) window, thereby excluding orientations far from the stimuli presented on those days where measurement noise is more dominant. For orientations distant from the stimulus, the model predicts primarily the baseline reduction shown in Fig. 1..h
between TAVAE and experimental results (Table Second, we reasoned that the abrupt change in the NoGo stimulus from 135" to 90" on D2 likely induced update in the prior. This interpreta- tion is supported by the observation that the false alarm rate of the animal on Day 2 is comparable to that on Day (Appx. Fig. Pa), indicating that the animal is adapting the new NoGo stimu- lus. Notably, the false alarm rate further decreases across behavioral quintiles within Day 2 (Appx. Fig. In contrast, from Day onward, the false alarm rate steadily increases (Appx. Fig. b) (in- dicating that the animal is not adapting further). Confirming this ussumption, a prior reflecting 45t Go and 90º NoGo stimuli reproduces the systematic biases observed in the experiments (Fig. 3b): reprodd
the population response peak shifts away from the actual NoGo stimulus, as observed experimen- tally during D3-D6, and bimodality is present, with a local minimum near the stimulus orientation, for the NoGo stimulus under the conditions used during D4. Indeed, for the shifted peak of the response profile we found that the absolute difference between the peak position and the stimulus orientation was significantly larger in the TAVAE than in the naive VAE for all stimuli except 90" where no ptior-stimulus mismatch was present (Appx. Fig. [0 and see Appx. [.8). To test whether the population response profile displayed bimodality with a dip at the orientation corresponding to the NoGo stimulus, we identified the positions of maximal population activity and compared to the activity at the stimulus orientation. For the 702 stimulus (corresponding to D4), the flank maxima exceeded the activity at the stimulus position on both sides (one-tailed t-test, left: p = 0.001; right: p 0.028; Asimmulus 48, T4 32, Fright 9), indicating the presence of a local minimum be- tween the two flanking orientations. Samples were drawn from individual model neuron responses within the orientation bins used for the comparison. Beyond the presence of signatures of system- atic biases, we also assessed the performance of TAVAE in predicting the experimental population response profile. When assuming a 45 Go and 90º NoGo prior, the correlation between TAVAE prediction and experiments exceeded alternatives (Table including the task-genera VAE. We fur- ther characterized the aligmment of model predictions with experimental recordings through Neural Predictability (Naycbi et al. 2023) and Centered Kernel Alignment (Kornblith et al. 2019) analyses (Appx. 10. These analyses confirmed that TAVAE across-stimulus predictive power exceeds that of the task-general VAE (Tables|4
To explore the contribution of model choices in TAVAE, we performed experiments with altered contrast levels (Appx. Fig. [1 and ablating the scaling variable from the baseline model (Appx. Fig. 12, see also Appx. B.9). Analyses showed that the decreased contrast contributed to better model fits (Table 2). Intuitively. this lowered contrast increases uncertainty, which has a defining role in rendering alternative options similarly viable. Note, that lowered contrast was inspired by matching the experimental settings (Corbo et al. 2025). Our analyses also highlighted that the GSM-style scaling was contributing to slightly better model fits (Table 2 but integrating TAVAE with a VAE without scaling also delivered significant correlations with the experimental data.
Signature of updating the contextual prior. Our results indicate that DI responses are consis- tent with a contextual prior peaking at 45 and 135". Further analysis of D2 through D6 sessions indicated a different contextual prior peaking at 45 and 90", Taking these findings together, animals seem to shift their priors from the first session (D1) when stimuli are identical with training stimuli to the second session when the NoGo stimulus is radically updated. Behavioral results indicate an adaptation during D2 (see the gradual reduction in the false alarm rate as the day progresses in Appx. Fig. 9p), motivating analysis of population activity throughout the experimental session of DD2.
8

---

Published as a conference paper at ICLR 2026
stimulus: 90"
stimulus: 75*
D4, stimulus: 70"
DS, stimul
D6, stimalus: 60
Population Acitkky
0.02
9.11
0.00
-0.01
45
90
135
180
45
90
135
180
45
90
135
180
45
90 Dranitatio
135
180
45
90 Orientaiton
135
180
Orientation (?)
Drientationn (?)
Ortentation I"1
(*)
b
1.0
Popuanter Actiityy
0.8
0.6
0.
0.0
150 Orientation(")
150 Oriertation
150
120
150
Orientation
Orientation
Oriantation
Figure 3: Systematic biases for mismatched training and test in experiment and model. a, VI responses over five experimental sessions during which the NoGo stimulus (red) is deviating from the training NoGo stimulus at 135%, confidence shades 2 sem. Responses from naive animals are shown for reference (blue). Bimodal peaks-to-middlle -idddde---ooughh activity differences (vertical black arrows). Mean absolute difference between stimulus and peak orientation (horizontal red and blue mirrored half arrows). b, Model responses on the same stimulus orientations as in the experiment across the five sessions. prior learned for 45 and 90 gratings assumed (grey dashed lines), bimodal coo--idddle-troup without smoothing (vertical black arrows). All panels: data points, smoothing and confidence intervals as Fig. asterisks indicate statistical significance (see text).
We model the shifting prior by fitting the task prior to a dataset with gratings characterized by unbalanced orientation classes. Namely, we investigated a dataset that contains 45 90% and 135 gratings with the ratio 1: y 1 7- We obtained contextual priors for (0.1, 0.25, 0.5, 0.75, 0.9) and calculated population response profiles to the D2 NoCio stimulus, oriented at 90%
Population response profile of high- 135 priors (i.e. Y 0.5) reveals two additional modes flanking the central stimulus-aligned mode (Fig. 4h.. These correspond to (shifted) modes con- tributed by the alternative hypotheses, i.e. that the stimulus is coming from the 45 and 135" com- ponents of the prior. A low weight of the 135 stimulus (high 7) in the contextual prior results in asymmetrie flanking modes whereby the mode corresponding the 45 component is greater than the 135 (Fig. 4b). For the example pair of ys (0.25 and 0.75, the asymmetry is found to be greater at 7 = 0.75 than for 7 = 0.25 (t-test, p < 10-6, n 806; sample pairs were constructed by sample individual model neurons from the two bins). In fact, if the prior is updated from D1 to D2, we expect a gradual decline in the flanking peak closer to the NoGo stimulus.
To test this prediction in the data, stratified D2 trials into five quintiles and analyzed population response profiles separately. Flanking modes were consistently present both early and late in the session (Fig. 4e). Further confirming hypothesis, the late trials display asymmetric flanking modes. Namely, as the session progresses from early trials (quintile 1, Q1) to late trails (Q5), the difference between Go and NoGo flanking peaks widen as shown in Fig/4 (t-test, p < 10-6 with n=1685 equal random sample at the 45 and 135 orientation bins, with number of trials 10-30). This shift is also reflected behaviorally by a gradual reduction in false alarm rates across quintiles (Appx. Fig. (b).
Importantly, expected from theory, the flanking modes are 'attracted" towards the orientation of the NoGo stimulus, i.e. to 90", in line with the formation of an orientation posterior by combining contextual prior over orientations with a stimulus-driver likelihood. We argue that shifts of 45 and 135" modes towards the NoGo stimulus on D2 is a signature of probabilistic inference under uncertainty: a wide likelihood combined with finite-width prior results shifted posterior. We then use this insight to estimate the likelihood and the orientation prior from the position of the modes from the population response profiles (see Appx. B. for derivation). The inferred likelihood is substantially wider than the orientation prior (Fig. 4e.f. F-test p<10->). which is consistent with the relatively small shift of the modes away from the originally trained stimulus orientations. This wide likelihood resembles the population response profile observed in untrained animals (F-test p>>0.13). In untrained animals, where the orientation prior is effectively flat and cannot sharpen the posterior, the population response profile provides close approximation of the likelihood itself.

---

Published as a conference paper at ICLR 2026
Figure 4: Signatures of updating a contextual prior. a. TAVAE population response profiles under a mixture con- textual prior, where the weight (7) of the 90º components is increased from 0.25 to 0.75. b. Progression of the heights of flanking modes with changing mixture components simulating the progression of the trials across the 90º stimulus ses- sion for TAVAF. Error bars: 95% CI for the mean estimate e, Population re- sponse profile on D2 when 90 stimu- lus is introduced, for early (Quintile I) and late trials (Quintile 5). d, as b but for experiment on D2. a-d: smooth- ing, confidence intervals as in Fig. 2 e. Orientation likelihood inferred from D2 QI population response profile (black, centered on 09) and population response profile of naive animals (blue, centered 0*). f, D2 prior inferred from popu- lation response profile.
DJ www.ee 1071
126
100
100
8:00
80
86
10
Ottertdoon (1
Sale souiee (1A7
120
Q2
0.2
0.33
0.3
0.0
FRied kallhood pH
-0222
Tor (etA 9-108
-100
50
300
30
93
120
150
380
4 DISCUSSION
Here we investigated the biases that emerge from task learning in a generative model and compared them to the biases observed in animals extensively trained on the same task. For this, we developed a variant of VAES, the TAVAE in which the prior can be flexibly adapted in computationally efficient way. This permitted the reuse of the representation learned for natural images. TAVAE could account for range of phenomena, including the sharpening of population responses with learning (without changing the receptive fields of neurons) and changes in baseline. Crucially, the model revealed bimodal responses, a hallmark of probabilistic inference under uncertainty, and this bimodality tightly aligned with that observed in VI when trained stimuli were not matched with actual observation. TAVAE could also capture the updating of the prior when the animal was faced with updated task contingencies.
TAVAE permits the explicit reuse of the recognition model when acquiring new task. Although we do not argue that this algorithmic solution is the one used by the neural circuitry. it provides insights into how the representation is reshaped through learning. The systematie distortions of neuronal responses were reported from layer 2/3 neurons, where bottom-up and top-down signals integrated. The laminar organization of the cortex might permit more complex computations, for instance layer 4 neurons might actually preserve the original recognition model to be modulated by top-down influences when the feed-forward input reaches layer 2/3. is a delicate question how a telkeee contextual prior is integrated with the contextual priors reflecting the regularities of natural images (Csikor et al.] 2025). The representational geometry of population responses might give clue about this (Lazar et al. 2024) but this remains an open question.
We chose the task prior as a simple modification of the natural prior, namely by retaining inde- pendence among latents and tuning only their variances. Interestingly, although the posterior in the high-dimensional latent space was unimodal, this still translated into a multimodal population response profile in the one-dimensional orientation space within certain parameter range. The TAVAE formalism capable of accommodating more complex priors with more flexible form. In particular, different modes in the latent space might correspond to different options. yielding multi- modal posteriors. Such priors can be learned through the posteriors associated with different options. In principle, the framework can also be applied to model contextual modulations in higher cortical areas. More broadly, deep generative models may become an important tool for modeling cortical activity under realistic conditions.
10

---

Published as a conference paper at ICLR 2026
Reproducibility statement. The mathematical derivation needed for the results be The code used for modeling be found in the Supplementary material. In the code, provided script that down- loads the model weights of the VAE used in the paper. Mouse VI activity data can be downloaded from htns://zenodoo.org/eeooor 09858, and used with the data analysis scripts: http://awwbbbooooooooooeeeeerrne --tasss-rioossss
found in the main text 2 and in the appendix A.1 A.2 B.5
Acckowleeggments. This work was supported by the European Union project RRF-2.3.1-211 2022-00004 within the framework of the Artificial Intelligence National Laboratory in Hungary (GO), the National Research, Development and Innovation Office under grant agreement AD- VANCED 150361 (GO), and the Fulbright U.S. Student Program (KM). The authors thank HUN- REN Cloud for providing compute support for the study (see Héder et al.] (2022); https://ccience cloud.hu/). We also thank anonymous ICLR reviewers for their constructive feedback and helpful suggestions.
REFERENCES
Josefina Catoni, Domonkos Martos, Ferene Csikor, Enzo Ferrante, Diego H Milone, Balázs Meszéna, Gergö Orbán, and Rodrigo Echeveste. Uncertainty in latent representations of vari- ational autoencoders optimized for visual tasks. arXiv preprint arXiv:2404.15390, 2024.
Minggui Chen, Yin Yan, Xiajing Gong, Charles Gilbert, Hualou Liang, and Wu Li. Incremental integration of global contours through interplay between visual cortical areas. Neuron, 82(3): 682-694, 2014.
Julien Corbo, John P McClure, O Batuhan Erkat, and Pherre-Olivit Polack. Dynamic distortion of orientation representation after learning the mouse primary visual cortex. Journal of Neuro- science, 42(21):4311-432 2022.
Julien Corbo, O Batuhan Erkat, John McClure Jr, Hussein Khdour, and Pierre-Olivier Polack. Dis- cretized representations in vl predict suboptimal orientation discrimination. Nature communica- tions, 16(1 (11114442240) 2025.
Ferenc Csikor, Balázs Meszéna, Bence Szabó, and Gergö Orbán. Top-down inference in an early visual cortex inspired hierarchical variational autoencoder. arXiv preprint arXiv:2206.000 00436, 2022.
Ferene Csikor, Balázs Meszéna, Katalin Ócsai, and Gergö Orbán. Top-down perceptual inference shaping the activity of early visual cortex. Nature Communications, 16(1):9995 2025.
Floris P De Lange, Micha Heilbron, and Peter Kok. How do expectations shape perception? Trends in cognitive sciences, 22(9):764-779, 2018.
Rodrigo Echeveste, Laurence Aitchison, Guillaume Hennequin, and Máté Lengyel. Cortical-like dynamics in recurrent circuits optimized for sampling-base probabilistic inference. Nature neu- roscience, 23(9): 138 113--149,, 2020.
Dylan Festa, Amir Aschner, Aida Davila, Adam Kohn, and Ruben Coen- DagllC Neuronal variability reflects probabilistic inference tuned to natural image statistics. Nature communications, 12(1): 3635, 2021.
József Fiser, Pietro Berkes, Gergö Orbán, and Máté Lengyel. Statistically optimal perception and learning: from behavior to neural representations. Trends in cognitive sciences, 14(3):119-130, 2010.
Victor Geadah, Gabriel Barello, Daniel Greenidge, Adam S Charles, and Jonathan W Pillow. Sparse- coding variational autoencoders. Neural computation, 36 36(12)2257--261, 2024.
Mihály Héder, Emo Rigó, Dorottya Medgyesi, Róbert Lovas, Szaboles Tenezer, Attila Farkas, Márk Benjámin Emödi, József Kadlecsik, and Péter Kacsuk. The past, present and future of the elkh cloud. Információs Társadalom: Tärsaaulomtudomány Folyoirat, 2282):128-137, 2022.

---

Published as a conference paper at ICLR 2026
Patrik Hoyer and Aapo Hyvärinen. Interpreting neural response variability monte carlo sampling of the posterior. Advances in neural information processing systems, 15, 2002.
David H Hubel and Torsten N Wiesel. Receptive fields of single neurones in the cat's striate cortex. The Journal of physiology, 148(3)5574, 1959.
Seyed Khaligh-Razavi and Nikolaus Kriegeskorte. Deep supervised, but not unsupervised, models may explain it cortical representation. PLoS computational biology, 10(11):e1003915 2014.
Diederik P Kingma and Max Welling. arXiv: 14, 2013.
Auto-encoding variational bayes. arXiv preprint
Peter Kok, Lauren Bains, Tim Van Mourik, David G Norris, and Floris de Lange. Selective activation of the deep layers of the human primary visual cortex by top-down feedback. Current Biology, 26(3):371-377 2016.
Simon Korbliihh, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural net- work representations revisited. In Proceedings of the 36th International Conference Machine Learning, pp. 3519-3529. PMLR, 2019.
Richard D Lange and Ralf M Haefner. Characterizing and interpreting the influence of internal variables on sensory activity. Current opinion in neurobiology, 46:84-89, 2017.
Richard Lange and Ralf M Haefner. Task-induced neural covariability as a signature of approxi- mate bayesian learning and inference. PLoS compulational biology, (8(3)ee100953 2022.
Andreea Lazar, Liane Klein, Johanna Klon-Lipok, Mihály Bányai, Gergö Orbán, and Wolf Singer. Paying attention to natural scenes in area v1. Iscience, 27(2), 2024.
Tai Sing Lee and David Mumford. Hicrarchical bayesian inference in the visual cortex. Journal of the Optical Society of America A, 20(7): 1448, 2003.
Tai Sing Lee and My Nguyen. Dynamics of subjective contour formation in the early visual cortex. Proceedings of the National Academy of Sciences, 98(4):1907-9111, 2001.
William Lotter, Gabriel Kreiman, and David Cox. A neural network trained for prediction mimics diverse features of biological neurons and perception. Nature machine intelligence, 2(4):210-219 2020.
Wei Ma, Jeffrey M Beck, Peter Latham, and Alexandre Pouget. Bayesian inference with prob- abilistic population codes. Nature neuroscience, 9(1)):1432-143, 2006.
Aran Nayebi et al. Mouse visual cortex limited resource system that self-learns ecologically- general representation. PLOS Computational Biology, 19110)=1011506, 2023. doi: 10.1371/
journal.pebi.100 htt.:/ppbi..111111111 1011506.
Bruno A Olshausen and David J Field. Emergence of simple-cell receptive field properties by learning sparse code for natural images. Nature, 381(6883)607-6099 1996.
Gergö Orbán, Pietro Berkes, József Fiser, and Máté Lengyel. Neural variability and sampling-base -bsed probabilistic representations in the visual cortex. Neuron, 92(2):33-554, 2016.
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and ap- proximate inference in deep generative models. In International conference on machine learning, pp. 1278-1286. PMLR, 2014.
Odelia Schwartz and Eero P Simoncelli. Natural signal statistics and sensory gain control. Nature neuroscience, 4(8):819-825, 825, 2001.
Suhas Shrinivasan, Konstantin-Klemens Lurz, Kelli Restivo, George Denfield, Andreas Tolias, Edgar Walker, and Fabian Sinz. Taking the neural sampling code very seriously: A data-driven approach for evaluating generative models the visual system. Advances in Neural Information Processing Systems, 36:21945-21959 2023.
12

---

Published as a conference paper at ICLR 2026
J Hans Van Hateren and Arjen der Schaaf. Independent component filters of natural images compared with simple cells in primary visual cortex. Proceedings of the Royal Society of London. Series B: Biological Sciences, 265(139)) 1998.
Martin J Wainwright and Eero Simoncelli. Scale mixtures of gaussians and the statistics of natural images. Advances in neural information processing systems, 12, 1999.
Daniel LK Yamins and James DiCarlo. Using goal-driven deep learning models to understand sensory cortex. Nature neuroscience, 19(3):356-36) 2016.
Alan Yuille and Daniel Kersten. Vision as bayesian inference: analysis by synthesis? Trends in cognitive sciences, 10(7):301-308 2006.
Chengxu Zhuang, Siming Yan, Aran Nayebi, Martin Schrimpf, Michael C Frank, James DiCarlo, and Daniel LK Yamins. Unsupervised neural network models of the ventral visual stream. Pro- ceedings of the National Academy of Sciences, (18())x20141991 18, 2021.
Corey M Ziemba, Richard Perez, Julia Pai, Jenna G Kelly, Luke E Hallum, Christopher Shooner, and J Anthony Movshon. Laminar differences in responses to naturalistic texture in macaque vl and v2. Journal of Neuroscience, 39(49):9748 9756, 2019.
A APPENDIX
A.1 LEARNING THE TAVAE PRIOR
We derive optimization objective determine the task prior. We would like maximize the log-likelihood under the latent prior, pr(2) using observations from the task Xy:
L
Σ x
log(pr (x))
Σ log
/ f dz p(æ 2) pr(2)
(9)
xxxr
we assume that the variational posterior of the natural images a good approximation the true posterior
pa 2)) ~ g(= a:) po(a)
(10)
Po(z)
then the lig-liiel,hood, with (a():) moved outside of the dz integral, can be written as:
Pu(x) + log / dz g(> pr())
L = Σ
(11)
Po(z)
xEXT
As the first term does not depend on the task prior, the functional to maximize is:
pr(x))
Σ log f dz g(2|2 ((2(x))pr(2)
='=
Σ log
(12)
Polar)
Po(z)
xEXT
xEXr
where Lo the log-like log-likelihood of the task data under the original model.
A.2 LEARNING THE PRIOR FOR THE DISCRIMINATION TASK
As explained in the main text, seek the task prior in the form of
Laplace distribution:
N
|
(T(=) = Laplacee2; 0,g.))
II
exp
(13)
20T,
OT,
i=1
The scales can be determined in principled way by taking the derivative of the log- likelihood with respect to the scales.
( x))
|
dz;
Po(2:) |
exp
aL'
oT,
(14)
d
OT
x
T
dz;
q(2; ()
exp
[z
po(2i)
dr.
13

---

Published as a conference paper at ICLR 2026
where n is the number of images in the task dataset. This can be rearranged to intuitive form:
1 Σ Egr. (a:lae) [|z:]]
TT
(15)
DEXT
This is self-consistency equation, since, of course, the right-hand side also depends on oG. We solve this iteratively. We start with the original prior p/ (z) Po(z). Then after the first iteration, the scales for each dimension will correspond to the scale from the variational posterior, which will be refined the subsequent iterations. The resulting approximate posterior (Eqq 66) is no longer a Laplace distribution but a piecewise exponential function. Because of this, the necessary integrals can be performed analytically to calculate the various moments, as we demonstrate in Section B.5
B EXPERIMENTAL ORIENTATION DISCRIMINATION TASK AND ITS VAE IMPLEMENTATION
B. GRATINGS DATASET
We generated synthetic dataset consisting of 40 grayscale images of sinusoidal gratings. Each image is defined by the function
s X, Y) = C sin f (X cos(O) + Y sin (0) + +)).
where / represents the spatial frequency (fixed at 3), 0 is the orientation angle (measured in radians), 6 stands for the phase, and C' serves as the contrast scaling factor.
The coordinates (X, Y) correspond to pixel locations [-1, I] in both dimensions. For fitting the scales of the task prior, we selected a contrast factor of I, while for inputting the test stimuli into the model, we opted for contrast factor of 0.3 to simulate the reduced contrast used during the test phase of the experiment. The dataset comprised 36 angles spanning from 0% to 180; along with 50 distinet phase values.
uniform 40 x 40 grid spanning the interval
B.2 RECEPTIVE FIELDS AND TUNING CURVES OF EAVAE AND TAVAE
The latent variables z of our baseline VAE model EAVAE have a linear receptive field (determined by the image produced when just one value is assigned I and all others are set zero) consist of localized oriented filters (see Fig. 5 a).
Thus, it is reasonable infer that (some) of these model neurons demonstrate orientation selectivity when gratings at different angles pass through the network, as illustrated by the examples in Fig. 5p.
Priors affect the tuning curves. In Fig. we show that neurons whose preferred orientations flank the prior orientation are suppressed, as indicated by decrease in their tuning curve amplitude.
B.3 EXPERIMENTAL DATA
Calcium imaging recordings were obtained from excitatory neurons in layer 2/3 of mouse V1 during the test phase of the experiment. Recordings were performed in 10 mice expressing GCaMP6f or GCaMP6s, across six experimental sessions corresponding to six stimulus conditions. Each session contained between 2007 and 2675 neurons and 50-150 trials.
Calcium signal preprocessing followed the procedure described Corbo et al. (0225). Fractional fluorescence (AF/P) signals were baseline-corrected using the median fluorescence during the pre- ceding intertrial interval and deconvolved to infer action potential-relate events (APrEs), which served as proxy for spiking activity.
Neural responses were computed by averaging activity over 40 frames corresponding to the central 1-1.5 of stimulus presentation. Temporal autocorrelation in calcium signal time-courses was counted for by correcting the effective sample size to n = 7 when computing standard errors of the mean (s.c.m.) and performing statistical tests.
Population response profiles were constructed by arranging neurons according to their preferred orientation, estimated from separate tuning block and averaging responses across neurons and
14

---

Published as a conference paper at ICLR 2026
Orientation (*)
Figure 5: a, Receptive fields for example model neurons EAVAE base model. Tuning curves and von Mises fits for these neurons. Note that the last neuron is classified not orientation selective.
Aggregated Mean Tunling Curves by Orientation Bin
Preferred ematton:
Preferred orfentationn
3.5
0.2
100
150
135
160
17%
175
Samalas Ortornation Isegreal
Sima Drentazion (digreoi
Zimula Crantationn Megrosa
0.8
No
Po
6.
0.6
1.-
32-
0.6
25
100
150
125
100
175
Sumeuus Ontorattonn (59g)
Stmaa us Drentaiion (digreol
Sinula Cranlation Migensal
Figure 6: Tuning curve modulation of individual model neurons (aggregated by preferred orienta- tion) for the naive model (blue) and the model trained with priors 45 and 135" grating stimuli (orange). Neurons whose preferred orientations flank the prior stimulus directions are suppressed. Shading: 95% Cl of the mean.
time. Profiles were pooled across animals to reduce recording noise, resulting in approximately 2-20 neurons per 1 orientation bin. Error bars and shaded regions denote either +2 s.c.m. 95% confidence intervals the mean, as specified in figure captions. For visualization only, population profiles were smoothed using Gaussian kemee with 6
15

---

Published as a conference paper at ICLR 2026
B.4 MODEL POPULATION RESPONSES
To compute population response profiles for the model, each latent variable 2; was assigned an orientation preference. This achieved by synthesizing dataset of grating stimuli spanning multiple orientations and phases. For each latent unit, responses were averaged across phases to construct an orientation tuning curve.
Each tuning curve was fit with both a von Mises function and constant function. A latent unit was classified orientation selective if the coefficient of determination (It) of the von Mises fit exceeded that of the constant fit by at least 0.5. Using this criterion, 1415 out of 1799 latent units were classified as orientation selective. Example receptive fields and tuning curves are shown in Appendix Fig. 5
Orientation-selective latent units were then ordered by their preferred orientation. Population re- sponse profiles were computed by binning units into 5 orientation bins and averaging responses within each bin. Variability across model neurons was quantified using 95% confidence intervals obtained via bootstrap resampling across latent units.
B.5 CALCULATING MOMENTS FOR PRODUCTS OF LAPLACIAN DISTRIBUTIONS
We are interested in computing the expectation values
E[2,, B[]ai]]
under an unnormalized distribution of the form
((=) (z) pT pr(2;0,0r)
ZERN
0,
Here q is multivariate Laplace with mean vector μ E IRN and scale vector R.N. Pr zero- mean Laplace with scale vector E RR,, and Po is the standard zero-mean, unit-scale Laplace.
Because Laplace densities factorize across coordinates, both expectations reduce to -ne-iimnsioona problems of the form
fz4) x cxp
+ 12:l).
The integrand is piecewise exponential with kinks at 2 0 and 24 p4g- On each segment,
fi(z;) = exp(d + k2;).
with slope and shift d determined by the interval and parameters (μi, az oT.). The normalization and required moments can then be expressed in closed form.
For segment (#1o, 2ni) (assuming that 2to and 2hi have the same sign, or that one of them is zero, so that the integration interval does not cross zero):
Normalization:
atrapi -e x edl Cahi - -210),
* K =
N
First moment:
K: + 0, 0
Mseg
(2hl
2/00;
Absolute-value moment:
Pii
ed Iedlissii
1e 21 majo
K K 0, sgn( dl now -sgn
( Innoo
Aseg
x2
Thiil
Zhil
-1o (0o12
0
By summing across all segments wise), we obtain
(-o0, 0). (0. pe), (Hi oo) for bs (and symmetrically other-
Σ
Eongmeents Marg
segments Aung
N; Σ
Nagg
E[2
E||
N
segments
16

---

Published as a conference paper at ICLR 2026
2.00
Reration Izeration iteration Iteration
1.75
1.50
Population Scale
1.25
1.00
0.75
0.50
0.25
0.00
0
30
60
90
120
150
180
Orientation
Figure 7: Convergence of the iterative solution of the self-consistency equation for the scale param- eters.
To avoid numerical overflow/underflow in practice before evaluating exponentials, subtract the maximum exponent among all segment endpoints. This does not affect the ratios, but makes the calculation stable.
B.6 CONVERGENCE OF THE LAPLACE PRIOR FITTING
When performed the iterative determination of the scale components (the solution of Eq. 14] found convergence after five iterations. In Fig. 7 we plot the progression of the scale. The com- ponents of the scale vector are categorized into orientations the same way as the model neurons.
B.7 ALTERNATIVE PRIOR CHOICES
We investigated alternative hypotheses about the mice's 's internal prior across D2-D6. As shown in Appx Fig. D when the model uses 135 NoGo prior, the response profiles differ qualitatively from the measured profiles. A similar mismatch appears when we model an animal that updates its prior daily (Appx. Fig. (8). This is consistent with the false alarm rate (the rate at which animal licks in response to NoGo signal) across days: on D2 the animal performs similarly to D1 (Appx. Fig. (h), indicating adaptation to the new environment,
on subsequent days performance worsens. The animal appears to be improving throughout D2 as it adapts the new NoGo signal (Appx. Fig. ()).
B.8 PEAK POSITION DEVIATION ACROSS TEST-STIMULI
To statistically quantify the distortion caused by the prior-stimulus mismatch this model configu- ration, tested whether the peaks of the response profiles were shifted away from the stimulus by the prior. To obtain model samples for this analysis, we constructed sample curves by selecting one neuron per orientation bin.
We found that the absolute difference between the peak position and the stimulus orientation was significantly larger in the TAVAE than in the naive VAE for all stimuli except 90", where there was no prior-stimulus mismatch (Fig. 10 one-tailed t-test, < 10 10-1, n 40).
B.9 CONTRAST DEPENDENCE IN TAVAE AND ALTERNATIVE MODEL
The rule for combining the model posterior (trained on natural statistics) with the adapted prior (Eq. specifies that the width of the VAE posterior determines the strength of the influence exerted by
17

---

Published as a conference paper at ICLR 2026
a
Stimuluus Angle: 90"
Stimulus 75*
Stimulus Angle: 70
Stimulus Angle:
Stimulus Angle: 60*
30
60
90
120 150 180
120 150 W
eo 90 120 150 160 Orientakion (*)
30 bo 90 Oriantation (*1
120 180
30 60
90 Orienaation (?)
30 60 90 120 180 Orientation (*)
Oriencation (*)
MAvIirror Poor arglin 0.8
0.4
0.0
150
Orientation (7)
Oneenstton P?)
Odtentaioo (*)
Orientation (?)
Orientation (?)
Figure 8: Population response profile of TAVAE on D2-D6 sessions for priors matched with the actual stimulus set, i.e. faithfully following the across-day changes of the NoGo stimulus. b, Same but for invariant prior across days, with prior reflecting the D1 conditions: Go 45" NoGO at 135%. Grey dashed lines: Prior positions; red dashed line: Stimulus orientation.; Red line: Task- trained prior condition; Blue line: Natural prior condition. All panels: data points, smoothing and confidence intervals as in Fig,
a
b
NoGo
NcGo Thine Alarm Rate
0.6
0.2
D-1 Reconding Day
D5
30%
57%
40 50%
00%
100%
Sescon pro.tmmx
Figure 9: a, D2-D6 animal behaviour characterized through the false alarm rate of the NoGo stimuli. b, Gradual adaptation of the animals to the new NoGo stimulus of D2. False alarm rates at different quintiles of the experimental session.
the new prior. The experiment used low-contrast test gratings. In biological neurons, lower-contrast images elicit V1 responses with higher noise correlations (Orbán et al. 2016). However, in VAEs without scaling factor, the contrast dependence of the posterior does not capture this property (Catoni et al. 2024).
We illustrated the effect of varying contrast in the TAVAE used in the paper in Fig. IT as well as in variant of TAVAE whose base VAE lacks a scaling factor (Fig. [2] As shown, when scaling factor present, the influence of adapting to new prior increases as contrast decreases. In contrast, the model without the scaling factor exhibits only a weaker modulation by the prior, even at low contrast, compared to the TAVAF.
We quantified the model predictions in Table 2 similarly to that in Table | [siing the correlation coefficient between the model orientation bin prediction and the experimental value that bin. We compared this metric when the model inferred its neuron responses from high- and low-contrast test images. Additionally, we also computed the correlation with a model in which the base VAE did not include the scaling factor. We modulation and the predicted modulation is only slightly better for the TAVAE with the scale factor than for the TAVAE without the scale factor.
that the correlation between the experimental response
18

---

Published as a conference paper at ICLR 2026
Peak Pasition Deviation: Task vs Naive
50
Absplute Deviation from Test Test. Andle (*)
4--
Task
20-
70
75*
Angle
Figure 10: Absolute deviation of the peak of the population response profile from the NoGo stim- ulus for different NoGo stimulus angles presented on different days. Asterisks denote significant deviations in the task condition relative to the naive case.
TAVAE (c 0.3)
TAVAE (c 1.0)
TAVAE (w/o scaling, 0.3)
r (TASKK r (TASK-NAIVE)
0.78 + 0.02
0.61 + 0.10
0.67 + 0.07
0.58 + 0.09
0.52 + 0.10
+
Table 2: Correlation between model and experimental population activity across orientation bins (averaged over stimuli). The standard error the mean is shown. Correlations between experi- mental and model response profiles are compared for different TAVAE variants. The full TAVAE evaluated at both reduced and full contrast levels. In addition to the full model, a constrained variant without the scaling variable is also tested. Top correlation between task-engaged perimental data and model predictions. Bottom row: correlation between experimental modulation (difference between task-engaged and naive conditions) and model modulation (difference between TAVAE with and without the prior).
B.10 NEURAL PREDICTIVITY AND CKA ESTIMATES
We quantified the similarity of model predictions and experimental measurements through the cor- relation between model and experiment across stimuli. We established both linear Centered Kernel Alignment (CKA) (Kornblith et al.) 2019) and Neural Predictivity (correlation between predicted and measured responses across held-out stimuli) Nayebi et al.) (022)) The stimuli in this analysis encompassed stimuli used across D1-D6, and included the Go stimulus from DI along with the NoGo stimuli across experimental sessions (moving gratings with orientations of 45% 60% 65", 70%, 75%, 90", and 135%). This stimulus set is considerably smaller than the sets typically used in passive viewing experiments, which explained by the limitation of the task that animals are exposed only to very specific set of stimuli.
As experimentally recorded neurons are solely characterized through their preferred orientation, responses of both recorded and model neurons with the same orientation preference were aggregated into bins, and metrics were calculated for averaged responses of neurons belonging to the same bin. Preferred orientations for experimentally recorded neurons obtained from separate block of trials. To climinate noise-dominated responses from the analysis, bins that were far off from the task stimuli were not included in the analysis. Consequently, similar to Section 3 used the range of bins spanning the (30%, 1059) interval.
We performed three comparisons between model and experiments. A baseline analysis per- formed with the standard VAE model, using data from non-trained (naive animals). As the naive animals do not display evident biases, standard VAE is expected to faithfully predict neuronal
19

---

Published as a conference paper at ICLR 2026
Contrant
Costrant
Contreat 0.5
Contrast
Contreat
03
20-
2.0
0.3
0.3
325
125
00
102
125
125
01
Figure 11: Contrast dependence population responses of TAVAE for different test stimuli. Grey dashed lines: Prior positions; red dashed line: Stimulus orientation.; Red line: Task-trained prior condition; Blue line: Natural prior condition. Data points, smoothing and confidence intervals as in Fig.
tivity in this condition. The standard analysis compared the TAVAE model to the neuronal responses recorded across D1-D6 in six sessions. A control was performed by using the standard VAE that cannot reflect task-specific response patterns to analyze data obtained from recordings during task execution.
CKA (Table and NP (Table 4 were performed in two ways. First, a trial-averaeed analysis was performed, where the experimental value was the average activity across all trials for each condition. Second, a trial-based analysis was performed, where model predictions were compared to individual experimental trials (40 trials per stimulus) and then averaged across trials. In this second scenario, also computed noise ceiling, comparing trial-averaged experimental responses to individual trials.
VAE (on naive data)
TAVAE (on trained data)
VAE (on trained data) 0.82
CKA (Mean)
0.85
0.86
CKA (Trials)
0.72 + 0.01 (0.90)
0.71 = 0.01 (0.77)
0.64 + 0.01 (0.77)
Table 3: Centered Kerne Alignment (CKA) scores for VAE and TAVAE models evaluated naive and trained mouse experimental data. Top row: CKA for the mean orientation-bin activity across all available trials. Bottom row: trial-based analysis (with the standard emrrr of the mean indicated). Values in parentheses denote the estimated noise ceiling for the corresponding dataset. The VAE prediction on the task data serves control.
20

---

Published as a conference paper at ICLR 2026
Contrast
Contract
Contreat 0.5
Contrast
Contraat
03-
00
20-
3.0
6.0
1.5
100
Figure 12: Contrast dependence of population responses for different test stimuli in a TAVAE lacking the scaling variable from the baseline model component. Grey dashed lines: Prior positions; red dashed line: Stimulus orientation.; Red line: Task-trained priors; Blue line: Natural prior. Data points, smoothing and confidence intervals in Fig. 2
VAE (on naive data)
TAVAE (on trained data)
VAE (on trained data)
N.P. (Mean)
0.907
0.804**
0.777 0.379 (0.528)
N.P. (Trials)
0.655 (0.772)
0.387* (0.528)
Table 4: Neural predictivity (N.P.) scores, averaged over orientation bins, for VAE and TAVAE models evaluated naive and trained mouse experimental data. Top trial-averaged analysis. Bottom row: trial-based analysis. Values in parentheses show the estimated noise ceilings of the corresponding dataset. Asterisks indicate significant differences between TAVAE (on trained data) and VAE (on trained data), based on paired (-tests across orientation bins: P 0.015 (n 16); p 10- (n 16).
CKA (Table
3 and neural predictivity (Table consistently show that:
1. The VAE, when compared with experimental data from naive mice, captures meaningful variance for our grating stimuli, supporting its use sensible baseline for V1 model in the passive viewing condition.
2. For data from task-engaged animals, the TAVAE better accounts for the neural responses than the VAE. However, the specificity of these metrics metrics-primarlly developed for larger stimulus sets-is lower than that of the metric used in Section and Table
21

---

Published as a conference paper at ICLR 2026
B.11 ESTIMATING LIKELIHOOD AND PRIOR THE SPACE OF STIMULUS ORIENTATIONS
Assuming that we observe an orientation posterior in the orientation space when recording a pop- ulation of neurons, intend to infer the likelihood and orientation prior in this space from the population response profile. Importantly, this prior-posterior formulation is distinct from that of the VAE framework, which defined over a nigh-dimensional neuronal activation space rather than onc-dimensional orientation space.
We focus on D2 responses and focus on one of the modes flanking the dominant mode centered on the stimulus. The flanking mode is mixture component of the posterior, which is combination of the contextual prior (pr), and the likelihood (l). Approximating the posterior, likelihood, and prior with Gaussian in the orientation space, the mean of the mode can be obtained:
pp Hpr of ln
oe + opp
Poost
o of
(16)
of
In a coordinate system centered around the NoGo stimulus of D2, some of the parameters can be readily determined: lmI = 0; Upr = 45%; and ppost can be measured from the population response profile.
We rely on the insight that if we know the means of the prior and the likelihood - which we do know then it is the relative width of the likelihood and the prior that determines the place of the posterior. Let's s assume that the prior width is Opr = r 0). Then Eq. 16 becomes
2² 2 + of Mpr of (1 + r?)
p2
14 + Ppr
Ppost
(17)
r2
Thus, we know pposss then we can calculate r.
We can solve for in Eq. 17 via:
Ppost 12 11 + Mpr
1 p2
Apost (1 + +²) = 12m1 + Ppr
Mpost + r² Mpost = r2 Mn + Hpr
22 Hpost
122 MI Hpp Mpoos
Upr Ppost
Mpost MI
Establishing the width of the likelihood. At this point, we can establish the actual widths of the likelihood and the prior. Consider the bump in the population activity profile when presenting the Go stimulus. There the prior and the likelihood have the same mean. such case the posterior is a product of two Gaussians:
1
exp (-
1 20p
(o uGGo)) [
N(o; MGo GGo) x exp
(o mcn))
(18)
2n pr
1 11+1 2
exp
2 o207 (o- - (cGH))
(19)
This highlights that the posterior width is
post
(20)
V++22
Using this, we obtain an estimate of the widths of the likelihood and prior from the width of the Go responses.
B.12 USE OF LLMS
We occasionally utilized LLMs to refine the paper's 's text. Additionally, we employed -ow-red tools for programming, particularly when creating scripts for the figures.
22


Reviews:
[{"review_id": "io5BUIwi8T", "reviewer": "Reviewer_4QfZ", "summary": "The authors present a VAE framework that explicitly describes a task-dependent prior and test it's performance along side a neural data from mouse V1. The authors present their model, describe data collection, and present qualitative similarities between the activations of latent variables in their model to the spiking activity of V1 neurons measured by calcium imaging.", "questions": "The authors should clarify their mechanistic claims about why their model matches the data in the ways they claim, the modeling choices, statistical inference, and all claims should be accompanied by statistical tests.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper presents an interesting and, to my knowledge, novel accounting of neural tuning properties in the face of changing stimulus statistics using the model they present in Section 2. They present this along side an approach for learning context specific priors in the variational framework. There do appear to be some qualitative similarities between neural data and model latents but validating these results is required before claims can be made about how their model maps mechanistically onto neural representations of stimuli.", "weaknesses": "I think there are 2 main dimensions on which this paper falls short of acceptance 1) validation of model structure, 2) statistical rigor, 3) clarity of question.\n\n1) The authors make claims about the qualitative properties of the latents of their model and how they match those of the real data. However, I'm not sure it's possible to attribute these features (even if they are statistically valid) to the prior structure of the model exclusively. Specifically, no ablation analysis of the model was conducted to determine which of their modeling choices was essential to their findings. For example, how critical was it that the latent responses were sparse? How important was the scaling latent? Neither of these choices were evaluated in any way and it is not clear they are germane to the properties they intend to model. \n\n2) There is virtually no statistical analysis beyond Figure 2. Error bars and shaded regions around population tuning curves are not defined. Data points in tuning curve plots (eg. red and blue dots in Figure 2a,b) are not defined. Moreover, if these really are data points and the shaded regions are supposed to be 95% confidence intervals then I suspect their inference is over-confident. \n\n3) It's not obvious what the authors are testing when they are examining neural activity along side latent activations. This seems to be an unreasonably course level of analysis and I would not expect a clear correspondence to exist beyond something incidental. Perhaps the authors meant to examine the posterior distribution over the stimulus? This would have real cognitive meaning in the context of a shift in prior probabilities.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kxKFeyuId7", "reviewer": "Reviewer_97W7", "summary": "This paper proposes TAVAE, a task-adapted VAE framework that modifies only the latent prior (not the encoder or decoder) to account for contextual modulation effects observed in mouse V1 during a visual discrimination task. By adapting the prior learned from natural images to task-specific contingencies., the model reproduces several well-known effects: sharpening, baseline suppression, and multimodal responses under stimulus-prior mismatch. The VAE is strongly constrained: linear decoder, Laplace prior, and overcomplete latent space, mirroring classic sparse-coding models rather than deep nonlinear architectures.", "questions": "1. Is it possible to extend this model to decode neural activity? like Maheswaranathan Neuron 2023? \n\n2. The encoder is linear, with overcomplete latent dimensions, and trained under a Laplace prior. How close is it to ICA or sparse coding rather than deep encoder following by variational sampling? \n\n3. Would you expect the same latent prior adaptation mechanism to work in tasks involving richer stimuli or additional visual features (e.g., natural scenes/motion)? Why or why not?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1. The model is a minimal model with biologically inspired constraints—linear decoder, sparse Laplace prior, overcomplete latent space, and GSM-style gain modulation while it mirrors classic models of V1 (e.g., Olshausen & Field).\n\n2. The model qualitatively reproduces several experimentally observed phenomena using a single mechanism (prior variance reweighting).", "weaknesses": "1. While the paper claims that adaptation in the prior alone is sufficient to account for several task-induced changes in neural population statistics. The lack of comparison to single neuron activity left this claim speculative \n\n2. Figure 3a: I really cannot see \"drastic\" difference between red and blue curves. There needs to be a metric or something to quantify how they are different. \n\n3. Figure 4a; The curves are visually nearly identical in shape, except for slightly lower side peaks and a slightly higher center as γ increases. If all that happens is one peak increases slightly, calling it “updating the inference toward the new context” feels like a strong claim for a weak effect.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2MXV1zPWYQ", "reviewer": "Reviewer_kx1v", "summary": "The paper suggests a modified image autoencoder to account on task adaptations in visual cortex. \nThey have a two-stage procedure when first the latent representations are trained for natural images and then they are adjusted with respect to the task prior (which is optimized).\nIn this particular paper they use mice performing go/no go tasks and show that the latent space of the visual autoencoder after introducing a task prior shows same qualitative phenomena as the actual responses in V1 when the task is changing, supporting the claim that the brain performs a probabilistic inference under a prior.", "questions": "1. If I get it correctly, then the autoencoder model is trained on images only and neural responses are used for validation only. Is it right? Also, you first train an autoencoder using eq (9) as the loss function to get $q(z|x)$ and $p(x|z)$ and then you only train $\\underline{\\sigma}_{T}$  (line 253) ?\n\nAnd it adjusts $q(z|x)$ to $q_{T}(z|x)$ ? Are there any other parts retrained?\n\n2. Why the trained baseline activity in Fig2 H is negative? I though you are taking the absolute values (lines 237-240)\n3. Why exactly  does Laplace prior give us localized, oriented receptive fields? (lines 226-228)\n4. Lines 237-240 identify that $z$ could be negative, which clearly misaligns the autoencoder latent space with the neuronal responses. Have you tried to restrict the $z$ to be strictly non-negative during training?\n5. Connected to the previous question - lines 212-214  say that *\"activations of latent variables, z, of the generative model were assumed to correspond to activations of individual neurons in V1\"*. How adequate is this assumption? Have you tried to regress learn linear regression from $z$ to the actual neuronal responses (like \"neural predictivity\" in [1] ) and see how well it performs or do something like CKA analysis [2-4]?\n6. How exactly was the neuronal data pooled across sessions? Have you selected the neurons which were enough orientation selective and then just averaged them across all sessions to make the lines in Fig 2 e, f for example? And for the autoencoder - you always used a single model (e.g. there were no several autoencoders to match the latent space for the number of neurons per session)?\n7. I would appreciate your thoughts on weakness 4.\n\nMinor:\n1. What are blue and green lines in Fig1 h?\n\nReferences:  \n[1] Nayebi, Aran, et al. \"Mouse visual cortex as a limited resource system that self-learns an ecologically-general representation.\" PLOS Computational Biology 19.10 (2023): e1011506.  \n[2] Murphy, Alex, Joel Zylberberg, and Alona Fyshe. \"Correcting biased centered kernel alignment measures in biological and artificial neural networks.\" arXiv preprint arXiv:2405.01012 (2024).  \n[3] Williams, Alex H., et al. \"Generalized shape metrics on neural representations.\" Advances in neural information processing systems 34 (2021): 4738-4750.  \n[4] Chun, Chanwoo, et al. \"Estimating Neural Representation Alignment from Sparsely Sampled Inputs and Features.\" arXiv preprint arXiv:2502.15104 (2025).", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. **Elegant framework**. Beautiful idea - fixing the likelihood $p(x|z)$ and only learning a new task-specific prior $p_T(z)$ is both elegant and powerful.  The paper makes a clear hypothesis: systematic biases in V1 during task performance are the result of probabilistic inference under a learned, task-specific contextual prior. The model provides a concrete implementation of this hypothesis and generates specific, falsifiable predictions that are then confirmed by the experimental data.\n2. **Qualitative comparisons**. The model reproduces the qualitative phenomena, eg splitting the distribution from unimodal to bi-modal when there is a mismatch between the train and test data (e.g. Fig 3)\n3. **Reproducibility**.  The code is provided in the supplementary materials.\n4. **Statistical rigor**. All the plots and tables report error-bars.", "weaknesses": "1. **Clarity**. The paper might benefit from a more clear high-level framework introduction, before getting to the formalism. If I get it correctly, then the autoencoder model is trained on images only and neural responses are used for validation only.  \n2. **Lack of quantification of qualitative results**. While Fig 3 generates nice qualitative insights, some statistical tests might support the claims, eg Hartigan's Dip Test to quantify when the red line stops being unimodal (and if it happens faster in real mice or in the model), one-Sample t-test to test that the peaks of the response are significantly shifted away from the actual stimulus orientation, and some pearson correlation to check how good the model predictions fit for the actual neuronal responses.\n3. **Representations alignment is not considered**. Lines 212-214 make an assumption that the autoencoder latent space $z$ is assumed to correspond to neural activities in V1, however, this correspondence is clearly violated by the fact that $z$ could be negative. Hence, this raises questions about the validity of this assumption and how aligned the representations are in general. \n4. **Limited direct applicability**. While this is a beautiful hypothesis testing framework, applying it for different stimuli can be very complicated. Specifically, eq (11) is nice and tractable as *\"in a typical gratings dataset we expect a symmetry in z around zero\"* (247-252). However, it is not that clear how to set up a meaningful prior in case of other tasks and out-of-distribution designs  (like distinguishing images by colors, primarily direction of \"random\" moving dot stimuli, etc)\n\nMinor:\n1. Inconsistent font sizes in the plots (see Fig 1 panel D and H, or Fig 2 panel A and E).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)