A data and task-constrained mechanistic model of the mouse outer retina shows robustness to contrast variations
Authors: Kyra L. Kadhim
Jonas Beck
Ziwei Huang
Jakob H. Macke
Fred Rieke
Thomas Euler
Michael Deistler
Philipp Berens
Venue: NeurIPS 2025 poster ()
Keywords: neural coding
retina
recurrent networks
biophysics
differential equations
Primary area: neuroscience_and_cognitive_science

Abstract: Visual processing starts in the outer retina where photoreceptors transform light into electrochemical signals. These signals are modulated by inhibition from horizontal cells and sent to the inner retina via excitatory bipolar cells. The outer retina is thought to play an important role in contrast invariant coding of visual information, but how the different cell types implement this computation together remains incompletely understood. To understand the role of each cell type, we developed a fully-differentiable biophysical model of a circular patch of mouse outer retina. The model includes 200 cone photoreceptors with a realistic phototransduction cascade and ribbon synapses as well as horizontal and bipolar cells, all with cell-type specific ion channels. Going beyond decades of work constraining biophysical models of neurons only by experimental data, we used a dual approach, constraining some parameters of the model with available measurements and others by a visual task: (1) We fit the parameters of the cone models to whole cell patch-clamp measurements of photocurrents and two-photon glutamate imaging measurements of synaptic release. (2) We then trained the spatiotemporal outer retina model with photoreceptors and the other cell types to perform a visual classification task with varying contrast and luminance levels. We found that our outer retina model could learn to solve the classification task despite contrast and luminance variance in the stimuli. Testing different cell type compositions and connectivity patterns, we found that feedback from horizontal cells did not further improve task performance beyond that of excitatory photoreceptors and bipolar cells. This is surprising given that horizontal cells are positioned to mediate communication across cones and that they add to the model's number of trainable parameters. Finally, we found that our model generalized better to out of distribution contrast levels than a linear classifier. Our work shows how the nonlinearities found in the outer retina can accomplish contrast invariant classification and teases apart the contributions of different cell types.

A data and task-constrained mechanistic model of the mouse outer retina shows robustness to contrast variations
Kyra L. Kadhim Jonas Beck! Ziwei Huang Jakob H. Macke2 Fred Rieke', Thomas Euler", Michael 2eistler and Philipp Berens
Hertie Institute for in Brain Health, University of Tübingen, Tübingen, Germany
TTübingen AI Center, Tübingen, Germany
3 Machine Learning in Science, University of Tübingen, Tübingen, Germany
MMax Planck Institute for Intelligent Systems, Tübingen, Germany
'Department of Physiology and Biophysics, University Washington, Seattle, United States Institute for Ophthalmic Research, University of Tübingen, Tübingen, Germany
Max Planck Institute for Biological Intelligence, Martinsried, Germany kadhima uni-tuubinggn..ee
Abstract
Visual processing starts in the outer retina where photoreceptors transform light into electrochemical signals. These signals are modulated by inhibition from horizontal cells and sent to the inner retina via excitatory bipolar cells. The outer retina is thought to play important role in contrast invariant coding of visual information, but how the different cell types implement this computation together remains incompletely understood. To understand the role of each cell type, developed a fully-differentiable biophysical model of a circular patch of mouse outer retina. The model includes 200 cone photoreceptors with realistic phototransduction cascade and ribbon synapses as well as horizontal and bipolar cells, all with cell- type specific ion channels. Going beyond decades of work constraining biophysical models of neurons only by experimental data, we used a dual approach, constraining some parameters of the model with available measurements and others by visual task: (I) We fit the parameters of the cone models to whole cell patch-clamp measurements photocurrents and two-photon glutamate imaging measurements of synaptic release. (2) We then trained the spatiotemporal outer retina model with photoreceptors and the other cell types to perform visual classification task with varying contrast and luminance levels. We found that our outer retina model could learn to solve the classification task despite contrast and luminance variance in the stimuli. Testing different cell type compositions and connectivity patterns, we found that feedback from horizontal cells did not further improve task performance beyond that of excitatory photoreceptors and bipolar cells. This is surprising given that horizontal cells are positioned to mediate communication across cones and that they add to the model's number of trainable parameters. Finally, we found that our model generalized better to out distribution contrast levels than a linear classifier. Our work shows how the nonlinearities found in the outer retina accomplish contrast invariant classification and teases apart the contributions different cell types.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Cone model to data
OPL model fit to task
Photorecaptors
Readouts
Light stmull
Photooument
M
25
25
Glutamato roloeso
Foadkmandd
Fooer
Figure Training detailed, data-constrained biophysical model of the mouse outer retina to solve a classification task: (left) A cone model with phototransduction cascade and ribbon synapse mechanisms is fit to glutamate and photocurrent recordings. (right) The optimized photoreceptors are spatially distributed and connected to horizontal cells and readout units by excitatory feedforward ribbon synapses. Horizontal cells are also connected to cones by feedback connections. The network is then trained to classify MNIST with varying contrast and luminance.
1 Introduction
The outer retina-thh first stage of visual processing- -aleady performs nonlinear processing of visual stimuli which shapes downstream vision, and hence, behavior. In particular, these nonlinearities are thought to play an important role in luminance normalization and contrast invariant encoding. Some of the most prominent nonlinearities the photoreceptors include the phototransduction cascade, which translates the absorption of photons to intracellular currents [. ion channels which govern the cell's membrane potential (2-5,, and the ribbon synapse where glutamate is released dependent on the intracellular calcium concentration (6-8). In addition, horizontal cells provide recurrent feedback to photoreceptors, further extending the computational capabilities of the outer retina (9.
Previous work has shown that photoreceptors are capable of adapting to drastically different global luminance levels and that this adaptation visible in the current produced by the phototransduction cascade [10]. It has also been suggested that horizontal cell feedback to photoreceptors facilitates adaptation and contrast enhancement [.. However, contrast adaptation has been found in bipolar cell dendrites even without input of horizontal cells and without voltage-gates ion channel activity in the bipolar cells [1]]. Despite of this experimental evidence, most computational models of the retina and higher level visual processing either omit the outer plexiform layer collapse it into spatiotemporal linear filters, relegating nonlinear processing to later stages [7]
Understanding the computations of the outer retina requires models that on the one hand incorporate all of the mechanisms previously described, and that on the other hand can be optimized to perform visual processing tasks to constrain parameters that cannot be identified from experimental mea- surements. However, current biophysically detailed models do not support task-based optimization of parameters [18]. Alternatively, more abstract models of retinal processing, which can be easily optimized for task performance, do not capture bicphysical mechanisms at the required level of detail [13 11) Leveraging newly developed differentiable simulators for biophysical models [22, we built a detailed model of a spatially extended patch of the outer retina including photoreceptors, hori- zontal cells and bipolar cells, and all of their biophysical mechanisms outlined above (Fig. D Using backpropagation error, we optimized biophysical parameters of the photoreceptors to fit whole cell patch-clamp measurements of photocurrents and two-photon glutamate imaging measurements of synaptic release. We then fit between 2,000 and 2,900 additional parameters of the model not constrained by these experimental measurements enable the network to perform classification task. Using this data- and cosstronsrr model, we then show how the nonlinearities of the outer retina enable small network of cells with realistic biophysics perform contrast invariant encoding of visual stimuli and decipher the contributions of different cell types.
2 Related work
Few previous works have combined biophysical models with deep learning and optimization tech- niques [2] 23 22.. Idrees et al. [21 developed convolutional neural network with biophysically

---

realistic photoreceptors as an input layer [2I]. This photoreceptor layer was shown to perform adaptation that improved prediction primate and rat retinal ganglion cell activity and enabled better predictions for luminance levels out of the training distribution. Their photoreceptor model, however, did not include ribbon synapses, and had no biophysical or structural realism beyond the photorecep- tor layer. They also didn't 't test for contrast invariant encoding of visual stimuli. Schröder et al. [14] developed a differentiable and biophysically realistic model of the inner retina with realistic synapse dynamics, feedback circuits and trained the model end-to-end, but their work focuses on temporal processing in the inner retina, and they do not constrain their model by task performance.
Other work has used visual tasks to constrain the parameters of more abstract vision models with highly simplified neuron models [24 23 (2]]. For example, Lappalainen et al. [23] recovered experi- mental properties of neurons in the fly visual system by combining connectome -aseddbaeddde constraints and training on an optic flow task, but they used very linear point neurons and simplified synaptic dy-
namics. We are not aware of any biophysical model of the retina whose parameters were constrained by experimental data and task training, nor of a biophysical model trained perform classification task at different contrast levels.
Finally, several biophysically realistic retina models exist 28,, but they are not implemented in frameworks that are amenable to task optimization, and they often lack biophysical mechanisms. Our model is dully-differentiable, highly-detailed, and implemented in Python, allowing it to interoperate with the Python machine learning ecosystem for efficient gradient-based optimization.
3 Methods
3.1 A differentiable, biophysically-detaaled model of the outer retina
In this section we will provide some background information about our model and its detailed biophysics. In our model, we use single compartment Hodgkin-Huxley type models [29]. These model the flow of different ions through voltage modulated channels in the semi cemi-permeable cell membrane with equations that derive from Ohm's law,
Lion (V, Tion m (V,t) (VV (V Eion).
(I)
where V is the voltage across the cell membrane, Jion the maximal conductance of a specific ion, m and h the gating variables that model the opening and closing probabilities of the channels, a and b the number of gates or protein subunits per channel, and Ejon is the reversal potential which there is no net-flow of ions. Given several of these ionic currents and an external input current Imput, the membrane potential V is governed by
dv dt
Σ Hions ion
= Imput
(2)
where C' is the capacitance. We model different cell-types by including different ionic currents. We model the photoreceptor ion currents after Kamiyama et al. as
dv = - + IKv + Inyper + Ica + IK(Ca)) + ICNCa) + Iteak).
(3)
dt
The current generated by the phototransduction cascade Iphoto is determined by plex electrochemical signaling pathway triggered by the absorption of photons by pho- topigments in the outer segments of photoreceptors. Upon photon absorption, cascade of intracellular reactions is initiated, ultimately leading to the modulation of ionic cur- rents across the photoreceptor membrane. We model the phototransduction cascade as
dR = S(()) RR(t) (Rt))
dP(() R(t) - PP(t) +n dt
(4)
(5)
dt
dG(t) dt + (C(t)/ooo)"
Smax
P(t)G(() (6) Smax 1 Gatari (1 1
Cdaak m
(7)
0
KGc
dC(t)
Cdark
Iphoto (t) = kt)
(8)
= 3 (Gda ((t) - -(t))
(9)
dt
3

---

Table I: Network architectures
Readout neurons
# Horizontal cells
Connectivity
Passive
Full
Active bipolar cell
Full
Active bipolar cell
0
n/a Local
Active bipolar cell
Active bipolar cell
200
One-to-one
Here, equation models the production and decay of opsin. S(() is the light stimulus and R(())
the activated opsin molecules. P(() denotes the second messenger phosphodiesterase activity;
increases with opsin activation and decays over time (Eqq 5). G(L) corresponds to the concentration of
cyclic GMP (cGMP), the key second messenger that controls ion channel opening (Eq. 6). I Ihhoto() (1)
is the resulting photocurrent, modeled as nonlinear function of cGMP concentration (Eg. 8). C(t)
tracks intracellular calcium concentration, which is modulated by the photocurrent and in turn feeds back to regulate cGMP synthesis (Eg. ( This feedback system captures the adaptive and nonlinear response dynamics of cone photoreceptors to varying light intensities 10
Cone photoreceptors also possess specialized ribbon synapses which enable rapid and continuous neurotransmitter release in response to graded intracellular voltage changes [8]. The ribbon synapse facilitates efficient transmission of visual signals to downstream neurons even under high-throughput conditions. It is organized into discrete vesicle pools that undergo cycles of replenishment, priming, and exocytosis. We model the dynamics of the ribbon synapse as
dRP
(t
RP(t) RP max
df dmax Exo(t)= - -maax
(10)
IP maxx
dIP
IP(()
RP(()
RRP())
IP(()
dt Tmax
imax
(11)
IP
R. R.maax
RRP
IP Pmax
dRRP imax 1
RRP(t)
IP Emax S(V(L))
RRP(()
(12)
dl
RRP max
IP max
RRP
dExo
RRP(t)
Cmax (()(
dmax Exo(t).
(13)
dt
RRP max
In this model RP(t), It).. and RRP(t) describe the reserve, intermediate, and readily releasable pool of vesicles, respectively. Exo(() indicates the exocytosed vesicles (i.e., neurotransmitter release). Eq. [0 describes the replenishment of the reserve pool from exocytosed vesicles and its depletion into the intermediate pool and the priming of vesicles from the intermediate pool into the readily relcasable pool. Eq. 12 & 13 describe the calcium- and voltage-dependent transfer of vesicles from the readily relcasable pool into the synaptic cleft and the dynamics of vesicle release and recycling. The term (t) a sigmoid function of the photoreceptor membrane voltage (t) capturing the voltage-dependent probability of vesicle release.
The phototransduction cascade and ribbon synapse together allow the transformation of light stimuli into graded voltage signals and their rapid encoding into neurotransmitter output, forming the foundation of early visual processing. With these details, we describe our cone photoreceptors with 26 differential equations, the most detailed computational model of photoreceptors of which we are aware. We implemented all of the mechanisms an open source Python library.
We model horizontal cells (HCs) based on dynamics of rabbit HCs 13] These dynamics are are defined by
C.V = - IKa- + IK., + INa + Ica + Iteak).
(14)
dt
For the synapses of cones onto HCs, we use the ribbon presynaptic mechanism described above and an ionotropic post-synaptic mechanism
Jann Ās Exo(t) (Vpoat Esyn)
(15)

---

Photocurent fit
b
Glubamate trace lits
Ribbon synapse paramelers
Dorsal Glueeo Vontral Glulgo
x10"
Slimuu (Pr1))
1.0
V
0.08
V
0.05
0.2
Sim.
0.3 fexx
0.8
Inttialzed
Data
Trained
0.0
Spectral selectivity comparison
0.9
0.03 k
0.
-30,0 Vsz
-20.0
Time te)
Time
Time
Data SC
Figure 2: Photoreceptor parameters fit experimental data using our differentiable simulator, a) Phototransduction cascade parameters were fit using gradient descent to normalized photocurrent data when exposed the light stimulus (top). b) Ribbon synapse parameters were fit glutamate recordings given green center flash stimulus (2s) and UV center flash stimulus (2s). Green traces show simulations to glutamate recordings from the dorsal retina whereas magenta traces show simulations fit to glutamate recordings from the ventral retina. Scale bars indicate one unit difference from the baseline. c) Distributions of fitted ribbon synapse parameters for 42 cones based on one and two dimensional kernel density estimates. d) Spectral selectivity, green vs. UV preference, of 42 data traces compared the trained simulations.
where gs is the maximal conductance, Exo(t) is the presynaptic exocytosed glutamate, Vpusi is the postsynaptic membrane voltage and Egyn is the reversal potential.
For the synapses of HCs onto cones, use the same ionotropic synaptic mechanism (Eg. 15 but instead of exocytosed glutamate Exo(!), the synaptic state evolves according to a standard model of ionotropic synapses found in the literature [30]:
ds(t) dt
(sx Ts
(sx - s()) S:0 1/ (1 exp ((Vi Vpre)/A)
Te = (1 (1-88))/ke soo)/k_.
The exact mechanism of feedback from HCs to cones is still debated nomenological feedback mechanism.
99. so we opted for phe-
The bipolar cell channel models were obtained from work observing five main membrane currents in the bipolar cells of several vertebrate species 44:
C dv dt
- + IK, + (K(Ca) + Inyper + ICa + Hleak).
(16)
The calcium dynamics were adapted from other work [5]. For the synapses of cones onto bipolar cells, the ribbon presynaptic mechanism and the ionotropic post-synaptic mechanism (Eq. [5] such that all bipolar cells are of the OFF subtype, meaning that they hyperpolarize to light-onset.
We evaluated different network configurations for which we varied the number of HCs, connectiv- ity, and bipolar cell dynamics to clucidate which components are important for contrast invariant classification. Table lists the different architectures the HCs are placed on a lattice spanning the diameter of the patch and each connected to its 50 nearest cones, approximating the cone-HC connection ratio in the mouse retina [3 The one-to-one connectivity of the 200 horizontal cells designed to model a case in which each cone photoreceptor gets feedback from electrically isolated segment of horizontal cell.
compare. Local connectivity means that
implemented all our models JAX using JAXLEY 22), a toolbox for differentiable bio- physical simulation. This allows our models to run efficiently GPU and to use gradient-based based optimization techniques for model parameter fitting. The code for producing and training our models can be found at lttth:t///// : //gi
5

---

a
Activity in network with passive readout neurons
Others Readout 6 Inttialized
Avg cone vallage (mVV)
Treined
Vec ceeeeee (mh))
Trained Initiaized
Reateut votage (mV)
DV axts (um)
2400
10
10
Intiaaizeed
-55
10
9
Gluse
8
-2400
-85
-30
0
-2400
200
200
axis (um)
Time (ms)
Time (me)
Time (me)
Avg vollage (mV)
b
Activity in network with active bipolar readout neurons
Avg cone volage (mV)
Readcut vottage (miV)
HC volaage (mi))
DV axis (im)
2400
10
-55
10
9
Gluan
6
20
-2400
-85
-30
-2400
200
200
axis (um)
(ms)
Time (ms)
Time (me)
Avg vollage (mV)
Figure 3: Neural activity in the biophysical model with 200 photoreceptors and one horizontal cell connected to all cones trained to classify MNIST with variance in contrast and luminance. a) Activity in the trained architecture with only passive leak channel on the readout neurons. b) Activity in the trained architecture with active bipolar cell channels on the readout neurons.
3.2 Data-constraining parameters of the photoreceptor models
The data used for constraining biophysical parameters of the model included recordings of pho- tocurretts from six cones when exposed to 16 seconds of noise stimuli with variable mean [10]. Experimental data was normalized by the experimentally-measee
dark photocurrent. To normalize our simulations, we estimated the dark current using model parameters as kGG dark. where Gdark is the concentration of cGMP the dark, n is the eGMP channel cooperativity, and is a constant for converting cGMP concentration to current.
We fit the 11 phototransduction cascade parameters of our cone model by cutting the stimuli and traces into 100 random four second clips, recording the photocurrent state variable in our simulations, and performing gradient descent with a mean squared error (MSE) loss function. All simulations in this work were run with a timestep of 0.025 ms for stable dynamics. We used the Adam optimizer 33 with a learning rate of 0.01 and initialized the parameters following previous work [10]. To ensure that the parameters stayed bounded during optimization and to stabilize training, we used a sigmoid transform, similar to Deistler et (22]. The exact bounds can be found Table [ST We performed leave-one-out cross-val validation, leaving the recording from one cone out in each fold to choose a final parameter set for all cones. The entire training required < 12 h on a NVIDIA A40.
Ribbon synapse parameters were constrained using fluorescence recordings of the glutamate biosensor iGluSnFR while the cells were exposed to flashes of green and UV light (34] We used total of 42 glutamate traces, each with a 1 s green flash followed by 1 s recovery period and 1 s UV flash followed by s recovery period. With frozen phototransduction cascade parameters, we recorded the exocytosed glutamate state variable in our simulations and fit six of the parameters in the ribbon synapse model and an additional parameter that scaled the cone's selectivity for green versus UV light (a). We again minimized the MSE loss between the simulation and experimentally recorded trace. We used the Adam optimizer with a learning rate of 0.01 and stopping criteria of <99% the previous average loss over ten epochs. We imposed a maximum number of epochs, 400, and the training for each ribbon synapse typically lasted 20 hrs on a NVIDIA GeForce RTX 2080 Ti. We also sigmoid transformed the parameters to stay within bounds estimated to maintain simulation stability. The bounds were (10-3 5, 1) for all rates of vesicle pool movement, (10-5,5) 5,5) for k, (-50,- 220) for V1/2- and (0, 1) for the selectivity parameter a. Of the 42 parameter sets obtained, we chose the ten best fits of glutamate traces from the ventral retina and the ten best fits of glutamate traces from the dorsal retina for remaining simulations. These were picked as having the lowest MSE while matching the response characteristics of the data (i.c. not only the mean activity).

---

3.3 Task-constraining remaining parameters of the outer retina model
After fitting individual cones to data, we assembled them into a network with varying numbers of horizontal cells and varying readout neuron models. To test the network's 's ability encode visual stimuli in a contrast invariant manner, we trained the network to classify MNIST digits with varying contrasts and luminances. Each network had ten bipolar cell readout neurons, one per digit.
For each digit, a contrast and luminance scaling factor was randomly chosen from the range (0.1, 1.0) and (0, 0.1), respectively. We then took the digits with contrast and luminance variation and scaled their intensities to photoisomerization rates up to 40,000 /*/s (photopic) for all digits. We stimulated cones with the photoisomerization rate associated with the pixel at its location for the first 50 ms of the simulation, and this signal then traveled to the readout neurons. The average voltage of the readout neurons over the simulation period was calculated, rectified, and transformed by the softmax function. The cross-entropy loss of the classification was then minimized via gradient descent. We trained the conductance parameter of each ribbon synapse and ionotropic horizontal cell-to-cone synapse in this setup.
For the training, we used the Adam optimizer on an exponential decay schedule with an initial learning rate of 0.01, a decay rate of 0.9 over 1000 transition steps, and gradient clipping by the global norm. We set a maximum of five epochs with early stopping at 99.5% reduction in average loss over the previous
batches. Access to NVIDIA H100 GPUs permitted a batch size of 32 digits. Training for five epochs on the full MNIST training set and evaluating on the test set required two to three days depending on the network architecture. This lengthy training time results from integrating up to 7,652 differential equations for 8,000 timesteps per gradient step.
We applied this training procedure to five different network architectures (Table I to study if the addition of bipolar cell channels and the number and structure of horizontal cell connectivity would improve classification performance on the held out MNIST test set. Later, we trained the architectures on the MNIST train split without contrast and luminance distortions and calculated accuracy on the test split at different contrast levels. We trained the network as before, and the evaluation different contrast levels required 2-6 hours of compute time on the NVIDIA H100 GPU.
Results
4.1 Biophysical cone models can be fit experimental data using gradient descent
We first fit the parameters of the phototransduction cascade on the recorded photocurrent data, and we then the parameters of the ribbon synapse model on the glutamate imaging data using gradient- based optimization (see Sec. 3.2p. We found that the photocurrent of the trained simulation fit the photocurrent data qualitatively well with a mean squared error of 0.0092 and fraction of explained variance of 0.86 for the held out test cell current recording (Fig. 2h.. Adaptation was visible in the photocurrent data as the reduction in current after the onset of a larger light stimulus. This effect was not visible at initialization adaptation only emerged after parameter optimization.
Table 2: Performance of different network architectures on classification of all MNIST digits with contrast and luminance distortions. All network architectures have 200 cones and the listed horizontal cell connectivity and readout neuron mechanisms. All network architectures are trained and tested with the same amount of contrast and luminance variation.
Readout neurons
# Horizontal cells
Connectivity
Test accuracy
Passive
Full
0.42
Active bipolar cell
Full
0.62
Active bipolar cell
0
n/a
0.63
Active bipolar cell
9
Local
0.64
Active bipolar cell 200
to
0.64
The trained ribbon synapse model matched the recorded glutamate traces both the ventral and dorsal retina (Fig. 2 Mean squared errors of the best chosen fits ranged from 0.025 to 0.437 and their Pearson correlations from 0.83 to 0.98. The fitted traces typically had the same spectral selectivity (green vs UV sensitivity) as the traces from the data with only slight deviations (Fig 2b),
7

---

r = 0.96. Deviations occurred when the traces were noisy or activity was likely influenced by surrounding cells in the experiment. Parameters were overall well constrained by the data and we did not find significant correlations between the fitted ribbon synapse parameters (Fig. 2+).
4.2 Outer retina models with and without horizontal cells can perform a visual task despite contrast and luminance variation
We then assembled 200 of these cones into network model together with horizontal cells and bipolar cells with different connectivity patterns (see Table and we trained this network to classify MNIST digits with contrast and luminance variation. We tested different model architectures with the goal of inferring which mechanism is most important for task performance. The mechanism that made the biggest difference in performance turned out to be the set of biophysically realistic bipolar cell ion channels (Table 2). The addition of horizontal cells, regardless of abundance or connectivity, did not improve classification accuracy substantially beyond that of cone photoreceptors alone. Notably, this despite the fact that they contributed 400-900 additional trainable parameters.
The addition of bipolar cell channels to the readout neurons changed the dynamics of the readout neurons substantially (Fig. 3 but did not change the dynamics of the cones or horizontal cells noticeably. The readout neurons do not synapse onto any of the other cells, so no change in the activity of the other cells was to be expected. We also investigated the learned conductances of the ribbon synapses from the photoreceptors to each of the bipolar cell readouts. We found that the spatial patterns in the distribution of conductances somewhat resembled the digit that each readout encodes, indicating that a contrast invariant representation of each digit was learned by the model (Fig. 4 The central spot of high conductance weights for readout neuron one likely arose because ones are often rotated differently but consistently occupy the center of the image.
Ribbon synapse conductances each the readout neurons
Readoou
Readout
Readout
Readout
Readout
H3
tegls (.) (sg)
Readout
Readout
-20
Figure 4: Trained conductances of the photoreceptor ribbon synapses onto each of the readouts in the architecture with 200 photoreceptors and nine locally connected horizontal cells.
4.3 Mechanistic outer retina models are robust to out-o--rriiinng-diitibbttoo contrast changes
For the best performing and most biophysically realistic architecture with 200 photoreceptors and nine locally connected horizontal cells, we tested whether the biophysical model could gener- alize to contrast levels outside of its training distribution. To this end, we trained the network on the MNIST dataset full contrast, and we then evaluated the model's performance at lower contrast levels. As references, fully connected to ten readout units, a linear classifier preceded by adaptive normalization, a linear classifier preceded by adaptive thresholding, two-layer multilayer perceptron, and the biophysical model with its initial parameters (Fig. 5l)). These models received the same input as the photoreceptors and softmax was applied to the readout activations to classify the digits.
compared our model to linear classifier with 200 units
8

---

Remarkably, we found that the biophysical model's 's classification accuracy degraded much more slowly than that of the linear classifier as the contrast level moved further out of the train- ing distribution. However, models that remove contrast variation in the stimuli via normaliza- tion or thresholding outperform the biophysical model across contrast levels. Given that the shape of their performance curves are similar, one could suspect that the biophysical model is performing a similar computation, though in biophysically explicable manner. The MPL. out- performs the biophysical model, though it uses far more trainable parameters to do so, and more importantly its computations are not tractable.
OOD test perforaance
nor
ocuraa
thresh func + Insar
mip
bipphys
Inear
blophys Int
contrast
b
OOD neurai activity
Test contrest
800
PM1
0
30
1.25
600
66
For our biophysical model, we are able to ex- plain robustness to out-of-distribution contrast levels with the evolution of state variables. As stimuli is processed, we see large variation in preliminary stages of the phototransduction cas- cade (activated opsins and phosphodiesterase activity), but then less variation in the follow- ing stages where cGMP concentration, calcium concentration, current, voltage, and glutamate release are modulated (Fig. 5p))
200
200
-60
Time (ms)
0.8
Glocodft
200
PRU
vin Qloudi
Time (ms)
5
Discussion
Here, we presented a mechanistic model of the mouse outer retina constrained by both exper- imental data and task performance. Because the model contains high level of biophysical detail, we were able to analyze how different nonlinear mechanisms found in the outer retina contribute to contrast invariant visual coding. We found architectures with bipolar cell chan- nels on the readout neurons to perform the best, but all network architectures were able to learn the task. Surprisingly, the addition of horizontal cells did not substantially improve classification accuracy in the face of contrast variations or gen- eralization across contrasts, contrary to common believe and despite an increase in the number of trainable parameters. This suggests that hor- izontal cells, which were experimentally found to modulate cone signals different role in visual processing. While it is possible that different training configurations could have clicited more horizontal cell activation, our study suggests that photoreceptor dynamics, ion channels and ribbon synapses alone are sufficient to transform a light stimulus into a well-scaled, contrast invariant quantity of glutamate release.
Figure 5: Biophysical model outperforms lincar classifier MNIST classifications outside of the contrast training distribution. a) Test accuracy at different contrast levels outside of the training dis- tribution 1.0) of the biophysical model and other reference models. Error bars for the lin- ear classifier and biophysical model across three small to be visible. b) Activ-
random seeds ity of the phototransduction cascade state variables and photoreceptor voltage for a randomly selected cone at the different test contrast levels (first three rows), and glutamate release from three random cones at the different test contrast levels (bottom left). Colors match the contrast levels in a. Coef- ficient of variation across contrast levels for each state variable, averaged over cones (bottom right) with standard error of the means all below 0.02.
[35 [I (3), may have
Our fitted parameters generally agree with previous estimates for models of the phototransduction cascade (Table (ST) and yielded traces that looked similar 100, despite the previous work having trained only subset of the parameters. Fits of the ribbon synapse parameters could have potentially been improved further with longer training, but we imposed a maximum number of epochs to reduce computational cost. An interesting avenue for future work could be the constraining of photoreceptor parameters with commulatoo-based-iffeeeee that reflect parameter uncertainty.
as done in previous work 66] and generating simulations
9

---

Previous work has shown that biophysically realistic photoreceptor model improves CNN predic- tions of retinal ganglion cell activity and enables the CNN predict activity given stimuli outside of the network's luminance training distribution [21]. This is key example of how inductive bias introduced by biophysical models can improve the processing of visual information outside of a model's is training distribution. We extend this work and show that photoreceptor layer with even more biophysical detail and other cell types found in the outer retina can accomplish the classification of images with contrasts it has never seen before.
Limitations of our modeling include the lack of more detailed horizontal cell to cone synapse. As previously mentioned, the exact mechanisms of this feedback remain unclear. Some hypotheses include various combinations of ephaptic feedback, pH-mediated feedback, and GABA reuptake of horizontal cells [.. Including more detailed synapse model would result in more accurate estimate of horizontal cell activity and its effect on cone dynamics. Such mechanisms could be incorporated in our model in future work and used to study the influence of horizontal cell activity on a variety of computations.
We also did not test the effect of the phototransduction cascade itself on classification at different contrast levels. It has been shown previously that the phototransduction cascade reduces contrast sensitivity due to the introduction of noise depending on the global luminance level [37]. Our model of the phototransduction cascade, however, does not include noise. Future work could study how the addition of noise to the photocurrents affects classification performance at the different contrast levels.
The study of how detailed biophysical models perform computational tasks is not yet commonplace. but then it is not determined what role different pieces of the model play in any computations, such as our analysis of different model architectures. Phenomenological models such as CNNs and linear-nonlinear models are also traditionally fit to data, and some computational insights can be obtained [39 40), but the underlying biophysies cannot be explained. Our work is step towards more detailed biophysical models that perform visual tasks. These models can explain computations by analysis of underlying states that map to wide variety of real physical quantities, as we show for contrast invariant encoding. In this paradigm, more complex tasks, for example including chromatic information and other statistics describing the natural environment, could provide further novel insights into retinal processing.
Biophysical models are often designed and only trained to fit data [26] 38 14
Acknowledgements
We thank Sarah Müller and Lisa Schmors for helpful discussions. This work was supported by the Hertie Foundation, German Research Foundation (DFG) through Germany's 's Excellence Strategy (EXC 2064- Project number 390727645) and the CRC 1233 "Robust Vision" and the European Union (ERC, "DeepCoMechTome"" ref. 101089288, "NextMechMod", ref. 1010391 15). Views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union or the European Research Council Executive Agency. Neither the European Union nor the granting authority can be held responsible for them. KLK, JB, and MD are members of the International Max Planck Research School for Intelligent Systems (IMPRS-IS).
References
[1] Edward Pugh Jr and TD Lamb. Amplification and kinetics the activation steps in phototransduction. Biochimica el Biophysica Acta (BBA)--ioenerggeticc, 1141(2-3):111-444 1993.
[2] Yoshimi Kamiyama, Samuel M Wu, and Shiro Usui. Simulation analysis of bandpass filtering properties of a rod photorecoptor network. Vision research, 4909)977-977, 2009.
[3] Toshihiro Aoyama, Yoshimi Kamiyama, Shiro Usui, Roman Blanco, Cecilia F Vaquero, and Pedro de la Villa. lonic current model of rabbit retinal horizontal cell. Neuroscience Research, 37(2): 2000,
[4] S Usui, A Ishihaiza, Y Kamiyama, and H Ishii. lonic current model of bipolar cells in the lower vertebrate retina. Vision research, 36(24):40694400 4076, 1996.
[5] Gregory Benison, Joel Keizer, Leo M Chalupa. and David W Robinson. Modeling temporal behavior of postnatal cat retinal ganglion cells. Journal of Theoretical Biology. 210(2):187-199) 2001.
10

---

[6] Cornelius Schroeder, Jonathan Oesterle, Philipp Berens, Takeshi Yoshimatsu, and Tom Baden. Distinct synaptic transfer functions in same-type photoreceptors. Elife, 10:e67851, 2021.
[7] Tobias Moser, Chad P Grabner, and Frank Schmitz. Sensory processing at ribhon synapses in the retina and the cochlea. Physiological reviews, 100(1):103-144, 2020.
[8] Peter Sterling and Gary Matthews. Structure and function of ribbon synapses. Trends in neurosciences, 28 (1):20-29, 2005.
Camille A. Chapot, Thomas Euler, and Timm Schubert. How do horizontal cells 'talk' cone photorecep- tors? different levels of complexity at cone-horizontal cell synapse. The Journal of Physiology. 595 (16):5495-5506, 2017.
[10] Qiang Chen, Norianne T Ingram, Jacob Baudin, Juan M Angueyra, Raunak Sinha, and Fred Rieke. Predictably manipulating photoreceptor light responses reveal their role downstream visual responses. Elife. 13:RP93795, 2024.
[11] Wallace B Thoreson and Stuart Mangel. Lateral interactions in the outer retina. Progress in retinal and eye research, 31(5):407-441 2012.
[12] Fred Rieke. Temporal contrast adaptation in salamander hipolar cells. Journal of neuroscience, 21(33): 9445-9454, 2001.
[13] Jeremy Freeman, Greg D Field, Peter H Li, Martin Greschner, Deborah Gumning, Keith Mathieson, Alexander Sher, Alan M Litke, Liam Paninski, Eero P Simoncelli, et al. Mapping nonlinear receptive field structure in primate retina single cone resolution. Elife, 4:c05241, 20155
[14] Cornelius Schröder, David Klindt, Sarah Strauss, Katrin Franke, Matthias Bethge, Thomas Euler, and Philipp Berens. System identification with biophysical constraints: A circuit model of the inner retina. Advances in Neural Information Processing Systems, 33:15499-15450, 2020.
[15] Gregory W Schwartz, Haruhisa Okawa, Felice A Dunn, Josh L Morgan, Daniel Kerschensteiner, Rachel O Wong, and Fred Rieke. The spatial structure of nonlinear receptive field. Nature neuroscience, 15(11): 1572-1580, 2012.
[16] Esteban Real, Hiroki Asari, Tim Gollisch, and Markus Meister. Neural circuit inference from function to structure. Current Biology, 189-)9889-198 2017.
[17] Chichilnisky. A simple white noise analysis of neuronal lightresponses. Network: computation in neural systems, 12(2):199, 2001.
[18] Christian Behrens, Shubhash Chandra Yadav, Maria M Korympidou, Yue Zhang, Silke Haverkamp, Stephan Irsen, Anna Schaedler, Xiaoyu Lu, Zhuohe Liu, Jan Lause, et al. Retinal horizontal cells use different synaptic sites for global feedforward and local feedback signaling. Current Biology, 32(3)5545-558, 2022.
[19] Niru Maheswaranathan, David B. Kastner, Stephen A. Baccus, and Surya Ganguli. Inferring hidden structure multilayered neural circuits. PLOS Computational Biology, 14(8):1-30, 2018. doi: 10. 10fora// pttppppcbi........ 1006291.
[20] Lane MeIntosh, Niru Maheswaranathan, Aran Nayebi, Surya Ganguli, and Stephen Baccus. Deep learning models the retinal response to natural scenes. Advances in neural information processing syxtems, 29, 2016.
[21] Saad Idrees, Michael Manookin, Fred Rieke, Greg D Field, and Joel Zylherherg. Biophysical neural adaptation mechanisms enable artificial neural networks to capture dynamic retinal computation. Nature Communications, 15(1):5957, 2024.
[22] Michael Deistler, Kyra Kadhim, Mathhij Pals, Jonas Beck, Ziwei Huang, Manuel Gloeckler, Janne Lappalainen, Cornelius Schröder, Philipp Berens, Pedro Gonçalves, Jakob H. Macke. Differentiable simulation enables large-scale training of detailed biophysical models of neural dynamics. bioRxiv, 2024.
doi: 10.1 101/200//222.0 0..0.999
[23] Janne Lappalainen. Fabian Tschopp. Sridhama Prakhya, Mason McGill, Aljoscha Nern, Kazunori Shinomiya, Shin-ya Takemura, Eyal Gruntman, Jakob H Macke, and Srinivas C Turaga. Connectome- constrained networks predict neural activity across the fly visual system. Nature, 132488036):1132-114444 2024.
[24] Daniel LK Yamins, Ha Hong, Charles Cadieu, Ethan A Solomon, Darren Seibert, and James DiCarlo. cerformance-opimmize
Performance optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the national academy of sciences, 111(23):8619 182444866118664 2014.

---

[25] Shahab Bakhtiari, Patrick Mineault, Timothy Lillicrap, Christopher Pack. and Blake Richards. The functional specialization of visual cortex emerges from training parallel pathways with self-supervised predictive learning. Advances in Neural Information Processing Systems, 34:25164-2511 178, 2021.
[26] Seungbum Back, Jason K Eshraghian, Wesley Thio, Yulia Sandamirskaya. Herbert HC lu, and Wei D Lu. A real-time retinomorphic simulator using a conductance- based discrete neuronal network. 2020 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (A/CAS), pages 79-83. IEEE, 2020.
[27] Adrien Wohrer and Pierre Komprobst. Virtual retina: a biological retina model and simulator, with contrast gain control. Journal of computational neuroscience, 26:219-22 209.22494422 2009.
[28] Keith Ly, Michael L Italiano, Mohit N Shivdasani, David Tsai, Jia-Yr Zhang, Chunhui Jiang, Nigel H Lovell, Socrates Dokos, and Tianruo Guo. Virtual human retina: Simulating neural signalling, degeneration, and responses to electrical stimulation. Brain Stimulation, 2025.
[29] Alan L Hodgkin and Andrew F Huxley. A quantitative description of membrane current and its application to conduction and excitation in nerve. The Journal of physiology, 117(4):500, 1952.
[30] LF Abbott and Eve Marder. Modeling small networks, 1998.
[31] Chang-Jin Jeon, Eurica Strettoi, and Richard Masland. The major cell populations of the mouse retina. Journal of Neuroscience, (8(21):8936-894) 1998.
[32] James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclau- rin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne and Qiao Zhang. JAX: composable transformations of Python+NumPP programs, 2018.
[33] Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6988 2014.
[34] Klaudia P Szatko, Maria M Korympidou, Yanli Ran, Philipp Berens, Deniz Dalkara, Timm Schubert, Thomas Euler, and Katrin Franke. Neural circuits in the mouse retina support color vision in the upper visual field. Nature communications. 11(1):341, 2020.
[35] Camille A Chapot, Christian Behrens, Luke E Rogerson, Tom Baden, Sinziana Pop, Philipp Berens, Thomas Euler, and Timm Schubert. Local signals in mouse horizontal cell dendrites. Current biology, 27 (23):3603- 1,155,33365,,,, 2017.
[36] 11 Keffer Hartlne and Floyd Ratliff. Inhibitory interaction of receptor units in the eye of limulus. The Journal of general physiology, 40(3):357-376, 1957.
[37] Nicolas P. Cottaris, Brian A. Wandell, Fred Ricke, and David H. Brainard. A computational observer model spatial contrast sensitivity: Effects photocurrent encoding, fixational eye movements, and inference engine. Journal of Vision, 20(7):17-17, 07 2020. ISSN 1534-7362.
[38] Cornelius Schröder, Ben James, Leon Lagnado, and Philipp Berens. Approximate bayesian inference for a mechanistic model of vesicle release at a ribbon synapse. Advances in Neural Information Processing Systems, 32, 2019.
[39] Larissa Höfling, Klaudia P Szatko, Christian Behrens, Yuyao Deng, Yongrong Qiu, David Alexander Klindt, Zachary Jessen, Gregory W Schwartz, Matthias Bethge, Philipp Berens, et al. A chromatic feature detector in the retina signals visual context changes. Elife, 13:c86860, 2024.
[40] Damon A Clark, Raphael Benichou, Markus Meister, and Rava Azeredo da Silveira. Dynamical adaptation in photoreceptors. PLoS computational biology, 9(1])=e003228) 2013.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The model we proposed solves a classification task with contrast variations well above chance performance, our comparison of model architectures yielded consistent results for multiple hyperparameter settings, and many versions of our model presented higher test accuracy than a linear classifier outside of the contrast training distribution though we show only one example.
Justification:
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We discuss limitations of our work in the discussion section, especially improvements to our model that could be made, and we mention limitations in the methods where applicable.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: Our paper does not present any purely theoretical results or proofs.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Our methods section and supplementary materials provide enough detailed information for the model to be recreated and retrained as we did with the data we used. Code to run the model will be provided.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide a link to the code required to reproduce our results and analysis in
section 3.1
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: How each dataset was used to train each piece of the model and the training
procedure was explained in sections 3.2 and 3.3
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
13

---

Justification: We ran our experiments with multiple random seeds and found negligible differences, not even visible on plots. Fig. error bars are slightly visible for the linear classifier. Such deterministic biophysical models are very consistent in the results they produce.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: In sections 3.2 and 3.3 state the GPUs and approximate execution times of
the trainings and experiments. Most cach GPU's VRAM is used during every training.
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: /ate:://neurippp.ee http://bbc//ttthccciiieeee...ee.
Answer: [Yes]
Justification: Our paper develops a model of the outer retina to answer basic science questions about the computational role of different cell types. We do not any conflitt with the Code of Ethics.
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [NA]
Justification: We do not see any immediate positive or negative socictal impact. In the future, one could imagine medical applications our model better understand disease processes in the retina.
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: We do not see any high risks associated with releasing our model.
Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Experimental researchers that acquired the retinal datasets used were appropri- ately cited. Code libraries used were also cited.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: All the models implemented as part of the paper will be publicly available. Some are already publicly available.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
14

---

Justification: We did not perform research with human subjects.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Since we only reused already published datasets, IRB approval neces- sary.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: LLMs were only used for basic code completions and some code suggestions, but this work was by no means dependent on them.
15

---

A Photoreceptor parameters
Table S1: Comparison of fitted photoreceptor parameters to previous estimates
Parameter
Description
Chen et al. 2024
Ours
Our bounds
Opsin decay rate
9.74
14.2
(5, 24)
PDE decay rate
9.74
12.4
(5, 24)
PDE dark activation
761
774
(750, 800)
Gdark
Dark cGMP concentration
15.9-20
23.5
(12, 25)
k
cGMP-to-current factor
0.01
0.015
(0.008, 0.022)
n
cGMP channel cooperativity
2.88
(2.8, 3.2)
Cdark
Dark Ca concentration
1.03
(0.8, 1.2)
B
Ca
extrusion rate
2.64
4.42
(2.5, 10)
m
Cooperativity of GC Ca²+ dependence
4.10
(3.75, 4.25)
KcG
Affinity of GC Ca dependence
0.4
0.348
(0.2, 0.6)
Opsin gain
10
10.6
(1,
B Ion Channel Equations
The implementations of all ion channels can be found in the repository stttps//giiie thub. com/ Baaleeyrere///aalllyymmm///re//e
B.1
Standard form
Gating variables m and take the following standard form unless otherwise defined.
dm mas mcs(V) (V) m
dh 110 (V)
dt Tm (V)
dt (V)
am (V) (V) = am (V) + BV)
(v)
100 (V) = (v) + Bn (V)
1
1 (V (v) + (v)
[V) am am(V) + (V)
B.2 Photoreceptors
Ion channel equations for photoreceptors were found in Kamiyama et al. 2
IKv gk.mmh ek)
5(100- V) (100-V)/42
Om (V)
(x = 0.15e V/222
BV( 0.4125 (10
Bn = -VV-0000)
gKv = 10-3 Slem?, eк -74 mV
Inyper Phyper + S2 + Sz)(V Chyper)
16

---

dC1
C₂ - 41
dt
dC2 4a 2 - (3an + B₂
di
8
(v e(V+78)/14
dSP
3ax + 33,,552 (2an, + 21
dt
18
Bn(V) e +
dS²
S1 + S3 (an, + 33,)SS2
dt
S2 - S3 dSg
dt
Jhyper 3x x 10-3 S/cm², Chyper -32 mV
Ica = Dcam - eCa)
3(80 V)
am (v)= = e(80-V)/25 -
(440-V)/18)
M(V) + V3/1-
10
(v) e(V+388)/7 +
gCa = 0.7 x 10-3 S/cm²
Calcium dynamics also involve the calculation of eça and Ca, as follows.
eCa 12.55 Tog(Caa/Ca.)
Ca, - Cac
IEs Jex e (V+14)777
Ca. Cae Kew
Ca. Cae
I = Jex2 Ca, Ca. Kex2
dCa, 10 (1ca(Iaa IEa IEa2)
DeaSi(Ca, - Cas) Lii Ca.(Ba - Cap.1s)
dt
2FV
8V1
Loz Laz Hi1Ca,(Br, Cab,ho) + HbzCas,he
dCa;
DCas,(Ca, Caa) L7i Cay - C(B,I)) + La2Cas.15
dt
8V22
Ha,Cay(Br Cab,as) HuzCab,hs
dCas.s LnCCa,(B, Cas,is) LugCas.1s
dt
Hor Cos - -Cabhaa) Hap dCar,h
dCav.1f
Ln Ca, (B, Can.f) Log Cap.1
dt

---

dCas.hf dt
H91 Caj - Cab,Ny) HabCab
= 9.648 x 10a C/mol, V1 3.812 x 10 dm 1/2 = 5.236 x 10 dm Dc = 6 10 dm²/s, 8 = 3 10 dm, 8 = 3.142 x 10 dm Lo1 0.4 1 L -0.2s-1
Ha1 = 100 s AM 11 Hbs = "
500 μM, B,, = 300 μM. Jez = 20 pA. Jezz = 20 pA.
KE 2.3 μM, Kerz 0.5 μM, Ca. 0.01 μM, Ca. 1600 μM
[K(Ck) = 9K(ca)m ek)
O'm O' (V) 15(80- V)
(800V)/40
Cas
n(Cag, K1)2) Cas + K1/2
(V) 20e3V/35
IK(Ca) = 5 x 10-3 Slem?, K1/2 = 0.3 mM, 6x = -74 mV.
ICu(Ca) (CI(CC) - ECI(Cw))
m(Cag, K1/2)= = + e(1/22 -a.l//0
Ca.1/0.09
(CI(Ca) =2 10-3 S/em², K1/2 = 0.37 μM, ecu(C)) -2 m..
1eak -eakk (V Eleak)
Jleak = 0.35 x 10 S/cm², Eleak = 77 mV.
B.3 Horizontal Cells
Ion channel equations for horizontal cells were found in Aoyama et al. [
Teak
Peeale V Cleak)
Dlenk 0.5 10-3 S/em², Cloak 80 mV.
NNa INamm ENa)
Om (V) 200(38 - V)
(v) = 1000e +8088)
e(38-V)/25 -
800
Bm 2000e 2000200000+00000 (65++
Bv( e(80-V)/75 +
UNa 2.4 x 10-3 S/em², eNa 55 mV.
IK IKar m m/h(V ek)
1500
(V) = 0.4(65 - V)
(V e(V +92)/7 + 1
(65-V
80
8m(V) = 4.8e
(V = 0.02
e(V+100)/15 +
gKdr 4.5 - S/em² eK 88 mV.
18

---

Ik 9k.. m² ex)
2400
(v -V660
(ml))
1 1+c 6 c-(--50)/28
20
(v) -V/36
Br(V) e e-V+-40/5 +
9k.o = 15
x x
10-3 S/em², EK = -0 mV.
IKa. - -K... (V ek)
m(V) + e(V+60)/12
_k = 4.5 x 10 Slem², 80 mV.
Ica 90am (V - -CCa)
240(68 V)
aml c(68-1)/22 V)/21
800
(m( = c555+V)/55 +
gCa 3.3 10-3 S/em², eCa 54.176 mV.
B.4 Bipolar Cells
Ion channels for the bipolar cell readouts use the leak, Ky, KA, and hyperpolarization-aciivvee channels of Usui al. 4 and the calcium dependent channels of Benison et al. (5). Calcium dependent channels come from this second source aimed at modeling retinal ganglion cells to reduce the computational complexity of the calcium pump.
heak 9leak (V Cleak)
Deakk 0.23 10-3 S/em2, Gleak -21 mV.
Jky 9ky m/h ek)
400
(V) = 0.0003e V/7
(v) e- 1-V-166)3 +
80
Bm(V) -
(V = 0.02 + e(V+115)/15 +
gKv 2 10-3 S/em?, eK -8 mV.
JAA gKAmph ex)
2400
(v 0.045e V/13
0m (m(( e- 2-VV-50)25 +1
75
(m( 12c V/10
(V 3VV+300)115
19

---

9KA 35 x 10-3 S/cm", = -58 mV.
Jnyper
Ihyper (S1 (hyper(S1 + ehyper)
BrCC - C1 dC1
dt
dC2
41 + 2,3551 - 3an - Br. C2
df
dSi
3a + 33, - 21 23
dt
dS²
21 + 43h S2 -3
dt
dSg
S2 -3
dt
3
an(V)= = c(V+110)/1 15
1.5
Bv(V) eV(V+115)/1 +
Phyper 0.975 10-3 S/em2 Chyper 17.7 mV.
ICar. gCa, mam cCa) (Ca)
0.061(V - 3)
am((V 1 CVVV-3312255
Pm(() 0.058e -VV-00/15
gCal 2 10-3 S/em².
ICAN 9Cay 9Cay m2h(V - eca)
(V) = 0.1(V - 20)
(n 0.01e 0.01e-(V+50)/1 -V+50)/10
1-6 e -0.1(VV-23)
0.1
(v) = 0.4e -V-+55)/18
Bv(V) + +V177/117
gCax 1.5 10-3 S/em?
I
IKCs = IKCa
(V - -ek)
1+x
CC2+1 2
I
gKca = x 10 Slem2, KKe. 0.6 10-3 mM, ek -5 m..
20

---

The calcium pump and equilibrium potential calculation is as follows.
iCa = =ccax + ICa,
Voll TT²1
Tstore
Tell
ft
[Ca 21
jpmmp
Upump
[Ca 2+ } + Kdd
pump
10000 ica 2F 0
if 10000 ics
0
2FV cell
Jehannel
otherwise
[Ca2+
[Ca2+] leg
Tstore (Jehannel }pump)
d[Ca²*],
Caa 2+ 11.00 -
[Ca2*]
dt
Teff
RT
[Ca2+]o 0 [Ca2t];
CCa
1000 In
2F
r= Jum, = ya, 96485.3329 C/mol, T 279.45 K, R= 8.314 J/mollK).
[Ca²*], 2.0 mM, (Ca2t oa 1 x 10-4 mM, Tstore 12.5 ms, K pumpp = 1 x 10-1 mM,
pump = 7.2 x 10 mM/ms, f. = 0.025.
21


Reviews:
[{"review_id": "iKkT40tArK", "reviewer": "Reviewer_RpSH", "summary": "This paper introduces a novel framework for training and inspecting models of biological neural circuits, applying it specifically to the under-appreciated processing of the outer retina. The contributions are twofold: first, the authors take the nonlinearities of phototransduction in the outer retina seriously and attempt to understand how they interact with downstream processing (whereas the vast majority of adjacent work make assumptions of linearity). Second, they demonstrate a dual-stage model-fitting procedure that optimizes both biophysical fidelity (using priors and data from literature and experiments) and subsequent performance on an ethologically-plausible task (contrast invariance). Their model, specifically, is a dynamic model of the outer-retina fit to both whole-cell electrophysiology from cones and two-photon glutamate sensor imaging from ribon-synapses, providing a detailed and data-constrained simulation of how light is communicated by the cone photoreceptors to downstream circuitry. They then extend this model to include a variety of architectures that add bipolar and horizontal cells in a manner that enables classification of the MNIST dataset by decoding readout bipolar cell activity. Importantly, they evaluate their model not on MNIST alone, but on out-of-distribution samples that have varied contrast and luminance. The biophyiscal front-end of their model is accurate and captures temporal phenomena (adaptation); the necessity / sufficiency of various aspects of the downstream decoding circuitry are used to make hypotheses about the role of different cell types in the known phenomena of contrast invariant processing.", "questions": "1. In figure 5, the comparison of a linear classifier with the biophysical model is a good control, but only one of a few that are needed. In order to test whether it is architectural (not actually due to the learned parameters) the authors should also include the initialized biophysical model, to rule out that the OOD performance is not merely a result of the addition of dynamics. The authors could also evaluate using the same pipeline, with a much simpler dynamical system (see the LNK model in Ozuysal and Baccus, 2014) in order to get a sense of whether the complexity of the dynamics model is actually necessary. Additionally, what if the models are trained on a variety of contrasts, not merely tested on them? One could imagine that varying contrasts could confuse a linear classifier during training, but not your biophysical model.\n\n2. In general, I think the authors are missing an opportunity to investigate the temporal nature of contrast / luminance adaptation. The authors present MNIST data to their biophysical models at varying levels of contrast, and do so in a temporal manner - however, contrast / luminance invariance is a dynamic process that involves the adaptation of a response to the mean contrast or luminance over a specific window of time (Baccus and Meister, 2002). Thus, I would appreciate some exploration of how the model behaves during the transition between high and low contrast regimes, not just evaluated at specific instances of varying contrast. Since contrast invariance is mediated by adaptation, this could be incorporated into the training of the readout network and structure of the task (if the input to the model were, for example, a sequence of frames for one digit with randomly fluctuating luminance and contrast). I imagine a linear model, even trained on inputs with fluctuations in contrast, would have quite a bit of trouble with this task. \n\n3. Given the highly complex biophysical model of cones, I would like to see some preliminary attempt to understand an aspect of the biophysical model of the cones. Since the model is differentiable, the authors could identify which of the parameters / states contributes most to contrast adaptation. Several strategies could be taken - randomly ablating specific states within the dynamics model could identify the crucial mechanisms governing its behavior. Alternatively, looking at the evolution of these state variables during adaptation (or the gradients) could offer clear hypotheses about what aspects of the phototransduction cascade govern contrast invariance.\n\n4. The authors suggest that the highly-complex biophysical model offers an inductive bias that imbues the network with new capabilities (OOD performance on differing contrasts and luminance on MNIST). This is quite a strong claim, given that the machine-learning community already has neural-network primitives that provide similar computational mechanisms without the need for such computationally costly dynamics models (layer-norm, for example). The authors should make explicit that the utility of the model is in gaining insights into their biophysical system - or, if the claim is that some aspect of the biophysical model enables a new computational capability (beyond a vanilla neural network), the authors should draw parallels between more standard machine-learning techniques (layernorm) and provide comparisons.\n\n5. In the retina, contrast adaptation is seen dramatically in amacrine and ganglion cells - but this reviewer's understanding is that it is a much smaller effect in photoreceptors. Please provide some contextualization of the relationship between contrast invariance in the outer retina and the inner retina, and provide a bit more justification for evidence that horizontal cells were ever considered as a putative mechanism thereof.", "limitations": "MNIST is semi-naturalistic but is not really a naturalistic stimulus for any organism. I would encourage the authors to discuss more how their results can and cant be extended to a general understanding of the processing of naturalistic vision with spatiotemporal correlations and structured fluctuations in contrast and luminance. In future work, different tasks could be used that better approximate truly naturalistic vision (cifar, imagenet, video-classification, etc).", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Overall, the main strength of this paper is its creative framework for evaluating biophysically-constrained networks on in-silico tasks, as a means of probing the functional impact of specific biophysical parameters. As mentioned by the authors, recent years have brought a large number of papers that optimize models of neural encoding directly to data and by comparing the representations of task-trained networks to experimental data - the author's approach (training to data, then evaluating on an ethologically-meaningful task) has significant potential across a wide range of questions. The paper is written clearly and the approach is well-motivated, however the scientific results of the paper, while well-reasoned, lack important controls and further analyses in order to demonstrate the potential of this new biophysical / task optimized model.\n\nSpecifically, the authors motivate their biophysical model architecture by way of its complexity and fidelity to known processing in the outer retina (in addition to the fact that it is differentiable, which this reviewer does not find particularly novel, given the widespread use of autograd functions). It is unclear, however, whether all of this complexity is actually needed for their ultimate conclusions. As described in the paper, their cone-photoreceptor model involves 26 differential equations and this faithfulness is suggested as a mechanism for their downstream task-performance on contrast and luminance invariance testing data. However, the author's do not cite other papers that capture such phenomena using many fewer parameters in a number of different retinal cell-types (including photoreceptors). For example, in \"Linking the computational structure of variance adaptation to biophysical mechanisms\" (Ozuysal and Baccus, 2014) - contrast adaptation can be captured using a much simpler dynamical system with many fewer parameters (though they do not appear to investigate photoreceptors). However, \"Dynamical Adaptation in Photoreceptors\" (Clark, ..., Meister, Azaredo da Silveira) do build a model of contrast adaptation in photoreceptors that captures the phenomena. Since much simpler models can capture the phenomena of contrast and luminance invariance - the burden lies with the authors to demonstrate that the complexity of the biophysical model is either useful functionally - or permits some deeper insight. The former could be demonstrated via additional controls. The latter is demonstrated cursorily by suggesting that horizontal cells are not required for contrast invariance.\n\nIn regards to the task-evaluation of the biophysical model, the actual performance demonstrated is quite low, given that even simple MLPs can achieve >95% accuracy on MNIST. The reason is presumably that the task is not vanilla MNIST, but the contrast-invariance test. Thus, providing some more standard baselines to show how well standard architectures can perform on OOD evaluation would reassure this reviewer as to the significance of an accuracy of 0.6 (in Table 2). The authors include the \"Linear\" model, but in order to account for the additional parameters in the biophysical model, the authors could include a biophysical model at initialization or biophysical models with scrambled dynamics parameters.", "was_revised": true, "final_justification": "After thoughtful responses from the authors, I have changed my score to an accept. This is specifically because I think their responses indicate that they have drawn important links to other areas of work and have explored the kinds of controls I was hoping to see in the original work.", "rebuttal": "We thank the reviewer for taking the time to review our paper and provide helpful feedback. We are encouraged that the reviewer called our framework “creative,” our model “detailed” and “accurate,” and our paper “well-motivated.” \n\nA main concern of the reviewer is that we do not include more simple controls. For testing out of distribution performance, we have now tested a variety of other linear and nonlinear architectures for comparison. Specifically, we have tested the linear classifier with layer normalization (which performs extremely well out of the training distribution, as expected), a two layer MLP which also performs well out of its training distribution but with a much greater number of trained parameters, the initialized biophysical model as suggested by the reviewer (see answer to Q1 below), and a couple of other references suggested by other reviewers. These experiments have shown us that our biophysical model could be implementing the computations of a linear model with layer normalization, but our biophysical detail allows us to provide an account of how this computation is implemented with the cell’s molecular components. \n\nSimilar to these artificial architectures, models such as the one proposed by Clark et al. are phenomenological models, and thus target a different level of understanding of the biophysical system. While phenomenological models can provide interesting insights, they do not provide intermediate steps of computation that can be validated as our model does. To highlight this, we have also plotted the state variables of the phototransduction cascade when performing the classification task on out of distribution stimuli. We are happy to include this as a supplementary figure for our paper, or potentially in the main text if space allows. This figure shows that the photopigment and phosphodiesterase activity change greatly as the contrast moves further out of distribution, but that the cGMP concentration is quite robust, and then in turn the downstream calcium concentration, photocurrent and photoreceptor voltage. Our Fig. 5 currently only shows the glutamate traces as contrast moves further out of distribution, but we are happy to show additional state variables that make predictions about the biophysical mechanisms supporting contrast invariant encoding in even more detail.\n\nThe reviewer also voiced concern that our classification accuracy is not high. While reasonable performance is important, the goal of our paper is rather to understand how a biophysically realistic system could implement contrast invariant visual processing and what quantities are most decisive. Biophysical systems are much more constrained than artificial architectures because information is transmitted by a variety of bounded physical quantities.  The biophysical model without the horizontal cells also does not have more trainable parameters for learning the task than the linear classifier, it has 2,000 weights while the linear model has 2,010, and the biophysical model with its initialization parameters only achieves chance accuracy at any contrast level. We have, however, included other reference models as previously mentioned which provide guidance on what computations need to be implemented to solve the task. \n\nFinally, the reviewer notes a limitation that MNIST is not naturalistic. We would like to emphasize, however, that our work is one of the first to train highly-detailed biophysical models on any task such as contrast-invariant classification and our contrast-modulated MNIST provides a reasonable test case for this. We are very excited to see in future work if our results still hold when processing other images and videos. \n\nTo address the questions of the reviewer:\n1. To assure the reviewer that the out of distribution performance is not due to model architecture, we see that the model only achieves chance performance on the task at initialization, even for stimuli within the training distribution. The model weights are initialized from normal distributions centered at values chosen to promote stable dynamics throughout training. To answer the subquestion of task performance if the models are trained on a variety of contrasts: We do train our biophysical models on contrasts chosen randomly in Tab. 2. When training the linear model this way, it actually achieves a high test accuracy of 87%. The linear model is able to achieve high classification accuracy on data within its training distribution.\n2. The reviewer asks about more analysis on the temporal nature of contrast and luminance adaptation: We think this is a great idea! The linear model we test actually has no concept of time; it simply takes a snapshot at the peak stimulus intensity and uses that for classification, so we would have to reformulate this reference model. However, for this rebuttal, we did briefly test whether our trained biophysical model would maintain its test accuracy on stimuli with contrast fluctuations throughout the stimulation period. The test accuracy in this case only reduced to about 48% (without any additional training). Future work could extend this analysis.\n3. The reviewer suggests a “preliminary attempt to understand an aspect of the biophysical model of the cones.” We thank the reviewer for this suggestion, and we refer them to our previous discussion of the other state variables in our model, a plot that we can add to the paper.\n4. We will be more explicit that our goal is to obtain biological insights into contrast-invariant visual encodings while adding our comparison to layer normalization and other architectures as discussed previously. \n5. A major goal of this paper is to enhance appreciation for the computational power of the outer retina which extends beyond the linear filtering of stimuli. Horizontal cells are positioned to provide lateral feedback to photoreceptors, a phenomenon first observed in the 1950s and 60s in Nobel prize winning work by H. K. Hartline and colleagues. Lateral feedback is in reference to the case where a strongly excited cell inhibits its neighbors, for example by exciting an inhibitory interneuron. This action can enhance contrast by creating a larger difference between the activity of the two cells. Such inhibition was observed in the photoreceptors of Hartline’s horseshoe crab and in other species since; however, other findings have conflicted our understanding of what horizontal cells do. For example, Fred Rieke wrote in 2001 that contrast adaptation was likely to first occur in bipolar cell dendrites after showing that a model of horizontal cell activity did not change at different contrast levels but the bipolar cell model did. The suppression of horizontal cell input to bipolar cells also did not change contrast adaptation in this study. We only briefly discussed this history and motivation in our introduction, and we would be happy to add more information. \n\nWe hope that the additional clarifications and experiments increase the confidence of the reviewer in our paper and potentially allows them to raise their score."}, {"review_id": "aBzCmldTio", "reviewer": "Reviewer_ApBk", "summary": "The manuscript presents a fully differentiable, biophysically detailed model of a mouse outer retinal patch—including cone photoreceptors, horizontal cells, and bipolar cells—implemented in JAX. Cone parameters are constrained by fitting to patch-clamp photocurrent and two-photon glutamate imaging data, after which the assembled network is trained end-to-end on a contrast- and luminance-varying MNIST classification task. The findings are interesting. Firstly, cone phototransduction and ribbon-synapse nonlinearities alone produce contrast-invariant representations sufficient for classification. Secondly, adding horizontal-cell feedback does not measurably improve performance beyond the cones plus bipolars model, contrary to prevailing hypotheses. Finally, the biophysical model generalizes better to out-of-distribution contrasts than a linear baseline.", "questions": "1. In Figure 1, on the left panel, I'm unclear what the y-axes for \"Photocurrent\" and \"Glutamate release\" actually measure. What do the green and magenta square signals represent? On the right panel, what do the different luminance levels of the MNIST squares correspond to, and how are those mapped to photoreceptor grayscale intensities? Why is only a single cell shown under the HC? The BC readout traces need clear axis labels- what do their x- and y-axes denote? Finally, I don't see Figure 1 referenced in the main text.\n2. The paper does not spell out the voltage- and time-dependent formulas for the gating variables $𝑚$ and $ℎ$ in Eq. (1).\n3. The authors defer the photoreceptor ion-current formulas to Kamiyama et al. (2024), providing the explicit voltage‐ and time‐dependent expressions for each current term (e.g. $ I_{Kv} $, $ I_{h} $, $ I_{Ca} $, $ I_{K(Ca)} $, $ I_{Cl(Ca)} $, $ I_{Leak} $) and for the gating variables $ m(V, t) $ and $ h(V, t) $ in Eq. (1) would greatly improve clarity. Writing those equations out explicitly (or at least in a Supplementary Material) would also help the readers to understand the model better.\n4. In Table 1, the switch from \"local\" to \"one-to-one\" connectivity (full connection) seems very abrupt. How does horizontal cell connectivity gradually evolve from local to full connection?\n5. In Eq. 9, it has two periods at the end of the equation.\n6. Some expressions are not consistent. For example, line 101 \"equation\", while other places use \"Eq.\" in line 118.\n7. I suggest that the authors put the main formulas in the main text and put the detailed derivations in the Supplementary Material.\n8. The acronym should be written in the first time it appears in the text. For example, HCs refers to horizontal cells in line 127.\n9. Can authors explain that why horizontal cells can perform a visual task despite contrast and luminance variation intuitively? Moreover, I suggest that the authors should add to the abstract to explain the mechanism of this finding.", "limitations": "1. MNIST offers a highly simplified and low-dimensional dataset, while natural scenes exhibit far richer spatial and color statistics, so it would strengthen the work to evaluate the model on more complex benchmarks (e.g., CIFAR-100, ImageNet, etc.). These tests would demonstrate whether the retina's contrast-gain and invariance mechanisms generalize beyond handwritten digits to real-world structures.\n2. Section 2 mentions prior models; thus, the discussion needs to discuss them in detail, especially explicitly comparing your findings with those frameworks.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**strengths:** The manuscript has novel integration of data and task constraints. Fitting 2,000 and 2,900 biophysical parameters by both experimental recordings and a high-level visual task is a technically impressive advance. Moreover, the photoreceptor model comprises 26 differential equations capturing the transduction cascade, calcium feedback, and ribbon-synapse dynamics. The finding that horizontal-cell feedback adds little to contrast-invariance challenges established theories and opens new questions about horizontal-cell function. Finally, the manuscript begins with code, which is suitable for reproducibility and community extension.\n\n**Weaknesses:** 1.  The cascade model lacks stochasticity, which is important for the retina to perform contrast adaptation.\n\nReference: Cottaris, N. P., Wandell, B. A., Rieke, F., & Brainard, D. H. (2020). A computational observer model of spatial contrast sensitivity: effects of photocurrent encoding, fixational eye movements, and inference engine. Journal of Vision, 20(7):17. https://doi.org/10.1167/jov.20.7.17 \n\n2. In line 136, $ \\tau_{s} = (1 - s_{\\infty})(k^{-})$ is not correct. Please check the equation in the cited paper (ref. 30, page 20, eq. 18). Please also check if the findings still hold for the correct equation (**This is my major concern**).", "was_revised": true, "final_justification": "The authors have thoughtfully addressed my concerns in their rebuttal materials, providing thorough and compelling responses. I recommend this paper for acceptance.", "rebuttal": "We thank the reviewer for the detailed assessment of our work and for their valuable feedback. They found our work to be “novel”, inviting to “community extension” and a “technically impressive advance” in the field.\n\nThe reviewer raises an interesting point about stochasticity as discussed by Cottaris et al. 2020.  We did include a citation to this paper in our discussion section and discussed how this was a limitation of our current model. However, we would like to reiterate that our model of photoreceptor activity is already highly detailed and the photocurrents are trained on actual data with a noisy stimulus. We also note that most biophysical models do not consider noise, likely because the mathematical form of realistic noise in these systems is difficult to define. This is an interesting problem for future work.\n\nThe second major concern of the reviewer was the equation of our ionotropic synapse model. This was an unfortunate typing error in our manuscript. The model synapse does in fact correctly implement the synapse as described in Abbott & Marder 1998, with $\\tau = (1-s_{\\infty}) / k_-$. We thank the reviewer for pointing this out and we will correct this in the text.\n\nTo address the reviewers questions:\n- The reviewer asked for clarifications about Figure 1: The photocurrent y-axis is the patch-clamp measured photocurrent measured in pA but normalized by the dark current and the glutamate y-axis is the change in fluorescence of the experimental glutamate indicator from the baseline fluorescence. The green and magenta images show the images shown to the region of interest on the retina in the experiment where the data was collected. The luminance levels of the MNIST squares are converted to photoreceptor stimulus intensities in P*/s, and the model photoreceptors each have an (x, y) location that indicates what pixel of the digit they see. We show only a single horizontal cell in the diagram because this is one of the architectures we train, and showing more horizontal cells would look a bit messy. The BC readout traces are simply example voltage traces of arbitrary size and timescale, depicted only to sketch the task setup. We did indeed not reference this figure in the manuscript and thank the reviewer for pointing this out to us. We will clarify this in the main text of the paper and add references to Fig. 1.\n- Regarding equations, we will add the general form of the gating variable equations to the main text. They differ slightly depending on the ion channel, but their general form is $\\frac{dm}{dt}= \\alpha(V)(1-m) - \\beta(V)m$. We will also add the ion channel equations to the supplementary materials. We also thank the reviewer for pointing out several small formatting errors which we will of course fix in the revised manuscript.\n- Local horizontal cell connections refers to an architecture where each horizontal cell is connected to its 50 nearest cones. This means that cones have the opportunity to influence one another’s activity via horizontal cells, likely being close to what is observed in the real retina. The one-to-one connectivity pattern is designed to model a case in which each cone photoreceptor gets feedback from an electrically isolated segment of horizontal cell. In this architecture, cones still receive horizontal feedback, but they are not able to influence one another via the horizontal cells. We will clarify this further in the text.\n- An intuitive explanation for why horizontal cells could contribute to contrast invariant encodings is that they are positioned to provide photoreceptors with lateral feedback. Lateral feedback is in reference to the case where a strongly excited cell inhibits its neighbors, for example by exciting an inhibitory interneuron. This action can enhance contrast by creating a larger difference between the activity of the two cells. Such inhibition was observed in wet-lab experiments with photoreceptors in the 1950s and 60s; however, other findings since have conflicted our understanding of what horizontal cells do. A potential intuitive explanation why horizontal cells would not contribute to contrast invariant encoding is that highly stimulated photoreceptors themselves perform enough local luminance adaptation so that they do not elicit enough horizontal cell activity to inhibit their neighbors. We are happy to add a summary of these explanations to the abstract.\n\nWe agree with the reviewer that more complex and natural stimuli will be an interesting direction for future work. We would like to emphasize, however, that our work is one of the first to train highly-detailed biophysical models on any task such as contrast-invariant classification and our contrast-modulated MNIST provides a reasonable test case for this.\n\nUsing more complex stimuli is, however, high on our list of things to study. Because we trained our photoreceptor models on stimuli with color, we have estimated how sensitive each cone is to green and UV light (the two opsins present in mouse cones). Thus, we could also study the role of different cone types in different retinal regions in pattern discrimination. In the current paper, we restrict ourselves to the outer retina’s processing of stimuli with varying contrast and luminance.\n\nTo another point of the reviewer, we are happy to provide more discussion of previous models in the discussion section. None of the models in previous work were both biophysically detailed and made predictions about cellular mechanisms responsible for the contrast invariant encoding of visual information. Baek et al. developed a model with similar detail to ours, but its ability to perform any sort of computation was not tested (2020). Another retina simulator developed by Wohrer and Kornprobst used a phenomenological model of the outer retina. This model includes a baked-in mechanism for contrast gain control, but it cannot be used to study what existing biophysical mechanisms might be responsible for such a computation (2009). We can continue to provide detail.\n\nWe thank the reviewer for their valuable feedback and kindly ask them to raise their score given our clarifications, additional experiments and modifications to our paper."}, {"review_id": "dG8KnlOqfL", "reviewer": "Reviewer_N9j5", "summary": "The paper introduces a fully differentiable biophysical model of the mouse outer retina to understand contrast invariant coding via different cell types. The authors constrained some parameters with measurements while other were optimized on MNIST, but with varying contrast and luminance. The authors show that their model can classify digits in spite of these stimulus variations and identify the most important model mechanism determining classification performance. They also compare their model to a linear classifier and find more robust task performance to out of distribution contrast levels.", "questions": "1. Can the authors assess the utility of the biophysical model for contrast invariant processing on this task? Do the complex non-linearities of the biophysical model actually matter for performing the classification task? To me it seems that a simple thresholding operation would be sufficient as a preprocessing step - or at the most normalization plus thresholding. This is the key point for me. I would increase my score if the authors can show that there is more to the model than what simple operations can do in the context of this task.\n\n2. In the out-of-distribution task, how well would the linear model do if trained on a range of different contrast levels? This would provide a better idea of the overall performance that the linear model can reach and help to put the results into context.\n\n3. In the out-of-distribution task, how well would the linear model do if preceded by a simple thresholding operation where the threshold is tuned? This ties into question 1 about how well a simple thresholding operation would help solving the task at different contrast levels but here focusses on the out-of-distribution aspect.", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "The paper attempts to address an important problem, which is reconciling biophysical realism and functional task performance. It gets around previous shortcomings of detailed biophysical models by employing differentiable simulators. Using these techniques, the authors construct a detailed model of a patch of the outer retina using single compartment Hodgkin-Huxley type models. They further propose a two phase procedure where first the biophysical parameters are fit to recorded data and then additional parameters are fit to the task. The biophysical model is exceptionally realistic which is a strength of the paper. Using this model the authors tune additional parameters on a variant of MNIST where they change contrast and luminance. Except for the differentiable detailed model, each of these steps uses fairly off-the-shelf methods - bounded gradient descent with MSE loss, and rectified and softmax transformed output voltages with cross-entropy loss. The resulting MNIST test accuracy is not great: between 42% and 64%. Notably, the authors identified bipolar cell ion channels as the most important mechanism for task performance.\n\nThe authors also trained their best model without contrast and luminance distortions and tested at different contrast levels, comparing to a baseline linear classifier, which was less robust to contrast variations but achieved higher performance at full and large contrast levels. To me this looks like a very weak baseline and doesn't provide a good reference for out-of-distribution robustness.\n\nQuality:\nIn my opinion, the task is not well selected and not suitable for supporting the claims about the benefits of the biophysical model for contrast invariant processing. The classification task uses MNIST where contrast and luminance are varied. However, MNIST has no resemblance to a real task that the biological system might face and or would be optimized for. For MNIST, the gray values of pixels do not matter that much: binarized image pixels suffice to reach relatively good performance. Checking just now and using a well trained model, I obtained 67.8% test set accuracy after binarizing the MNIST test set pixels without retraining the model and without optimizing the threshold. This is a better accuracy than what the best of the proposed biophysical models achieves. Simple operations might be all that is needed to achieve the reported MNIST performance. It is not clear whether the task is suitable for assessing contrast invariant processing of the biophysical model.\n\nClarity:\nThe paper is very well written. Introduction, Related Work, Methods and Results build on each other and provide context and details. The figures are informative and complement the text but do not always use the available space well (generally lots of whitespace, especially for Fig. 5). I did not try but I do believe that the paper provides enough information to reproduce its results.\n\nSignificance:\nThe paper addresses a general shortcoming of previous work: biophysically detailed models do not support task-based optimization of parameters. Overcoming this problem could open up new possibilities for modelling and understanding biophysical processes. It is not clear how well the proposed approach extends to more complex and more useful tasks and how important is compatibility of the task and the biophysical model. Even the MNIST performance of the proposed models is relatively poor.\n\nOriginality:\nThe authors claim that their biophysical model is the most detailed computational model of photoreceptors. I am not aware of an even more detailed model, so I take that at face value. However, the main contribution of the work is the training procedure where part of the model is constrained by recordings while other parameters are constrained by a task. The related work section does a good job of listing other approaches. From what I can tell, the proposed procedure is novel but might be of limited utility for reasons mentioned above.\n\nMinor:\n\nEq. 1: Superscript M not explained.\n\nEq. 3: The ion h can be confused with the closing variable h of Eq. 1.\n\nEq. 9: Double full stop.\n\nFigure 3 caption: \"with only with\".", "was_revised": true, "final_justification": "I thank the authors for their detailed responses and engagement in the discussion. My concerns regarding the modified MNIST task still stand. The additional experiments and the promise to frame their findings in a different way alleviate these concerns somewhat but not completely. I still believe that a modified MNIST task was not an appropriate selection, in line with Reviewer RpSH but from a more technical viewpoint. I believe that the required changes to the manuscript are too substantial. I raise my score a little but still tend more towards recommending rejection.", "rebuttal": "We thank the reviewer for their detailed and valuable feedback. The reviewer found our manuscript to be “well written” and providing sufficient details for reproducing our findings. The reviewer also positively noted that we were able to “get around previous shortcomings of detailed biophysical models” and that “overcoming this problem could open up new possibilities for modelling and understanding biophysical processes.” We further appreciate that our model was called “exceptionally realistic” and that our proposed procedure for fitting the parameters was considered novel. Given this positive appraisal of our work, we were surprised by the low score.\n\nThe reviewer’s main critique was that our biophysical model did not achieve classification accuracy on par with artificial neural networks and that the biophysical model is more complex than it needs to be to solve the task. We would like to clarify that the main goal of our work is not to compete with ANNs, but to investigate how a biophysically realistic model of the outer retina could implement contrast invariant visual processing and what biophysical mechanisms are at play.\n\nBiophysical systems are much more constrained than artificial architectures because information is transmitted by a variety of bounded physical quantities. We also used experimental data to fit most model parameters which further constrained the model. In this biophysical system, “simple operations” employed by artificial architectures must be implemented as the change of physical quantities, and this is not trivial. We are happy to make this perspective and motivation more clear in the introduction and discussion of our paper.\n\nTo provide more biological insights, we also plotted all of the state variables in the phototransduction cascade throughout the classification of out of distribution stimuli at different contrast levels. This figure shows that the photopigment and phosphodiesterase activity change greatly as the contrast moves further out of distribution, but that the cGMP concentration is quite robust, and then in turn the downstream calcium concentration, photocurrent and photoreceptor voltage. Our Fig. 5 currently only shows the glutamate traces as contrast moves further out of distribution, but we are happy to show additional state variables that make predictions about the biophysical mechanisms supporting contrast invariant encoding in even more detail.\n\nThe reviewer also cited a lack of strong baselines.  While we consider the linear model as a point of reference rather than as a baseline for our model to beat in terms of absolute classification accuracy, we agree with the reviewer that additional reference models would help to contextualize the performance of the biophysical model. We thank the reviewer for this suggestion and have run additional experiments that we compare to in the revised version of the manuscript.  The additional non-biophysical reference models we explored for this rebuttal include a two-layer MLP, layer normalization followed by linear classification, and adaptive and static thresholding followed by linear classification. \n\nInterestingly, a two-layer MLP achieves much better classification performance for all contrasts than our model, but it uses far more trainable parameters by design to achieve this. Furthermore, the computation this network implements is quite opaque, and the mechanism by which it achieves this feat is unclear. \n\nWe find that the use of layer normalization in addition to linear classification performs the best, with a contrast-accuracy curve similar to that of our model. While it provides an explicit description of the computation, it does not provide insights into how such a computation could be implemented by the outer retina, as our model does. \n\nUsing static thresholding plus linear classification does not work for classifying out of distribution stimuli in this case because the models are trained at full contrast, and thus the threshold will be too high or too low for all of the other contrast levels depending on where it is initialized. That is, there is no proper way to learn a static threshold that classifies the stimuli well outside of the training distribution. An adaptive threshold, however, that is multiplied by the maximum stimulus intensity, performs almost the same as layer normalization with linear classification and thus implements a similar computation, but likewise does not provide insights into the biophysical implementation of such a computation. \n\nThus, in summary, the rationale of our revised paper is:\n1. We hope that our previous discussion about the biophysical realism of our system convinces the reviewer that seemingly simple computational operations are not actually so simple in constrained biophysical systems and our model helps to solve that question. In addition, it is important to note that our model considers time, and this is critical to our testing of the hypothesis that horizontal cells contribute to task performance, because their feedback would be required on an adequate timescale. While this makes our model suitable for answering a host of other interesting questions, in the context of this task, assessing the role of the different cell types is not possible without the notion of time. \n\n2. As a reference model, we used a linear classifier to see if our biophysical model simply implements linear functions. We found that the linear model performs very well in its training distribution (87% when trained at varying contrasts), but its performance degrades much more quickly as contrast moves outside of the training distribution than for our biophysical model. We thus concluded that the biophysical mechanisms implemented in our model underlie this ability to generalize.\n\n3. Prompted by the reviewer, we used other reference models and found that the linear model does perform much better when preceded by a simple adaptive thresholding operation on out-of-distribution stimuli, so we suppose that this is the sort of operation that the biophysical model implements, though in a biophysically plausible way with real physical quantities. We are happy to include this in our figure and discussion in the paper.\n\nWe are grateful to the reviewer for their feedback and hope that they will indeed raise their score as mentioned in the review in light of these clarifications and forthcoming additions to the paper."}, {"review_id": "19oPazhxsJ", "reviewer": "Reviewer_x6ob", "summary": "The authors leveraging a biophysically detailed model they fitted outer-retina cone cells using on one hand experimental measurements (patch-clamp and 2photon recordings) and on the other hand a task-driven loss. They implemented their training in a multi-stage process, first fitting the phototransduction cascade using the patch-clamp data, then fitting the synaptic transmission using the 2-photon recordings, and finally optimizing the model to perform a digit classification task with various contrasts.\nThe model is able to reproduce the experimental data and generalizes well to different contrast levels, showing robustness. Importantly, this contrast robustness is not coming from the horizontal cells, which were thought to be responsible for it.", "questions": "- I am not sure about the meaning of the last sentence of section 4.3, could you elaborate further? \n- Is the argument of Figure 4 that basically the contrast is filtered out by the photoreceptor cells?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The multi-stage training process allows for a more structured approach to model fitting, ensuring that each component is accurately represented.\n- The model's robustness to contrast variations not being depedent on the horizontal cells seems an interesting and potential important finding.\n- The use of both experimental data and task-driven loss provides a comprehensive approach to model fitting, ensuring that the model is not only accurate but also functionally relevant.\n- The resulting photoreceptor parameters are consistent with previous studies, which adds credibility to the model.\n\nWeaknesses:\n- I found it unfortunate that I could not see Table 3 and 4 that were mentioned in the text, as they would have provided additional insights into the model. On that note, this was the biggest part that I was missing, more detailed ablation studies to understand the contributions of different components of the model to its performance. For example, is a HH model necessary, can we use simpler models for the phototransduction cascade or synaptic transmission?\nAs we see in the text, running this model is computationally expensive, so it would be interesting to see which model components could be simplified without losing performance.\n\n- I might not sure that the control in Figure 5 is optimal, maybe a two-layer network would be more appropriate.", "was_revised": true, "final_justification": "As I wrote from my initial review, I found that this paper is technically solid and is interesting for the field. The author's rebuttal further clarified some points, and I am pleased to keep my initial score.", "rebuttal": "We thank the reviewer for their time in reviewing our manuscript. We appreciate that the reviewer considered our model “accurate but also functionally relevant” and our multi-stage training process “a more structured approach to model fitting.” \n\nTo address the reviewer’s first concern about the missing Tables 3 and 4, we apologize that those references were supposed to be to Tables 1 and 2. Table 1 was designed to list the different connectivity patterns and Table 2 was designed to show the performance of each connectivity pattern on the task. We will fix that.\n\nAnother concern of the reviewer was that our model is very detailed and thus computationally expensive. We acknowledge this point, and simplifications to the model could be made but at the expense of our scientific understanding of the processing. We are currently able to look at the role of each state variable throughout task performance, and we have created additional plots that provide further insight into the contrast invariant encoding. Specifically, we found that the photopigment and phosphodiesterase activity change greatly as the contrast moves further out of distribution, but that the cGMP concentration is quite robust, and then in turn the downstream calcium concentration, photocurrent and photoreceptor voltage. Such findings would not be possible in a simpler model.\n\nTo address the concern about our baseline in Figure 5, we trained and tested some other artificial architectures in addition. We rather refer to these alternative models as “reference models”, as we are not so much interested in their absolute performance, but rather e.g. in their capability to handle out of distribution contrasts. As the reviewer suggested, we tested a two layer MLP with a strong optimization routine but no layer normalization and a linear classifier preceded by layer normalization. Both of these architectures perform better outside of the contrast training distribution than the linear classifier; however, they come with their own set of drawbacks. A two-layer MLP, for example, without layer normalization but with a strong optimization procedure, requires far more trainable parameters by design to achieve higher accuracy. We chose the linear classifier in part because it has a similar number of trainable parameters to the biophysical model, 2,010 vs 2,000. We find that the use of layer normalization in addition to linear classification performs the best, not surprisingly, and the contrast-accuracy curve has a similar shape to that of the biophysical model. Thus, this is likely the computation implemented by our biophysical system, and we can now determine which components are responsible for it.  We are happy to add these reference models to our figure for more context. \n\nTo address the reviewer’s questions:\n1. The last sentence in Section 4.3 describes panel b of Figure 5 where the glutamate release of three photoreceptors is plotted. The glutamate release is plotted for the full contrast stimulus (yellow) for which the model is trained, and then the reduced contrast stimulus at the different colors. These colors match the colors of the line indicating the biophysical model in panel a. This is an illustrative figure showing that as contrast is reduced, some cones release more glutamate than at high contrast whereas others reduce less glutamate than at high contrast. What we should have mentioned, however, is that cones where the digit pixels are present (illuminated cones) tend to increase their glutamate at lower contrasts and cones where the digit pixels are not present (non-illuminated cones in the background) tend to reduce their glutamate release at lower contrasts. This is expected because greater illumination results in hyperpolarization and less glutamate release. These changes in glutamate release maintain the digit representation until contrasts of below 0.3 where the classification accuracy drops as shown in panel a. \n2. This is closely related to your first question, and hopefully the previous discussion was helpful. The answer is basically yes, the photoreceptors are “filtering out the contrast” in their own way.\n\nWe hope that our additional experiments and modifications further increase the confidence of the reviewer in our paper."}]

Author rebuttal:

Meta-review: This paper proposes a biophysically plausible model of the outer retina adapted to gradient-based learning with modern machine learning tools. While previous models of several components used here exist, along with some similar efforts, this model is unique in combining realistic biophysics with tunable nonlinearities in a fully differentiable framework that can be used to investigate early-stage visual processing. The authors perform a number of experiments in this framework to delineate the contributions of various circuit elements, finding that inhibitory feedback through horizontal cells provided only limited improvement in the tasks they consider. 

Reviewers applauded the careful nature of the modeling, which gathers together a good deal of previous literature in a modern package that can be optimized using JAX. They were likewise positively impressed with the potential for understanding processing in neural circuits. Weaknesses centered on the performance evaluation and baselines, which included fairly poor performance on MNIST and comparison only with a linear model.

Discussion primarily focused on two points:
1. **Is MNIST, in which gray pixels play a minimally informative role, a good task on which to judge contrast-invariant processing in the model?** That is, does contrast significantly modulate task difficulty in a way that illustrates the model's ability to compensate. Here, the authors acknowledged that a simple binarization threshold is sufficient to remove effects of contrast, though they argued that adaptively ignoring contrast changes requires such a threshold to be set, and this is something analogous to what the model does.

2. **The performance of the model on MNIST is well below that of even simple neural networks, and only a linear model is used as a baseline comparison.** The authors responded by training additional simple baselines (including logistic classification on top of a layer norm), though they argued that performance metrics _per se_ were not the point of the modeling. 

While reviewers were appreciative of the second point, objections remained to the first. In a task where contrast is not directly relevant to performance (or can be easily compensated for), it is debatable whether any level of performance assesses contrast (in)sensitivity. Authors would likely do better to choose a separate task in which contrast more clearly modulates problem difficulty.

On balance, however, the careful nature of the modeling and its potential utility in neuroscience research argue for acceptance of the paper.
Decision: Accept (poster)