Title: A data and task-constrained mechanistic model of the mouse outer retina shows robustness to contrast variations
Venue: NeurIPS 2025
Accepted: yes
Keywords: neural coding

retina

recurrent networks

biophysics

differential equations
Abstract: Visual processing starts in the outer retina where photoreceptors transform light into electrochemical signals. These signals are modulated by inhibition from horizontal cells and sent to the inner retina via excitatory bipolar cells. The outer retina is thought to play an important role in contrast invariant coding of visual information, but how the different cell types implement this computation together remains incompletely understood. To understand the role of each cell type, we developed a fully-differentiable biophysical model of a circular patch of mouse outer retina. The model includes 200 cone photoreceptors with a realistic phototransduction cascade and ribbon synapses as well as horizontal and bipolar cells, all with cell-type specific ion channels. Going beyond decades of work constraining biophysical models of neurons only by experimental data, we used a dual approach, constraining some parameters of the model with available measurements and others by a visual task: (1) We fit the parameters of the cone models to whole cell patch-clamp measurements of photocurrents and two-photon glutamate imaging measurements of synaptic release. (2) We then trained the spatiotemporal outer retina model with photoreceptors and the other cell types to perform a visual classification task with varying contrast and luminance levels. We found that our outer retina model could learn to solve the classification task despite contrast and luminance variance in the stimuli. Testing different cell type compositions and connectivity patterns, we found that feedback from horizontal cells did not further improve task performance beyond that of excitatory photoreceptors and bipolar cells. This is surprising given that horizontal cells are positioned to mediate communication across cones and that they add to the model's number of trainable parameters. Finally, we found that our model generalized better to out of distribution contrast levels than a linear classifier. Our work shows how the nonlinearities found in the outer retina can accomplish contrast invariant classification and teases apart the contributions of different cell types.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors leveraging a biophysically detailed model they fitted outer-retina cone cells using on one hand experimental measurements (patch-clamp and 2photon recordings) and on the other hand a task-driven loss. They implemented their training in a multi-stage process, first fitting the phototransduction cascade using the patch-clamp data, then fitting the synaptic transmission using the 2-photon recordings, and finally optimizing the model to perform a digit classification task with various contrasts.\nThe model is able to reproduce the experimental data and generalizes well to different contrast levels, showing robustness. Importantly, this contrast robustness is not coming from the horizontal cells, which were thought to be responsible for it.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The multi-stage training process allows for a more structured approach to model fitting, ensuring that each component is accurately represented.\n- The model's robustness to contrast variations not being depedent on the horizontal cells seems an interesting and potential important finding.\n- The use of both experimental data and task-driven loss provides a comprehensive approach to model fitting, ensuring that the model is not only accurate but also functionally relevant.\n- The resulting photoreceptor parameters are consistent with previous studies, which adds credibility to the model.\n\nWeaknesses:\n- I found it unfortunate that I could not see Table 3 and 4 that were mentioned in the text, as they would have provided additional insights into the model. On that note, this was the biggest part that I was missing, more detailed ablation studies to understand the contributions of different components of the model to its performance. For example, is a HH model necessary, can we use simpler models for the phototransduction cascade or synaptic transmission?\nAs we see in the text, running this model is computationally expensive, so it would be interesting to see which model components could be simplified without losing performance.\n\n- I might not sure that the control in Figure 5 is optimal, maybe a two-layer network would be more appropriate.\n\nQUESTIONS:\n- I am not sure about the meaning of the last sentence of section 4.3, could you elaborate further? \n- Is the argument of Figure 4 that basically the contrast is filtered out by the photoreceptor cells?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-14"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a fully differentiable biophysical model of the mouse outer retina to understand contrast invariant coding via different cell types. The authors constrained some parameters with measurements while other were optimized on MNIST, but with varying contrast and luminance. The authors show that their model can classify digits in spite of these stimulus variations and identify the most important model mechanism determining classification performance. They also compare their model to a linear classifier and find more robust task performance to out of distribution contrast levels.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper attempts to address an important problem, which is reconciling biophysical realism and functional task performance. It gets around previous shortcomings of detailed biophysical models by employing differentiable simulators. Using these techniques, the authors construct a detailed model of a patch of the outer retina using single compartment Hodgkin-Huxley type models. They further propose a two phase procedure where first the biophysical parameters are fit to recorded data and then additional parameters are fit to the task. The biophysical model is exceptionally realistic which is a strength of the paper. Using this model the authors tune additional parameters on a variant of MNIST where they change contrast and luminance. Except for the differentiable detailed model, each of these steps uses fairly off-the-shelf methods - bounded gradient descent with MSE loss, and rectified and softmax transformed output voltages with cross-entropy loss. The resulting MNIST test accuracy is not great: between 42% and 64%. Notably, the authors identified bipolar cell ion channels as the most important mechanism for task performance.\n\nThe authors also trained their best model without contrast and luminance distortions and tested at different contrast levels, comparing to a baseline linear classifier, which was less robust to contrast variations but achieved higher performance at full and large contrast levels. To me this looks like a very weak baseline and doesn't provide a good reference for out-of-distribution robustness.\n\nQuality:\nIn my opinion, the task is not well selected and not suitable for supporting the claims about the benefits of the biophysical model for contrast invariant processing. The classification task uses MNIST where contrast and luminance are varied. However, MNIST has no resemblance to a real task that the biological system might face and or would be optimized for. For MNIST, the gray values of pixels do not matter that much: binarized image pixels suffice to reach relatively good performance. Checking just now and using a well trained model, I obtained 67.8% test set accuracy after binarizing the MNIST test set pixels without retraining the model and without optimizing the threshold. This is a better accuracy than what the best of the proposed biophysical models achieves. Simple operations might be all that is needed to achieve the reported MNIST performance. It is not clear whether the task is suitable for assessing contrast invariant processing of the biophysical model.\n\nClarity:\nThe paper is very well written. Introduction, Related Work, Methods and Results build on each other and provide context and details. The figures are informative and complement the text but do not always use the available space well (generally lots of whitespace, especially for Fig. 5). I did not try but I do believe that the paper provides enough information to reproduce its results.\n\nSignificance:\nThe paper addresses a general shortcoming of previous work: biophysically detailed models do not support task-based optimization of parameters. Overcoming this problem could open up new possibilities for modelling and understanding biophysical processes. It is not clear how well the proposed approach extends to more complex and more useful tasks and how important is compatibility of the task and the biophysical model. Even the MNIST performance of the proposed models is relatively poor.\n\nOriginality:\nThe authors claim that their biophysical model is the most detailed computational model of photoreceptors. I am not aware of an even more detailed model, so I take that at face value. However, the main contribution of the work is the training procedure where part of the model is constrained by recordings while other parameters are constrained by a task. The related work section does a good job of listing other approaches. From what I can tell, the proposed procedure is novel but might be of limited utility for reasons mentioned above.\n\nMinor:\n\nEq. 1: Superscript M not explained.\n\nEq. 3: The ion h can be confused with the closing variable h of Eq. 1.\n\nEq. 9: Double full stop.\n\nFigure 3 caption: \"with only with\".\n\nQUESTIONS:\n1. Can the authors assess the utility of the biophysical model for contrast invariant processing on this task? Do the complex non-linearities of the biophysical model actually matter for performing the classification task? To me it seems that a simple thresholding operation would be sufficient as a preprocessing step - or at the most normalization plus thresholding. This is the key point for me. I would increase my score if the authors can show that there is more to the model than what simple operations can do in the context of this task.\n\n2. In the out-of-distribution task, how well would the linear model do if trained on a range of different contrast levels? This would provide a better idea of the overall performance that the linear model can reach and help to put the results into context.\n\n3. In the out-of-distribution task, how well would the linear model do if preceded by a simple thresholding operation where the threshold is tuned? This ties into question 1 about how well a simple thresholding operation would help solving the task at different contrast levels but here focusses on the out-of-distribution aspect.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe manuscript presents a fully differentiable, biophysically detailed model of a mouse outer retinal patch—including cone photoreceptors, horizontal cells, and bipolar cells—implemented in JAX. Cone parameters are constrained by fitting to patch-clamp photocurrent and two-photon glutamate imaging data, after which the assembled network is trained end-to-end on a contrast- and luminance-varying MNIST classification task. The findings are interesting. Firstly, cone phototransduction and ribbon-synapse nonlinearities alone produce contrast-invariant representations sufficient for classification. Secondly, adding horizontal-cell feedback does not measurably improve performance beyond the cones plus bipolars model, contrary to prevailing hypotheses. Finally, the biophysical model generalizes better to out-of-distribution contrasts than a linear baseline.\n\nSTRENGTHS AND WEAKNESSES:\n**strengths:** The manuscript has novel integration of data and task constraints. Fitting 2,000 and 2,900 biophysical parameters by both experimental recordings and a high-level visual task is a technically impressive advance. Moreover, the photoreceptor model comprises 26 differential equations capturing the transduction cascade, calcium feedback, and ribbon-synapse dynamics. The finding that horizontal-cell feedback adds little to contrast-invariance challenges established theories and opens new questions about horizontal-cell function. Finally, the manuscript begins with code, which is suitable for reproducibility and community extension.\n\n**Weaknesses:** 1.  The cascade model lacks stochasticity, which is important for the retina to perform contrast adaptation.\n\nReference: Cottaris, N. P., Wandell, B. A., Rieke, F., & Brainard, D. H. (2020). A computational observer model of spatial contrast sensitivity: effects of photocurrent encoding, fixational eye movements, and inference engine. Journal of Vision, 20(7):17. https://doi.org/10.1167/jov.20.7.17 \n\n2. In line 136, $ \\tau_{s} = (1 - s_{\\infty})(k^{-})$ is not correct. Please check the equation in the cited paper (ref. 30, page 20, eq. 18). Please also check if the findings still hold for the correct equation (**This is my major concern**).\n\nQUESTIONS:\n1. In Figure 1, on the left panel, I'm unclear what the y-axes for \"Photocurrent\" and \"Glutamate release\" actually measure. What do the green and magenta square signals represent? On the right panel, what do the different luminance levels of the MNIST squares correspond to, and how are those mapped to photoreceptor grayscale intensities? Why is only a single cell shown under the HC? The BC readout traces need clear axis labels- what do their x- and y-axes denote? Finally, I don't see Figure 1 referenced in the main text.\n2. The paper does not spell out the voltage- and time-dependent formulas for the gating variables $𝑚$ and $ℎ$ in Eq. (1).\n3. The authors defer the photoreceptor ion-current formulas to Kamiyama et al. (2024), providing the explicit voltage‐ and time‐dependent expressions for each current term (e.g. $ I_{Kv} $, $ I_{h} $, $ I_{Ca} $, $ I_{K(Ca)} $, $ I_{Cl(Ca)} $, $ I_{Leak} $) and for the gating variables $ m(V, t) $ and $ h(V, t) $ in Eq. (1) would greatly improve clarity. Writing those equations out explicitly (or at least in a Supplementary Material) would also help the readers to understand the model better.\n4. In Table 1, the switch from \"local\" to \"one-to-one\" connectivity (full connection) seems very abrupt. How does horizontal cell connectivity gradually evolve from local to full connection?\n5. In Eq. 9, it has two periods at the end of the equation.\n6. Some expressions are not consistent. For example, line 101 \"equation\", while other places use \"Eq.\" in line 118.\n7. I suggest that the authors put the main formulas in the main text and put the detailed derivations in the Supplementary Material.\n8. The acronym should be written in the first time it appears in the text. For example, HCs refers to horizontal cells in line 127.\n9. Can authors explain that why horizontal cells can perform a visual task despite contrast and luminance variation intuitively? Moreover, I suggest that the authors should add to the abstract to explain the mechanism of this finding.\n\nLIMITATIONS:\n1. MNIST offers a highly simplified and low-dimensional dataset, while natural scenes exhibit far richer spatial and color statistics, so it would strengthen the work to evaluate the model on more complex benchmarks (e.g., CIFAR-100, ImageNet, etc.). These tests would demonstrate whether the retina's contrast-gain and invariance mechanisms generalize beyond handwritten digits to real-world structures.\n2. Section 2 mentions prior models; thus, the discussion needs to discuss them in detail, especially explicitly comparing your findings with those frameworks.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel framework for training and inspecting models of biological neural circuits, applying it specifically to the under-appreciated processing of the outer retina. The contributions are twofold: first, the authors take the nonlinearities of phototransduction in the outer retina seriously and attempt to understand how they interact with downstream processing (whereas the vast majority of adjacent work make assumptions of linearity). Second, they demonstrate a dual-stage model-fitting procedure that optimizes both biophysical fidelity (using priors and data from literature and experiments) and subsequent performance on an ethologically-plausible task (contrast invariance). Their model, specifically, is a dynamic model of the outer-retina fit to both whole-cell electrophysiology from cones and two-photon glutamate sensor imaging from ribon-synapses, providing a detailed and data-constrained simulation of how light is communicated by the cone photoreceptors to downstream circuitry. They then extend this model to include a variety of architectures that add bipolar and horizontal cells in a manner that enables classification of the MNIST dataset by decoding readout bipolar cell activity. Importantly, they evaluate their model not on MNIST alone, but on out-of-distribution samples that have varied contrast and luminance. The biophyiscal front-end of their model is accurate and captures temporal phenomena (adaptation); the necessity / sufficiency of various aspects of the downstream decoding circuitry are used to make hypotheses about the role of different cell types in the known phenomena of contrast invariant processing.\n\nSTRENGTHS AND WEAKNESSES:\nOverall, the main strength of this paper is its creative framework for evaluating biophysically-constrained networks on in-silico tasks, as a means of probing the functional impact of specific biophysical parameters. As mentioned by the authors, recent years have brought a large number of papers that optimize models of neural encoding directly to data and by comparing the representations of task-trained networks to experimental data - the author's approach (training to data, then evaluating on an ethologically-meaningful task) has significant potential across a wide range of questions. The paper is written clearly and the approach is well-motivated, however the scientific results of the paper, while well-reasoned, lack important controls and further analyses in order to demonstrate the potential of this new biophysical / task optimized model.\n\nSpecifically, the authors motivate their biophysical model architecture by way of its complexity and fidelity to known processing in the outer retina (in addition to the fact that it is differentiable, which this reviewer does not find particularly novel, given the widespread use of autograd functions). It is unclear, however, whether all of this complexity is actually needed for their ultimate conclusions. As described in the paper, their cone-photoreceptor model involves 26 differential equations and this faithfulness is suggested as a mechanism for their downstream task-performance on contrast and luminance invariance testing data. However, the author's do not cite other papers that capture such phenomena using many fewer parameters in a number of different retinal cell-types (including photoreceptors). For example, in \"Linking the computational structure of variance adaptation to biophysical mechanisms\" (Ozuysal and Baccus, 2014) - contrast adaptation can be captured using a much simpler dynamical system with many fewer parameters (though they do not appear to investigate photoreceptors). However, \"Dynamical Adaptation in Photoreceptors\" (Clark, ..., Meister, Azaredo da Silveira) do build a model of contrast adaptation in photoreceptors that captures the phenomena. Since much simpler models can capture the phenomena of contrast and luminance invariance - the burden lies with the authors to demonstrate that the complexity of the biophysical model is either useful functionally - or permits some deeper insight. The former could be demonstrated via additional controls. The latter is demonstrated cursorily by suggesting that horizontal cells are not required for contrast invariance.\n\nIn regards to the task-evaluation of the biophysical model, the actual performance demonstrated is quite low, given that even simple MLPs can achieve >95% accuracy on MNIST. The reason is presumably that the task is not vanilla MNIST, but the contrast-invariance test. Thus, providing some more standard baselines to show how well standard architectures can perform on OOD evaluation would reassure this reviewer as to the significance of an accuracy of 0.6 (in Table 2). The authors include the \"Linear\" model, but in order to account for the additional parameters in the biophysical model, the authors could include a biophysical model at initialization or biophysical models with scrambled dynamics parameters.\n\nQUESTIONS:\n1. In figure 5, the comparison of a linear classifier with the biophysical model is a good control, but only one of a few that are needed. In order to test whether it is architectural (not actually due to the learned parameters) the authors should also include the initialized biophysical model, to rule out that the OOD performance is not merely a result of the addition of dynamics. The authors could also evaluate using the same pipeline, with a much simpler dynamical system (see the LNK model in Ozuysal and Baccus, 2014) in order to get a sense of whether the complexity of the dynamics model is actually necessary. Additionally, what if the models are trained on a variety of contrasts, not merely tested on them? One could imagine that varying contrasts could confuse a linear classifier during training, but not your biophysical model.\n\n2. In general, I think the authors are missing an opportunity to investigate the temporal nature of contrast / luminance adaptation. The authors present MNIST data to their biophysical models at varying levels of contrast, and do so in a temporal manner - however, contrast / luminance invariance is a dynamic process that involves the adaptation of a response to the mean contrast or luminance over a specific window of time (Baccus and Meister, 2002). Thus, I would appreciate some exploration of how the model behaves during the transition between high and low contrast regimes, not just evaluated at specific instances of varying contrast. Since contrast invariance is mediated by adaptation, this could be incorporated into the training of the readout network and structure of the task (if the input to the model were, for example, a sequence of frames for one digit with randomly fluctuating luminance and contrast). I imagine a linear model, even trained on inputs with fluctuations in contrast, would have quite a bit of trouble with this task. \n\n3. Given the highly complex biophysical model of cones, I would like to see some preliminary attempt to understand an aspect of the biophysical model of the cones. Since the model is differentiable, the authors could identify which of the parameters / states contributes most to contrast adaptation. Several strategies could be taken - randomly ablating specific states within the dynamics model could identify the crucial mechanisms governing its behavior. Alternatively, looking at the evolution of these state variables during adaptation (or the gradients) could offer clear hypotheses about what aspects of the phototransduction cascade govern contrast invariance.\n\n4. The authors suggest that the highly-complex biophysical model offers an inductive bias that imbues the network with new capabilities (OOD performance on differing contrasts and luminance on MNIST). This is quite a strong claim, given that the machine-learning community already has neural-network primitives that provide similar computational mechanisms without the need for such computationally costly dynamics models (layer-norm, for example). The authors should make explicit that the utility of the model is in gaining insights into their biophysical system - or, if the claim is that some aspect of the biophysical model enables a new computational capability (beyond a vanilla neural network), the authors should draw parallels between more standard machine-learning techniques (layernorm) and provide comparisons.\n\n5. In the retina, contrast adaptation is seen dramatically in amacrine and ganglion cells - but this reviewer's understanding is that it is a much smaller effect in photoreceptors. Please provide some contextualization of the relationship between contrast invariance in the outer retina and the inner retina, and provide a bit more justification for evidence that horizontal cells were ever considered as a putative mechanism thereof.\n\nLIMITATIONS:\nMNIST is semi-naturalistic but is not really a naturalistic stimulus for any organism. I would encourage the authors to discuss more how their results can and cant be extended to a general understanding of the processing of naturalistic vision with spatiotemporal correlations and structured fluctuations in contrast and luminance. In future work, different tasks could be used that better approximate truly naturalistic vision (cifar, imagenet, video-classification, etc).", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-15"}