COMMENTARY

Ordering in heterogeneous connectome weights for visual
information processing
Taro Toyoizumia,b,1

What defines ourselves? People believe that neuronal circuitsin the brain characterize who we are and how we behave.
Following the triumph of the Human Genome Project, connectomics projects started as the next big biology initiative
to map out the wiring diagrams between neurons in thebrain. Since the completion of the first connectome of a tiny
worm, Caenorhabditis elegans, in 1986 (1), large-scaled connectome data have been accumulated from nanoscale tomacroscale across different animal species (2). However,understanding these data remains challenging (3). The reasons include the difficulty in identifying important parameters in high-dimensional observations, data variability acrossexperimental conditions, and relating the anatomical structure to brain function. These problems also exist if we focus
on the connectome data from the primary visual cortex, V1,of mice (4–6). The connection statistics differ depending on
the source and target cell types across cortical layers. Evenwithin the same classification type, some statistics deviatesignificantly across datasets. Thus, finding general rules ischallenging. Finally, it is nontrivial to understand how theanatomical circuit is related to visual information processing.
In PNAS, Kraynyukova et al. (7) conducted model-based analyses that address these problems.
Kraynyukova et al. studied the so-called stabilized supralinear network (SSN) model (8). It consists of (Fig. 1, Left) the excitatory, E, and inhibitory, I, neural populations in V1 that receive
input from the visual thalamus (dorsal lateral geniculatedLGNSupralinearIJ IE

gESupralinear

EJ EE

Copyright © 2022 the Author(s). Published by PNAS. This article is distributed under
Creative Commons Attribution-NonCommercial-NoDerivatives License 4.0 (CC BY-NC-ND).
See companion article, “In vivo extracellular recordings of thalamic and cortical visual
responses reveal V1 connectivity rules,” 10.1073/pnas.2207032119.
1

Email: taro.toyoizumi@riken.jp.

Published November 21, 2022.

The model fitting result based on visual responses

gE < gI < J EI < J EE < J II < J IE
J EIConnectome weights<<J EI

The author declares no competing interest.

>

gIAuthor contributions: T.T. wrote the paper.

gE < J EE>

J IIAuthor affiliations: aLaboratory for Neural Computation and Adaptation, RIKEN Center
for Brain Science, Wako, Saitama 351-0106, Japan; and bDepartment of Mathematical
Informatics, Graduate School of Information Science and Technology, The University ofTokyo, Bunkyo, Tokyo 113-8656, JapanJ IE<<V1nucleus, i.e., dLGN). Each population is characterized by its firingrate rX (X is either E or I), which monotonically and supralinearly
increases with its synaptic input, given in the form of excitation
minus inhibition. The model has six connection weights: twothalamocortical weights, gE and gI, to and four cortical weights,JEE, JEI, JIE, and JII, between the two cortical populations. In addition, the model has a power exponent n that describes thenonlinearity of the two cortical populations. Kraynyukova et al.
performed in vivo electrophysiological recordings of mice V1and dLGN neurons in response to the presentation of visual
grating stimuli with various contrast levels and orientations tofit this SSN model. They quantified, for each of the dLGN andV1 putative excitatory and inhibitory populations, the population response by averaging the corresponding neurons’ two-dimensional receptive field, i.e., the firing rate profile, afterJ IIFig. 1. (Left) A diagram of the SSN model of V1 (Kraynyukova et al., 2022). The cortical excitatory population (E: orange circle) and inhibitory population(I: light green circle) receive input from dLGN with weights gE and gI, respectively. The four types of cortical weights connecting the two populations are denoted
as JEE, JEI, JIE, and JII, where JEI describes the weights from I to E, for example. The line width of the thalamocortical and corticocortical connections is proportional tothe mean estimated weight based on the in vivo data. In the steady state, the firing rate rX of population X (X is either E or I) is given by rX = [ JXE rE - JXI rI + gX T ]+n,where T is the dLGN firing rate, the operation [·]+ transforms a negative argument to zero so that the firing rate is positive, and n is a nonlinear exponent. (Right)
Kraynyukova et al. fitted the SSN model to the in vivo data. They found specific ordering among them (Top). This ordering was verified using three in vitro connectome
datasets (Bottom). The transparency level of the inequality signs reflects the consistency among the datasets. In addition, they got n ~ 1.9, which indicates thatthe input–output function is supralinear.

PNAS

2022

Vol. 119No. 48e2216092119

https://doi.org/10.1073/pnas.2216092119 1 of 3

aligning the neurons’ preferred orientations. The orientation
tuning in these populations was contrast invariant to good accuracy (i.e., the shape of orientation tuning did not change withthe contrast), which warrants the examination of their population contrast response separately from their orientation tuning.
Given the dLGN population contrast response, they fit theseven parameters of the SSN model to reproduce the V1 population contrast response. This analysis consistently gives a
specific ordering of inferred weights (Fig. 1, Right). Remarkably,
this ordering is consistent with the connectome datasets (4–6)
despite the variability in individual weights.
The nonlinear exponent n of the SSN model also plays acrucial role. When a linear input–output function is used (i.e., n= 1) instead, the inferred weights follow dissimilar ordering fromthe connectome data. This result argues that the simple SSN
model involving only the two major cortical populations is suitable for relating contrast response in V1 and the connectome
data. Theoretically, the supralinear input–output function
increases the cortical firing rate progressively greater than thefiring rate of the thalamus at higher contrast. Consequently,
the network is mainly external-input-driven at low contrast andcortical-input-driven at high contrast. The network is said tooperate as an inhibition-stabilized network (ISN) (9–11) in thelatter case if the network activity diverges (at least until anotherstabilization mechanism kicks in) under the blockade of inhibitory activity due to its strong recurrent excitation. ISNs have asalient feature, a.k.a., the paradoxical effect, where an increasein the input to the inhibitory population transiently increases
its activity but subsequently reduces both the excitatory andinhibitory activity. This prediction was experimentally verified(12) across cortical areas. While alternative mechanisms, suchas the disinhibition mechanism that involves more than two
populations, are not excluded (11, 13, 14), the ISN regime is a
simple and attractive mechanism that reproduces the normalization and surround suppression properties commonlyobserved in the cortex (15). Kraynyukova et al. showed that
their fitting yields ISN at >4% contrast level, suggesting that ISN
properties should contribute to a variety of visual stimuli.

associated with the visual contrast response even though
individual weights are highly variable, arguing for the importance of supralinear contrast response in V1. But how canwe understand the variability in connectome weights further? Possible reasons for the variability include a bias anda finite number of samples in each dataset. Unbiased andinfinite-size data are elusive. Moreover, the distribution ofindividual synaptic strength is often heavy-tailed (16, 17),
hampering the convergence of the empirical mean with thesample size. The variability may also reflect the animal’s
experience and developmental trace (18) or transientchanges in the circuit properties like representational drift(19) and intrinsic synaptic dynamics (17). Another candidate
for the variability is the scaling property of the SSN model.
In the SSN model, for any positive real number a, multiplying
all cortical weights by factor a1/n−1 and all thalamocorticalweights by factor a1/n simply changes both the cortical excitatory and inhibitory firing rates by the factor a (c.f., thesteady-state equation in Fig. 1 caption). This transformation
corresponds to the invariant subspace of the SSN parameters. More generally, there are diverse solutions (i.e., configurations of biological elements) that can achieve a certain
network function. This diversity has been suggested to beimportant for the resilience of network functions and recovery from failures (20). The solutions to achieving a certain
network function could be clustered or scattered in the network parameter space. Understanding the distribution ofsolutions is also important in machine learning as it describes
if a solution is easily found (21). Analyses of deep networks
suggest that a learning path may arbitrarily follow one ofmany equally good paths involving multiple branching points
or follow a highly restricted path if structural pruning isapplied a priori. Interestingly, given that structural pruning
is successful, learning is faster and achieves higher generalization capability than without pruning (22). Thus, the connectome variability around good enough solutions may be
due to arbitrary fluctuations, highly structured individualdifferences, or their mixture. The finding of Kraynyukova etal. is a major step toward the understanding of
connectome variability in identifying general“The finding of Kraynyukova et al. is a major step
rules underlying network functions.
toward the understanding of connectome­ ariability in identifying general rules underlyingv
network functions.”
The findings of Kraynyukova et al. are attractive because
it shows a robust ordering of connectome weights

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.

ACKNOWLEDGMENTS. I thank Ho Ka Chan and Louis Kangfor their help in proofreading the manuscript. The study wassupported by RIKEN Center for Brain Science, Brain/MINDS fromAMED under Grant JP15dm0207001, and JSPS KAKENHI Grant JP18H05432.

J. G. White, E. Southgate, J. N. Thomson, S. Brenner, The structure of the nervous system of the nematode Caenorhabditis elegans. Philos. Trans. R. Soc. Lond. B. Biol. Sci. 314, 1–340 (1986).
J. L. Lanciego, F. G. Wouterlood, A half century of experimental neuroanatomical tracing. J. Chem. Neuroanat. 42, 157–183 (2011).
Y. Frégnac, Big data and the industrialization of neuroscience: A safe roadmap for understanding the brain? Science 358, 470–477 (2017).
Allen Institute for Brain Science, Synaptic physiology coarse matrix dataset. https://brain-map.org/explore/connectivity/synaptic-physiology. Accessed 23 September 2022.
X. Jiang et al., Principles of connectivity among morphologically defined cell types in adult neocortex. Science 350, aac9462 (2015).
S. B. Hofer et al., Differential connectivity and response dynamics of excitatory and inhibitory neurons in visual cortex. Nat. Neurosci. 14, 1045–1052 (2011).
N. Kraynyukova et al., In vivo extracellular recordings of thalamic and cortical visual responses reveal V1 connectivity rules. Proc. Natl. Acad. Sci. U.S.A. 119, e2207032119 (2022).
Y. Ahmadian, D. B. Rubin, K. D. Miller, Analysis of the stabilized supralinear network. Neural Comput. 25, 1994–2037 (2013).
H. Ozeki, I. M. Finn, E. S. Schaffer, K. D. Miller, D. Ferster, Inhibitory stabilization of the cortical network underlies visual surround suppression. Neuron 62, 578–592 (2009).
Y. Ahmadian, K. D. Miller, What is the dynamical regime of cerebral cortex? Neuron 109, 3373–3391 (2021).
S. Sadeh, C. Clopath, Inhibitory stabilization and cortical computation. Nat. Rev. Neurosci. 22, 21–37 (2021).
A. Sanzeni et al., Inhibition stabilization is a widespread property of cortical networks. Elife 9, e54875 (2020).
A. Litwin-Kumar, R. Rosenbaum, B. Doiron, Inhibitory stabilization and visual coding in cortical circuits with multiple interneuron subtypes. J. Neurophysiol. 115, 1399–1409 (2016).
A. Mahrach, G. Chen, N. Li, C. van Vreeswijk, D. Hansel, Mechanisms underlying the response of mouse cortical networks to optogenetic manipulation. Elife. 9, e49967 (2020).
D. B. Rubin, S. D. Van Hooser, K. D. Miller, The stabilized supralinear network: A unifying circuit motif underlying multi-input integration in sensory cortex. Neuron 85, 402–417 (2015).
L. Cossell et al., Functional organization of excitatory synaptic strength in primary visual cortex. Nature 518, 399–403 (2015).
H. Kasai, N. E. Ziv, H. Okazaki, S. Yagishita, T. Toyoizumi, Spine dynamics in the brain, mental disorders and artificial neural networks. Nat. Rev. Neurosci. 22, 407–422 (2021).

2 of 3 https://doi.org/10.1073/pnas.2216092119

pnas.org18.
19.
20.
21.
22.

B. A. Linkenhoker, C. G. von der Ohe, E. I. Knudsen, Anatomical traces of juvenile learning in the auditory system of adult barn owls. Nat. Neurosci. 8, 93–98 (2005).
M. E. Rule, T. O’Leary, C. D. Harvey, Causes and consequences of representational drift. Curr. Opin. Neurobiol. 58, 141–147 (2019).
J. M. Goaillard, E. Marder, Ion channel degeneracy, variability, and covariation in neuron and circuit resilience. Ann. Rev. Neurosci. 44, 335–57 (2021).
H. Li, Z. Xu, G. Taylor, C. Studer, T. Goldstein, Visualizing the loss landscape of neural nets. Adv. Neural Inf. Proc. Sys. 31, (2018).
J. Frankle, M. Carbin, The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv [Preprint] (2018). https://doi.org/10.48550/arXiv.1803.03635. Accessed 23 September 2022.

PNAS2022

Vol. 119No. 48e2216092119

https://doi.org/10.1073/pnas.2216092119 3 of 3