Efficient coding, channel capacity, and the emergence of retinal mosaics
Authors: Na Young Jun
Greg D Field
John Pearson
Venue: NeurIPS 2022 Accept (2022)
Keywords: efficient coding
retina
neuroscience
information theory

Abstract: Among the most striking features of retinal organization is the grouping of its output neurons, the retinal ganglion cells (RGCs), into a diversity of functional types. Each of these types exhibits a mosaic-like organization of receptive fields (RFs) that tiles the retina and visual space. Previous work has shown that many features of RGC organization, including the existence of ON and OFF cell types, the structure of spatial RFs, and their relative arrangement, can be predicted on the basis of efficient coding theory. This theory posits that the nervous system is organized to maximize information in its encoding of stimuli while minimizing metabolic costs.  Here, we use efficient coding theory to present a comprehensive account of mosaic organization in the case of natural videos as the retinal channel capacity---the number of simulated RGCs available for encoding---is varied. We show that  mosaic density increases with channel capacity up to a series of critical points at which, surprisingly, new cell types emerge. Each successive cell type focuses on increasingly high temporal frequencies and integrates signals over larger spatial areas. In addition, we show theoretically and in simulation that a transition from mosaic alignment to anti-alignment across pairs of cell types is observed with increasing output noise and decreasing input noise. Together, these results offer a unified perspective on the relationship between retinal mosaics, efficient coding, and channel capacity that can help to explain the stunning functional diversity of retinal cell types.

Efficient coding, channel capacity, and the emergence of retinal mosaics
Na Young Jun Department of Neurobiology
Greg D. Field
Department of Neurobiology
Duke University Durham, NCC 27710 nayoung jun@duke. edu
Duke University Durham, NC 27710 f field@neuro duke. edu
John M. Pearson
Department of Biostatistics & Bioinformatics
Department of Neurobiology
Department of Electrical and Computer Engineering
Duke University
Durham, NC 27710
wohn.pearson@dulke. edu
Abstract
Among the most striking features of retinal organization is the grouping of its output neurons, the retinal ganglion cells (RGCs), into a diversity of functional types. Each of these types exhibits mosaic-like organization of receptive fields (RFs) that tiles the retina and visual space. Previous work has shown that many features of RGC organization, including the existence of ON and OFF cell types, the structure of spatial RFs, and their relative arrangement, can be predicted on the basis of efficient coding theory. This theory posits that the nervous system organized to maximize information in its encoding of stimuli while minimizing metabolic costs. Here, we use efficient coding theory to present comprehensive account of mosaic organization in the case of natural videos as the retinal channel capacity- the number of simulated RGCs available for encoding varied. We show that mosaic density increases with channel capacity up to series of critical points at which, surprisingly, new cell types emerge. Each successive cell type focuses increasingly high temporal frequencies and integrates signals over larger spatial areas. In addition, we show theoretically and simulation that a transition from mosaic alignment to anti-alignment across pairs of cell types is observed with increasing output noise and decreasing input noise. Together, these results offer a unified perspective the relationship between retinal mosaics, efficient coding, and channel capacity that can help to explain the stunning functional diversity retinal cell types.
Introduction
The retina is one of the most intensely studied neural circuits, yet we still lack a computational understanding of its organization in relation its function. At structural level, the retina forms a three-layer circuit, with its primary feedforward pathway consisting of photoreceptors to bipolar cells retinal ganglion cells (RGCs), the axons of which form the optic nerve [ RGCs can be divided into 30-50 functionally distinct cell types (depending on species) with each cell responsive localized area of visual space (its receptive field (RF)), and the collection of RFs for each type tiling space to form "mosaic"
Each mosaic represents the extraction of specific type
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

A
B
Time
@1
f
P
2
spelial kernels
temporal convolutions with noniinearlyy
Poat
+ Pout
Poot
Pour
V Poot
Peu
firing rates
((1)
((2)
((3)
r(4)
((S)
r(6)
r(7)
C
optimize
Figure 1: ON and OFF RF mosaics and their temporal kernels predicted by efficient coding of natural videos. (A) Frames of natural videos x(1) plus input noise nin are linearly filtered with the spatial kerness w, and then passed through one-dimensional temporal convolutions Φ followed by nonlinearity, resulting firing rates (t) for each of J RGCs. (B) The same calculations shown along the time axis, visualizing the temporal convolutions. (C) Examples of initial and optimized spatial filters, temporal filters, and nonlinearities: (left) fast OFF kernel, (right) slow ON kernel. (D) Unconstrained spatial filters (J 160) learned center- surro-- shapes, about half of which are ON RFs. (E) Temporal filters (J = 160) using the parameterization (6) converged to four distinct clusters.
of information across the visual scene by particular cell type, with different mosaics responding to light increments or decrements (ON and OFF cells). high or low spatial and temporal frequencies, color, motion, and a host of other features. While much is known about the response properties of each RGC type, the computational principles that drive RGC diversity remain unclear.
Efficient coding theory has proven of the most powerful ideas for understanding retinal organi- zation and sensory processing. Efficient coding posits that the nervous system attempts to encode sensory input by minimizing redundancy subject biological costs and constraints (6. 7 As more commonly formulated, it seeks to maximize the mutual information between sensory data and neural representations, with the most common cost in the retinal case being the energetic cost of action potentials transmitted by the RGCs, Despite its simplicity, this principle has proven useful, predicting the center-surroune structure of RFs the frequency response profile of contrast sensitivity (9) the structure of retinal mosaics [1 I the role of nonlinear rectification [12. different spatiotemporal
kernels [ and inter-mosaie arrangements
[4 [5).
While previous studies have largely focused on either spatial or temporal aspects of efficient coding, optimize efficient coding model of retinal processing in both space and time to natural videos We systematically varied the number of cells available to the system and found that larger numbers of available cells led more cell types. Each of these functionally distinct types formed its own mosaic of RFs that tiled space. We show that when and how new cell types emerge and form mosaics the result of tradeoff's between power constraints and the benefits of specialized encoding that shift as more cells are available to the system. We show that cell types begin by capturing low-frequency temporal information and capture increasingly information over larger spatial RFs as new cell types form. Finally, we investigated the relative arrangement of these mosaics and their dependence on noise. We show that mosaic pairs can be aligned or anti-aligned depending on input and output noise in the system [4.. Together, these results demonstrate for the first time how efficient coding principles can explain, even predict, the formation of cell types and which types most informative when channel capacity is limited.
16
gigher-frequecyy temporal

---

2
Model
The model we develop is extension of [4. retinal model for efficient coding of natural images, which is based on a mutual information maximization objective proposed in [TO. The retinal model takes D-pixel patches of natural images x E RDd corrupted by input noise Din (O, Cr.)): ). filters these with unit-norm linear kernels {w; E IR= || 1jj=,....JJ representing J RGCs, and then feeds the resulting signals y, = w (x+) through softplus nonlinearities v)(y) = log (1 + e! / (we used B 0.25) with gain 2 and threshold ,,. Finally, these signals are further corrupted by additive output noise Nout ~ (0, C ). to produce firing rates ry:
Tj = 7i 7y - -0,) + Rout.j.
(1)
The model learns parameters wj. 7j. and fo to maximize the mutual information between the inputs x and the outputs r, under a mean firing rate constraint [TC 14:
log det (GW (Cx + C. WG + C
maximize
(2)
det (GW Crn.. WG + +Cnou)
subject to E[ry]= 1.
(3)
Here Cx is the covariance matrix of the input distribution, W E RDX contains the filters wj as its columns, the gain matrix G diag (t da
dy b, and the noise covariances are Cr., = o in 1DDD
and Cr..t = Pour Jx This objective equivalent to the formulation in IC,, which assumes normally distributed inputs and locally linear responses in order to approximate the mutual information in closed form.
Here, we extend this model to time-varying inputs x() E R representing natural videos (Figure A-B), which are convolved with linear spatiotemporal kernels (w,(t)]==,....))
(t) = w (1) x() f
wr(T) x(t - r)dr.
(4)
We additionally assume that the convolutional kernels are separable in time and space:
|| 1, 6,(1) R. / (()) =1
wy(() ())
(5)
and the temporal kernels are unit-norm impulse responses taking the following parametric form:
())( x { ay - aaf -t// if delay 1 2
otherwise
where 0j, a'y. Ty > 0, T/ > O are learnable parameters, and n E N is fixed. Previous work assumed an unconstrained form for these filters, adding zero-padding before and after the model's image inputs to produce the characteristic shape of the temporal filters in primate midget and parasol cells [3], but this pero-padding represents biologically implausible constraint, and the results fail to correctly reproduce the observed delay in retinal responses [1 I8 [9) Rather, optimizing ( with unconstrained temporal filters produces filter bank uniformly tiling time (Supplementary Figure
By contrast, 66 is motivated by the arguments of 200, which showed that the optimal mniimum-phase temporal filters of retinal bipolar cells, the inputs the RGCs, take the form
t/r (wt)
@( > 0) x e t///Tssinwt wt cos As -t/r
3
when WT < 1. Thus, we model RGC temporal filters as linear combination of these forms. In practice, we take only two filters and shown to perform well in capturing observed retinal responses [9] The results produced by more filters or different exponents are qualitatively unchanged (Supplementary Figure 7. For training on video data, we use discrete temporal filters and convolutions with Σ 6,1122 Finally, while unconstrained spatial kernels wj converge characteristic center-surround shapes under optimization of 2 (Figure [C), for computational efficiency and stability, we parameterized these filters using a mndally-symmetrie difference of Gaussians
6 rather than
since these have been
(r(r) x e- -ayr" eee byr²
b > 0, 0 0<ca <1
(8)
3

---

where r measures the spatial distance to the center of the RF, and the parameters aj, bj. Cj that determine the center location and spatial kernel shape are potentially different for each RGC j. The result of optimizing 2 using these forms is set of spatial and temporal kernels (Figure D-E) that replicate eeperimentalll--oseeee shapes and spatial RF tiling.
3 Efficient coding as function of channel capacity: linear theory
Before presenting results from our numerical experiments optimizing the model B we begin by deriving intuitions about its behavior by studing the case of linear filters analytically. That is, we assume single gain y for all cells, no bias (# = 0), and linear transfer function n7y) y. As we will see, this linear analysis correctly predicts the same types of mosaic formation and filling observed in the full nonlinear model. Here, we sketch the main results, deferring full details to Appendix
3.1 Linear model in the infinite retina limit
For analytical simplicity, we begin by assuming infinite retina which RFs form mosaics described by regular lattice. Under these conditions, we can write the log determinants in 2 as integrals and optimize over the unnormalized filter yo subject power constraint:
dkk Jag (2m)²
log Eeea lo(k (C, + aa
- Σ REG
lu(k + g)
(9)
Deeo lu(k +
=) +
where (zk) is the Fourier transform of the stationary image covariance C2 zz). the integral is over all frequencies k E Go unique up aliasing caused by the spatial regularity of the mosaic, and the sums over g account for aliased frequencies (Appendix [A.I]. In [ the range [-T, π is used for
the integral, corresponding to one-dimensional lattice and units of mosaic spacing A
Now, solving the optimization
results in a spatial kernel with the spectral form (Appendix A.2
+1) (=+=2_=A(()))
22 in 2 (k)
1
Cx)
(V++ 2 4 ou V (k)
)
|o(k)]
k E Go.
(10)
where k = |||] and is chosen to enforce the constraint total power. This is exactly the solution found in [8] linking (in the linear case) to the model of LC I.. Note, however, that (TO) is only nonzero within Gg. since RF spacing sets an upper limit on the passband of the resulting filters.
The generalization of this formulation to the spacetime case is straightforward. Given a spacetime stationary image spectrum Cz z', - t') and mnmiall-symmmttric, causal filter w(z, t). the same infinite retina limit as above requires calculating determinants across both neurons i.j and time points t, t' of matrices with entries of the form
Pijte / dzdz' dr 2orn222u(ee 2, (.1-T)Crle - +' r')u(z, 22,'' -2'('-7) 7')
/ d2k dw ek 2z) ti)+t (o(kww)) [ (k
(1I)
(2T)² 2T
Again, such matrices can be diagonalized in the Fourier basis, with the result that the optimal spacetime filter once again takes the form 10) with the substitutions u(k) + o(k,w), C2 (k) + (k. (Appendix A.3p. Figure A depicts the frequency response of this filter in d = spatial dimensions, with corresponding spatial and temporal sections plotted in Figures BC
3.2 Multiple cell types and the effects of channel capacity
Up to this point, we have only considered single type of filter v(k, w), corresponding to a single cell type. However, multiple cell types might increase the coding efficiency of the entire retina if they specialize, devoting their limited energy budget to non-overlapping regions of frequency space. Indeed, optimal encoding in the multi-cell-type case selects filters u and 1' that satisfy v" (k; (k, w) = O, corresponding encoding independent visual information (Appendix A.4).
This result naturally raises two questions: First, how many filter types are optimal? And second, how should given budget of J RGCs be allocated across multiple filter types? As detailed in Appendix

---

Optimal filte lolk
Spotial filter s(2)
Temporal filter filter a(O
lo5ap
E
F
Spatial fiter v(2)
Marginal information gain per neuron
Neuron allocation across mossics
Neurons
0.8
Naurars in mosali
40-
0.-
-0-
Information
20-
0.4
mosacc
Figure 2: Optimal filters in the linear spacetime case. (A) Spectrum of the optimal linear spacetime filter in d = spatial dimension. Contour lines indicate constant (log) power. (B) Spatial filters at representative temporal frequencies. Each filter represents vertical section the correspondingly colored dot in A. (C) Temporal filters at representative spatial frequencies. Each filter represents horizontal section at the correspondingly colored dot in (D) Spatial filters at w 0 for increasing numbers of RGCs J. The spatial extent of the center narrows with more RGCs added to the mosaic. (E) Gain information per RGC added to each mosaic as a function of current RGC number J. New cell types begin when the marginal benefit of adding an RGC to existing mosaic equals the benefit of adding the first RGC a new cell type. Color indicates wp, the temporal frequency of the narrow- band filter. (F) Total number of RGCs in each mosaic
function of total RGCs across all mosaics. As new cell types arise and form mosaics, new RGCs are allocated to existing mosaics at decreasing rate. For both plots, A 100, Oin 0.4, Jous 1.25, and logi0 wh 1.5, 1.52, 1.54, 1.56, 1.6. Details of calculations in Appendix A.5
can proceed by analyzing the case of finite retina in the Fourier domain, approximating the information encoded by mosaic of J RGCs with spatial filters given by (0) and nonoverlapping bandpass temporal filters that divide the available spectrum (e-g.. Figure B C). Following (21.. we approximate the correlation spectrum of images by the factorized power law (k, = kn with 1.3 and find that in this case, the optimal filter response exhibits two regimes as function of
A.5
spatial frequency (Supplementary Figure A)): First, below ky (??) 1/a the optimal filter is separable and log-linear, and the filtered image spectrum is white:
kawa Av
12
( ~01
where v, the Lagrange multiplier (9) that enforces the power constraint, scales as I/P for small
values of maximal power P and 1/ p for larger values (Supplementary Figure (D). Second, for
A 2 ky, the filter response decreases as --a until reaching its upper cutoff at lic ky ( with the filtered image spectrum falling off the same rate (Supplementary Figure 3
But what do these regimes have to do with mosaic formation? The link between the two is given by the fact that, for a finite retina with regularly spaced RFs, adding RGCs decreases the distance between RF centers and so increases the resolving power of the mosaic. That is, the maximal value of
k grows roughly as l ~ JJ in d = 2, such that larger numbers of RGCs capture more information increasingly higher spatial frequencies (Supplementary Figure [A). However, while information gain is roughly uniform in the whitening regime, it falls off sharply for k Z ky (Supplementary Figure suggesting the interpretation that the S ky regime is a "mosaic filling" phase which information accumulates almost linearly as RFs capture new locations in visual space, while the 2 ky regime constitutes "compression phase" in which information gains are slower as RFs shrink to accommodate higher numbers (Figure 2D). Indeed, one can derive the scaling of total

---

A
B
spotial spectrum
All videos
Slow videos only
(4)
(3)
(4)
[51
Aideoss
number of videos
spobil frequency (Lipbe))
Temporal all
20
S5000 dunale (881
temporal spectrum
0.7 0.5
1.0
spectrol attenuation
Past
Temporal RFs all OFF cells
bemporal Doj
Statistics of natural videos affect learned RFs. (A) Histogram of spectral attenuation (fraction of power < Hz) for each video clip from the Chicago Motion Database. A significant portion of the dataset exhibits predominaly low-frequency spectral content in time. Videos with spectral attenuation above 0.9, 0.8, and 0.7, are denoted (4), (3), and (2), respectively, while (1) refers to all videos in the dataset. (B) Spatial (top) and temporal (bottom) spectral density of the four subsets. (C) Temporal filters learned by training on each of the four subsets. Training on slow videos produced only smoothing kernels, while training on all videos produced a variety of temporal filters.
Figure 3:
information function of J:
log
[1+ G
2P (o+2)
1 Jj
k: kj
I~
(12)
J
2
k こ ky
where Po the power budget per RGC and f is the RGC number corresponding k ky. Thus, mosaic filling exhibits diminishing marginal returns (Figure 2E), such that new cell types are favored when the marginal gain for growing mosaics with lower temporal frequency drops below the gain from initiating new cell type specialized for higher temporal frequencies. Moreover, the difference between these gain curves implies that new RFs are not added to all mosaies at equal rates, but in proportion to their marginal information (Figure F As we demonstrate in the next section, these features of cell type and mosaic formation continue to hold in the full nonlinear model in simulation.
4 Experiments
We analyzed the characteristics of the optimal spatiotemporal RFs obtained from the model
trained on videos from the Chicago Motion Database (22]. Model parameters for spatial kernels.
temporal kernels, and the nonlinearities were jointly optimized using Adam 23 to maximize (
subject to the mean firing rate constraint (3) using the augmented Lagrangian method with the
quadratic penalty = 24 Further technical details of model training are in Appendix
E
All model code and reproducible examples are available at htttts:s//giiaataa http://tttuub.co//parraeee cw.ppearsoolab/
efficientcoding
As previously noted, the power spectral density of natural videos can be well approximated by product of spatial and temporal power-law densities, implying an anticorrelation between high spatial and temporal frequency content 21 Supplementary Figure shows the data spectrum of the videos in our experiments also well-approximated by separable power-law fits. To examine the effect of these statistics on the learned RFs, we divided the dataset into four progressively smaller subsets by the proportion of their temporal spectral content below 3 Hz, their spectral attenuation. Using values of 70%, 80%, and 90% then yielded progression of datasets ranging from most videos to only the slowest videos (Figure 3K B). Indeed, when the model was trained on these progressively slower data subsets, produced only temporal smoothing filters, whereas the same model trained on all videos produced variety of "fast" temporal filter types (Figure We also note that these experiments used unconstrained spatial kernels in place (8) yet still converged on spatial RFs with typical center-surround structure as in [10 15 [4]. Thus, these preliminary experiments suggest that the optimal encoding strategy- in particular, the number of distinct cell types found- dedpddss critically on the statistics of the video distribution to be encoded.

---

4.1 Mosaics fill in order of temporal frequency
As the number of RGCs available to the model increased, we observed the formation of new cell types with new spectral properties (Figure D. We characterized the learned filters for each RGC in terms of their spectral centroid, defined as the center of mass of the Fourier (spatial) or Discrete Cosine (temporal) transform. Despite the fact that each model RGC was given its own spatial and temporal filter parameters S 6
the learned filter shapes strongly clustered, forming mosaics with nearly uniform response properties (Figure (Fi) Critically, the emergence of new cell types shifted the spectral responses of previously established ones, with new cell types compressing the spectral windows of one another they further specialized. Moreover, mosaic density increased with increasing RGC number, shifting the centroids of early mosaics toward increasingly higher spatial frequencies. This is also apparent in the forms of the typical learned filters and their power spectra: new filters selected for increasingly high-frequency content in the temporal domain (Figure D DD).
We likewise analyzed the coverage factors of both individual mosaics and the entire collection, defined as the proportion of visual space covered by the learned RFs. More specifically, we defined the spatial radius of an RF as the distance from its center at which intensity dropped to 20% of its peak and used this area to compute a coverage factor, the ratio of total RF area to total visual space (T/4 of the square's area due to circular masking). Since coverage factors depend not simply RGC number but on RF density, they provide an alternative measure of the effective number of distinct cell types learned by the model. As Figure E shows, coverage increases nearly linearly with RGC number, while coverage for newly formed mosaics increases linearly before leveling off. In other words, new cell types initially increase coverage of visual space by adding new RFs, but marginal gains in coverage diminish as density increases. In all cases, the model dynamically adjusts the number of learned cell types and the proportion of RGCs assigned to them as channel capacity increases.
4.2 Phase changes in mosaic arrangement
In addition to retinal organization at the level of mosaics, a pair of recent papers reported both experimental [15] and theoretical
[14] evidence for an additional degree of freedom in optimizing information encoding: the relative arrangement of ON and OFF mosaics. Jun et al. studied this for the case of natural images in [4]. demonstrating that the optimal configuration of ON and OFF mosaics alignment (RFs co-located) at low output noise levels and anti-alignment (OFF RFs between ON RFs and vice-versa) under higher levels of retinal output noise. Moreover, this transition is abrupt, constituting a phase change in optimal mosaic arrangement.
We thus asked whether learned mosaics exhibited a similar phase transition for natural video encoding. To do so, following [4] repeatedly optimized a small model (J = 14, 7 ON, 7 OFF) for multiple learned filter types while systematically varying levels of input and output noise. In each case, one ON-OFF pair was fixed at the center of the space, while the locations of the others were allowed to vary. We used RF size D = 82 pixels for Slow and D = 122 for FastA and FastB cell types to allow the size of spatial kernels to be similar those of the previous experiments, and we imposed the additional constraint that the shape parameters aj. bj. and cy in (8) be shared across RGCs.
Under these conditions, the six free pairs of RFs converged to either aligned (overlapping) or anti-aligned (alternating) positions along the edges of the circular visual space, allowing for a straightforward examination of the effect of input and output noises mosaic arrangement. Figure SA-C shows that the phase transition boundaries closely follow the pattern observed 14: increasing output noise shifts the optimal configuration from alignment to anti-alignment. Moreover, for each of the tested filters, increasing input noise discourages this transition. This effect also follows from the analysis presented in [4] since higher input noise increases coactivation of nearby pairs of RFs, requiring larger thresholds to render ON-OFF pairs approximately indpendent (Appendix B).
5 Discussion
Related work: As reviewed in the introduction, this study builds on long line of work using efficient coding principles to understand retinal processing. addition, is related to work examining encoding of natural videos 25 2 [6 and prediction in space-time. The most closely related work to this one is that of [3], which also considered efficient coding of natural videos and considered the tradeoffss involved in multiple cell types. Our treatment here differs from that work in several
7

---

A
spatial frequency (pixet") spatial spatial spectral centrolo (plkel "Jsspataal spectral centrola (pikel *Jspatal spectral centroid (pixer")
Slow
FastA
FastB
FastC
FastD
0.16
80 neurons
Slowe OM
Cone OFF
ON
Pask
0.10
0.08
0.06
9.04
140 neurons
ON
Paolk ON
0.12
PietA
Parts
0.10
0.08
FastE Of*
OFF
0.06
0.16
neurons
0.14
ON
0.12
Flask
0.10
Fasmil on
Fachs
A22
5.53
0.08
Faeto OM
Pistl O++
OFF
0.04
65.4
0.45
4
10 12
temporal spectral centrold (Hz)
0.16
200 neurons
0.08
0.04
0.00
temporal frequency (Hz)
coverage factor per mosaic
total coverage factor
25
number of
Figure 4: Emergence of new RF types with increasing RGC number. (A) Distribution of spatial and temporal spectral centroids for J = 80, 140, 200 RGCs. ON and OFF RFs form distinct clusters corresponding to different learned filter types. (B) ON and OFF mosaics corresponding to each cell type. The number in the lower right of each plot is the coverage factor for the mosaic. (C) Power spectral density of a typical kernel in each mosaic for J = 200. As predicted, learned kernels filter over roughly nonoverlapping regions of spatiotemporal frequency. Contour lines represent isopower lines of the signal correlation C₂2 (k, w.) (D) Learned shapes of example spatial ON and OFF filters (top) and corresponding temporal filters (bottom) from each RF type for the J 200 case. (E) Total (left) and per-m mosaic (right) coverage factors as the number of RGCs J increases from 40 to 200. New mosaics increase coverage linearly with the number of RFs, while nearly full mosaies see diminishing returns in coverage from density increases. See Supplementary Figures for similar plots for all RGC numbers.

---

A
B
C
Slow
FastA
FastB
2.2 2.2
2.2
2.2
2.4
2.4
2.4
2.5
2.5
2.6
2.6
2.6
2.7
2.7
2.7
2.8
2.8
2.8
2.9
2.9
2.9
3.0
3.0
3.0
3.1
3.1
3.1
3.2
3.2
3.2
OFF RF centers
medlan NN distance
Figure 5: Learned mosaics exhibit a phase transition as a function of input and output noise. (A-C) Spatial kernel centers for Slow (A), FastA (B), and FastB (C) function of on and oou In all three cases, the optimal configuration changes from aligned to anti-alige aati-aligned when output noise increases or input noise decreases. Blue bars denote alignment measured by median distance to the nearest RF center the opposite type.
key ways: First, while [R] concerned with demonstrating that multiple cell types could prove beneficial for encoding (in a framework focused on reconstruction error), that study predetermined the number of cell types and mosaic structure, only optimizing their relative spacing. By contrast, this work is focused on how the number of cell types is dynamically determined, and how the resulting mosaics arrange themselves, (i.e., the channel capacity). Specifically, we follow previous efficient coding models [8] T0 in maximizing mutual information and do not assume an priori mosaic arrangement, a particular cell spacing, or a particular number of cell types- all of these emerge via optimization in our formulation. Second, while the computational model of [ optimized strides for pair of rectangular arrays of RGCs, we individually optimize RF locations and shapes, allowing us to study changes in optimal RF size and density new, partial mosaics begin to form. Third, while 13] used zero-padding of natural videos to bias learned temporal filters toward those of observed RGCs, we link the form of temporal RFs biophysical limits on the filtering properties of bipolar cells, producing temporal filters with the delay properties observed in real data. Finally, while [iK] only considered single noise source in their model, we consider noise in both photoreceptor responses (input noise) and RGC responses (output noise), allowing to investigate transitions in the optimal relative arrangement of mosaics
function of the number of units available for encoding
14
We have shown that efficient coding of natural videos produces multiple cell types with complemen- tary RF properties. In addition, we have shown for the first time that the number and characteristics of these cell types depend crucially on the channel capacity: the number of available RGCs. As new simulated RGCs become available, they are initially concentrated into mosaies with more densely packed RFs, improving the spatial frequency bandwidth over which information is encoded. However, as this strategy produces diminishing returns, new cell types encoding higher-frequency temporal features emerge in the optimization process. These new cell types capture information over distinct spatiotemporal frequency bands, and their formation leads to upward shifts the spatial frequency responses of previously formed cell types. Moreover, pairs of ON and OFF mosaics continue to

---

exhibit the phase transition between alignment and anti- alignmelnmmnt revealed in purely spatial op- timization of efficient coding [4]. suggesting that mosaic coordination is general strategy for increasing coding efficiency. Furthermore, despite the assumptions of this model-linea filtering, separable filters, firing rates instead of spikes- our results are consistent with observed retinal data. For example, RGCs with small spatial RFs exhibit more prolonged temporal integration: they are also more low-pass in their temporal frequency tuning. Second, there is greater variability in the size and shape of spatial RFs at given retinal location, but temporal RFs exhibit remarkably little variability our simulations and in data [9]. Thus, these results further testify to the power efficient coding principles in providing conceptual framework for understanding the nervous system.
Acknowledgments and Disclosure of Funding
This work was supported by NIH/National Eye Institute Grant ROI EY031390
10

---

References
[1] Mrinalini Hoon, Haruhisa Okawa, Luca Della Santina, and Rachel OL Wong. Functional architecture of the retina: development and disease. Progress in retinal and eye research, 42:44-80 84, 2014.
[2] H Wässle, L Peichl, and Brian Blundell Boycott. Morphology and topography of on-and oft-alpha cells in the cat retina. Proceedings of the Royal Society of London. Series B. Biological Sciences, 212((187): 157-1777 1981.
[3] Steven H Devries and Denis A Baylor. Mosaic arrangement of ganglion cell receptive fields in rabbit retina. Journal of neurophysiology, 78(4):2048 2060, 1997.
[4] Greg D Field, EJ Chichilnisky, et al. Information processing in the primate retina: circuitry and coding. Annual review of neuroscience, 30(1):1-30, 2007.
[5] Joshua R Sanes and Richard Masland. The types of retinal ganglion cells: current status and implications for neuronal classification. Annuul review of neuroscience, 38:221-246, 2015.
[6] Fred Attneave. Some informational aspects of visual perception. Psychological review, 61(3)1883 1954.
[7] Horace B Barlow. Possible principles underlying the transformation of sensory messages. Sensory communication, 1:217-234, 1961.
[8] Joseph J Atick and A Norman Redlich. Towards a theory of early visual processing. Neural computation, 2(3):308-320, 1990.
[9] Joseph Atick and A Norman Redlich. What does the retina know about natural scenes? Neural computation, 4(2): 1 19--660, 1992.
[10] Yan Karklin and Eero P Simoncelli. Efficient coding of natural images with population of noisy
neurons. Advances in neural information processing systems, pages 999-1007, 2011.
[11] Eizaburo Doi. Jeffrey L Gauthier, Greg Field, Jonathon Shlens, Alexander Sher, Martin Greschner, Timothy A Machado, Lauren H Jepson, Keith Mathieson, Deborah E Gunning, al. Efficient coding of spatial information in the primate retina. Journal of Neuroscience, 1226-1666666 2012.
[12] Xaq Pitkow and Markus Meister. Decorrelation and efficient coding by retinal ganglion cells. Nature neuroscience, 15(4):628 635, 2012.
[13] Samuel Ocko, Jack Lindsey, Surya Ganguli, and Stephane Deny. The emergence of multiple retinal cell types through efficient coding of natural movies. In Advances in Neural Information Processing Systems, pages 9389-9400, 2018.
[14] Na Young Jun, Greg D Field, and John Pearson. Scene statistics and noise determine the relative arrangement of receptive field mosaics. Proceedings of the National Academy of Sciences, 1 118(39), 2021.
[15] Suva Roy, Na Young Jun, Emily L Davis, John Pearson, and Greg D Field. Inter-mosaic coordination of retinal receptive fields. Nature, 592(7854):409-413 2021.
[16] Audrey J Sederberg, Jason N MacLean, and Stephanie E Palmer. Learning to make external sensory stimulus predictions using internal correlations populations neurons. Proceedings of the National Academy of Sciences, 115 2018.
[17] EJ Chichilnisky and Rachel S Kalmar. Functional asymmetries in and off ganglion cells of primate retina. Journal of Neuroscience, 22(7):2737-2777, 2002.
[18] Tom Baden, Philipp Berens, Katrin Franke, Miroslav Román Rosón, Matthias Bethge, and Thomas Fuler. The functional diversity of retinal ganglion cells in the mouse. Nature,
52957586):44--550, 2016.

---

[19] Sneha Ravi, Daniel Ahn, Martin Greschner, EJ Chichilnisky, and Greg D Field. Pathway-specine asymmetries between on and off visual signals. Journal of Neuroscience, 38(45):9728-97) 97400 2018.
[20] W. Bialek and W.G. Owen. Temporal filtering in retinal bipolar cells. elements of an optimal computation? Biophysical Journal, 58(5):1227-1233, 1990.
[21] Dawei W Dong and Joseph Atick. Statistics of natural time-varying images. Network: Computation in Neural Systems, 6(3):345, 1995.
[22] Jared M. Salisbury and Stephanie Palmer. Optimal prediction in the retina and natural motion statistics, Journal of Statistical Physics, (2255):300--133 January 2016.
[23] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:/412.6680 2014.
[24] Jorge Nocedal and Stephen Wright. Numerical optimization. Springer Science, 35(67-68):7, 1999.
[25] Stephanie E Palmer, Olivier Marre, Michael J Berry, and William Bialek. Predictive information in sensory population. Proceedings of the National Academy of Sciences, 112(22):6908 69116699988661,,
2015.
[26] Alan V Oppenheim, Ronald W Schafer, Mark Yoder, and Wayne T Padgett. Discrete-time signal processing. Pearson, Upper Saddle River, NJ, edition, August 2009.
[27] Michael S Lewicki. Efficient coding of natural sounds. Nature neuroscience, 5(4):356-363, 2002.
[28] Stephen W Kuffler. Discharge patterns and functional organization of mammalian retina.
Journal of neurophysiology, 16(1):37-68, 1953.
12

---

Checklist
The checklist follows the references. Please read the checklist guidelines carefully for information on how to answer these questions. For each question, change the default [TODO] to [Yes], [No] or [N/A] You are strongly encouraged to include justification to your answer, either by referencing the appropriate section of your paper or providing brief inline description. For example:
Did you include the license to the code and datasets? [Yes] See Section ??.
Did you include the license to the code and datasets? [No] The code and the data are proprietary.
Did you include the license the code and datasets? [N/A]
Please do not modify the questions and only use the provided macros for your answers. Note that the Checklist section does not count towards the page limit. In your paper. please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] We address this in the Discussion.
(c) Did you discuss any potential negative societal impacts of your work? [No] The focus of this work information -heooretic analyses on simple computational model for the retina. As such, we do not anticipate any direct, negative societal impacts.
(d) Have you read the ethics review guidelines and ensured that your paper conforms them? (Ys]]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] Detailed treatments Appendices and
(b) Did you include complete proofs all theoretical results? [Yes] Detailed treatments are Appendices A and
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material or as URL)? [Yes] The code and instructions to run the complete implementation of this work are included in the supplementary materials, with the necessary links to download the datascts for full replication.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] (and partially N/A) our work did not include data splits, and other, empirically chosen hyperparameters are detailed in the main text and the supplement.
(c) Did you report error bars (e.g.. with respect to the random seed after running experi- ments multiple times)? [No] Our figures were either conceptual theoretical plots or visualizing exemplar data, where error bars are not suitable.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] We detail the compute usage together with other training details in the supplement.
4. If you are using existing assets (e.g., code, data, models) or curating/releasngg new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] We primarily use the Chicago Motion Database 22 and data derived from
(b) Did you mention the license the assets? [No] The license is not specified in the paper and the website https:// c/cdd//mm.. rcc. uchicago edu, but the website's about page says it is "intended for use in visual neuroscience experiments, as training data for machine vision applications,
(c) Did you include any new assets either the supplemental material or as URL? [Yes] Code is available at
13

---

(d) Did you discuss whether and how consent was obtained from people whose data you't 're using/curating? [N/A] We did not use human data.
(c) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A] Our data consisted of images of natural scenes.
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A] We did use crowdsoureing human subjects
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A] We did not use crowdsourcing or human subjects
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A] We did not use crowdsourcing or human subjects
14


Reviews:
[{"review_id": "qhGTII3kWLI", "reviewer": "Reviewer_kZcp", "summary": "This paper presents a model for retinal mosaic organization, derived\nby application of the efficient coding principle to natural movies. In\nparticular, the model shows that the total number of retinal ganglion\ncells is a key parameter that controls the emergence of distinct cell\ntype. The model is also used to extract predictions about the relative\nphase of distinct retinal mosaics as a function of input noise.\n", "questions": "1. Is it possible to clarify better how the differences between this\n   work and [10] (listed in the \"related work\" paragraph) affect the\n   main conclusions of the paper? Or is the main point that the\n   results here are qualitatively similar to those in [10], despite\n   the differences listed under \"related work\"? (I am referring mostly\n   to the material in Section 3). For instance, the first sentence of\n   the Discussion reads: *\"Here, we have shown that efficient coding of\n   natural movies results in the formation of multiple receptive field\n   mosaics with complementary filtering properties [10].\"* Is this\n   suggesting that the main result of the paper is something for which\n   ref [10] should be credited?\n2. Lines 50-53 read: *\"In the case of linear encoding, we show\n   analytically how a tradeoff between information gains from\n   increasing mosaic density and informationg gains from adding new,\n   specialized cell types leads to the addition of new mosaics\n   capturing higher temporal frequency information.\"* Can you make sure\n   that the results backing up this statement are easier to find in\n   the text? If I am not mistaken, the passage should be lines\n   182--189. That part of the text hinges on the fact that *\"as\n   detailed above, adding filters covering non-overlapping spectral\n   windows allows mosaics to specialize to different regions of\n   spatial and temporal frequency\"*. The \"above\" here probably refers\n   to the previous paragraph, that ends with the following: *\"In [10],\n   the authors considered a form of this with spatial bandwidth\n   partitioning (v(k)\\*v'(k) = 0), but as we show below, this special\n   case encounters a limit beyond the first two mosaics, when the\n   spatial passband spectra of new filters substantially\n   overlap. Rather, the more general solution is that mosaic filters\n   are band-limited in both space and time, arranging themselves to\n   tile minimally overlapping regions with highest power in the (k,ω)\n   plane (Appendix A).\"* This passage, in turn, contains a reference to\n   ref [10], one to \"below\", and one to the Appendix. Forgetting about\n   [10] and assuming that \"below\" here refers to Section 4 (which is\n   not valid for the original point we were trying to find support\n   for, as we're looking for an analytical solution, as claimed in the\n   Introduction), we therefore follow this chain of references to the\n   Appendix. The relevant section seems to be section A.4, \"Effects of\n   channel capacity and new mosaic formation\", but it would be great\n   to have this spelled out more clearly, and perhaps the result in\n   the Appendix also connected back to the claim in the main text in a\n   more explicit way.\n3. The Discussion states (lines 278--279) that the results are \"in\n   strong agreement with observed retinal data\". I found this\n   statement surprising, given the absence of quantitative comparisons\n   with empirical data in the paper. Is it possible to clarify what is\n   meant by this, and to substantiate the statement with direct\n   comparisons?", "limitations": "The limitations of the work are acknowledged very briefly in the\ndiscussion (*\"despite the strong assumptions of our model - linear\nfiltering, separable filters, firing rates instead of spikes -\"*), but\nnot unsuitably so for a modelling work of this type.\n", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\nThe paper tackles a relevant problem, that is modeling the\nemergence of distinct retinal mosaics from an efficient coding\nperspective, by building on a solid foundation (reference [10]).\n\n### Weaknesses\nThe paper is very hard to read, mostly because (1) its\ntight coupling with reference [10], and (2) how much of it is\nrelegated to the Appendix. With respect to (1), I am not sure I\nunderstand fully what parts of the model, and which results, are fully\nnovel here and which are minor refinements or variants of the results\nin [10]. Regarding (2), the Appendix includes not only derivations for\nmost of the equations shown in the main text, or technical details for\nthe realization of some of the plots, but also - to my understanding -\nthe basic logic of some of the results (I'll give more details on this\nbelow, under \"questions\").\n\nAdditionally, the paper claims \"strong agreement with observed data\" but contains no data and no quantitative assessment of such claim.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "nzxCMK48atS", "reviewer": "Reviewer_QXXa", "summary": "The authors study how retinal mosaics emerge from an efficient coding framework. They extend the framework to spatial-temporal kernels and video inputs, which is a crucial extensions to understand how natural environmental statistics influence coding in neural systems.  Also, the approach is highly innovative as it allows to derive multiple optimized mosaics.", "questions": "* Lines 124: The Bravais lattice is not sufficiently explained for non-physicists. The section up to Line 129 could be better explained. What are principal and reciprocal vectors? \n\n* Fig 2A: The colored points should be defined in the legend.\n\n* Fig. 5 is very dense and hard to understand - maybe more intuitive labels would help?\n\n\n", "limitations": "The authors should discuss the limitation of the efficient coding objective - is that really all that the organism cares about?", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* Extend previous models to spatiotemporal signals \n* Study relationship between natural video statistics and optimal neural codes\n* Interesting approach to derive multiple mosaics with interesting results\n\nWeaknesses:\n* Sometimes a bit jargony/dense - the authors should check how understandable each section is and make sure they can be followed by non-theoreticians\n* Limited by focus on efficient coding objective - the authors could discucss whether reconstruction of the visual environment is really the ultimate goal of the visual system", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "FOPUuxp3IAV", "reviewer": "Reviewer_8e9N", "summary": "Presents mathematical analysis and computer simulation of model that maximizes mutual information between photoreceptors and the rgc outputs of the retina.  Shows that several different spatiotemporal filters are derived that mosaic the retina in different ways.", "questions": "see above", "limitations": "see above;  no societal impact issues.\n", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\nA nice formulation and modeling approach to understand retinal function.  Impressive technical work.\n\nWeakness\n\nNot sure what the main take away is.  The goal appears to be to understand the neural encoding in the retina, but after that the analysis and results, there is no attempt to tie these back to neurobiological mechanisms.  It seems one could, but the paper just ends with the statement, \"our results are in strong agreement with observed retinal data,\" which leaves you hanging.\n\nSpecific issues:\n\nThe difference of Gaussians model in eq. 8: it mentions that the center position of each kernel is different for each neuron, but is this also learned?  not mentioned.\n\nSection 3:  linear model in the continuum limit - this is very unclear.  what is being continuized?  space?  The integral is over frequency space - not following what's going on.  principal vectors a_1, a_2 and reciprocal vectors b_1, b_2 - what are these?  \n\nSection 4.1: \" power spectral density can be well approximated by a product of spatial and temporal power-law densities\" -  Dong & Atick is cited, but curiously the claim the exact opposite, it is not separable.\n\nFigure 4, panel A shows striking clustering in temporal spectral centroids - they are all stacked neatly in tight columns, no scatter.  is this what emerges from the learned filters, or is somehow the quantization imposed?  \n\nThe mosaics are interesting to look at, but not clear what to take away from this.  \n\nOverall this seems like a very promising direction, I want to like this paper, but I find it a bit confusing and lacking a clear message. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "0xGSKfAShiE", "reviewer": "Reviewer_gDAm", "summary": "The authors present a unified perspective on the appearance of different ganglion cell types (in terms of receptive field size, temporal properties and polarity) in the framework of efficient coding. They deploy a previously developed model and efficient coding framework to spatio-temporal movies. They show, first analytically in the case of a simplified linear model, later experimentally in the non-linear case, how optimal spatial and temporal filters change as the number of neurons (‘channel capacity’) increases. Additionally they investigate the resulting mosaics of similar types for (anti-) alignment and show that input as well as output noise has an important but opposed influence on the alignment of ON/OFF receptive fields. ", "questions": "1. Fig. 2 a: How do the ‘experimental dots’ relate to the presented isoclines?\n2. Could the authors comment on how their argumentation in line l. 186 (and also later) is depend on the assumption of time-space separability? While a detailed analysis is beyond the scope of the manuscript, the specialization to different regions of spatial and temporal frequencies is certainly influenced by this assumption. \n3. l. 196: The data distribution was approximated by a multivariate Gaussian. Is this an appropriate approximations given the known power laws for natural scene statistics? How would a different distribution change the results?", "limitations": "The authors did not comment on limitations or negative societal impact. While latter remains quite abstract, the limitations could be indeed discussed in more detail. ", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n\nThe authors embed their work excellently into the previous literature and work clearly out the parallels and their novel contributions. The theoretical as well as the experimental approaches are well described and cover different interesting arrangements and detailed analyses. All experiments are carried out carefully and are of very high quality, as well as the figures. The authors push the understanding of retinal layout further and try to unify previous approaches. \n\nHowever, there are some minor weaknesses: \n\n### Weaknesses\n1. There are some open questions (see below).\n2. A simple schema for the model would help to understand the setup. \n3. From equation (2) it does not become clear what the differences to [7] are, and how the formula is derived (as I could not find the exact formulation in [7]). \n4. Some parts of section 3 (and especially Suppl. A) are quite technical and it is sometimes difficult to follow. The authors could try to strengthen a ‘red thread’ and give also an intuition where possible.\n5. Sections 3 and 4 seem to be rather detached from each other. While formulas are derived thoroughly, their interpretation and link to section 4 could be improved. \n6. Their final conclusion that previous cell types change down in temporal frequency (l. 274) is not well supported by the presented experiments. While it is more obvious for the spatial frequency, an additional analysis for the temporal domain could help to support this argument. \n\n\n### Minor comments:\n- l. 256: shouldn’t it reference to eq (8)?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept