The functional specialization of visual cortex emerges from training parallel pathways with self-supervised predictive learning
Authors: Shahab Bakhtiari
Patrick J Mineault
Tim Lillicrap
Christopher C Pack
Blake Aaron Richards
Venue: NeurIPS 2021 Spotlight (2021)
Keywords: neuroscience
visual system
self-supervised learning
visual cortex

Abstract: The visual system of mammals is comprised of parallel, hierarchical specialized pathways. Different pathways are specialized in so far as they use representations that are more suitable for supporting specific downstream behaviours. In particular, the clearest example is the specialization of the ventral ("what") and dorsal ("where") pathways of the visual cortex. These two pathways support behaviours related to visual recognition and movement, respectively. To-date, deep neural networks have mostly been used as models of the ventral, recognition pathway. However, it is unknown whether both pathways can be modelled with a single deep ANN. Here, we ask whether a single model with a single loss function can capture the properties of both the ventral and the dorsal pathways. We explore this question using data from mice, who like other mammals, have specialized pathways that appear to support recognition and movement behaviours. We show that when we train a deep neural network architecture with two parallel pathways using a self-supervised predictive loss function, we can outperform other models in fitting mouse visual cortex. Moreover, we can model both the dorsal and ventral pathways. These results demonstrate that a self-supervised predictive learning approach applied to parallel pathway architectures can account for some of the functional specialization seen in mammalian visual systems.

The functional specialization of visual cortex emerges from training parallel pathways with self-supervised predictive learning
Shahab Bakhtiari
Patrick Mineault patrick. miw..climaiiaaattmmaaalla com
Mila & McGill University bakhtias@mila quebec
Tim Lillicrap
Christopher C. Pack
Deep
McGill University
cimothy111lliccappgooogee
christopher
hetiisoopher.pcckmmmglll... pwckemcgill.ca
Blake A. Richards
CIFAR, Mila & McGill University
blake. blake.ri www.hrisaaaaaaaaaaacaaaaaaacaaaa quebec
Abstract
The visual system of mammals comprised of parallel, hierarchical specialized pathways. Different pathways are specialized in so far as they use representations that are more suitable for supporting specific downstream behaviours. partic- ular, the clearest example is the specialization of the ventral ("what") and dorsal ("where") pathways of the visual cortex. These two pathways support behaviours related to visual recognition and movement, respectively. To-date, deep neural networks have mostly been used as models of the ventral, recognition pathway. However, it is unknown whether both pathways can be modelled with single deep ANN. Here, we ask whether a single model with a single loss function can capture the properties of both the ventral and the dorsal pathways. We explore this question using data from mice, who like other mammals, have specialized pathways that appear support recognition and movement behaviours. We show that when we train deep neural network architecture with two parallel pathways using self- supervised predictive loss function, in fitting mouse visual cortex. Moreover, we can model both the dorsal and ventral pathways. These results demonstrate that self-supervised predictive learning approach applied to parallel pathway architectures can account for some the functional specialization seen in mammalian visual systems.
outperform other models
Introduction
In the mammalian visual cortex information is processed in hierarchical manner using two special- ized pathways [15] [52] the ventral, or "where" pathway, and the dorsal, or "what" pathway. These two
pathways specialized for visual recognition and movement, respectively [42 24
19
58
59 17].
For example, damage to the ventral pathway may impair object recognition, whereas damage to the dorsal pathway may impair motion perception (64]
Deep artificial neural networks (ANNs) trained in a supervised manner on object categorization have been successful matching the representations of the ventral visual stream [22 57 (2) They have
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

been shown to develop representations that map onto the ventral hierarchy, and which can be used to predict [62 and control [ 49
neural activity in the ventral pathway. However, when we look at the other principal visual pathway in the mammalian brain, i.e. the dorsal pathway, the situation is different. Very few studies have examined the ability of deep ANNS to develop representations that match the dorsal hierarchy (though see the following fMRI study: (2I).. Moreover, to the best of our knowledge, no studies have demonstrated both ventral-like and dorsal-like representations in a single network.
This lack of deep ANN models that capture both ventral and dorsal pathways leads naturally to an important question: under what training conditions would specialized ventral-like like and dikee pathways emerge in deep ANN? Would second loss function be required to obtain matches to dorsal pathways, or is there a single loss function that could induce both types of representations?
Recent work has shown that self-supervised learning can produce similar results to supervised learning for the ventral pathway [63 (3 [26]. Moreover, there large body of work showing that mammals possess predictive processing mechanisms in their cortex [31 30) [8] [16., including in the dorsal pathway 22 331, which suggests that predictive form of self-supervised learning could potentially
One promising set of candidates are predictive self-supervised loss functions 45
22 38
lead to the emergence of both ventral and dorsal-like representations.
Addressing this question requires recordings from different ventral and dorsal visual areas in the brain. Here, we explore these issues using publicly available data from the Allen Brain Observatory (9]. which provides recordings from a large number of areas in mouse visual cortex. We examine the ability of self-supervised predictive loss function (contrastive predictive coding 45 22 to induce representations that match mouse visual cortex. When train a network with single pathway, we find that it possesses more ventral-like representations. However, when we train a network with two parallel pathways we find that the predictive loss function induces distinct representations that map onto the ventral/dorsal division. This allows the network to better support both object categorization and motion recognition downstream tasks via the respective specialized pathways. In contrast, supervised training with an action categorization task only leads to matches to the ventral pathway, and not the dorsal pathway.
Altogether, this work demonstrates that the two specialized pathways of visual cortex can be modelled with the same ANN if self-supervised predictive loss function is applied to architecture with parallel pathways. This suggests that self-supervised predictive loss functions may hold great promise for explaining the functional properties of mammalian visual cortex.
2 Background and related work
Self-supervised ANN models of the ventral visual stream Recently, 663 and 34 showed that
the representations learned by self-supervised models trained on static images produce good matches to the ventral pathway. Our work builds on this by exploring the potential for self-supervised learning to also explain the dorsal pathway.
ANN models of the mouse visual system Mice have become a common animal model in visual neuroscience due to the sophisticated array of experimental tools 44 27). As such, [53] compared the responses of different areas in mouse visual cortex with the representations of a VGG16 trained on ImageNet. In this paper, we show that self-supervised learning can produce better fits to both ventral and dorsal areas than supervised learning.
ANN models of the dorsal pathway The MotionNet ANN model [51]. is feedforward ANN trained to predict the motion direction of segments of natural images, which was inspired by the role of the dorsal pathway in motion perception. In (6) they show that learning both ventral and dorsal-like representations in a single ANN with two pathways is possible if one forces the two pathways to process the phase and amplitude of a complex decomposition of the stimuli separately. In (21], they show that training a deep ANN on supervised action recognition can induce some match to dorsal pathway fMRI recordings. In this study, we show that with prediction as the learning objective and an architecture that has two parallel pathways, both ventral-like and dorsal-liks representations can be learned.

---

3 Methods
3.1 Datasets
We use the Allen Brain Observatory open 2-photon calcium imaging dataset for the experiments in this study. We select subsets of the dataset based on brain area, recording depth, and visual stimuli used. Recordings from five areas of mouse visual cortex are used (VISp. VISIm, VISal, VISpm, VISam; Figure [p... We only exclude one area from our analyses (VISrl) because it is sentory area, and visual stimuli alone do not drive it well [9]. We use recordings from cortical depths of 175-250um (which corresponds to cortical layers 2-3) as these are the recordings with the largest number of neurons. When selecting visual stimuli, we use recordings elicited by the presentation of natural movies, because unlike static images, movies can elicit clear responses in both ventral and dorsal areas 19. Thus, we only use those parts of the dataset in which natural movies (30 seconds long) were presented as visual stimuli. More details about the dataset can be found in [ For training the deep ANNs, we use the UCF101 dataset [56] (see section A.4 supplementary materials).
3.2 Analysis techniques
Representational Similarity Analysis (RSA) To measure the representational similarities between real brains and ANNs we use the RSA method. RSA has been commonly used in both the neuroscience I [33 and deep learning literature [41] 43). The details of RSA can be found in those citations, but we will summarize it briefly here. First, we create response matrices R E IR" NxM for every brain area and every layer of our ANNs, where N is the number of neurons and M is the number of video blocks, with each block comprising 15 frames of the video. Element ij of the response matrix represents the response of the 7th neuron to the jth video sequence. We then use Pearson correlation to calculate the similarity of every pair of columns (e.g. 1nth and 1 columns) in matrix R, and form the M by M Representation Similarity Matrix (RSM E IR MRMxM, M, in which every element (kd) quantifies the similarity of the responses to the hdh and 1th videos blocks. Thus, the RSMs describe the representation space in a network, be it brain area RSMs, we then use Kendall's 's r between the vectorized RS to quantify the similarity between the two representations. It important to note that measurement noise can potentially induce bias in the RSM estimations, but since the variance of measurement noise is not expected to be different across different video blocks, using Kendall's 7 rank correlation should cancel the bias in our RSM estimations (see [12] for more details). As an additional sanity check, we use RSA to compare the representations between mice. If RSA is identifying salient representational geometries, then the RSMs between different areas should be lower than the RSMs for the same areas. Indeed, as shown in Figure [b, the representational similarity is highest for the same regions across animals (i.e. the diagonal values are larger). These diagonal values also represent the noise ceiling for these areas (see section [. Throughout the paper (with the exception of Figure [p). RSM similarities are reported as the percentage of the noise ceiling.
ANN layer. Given two
Identification of brain regions in the hierarchy The hierarchical organization of mouse visual areas can be inferred based on the anatomical and functional properties of each brain region. We adopted approximation of hierarchical indices as reported in 23 The relative hierarchical placing of the visual areas included in this study are shown in Figure Tc.
Identification of brain regions into ventral and dorsal streams We group the brain regions into two sets, i.e. ventral and dorsal areas. Although the ventral/dorsal specialization of visual pathways in mice is not as clear and well understood as it is in primates, many anatomical and physiological studies do suggest that mice also possess such specialized pathways 40,58 58 39 37). According to previous anatomical and physiological studies [17 58). VISIm and VISam can be considered as the most ventral and dorsal areas of mouse visual cortex, respectively. We then use the similarity of representations between other areas and VISIm and VISam to estimate a ventral score (V-score- = / + +r/h)) and a dorsal score (D-score = A /a/ /ra/ + (r())) for each area. ram is the representational similarity between area and VISam, and r the representational similarity between area A and VISlm. The D-score and V-score values for the five visual areas are plotted in Figure TT. Based on the D-score and V-score values, we grouped VISIm, VISp, and VISpm as more ventral and VISal, and VISam as more dorsal in our analysis. This grouping is in keeping
3

---

(a)
(b)
(c)
(d)
V-score D-score
V15p
0:43
Top
Ventra//doraal scores
0.90
ViSem
0.8
0:43
0.47
0.7
Anatomical hererchy
ViSp
0.6
ViSal
3.43
0:43
0.54
0.39
4.41
16:00
ViSim
0.4
Vi5pm
3.43
0:47
0.39
0.61
0.39
RSM
-0.25
0.2
Vi5am
0.33
0.36
0:41
0.39
vi5p
V15
VI5ai
VIBSo
Bottor
0.0
viSim
ViSp
ViSpm
VISal
ViSam
ventral
dorsal
Figure 1: (a) The schematic of mouse visual cortex. (b) Representational similarity analysis between the visual areas included in our analysis. (c) The anatomical hierarchy score of the visual areas adopted from 23 (d) Ventral and dorsal scores of the visual areas. Areas sorted from the most ventral (VISIm- left) to the most dorsal (VISam right) areas.
with recent study that showed VISam and VISal are the first and second most responsive areas motion stimuli: an important characteristic of dorsal areas [54].
3.3 Model
Self-supervised predictive learning We used Contrastive Predictive Coding (CPC) for self- supervised learning, which developed for modeling sequential data, including video datasets [5]. The loss function relies on predicting the future latent representations of a video sequence, given its present and past representations. See Figure for schematic of the model, and A.2 for more details regarding the CPC loss function.
ANN Architecture All the ANN backbones used in our study are variants of the ResNet architec- ture, similar to the ones used in [B] Our ResNet architectures have either one The one-pathway ResNet (ResNet-Ip) is regular 3D ResNet (Figure 2p). The two-pathway ResNet (ResNet-2p)) is composed of two parallel ResNet branches, which split after single convolutional layer, and merge after their final layers through concatenating their outputs along the channel di- mension (Figure 3). Both pathways of ResNet-2p receive copy of the first convolutional layer output, and each has 10 Res-blocks. To keep the total number of output channels the same in both architectures, each pathway in Re.. For the ANNs trained on object categorization, we use ResNet- 18 pretrained on the ImageNet database 25 Our 3D ResNet architectures are summarized Table ST
pathways.
ResNet-2p has half the number of channels of the single pathway
Baseline models In terms of alignment with brain representations of video sequences, we compare the CPC trained ANNs with four other models: (1) simple model based on Gabor filters, (2) randomized deep ANN, (3) a ResNet-18 trained on ImageNet, and (4) 3D ResNets trained on action recognition in a supervised manner. See section A.3 of supplementary materials for more details.
Training We use the backpropagation algorithm and Adam optimizer. CPC is trained with a batch size of 40 samples, a learning rate of 10-3, and 100 epochs. Supervised action recognition trained with batch size of 256 samples, learning rate of 5 10-4, and 300 epochs. All the models are implemented with PyTorch 1.7 and trained on RTX8000 NVIDIA GPUs.
Downstream tasks In addition to comparisons with dorsal and ventral representations mouse brain, we examine the two pathways of our trained ResNet-2p on two downstream tasks: object categorization and motion discrimination, which are supported in the real brain by the ventral and dorsal pathways, respectively. See section A.5 of supplementary materials for more details.

---

4 Results
4.1 CPC with single pathway architecture produces better matches to mouse ventral stream
We first compare the representations learned with CPC single pathway architecture (Fig. 2 with the three ventral areas of mouse visual cortex (VISIm, VISp, and VISpm; based on D-scores/ V-scores in Figure [t). In Figure 2 (top), the similarity of RSMs between different layers of the ANN and these three areas are shown. Comparing the maximum representational similarities between models in Figure 2. we can see that, for the three ventral areas, CPC shows a higher maximum similarity compared to the other models. Compared to the baseline models (untrained ANN and Gabors), the ANN trained with object categorization has higher representational similarity to VISIm and VISp. the two most ventral areas (see Figure It). which is consistent with the suggested role of ventral areas in form and shape representation [0,17. This in contrast with previous study [5] that could not find any significant difference between ANNs trained on object categorization and untrained ANNs in modeling mouse visual cortex. There are several sources of variability that could explain this contradiction (e.g. architecture, datasets, etc.), but an important possibility is that the two studies used different visual stimuli, namely, natural videos in our study vs static natural images in [5] Natural videos are better suited for eliciting sufficiently strong responses that can distinguish between the representations of the object categorization-trann
and randomly initialized ANNS 9
As noted, and as can be seen in Figure 2p, the maximum similarity happens in different hierarchical levels of the CPC-trained ANN for each area. We quantify the hierarchy index for every area by dividing the layer number with the highest similarity by the maximum number of layers. Based on this measure, the areas at the very top and bottom of the visual hierarchy have a hierarchy index of and O, respectively. The hierarchy index for the three areas are shown in Figure 2H. In accordance with the anatomical and functional data (Figure DD VISp has a lower hierarchical index (0.41 + 0.15) than VISIm (0.61 + 0.08) and VISpm (0.55 + 0.08). It should be noted, that despite the fact that anatomical/functional hierarchy indices position VISpm higher in the hierarchy than VISIm, our model based measure of hierarchy places the two areas at around a similar level, with VISIm having a marginally higher value than VISpm. We speculate that this may be a result of the fact that VISpm has noisier representations, and therefore lower noise ceiling (Fig. [pp. This may be due to VISpm actually being more multi-modes than purely visual.
4.2 CPC on a single pathway does not match mouse dorsal stream
We compare the similarity of RSMs between ResNet-Ip trained with CPC (Fig. 2h) and the dorsal areas (VISam and VISal: based on D-scores/V-scores in Figure TD). As shown in Figure (bottom), the ANN trained with CPC just passes the pixel-level representations in its early layers (the first gray circle in the plots). and then the similarity values continuously decrease for deeper layers. Notably, for the most dorsal area (VISam), the maximum similarity of RSMs does not go above the untrained model (Figure D For VISal, even though CPC shows some improvement compared to an untrained ANN, the performance is much lower than for the ventral areas, as shown in the previous section (Fig. 2... These findings show that the representations learned by CPC and the ResNet- Ip architecture are more ventral-like, and do not easily explain dorsal area representations (see supplementary section
Similarly, an object categorization trained ANN has a very low similarity with the dorsal areas (even lower than the untrained ANN for VISam) indicating that object categorization cannot be considered as an appropriate loss function for the dorsal pathway, which is consistent with our current understanding in neuroscience.
4.3
An architecture with parallel pathways trained with CPC can model both ventral and dorsal areas
CPC's tendency to learn ventral-like representations, as seen in the previous section, could be due to a limitation of the backbone architecture. Predicting the next frame of a video with contrastive loss requires learning invariances to every kind of transformation (or augmentation) that could happen from one frame to the next. For example, representing objects' shapes requires invariance to objects' motion, and representing objects' motion requires invariance to objects' shape. As suggested in ()) these two types of invariances often underlie the distinction between the dorsal and ventral representations in the brain. Therefore, one possibility is that ventral and dorsal-lik, like representations
5

---

(a)
predie:
(b)
(ventral)
visim
V1Sp
vISpm
RON snttary
0.5
0.30
640
0.50
(dorsal)
VISal
ViSam
input
Layers
6:32
0.9
6:0
Input
Leyers
t0p
input
Leyers
top
(c)
ViSim
V1Sp
v1Spm
ViSal
ViSam
(d)
Max RSM similrity
0.6
0:14
0.6
3.54
144
(noise
0.5
642
0.50
viSp
viSim
viSpm
0.0
3d Gabors
Untr
Object Catoa
SimCun
CPC (hositee up)
Figure 2: Representational Similarity Analysis between all the visual areas and the ANN trained with CPC. (a) The schematic of the ANN architecture with one pathway (ResNet-Ip) used as the backbone of CPC. (b) Representational similarity between all the layers of the ANN with one pathways (trained with CPC) and the ventral (top: VISIm, VISp. VISpm) and the dorsal (bottom: VISal, VISam) areas. (c) The maximum representational similarity values between the ANN and the ventral and dorsal areas. (d) Hierarchy index the ventral areas based on their fit to the ANN. Error bars represent bootstrapped standard deviation.
compete for resources in the network, and ultimately, the ventral-like representations win out, possibly because they provide greater overall reduction in loss. this hypothesis is true, then a network with two separate pathways may be able to reduce the competition by assigning one pathway to be more ventral-like, and one more dorsal-like.
To test this hypothesis, we use the simplest extension of ResNet- ResNet-2p. a ResNet architecture that composed of two identical, parallel ResNets which split after the first layer and merge after their final layers (Fig. 3a; also section (.3). Each pathway of ResNet-2p has half the number of channels of ResNet-Ip Ip. keeping the total number of channels per layer equal between the two architectures. We choose this ResNet-2p architecture it shares all of the features of ResNet- Ip, but with two separate pathways that could potentially be assigned specialized functions. We then check whether ResNet-2p can learn both ventral- and dorsal-like representations. We compare the representations of all the layers of the two pathways in ResNet-2p (blue and red in the schematic in Fig. 3h) with all the visual areas. The results show that the representations along one of the pathways (the blue pathway in Fig. 3h-b top) are more ventral-like, and the representations along the other pathway (the red pathway Fig- 3h-b, bottom) are more dorsal-like. Therefore, the two pathways together can model both ventral and dorsal areas of mouse visual cortex. Compared to an untrained ANN, both ResNet-Ip and ResNet-2p achieve high RSM similarity values for ventral areas (Figure with ResNet-2p showing a slight decrease in VISIm and VISpm compared to ResNet- Ip. However, unlike ResNet-Ip, Ip, which fails model the dorsal areas, ResNet-2p outperforms ResNet- Ip and the untrained ANN for area VISam by a wide margin (Fig. Be). For VISal, maximum RSM similarity values for ResNet- and ResNet-2p are around the same level (Fig. (3p), though the similarity values of the red pathway representations do not drop much throughout the network as the blue pathway representations do, indicating the general similarity of the red pathway to the dorsal areas. Examination of the RSM similarity between the two pathways shows that the representational geometries are quite different (Fig. S2). Moreover, when we compare the RSMs of the two pathways with those from ResNet- 1p,
see that ResNet- Ip has representations that are better match

---

(8)
predict
(b)
(ventral)
VISim
VISp
ViSpm
(nobbe corrected)
4.5
RSH simiarity
855
(dorsal)
130
VISal
ViSam
Layers
10P
3.3
140
Ingut
Layers
top
input
Layers
top
(c)
VISim
VISp
V1Spm
ViSal
VISam
(d)
RSM similarty
corrected)
077
678
Hleranchy Index
600
urcramed
CPC (ao)
Uncomed
DPC (Pm)
(20)
Untrihed
(1)
Untoired
CPC tip:
OPC pap)
ViSim
viSem
ViSal
ViSam
CPC
CPC
DC
Figure 3: Representational Similarity Analysis between all the visual areas and the ANN trained with CPC. (a) The schematic of the ANN architecture with two pathways (ResNet-2p) used as the backbone of CPC. (b) Representational similarity between all the layers of the ANN with two pathways (trained with CPC) and the ventral (top: VISIm, VISp, VISpm) and the dorsal (bottom: VISal, VISam) areas. (c) The maximum representational similarity values between the ANN and the ventral and dorsal areas. CPC (Ip) and CPC (2p) are ResNet- Ip and ResNet-2p, respectively, both trained with CPC loss (anction.(d) Hierarchy index of the ventral (left; in red) and dorsal (right; in blue) areas based on their fit to the ANN. Error bars represent bootstrapped standard deviation.
to the ventral-like pathway from ResNet-2p (Fig. S3 This supports the idea that, in the single pathway model, there is competition between the two forms of representation that favours ventral- like functions, and which leads to specialized functions in the two-pathway architecture. Using architectures with more than two parallel pathways also does improve representational similarities (Fig. (S)..
The hierarchy index values for the ventral and dorsal areas shown in Figure Bd. The hierarchy index for every area calculated based on the model pathway that aligns best with that area in terms of representation similarity (blue pathway for VISIm, VISp, VISpm, and red pathway for VISam, VISal). The hierarchy indices are shown separately for the ventral and dorsal pathways. Similar to the results with ResNet-1 p for the ventral pathway, VISp has lower hierarchy index (0.25 0) than VISIm (0.31 + 0.05) and VISpm (0.29 + 0.07). For the dorsal pathway, VISam has a larger hierarchy index than VISal (VISam: 0.82 + 0.16 vs. VISal: 0.19 0.15) which is also consistent with the anatomical/function hierarchy index (Fig. [..
In terms of the predictive loss function, the performances of ResNet- Ip and ResNet-2p not significantly different (top-3 accuracy for ResNet-Ip: 94.64 (0.68) and ResNet-2p: 93.472 (0.83)). However, it is worth noting that ResNet-2p has, in total, fewer parameters than ResNet- (ResNet- 435k vs ResNet-2p: 285k). Therefore, considering the lower capacity of ResNet-2p and its similar predictive performance to ResNet- Ip, we can conclude that the inductive bias of parallel architecture could help in predictive processing.
4.4 Supervised learning of action recognition with paraalee-pathways is not sufficient for dorsal match
The inductive bias of an architecture with parallel pathways combined with the spatiotemporal dynamics of the video data could be enough to trigger the emergence of both ventral and dorsal-like representations, as defined in the previous section, regardless of the loss function used. To understand the role of the loss function, trained the same ResNet-Ip and ResNet-2p backbones with
7

---

(a)
(ventra)
VISim
VISp
VISpm
action dessification
RSM similerity (notse corniced)
2.00
0.00
1.30
0.52
0.50
(dorsal)
VISal
VISam
Layers
0.0
E52
1390
input
Layers
Iop
input
Layers
top
(c)
VISIm
V1Sp
VISpm
VISal
VISam
(d)
Max RSM similartty correctec)
Heerarchy index
8
Untroined
AR 29p3
AR c20)
Untol Untoleed
AR (1:33)
Untratned
AR 240
AR 270
Untraned
AR (1p)
AR (20
Untroied
AR (1p)
AR (77)
vi5p
ViSim
ViSpm
VISel
KR
Figure 4: Representational Similarity Analysis between all the visual areas and the ANN trained with supervised action recognition loss function. Representational similarity between all the layers of the ANN with two pathways (trained with action recognition objective) and the ventral-like (top: VISIm, VISp. VISpm) and the dorsal-like (bottom: VISal, VISam) areas. The maximum representational similarity values between the ANN and the ventral and dorsal areas. AR (Ip) and AR (2p) are ResNet- Ip and ResNet-2p, respectively, both trained with action recognition loss function. (c) Hierarchy index of the ventral (left; in red) and dorsal (right; in blue) areas based on their fit to the ANN. Error bars represent bootstrapped standard deviation.
supervised action classification loss on the UCF101 dataset (Fig. 4a). Based on the representation similarity plots in Figure 4p, both blue and red pathways seem to learn similar representations that are more ventral-like (also see Fig. SS2). The maximum RSM similarity values Figure 4 show that both ResNet- 1p and ResNet-2p reach similarity values higher than the untrained ANN for ventral-like areas, but for the dorsal areas, specifically VISam, neither architecture achieves good representational similarity. The low performance of the action classification models here could not be due to the low spatial resolution of the training videos as our comparisons with ANNs pretrianed with higher spatial resolution videos also show similar results (see supplementary section F)
The hierarchy indices calculated using the ANN trained with an action classification objective (Fig. fi), reproduce the CPC results (Fig. 31 and Zd) for ventral-like areas, and roughly match the anatomical/functionaa hierarchy index. However, the action classification model fails predict the hierarchical organization of the dorsal areas, which is to be expected given the model's 's poor alignment with these areas. Overall, these findings demonstrate the importance of the CPC loss function for learning both ventral and dorsal-like representations across the ResNet-2p architecture.
4.5 Functional specialization of the ventral and dorsal-like pathways CPC trained model
The ventral pathway is responsible for object and scene-based tasks (28), while the dorsal pathway responsible for motion-based tasks
48 54]. Based on our knowledge of the functional specialization of the two pathways in the real brain, we run a linear evaluation on the two pathways of
ResNet-2p trained with CPC on two downstream tasks: (1) object categorization (CIFARIO dataset) and (2) motion discrimination with random dot kinematograms (RDKs) (see Fig. 5l... In vision neuroscience, RDKs have been commonly used to characterise motion representation the dorsal pathway 4 We use this stimulus to evaluate the CPC-trained ResNet-2p (the red pathway in Fig. 3 on motion direction discrimination (four directions: up, down, left, right). Figure shows the results for the two pathways, well for untrained ResNet. As expected based on the comparisons with

---

(a)
(b)
CIFAR-10
RDK
(c)
(d)
30
70
ventral-Iike dorsal-like
Accuracy
40-
20
untrained
ventral like
dorsal-llke like
untra iraaieed
ventral mentral-like likkk
dorsal likk
0.2
0.4
0.6
0.8 1.0
10
20
30
Figure Random dots motion discrimination task. (a) Schematics of the RDK task. (b) The performance of the two pathways and an untrained ResNet on the object categorization (CIFAR-10) and motion discrimination (RDK) tasks. The black dashed lines show the chance levels for the two tasks. (c) The accuracy of the dorsal-like and the ventral-like paths for different levels of the random dots coherence. (d) The accuracy of the two pathways for different number of frames of the random dots stimulus.
mouse visual areas, ventral-like pathway is better in object categorization, while the dorsal-like pathway outperforms the ventral-like pathway in motion discrimination. Therefore, in addition to their better fits to neural data, the two pathways are indeed more ventral-like and dorsal-like according to their ability to support downstream tasks.
As previously stated, decreasing the dots coherence (increasing spatial noise) makes the random- dots task more difficult. In real brains, dorsal areas can average out noise and extract motion signals by integrating motion over both time and space [46 60 It is thought that this is key to the importance of dorsal areas for RDK tasks. We measured ResNet-2p ventral-like and dorsal- like pathway performances for different coherence levels of random-dots (Fig. (5t). Ventral-like performance was dramatically reduced when the coherence was lowered from 100% to 50%. In contrast, the decrease dorsal-like pathway performance was much smaller, demonstrating the dorsal-like pathway's ability to spatially integrate motion, which is consistent with our expectations of dorsal areas. Indeed, the dorsal-like pathway can still achieve just under 40% accuracy even with 20% coherence. We also measured ventral-like and dorsal-like pathway performances for different numbers frames of the random-dots stimuli at 50% coherence (Fig. 5D). The dorsal-like pathway's s performance improved significantly when the number of frames increased, illustrating the ability of the dorsal-like pathway to integrate motion information over time. On the other hand, increasing the number of frames did not increase ventral-like pathway accuracy, but rather reduced it around chance level. This shows that unlike the dorsal-like pathway, the ventral-like pathway ResNet-2p does integrate motion information over time.
5 Discussion
In this paper, showed that self- celf-supervised learning with CPC can produce representations that more analogous to mouse visual cortex than either simple models or ANNs trained in a supervised manner. Furthermore, we showed that CPC applied to architecture that has two parallel pathways can model both the ventral and dorsal areas of mouse visual cortex. The downstream tasks of object recognition and motion discrimination also support the ventral-like and dorsal-like representations of the two pathways the model. Our experiments with supervised training on action classification indicated that the two-pathway architecture and video dataset are necessary but sufficient for learning both types of representations, showing an interaction between the self-supervised objective function and the architecture. This finding shows that self-supervised predictive learning required component of our model for obtaining both ventral- and dorsal-like representations. Our observation that supervised action classification cannot generate dorsal representations is in contradiction with previous fMRI study [11] The different results of the two studies boil down to the different data modalities used the two studies: two-photon calcium imaging in our study and fMRI [21]. Capturing the responses of the dorsal areas elicited by natural videos with high temporal dynamics requires neuronal recordings with high temporal sampling rate. Therefore, we hypothesize that some aspects of dorsal representations of movement would not be reflected in the fMRI data, which

---

could bias the fMRI-based analysis toward more static and lower temporal frequency features of the stimulus and neuronal responses (e.g. very slow-varying features; (1]).
Even though our results demonstrated the importance of self-supervised predictive learning for generating ventral- and dorsal-like representations, it is important note that our conclusions limited to the supervised loss functions that we included in our comparisons (i.e. object categorization and action recognition). We acknowledge that a different combination of more ecologically relevant supervised loss functions might be sufficient for learning ventral and dorsal representations in ANNs.
Learning representations of input data (images, videos, etc.) that are invariant certain data augmen- tations (e.g. rotation, cropping, etc.) is a common goal of modern self-supervised learning methods. In models such SimCLR 7 and BYOL 20), the augmentations most appropriate representations for downstream tasks. In CPC, however, the augmentations are inherent to the data being used. As noted above, predicting the next frame in movie requires two different invariances: (1) invariance to motion, but selective for shape, and (2) invariance to shape, but selective for motion. Our results suggest that these two types of invariances are mutually exclusive, which can explain both the need for two separate pathways to get good matches to both ventral and dorsal areas and the inverse relationship between ventral-likeness and dorsal-likeness of the learned representation (see section D and Fig. S4] in supplementary materials). Thus, our results suggest that the functional specialization observed in the mammalian brain may be a natural consequence of a predictive objective applied to an architecture with two distinct pathways.
engineered for learning the
6 Limitations
One limitation of this work is the lack of comparisons with ANNs that are trained with other predictive loss functions, such as PredNet (38. Other supervueee video-based learning models (for example, see (14) that do not optimize predictive loss function could also be compared with CPC in terms of matching the representations of mouse visual cortex. Another limitation concerns the backbone architectures that we used in this study. Different parameters of the architectures (such as the number of residual blocks, the number of layers before the split and after the merger the two pathways in in ResNet-2p, etc.) could be searched more thoroughly to determine the optimal setting for modeling mouse visual cortex. Furthermore, training ResNet-2p with CPC on a synthetic video dataset in which the motion and shape contents of the videos can be controlled could demonstrate more directly that the two pathways of ResNet-2p learn motion selectivity, respectively.
shape selectivity and shape-inver invariin
Broader Impact
ANNs can serve framework for understanding brains [50), demonstrated here. This under- standing would be based on finding the loss functions, architecture, and learning rules that best capture brain representations. Technologies that directly indirectly interface with the brain, such as brain machine interfaces, can benefit from an in silico model of the brain. ANNs, if being used as such models, can facilitate designing and optimizing these technologies. However, the downside is that the ANN models of the brain are prone to the same limitations encountered by other ANNs, such as adversarial attack or implicit bias. These limitations can potentially leak into the applications in which these models would be used with human subjects.
Acknowledgment and Disclosure of Funding
We thank Iris Jianghong Shi, Michael Buice, Stefan Mihalas, Eric Shea-Brown, and Bryan Tripp for helpful discussions. We also thank Pouya Bashivan and Alex Hernandez-Gareia for their suggestions on the manuscript. This work was supported by a NSERC (Discovery Grant: RGPIN-2020-05100 Healthy Brains, Healthy Lives (New
Discovery Accelerator Supplement: RCGPAS-2020-000 Investigator Award: 2b-NISU-8; Innovative Ideas Grant: Ic-II-15) and CIFAR (Canada AI Chair; Learning Machine and Brains Fellowship). CCP was funded by a CIHR grant (MOP-115178). This work was also funded by the Canada First Research Excellence Fund (CFREF Competition 2, 2015-2016) awarded to the Healthy Brains, Healthy Lives initiative McGill University, through the Helmholtz International BigBrain Analytics and Learning Laboratory (HIBALL).
10

---

References
[1] Pouya Bashivan, Kohitij Kar, and James DiCarlo. Neural population control via deep image synthesis. Science, 364(6439), 2019.
[2] Doris Braun, Duane K Boman, and John R Hotson. Anticipatory smooth eye movements and predictive pursuit after unilateral lesions in human brain. Experimental brain research, 110(1):111-16, 1996.
[3] Kenneth Britten, William T Newsome, Michael Shadlen, Simona Celebrini, Anthony Movshon, et al. A relationship between behavioral choice and the visual responses of neurons macaque mt. Visual neuroscience. 13(1):87-100, 1996.
[4] Kenneth Britten, Michael N Shadlen, William Newsome, and Anthony Movshon. The analysis of visual motion: a comparison of neuronal and psychophysical performance. Journal of Neuroscience, 12(12):4745-4765, 1992.
[5] Santiago A Cadema, Fabian Sinz, Taliah Muhammad, Emmanouil Froudarakis, Erick Cobos, Edgar Y Walker, Jake Reimer, Matthias Bethge, Andreas Tolias, and Alexander S Ecker. How well do deep neural networks trained on object recognition characterize the mouse visual system? 2019.
[6] Charles F Cadieu and Bruno A Olshausen. Learning intermediate-level representations of form and motion from natural movies. Neural computation, 24(4):827-866, 2012.
[7] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597-1607. PMLR, 2020.
[8] Andy Clark. Whatever next? predictive brains, situated agents, and the future of cognitive science. Behavioral and brain sciences, 36(3):181-204 2013.
[9] Saskia EJ de Vries, Jerome Lecoq, Michael Buice, Peter A Groblewski, Gabriel Ocker, Michael Oliver, David Feng, Nicholas Cain, Peter Ledochowitsch, Daniel Millman, al. A large-scale standardized physiological survey reveals functional organization of the mouse visual cortex. Nature neuroscience, 23(1):138-151, 2020
[10] Katrien Denys, Wim Vanduffel, Denis Fize, Koen Nelissen, Hendrik Peuskens, David Van Essen, and Guy A Orban. The processing of visual shape in the cerebral cortex of human and nonhuman primates: functional magnetic resonance imaging study. Journal of Neuroxcience, 24(10):2551- 2565, 2004.
[11] Barry Devereux, Alex Clarke, Andreas Marouchos, and Lorraine K Tyler. Representational similarity analysis reveals commonalities and differences in the semantic processing of words 2013.
and objects. Journal of Neuroscience, 18946):890661888
[12] Jöm Diedrichsen, Eva Berlot, Marieke Mur, Heiko H Schütt, Mahdiyar Shahbazi, and Nikolaus Kriegeskorte. Comparing representational geometries using whitened unbiased-distance-marii. similarity. arXiv preprint urXiv:2007.02788) 2020.
[13] Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6202-6211, 2019.
[14] Christoph Feichtenhofer, Haoqi Fan, Bo Xiong, Ross Girshick, and Kaiming He. A large-scale study on unsupervised spatiotemporal representation learning, arXiv preprint arXiv:2104.1455 145588. 2021.
[15] Daniel Felleman and David C Van Essen. Distributed hierarchical processing in the primate cerebral cortex. Cerebral cortex (New York, NY: 199/), 1(1):1-47, 1991.
[16] Aris Fiser, David Mahringer, Hassana K Oyibo, Anders V Petersen, Marcus Leinweber, and Georg Keller. Experience-dependent spatial expectations in mouse visual cortex. Nature neuroscience, (118))658-1164 2016.

---

[17]
Emmanouil Froudarakis, Paul G Fahey, Jacob Reimer, Stelios M Smimakis, Edward J Tehovnik, and Andreas S Tolias. The visual cortex in context. Annual review of vision science, 5:317-339, 2019.
[18] Colleen J Gillon, Jason E Pina, Jéröme A Lecoq, Ruweida Ahmed, Yazan Billeh, Shiella Caldejon, Peter Groblewski, Tim M Henley, Eric Lee, Jennifer Luviano, et al. Learning from unexpected events in the neocortical microcireuit. bioRxiv, 2021.
[19] Melvyn Goodale and A David Milner. Separate visual pathways for perception and action. Trends in neurosciences, 15(1):20-22) 25, 1992.
[20]
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, al. Bootstrap your own latent: A new approach to self-supervised learning. arXiv preprint 20Xiv:2006.07733, 2020.
[21] Umut Güçlü and Marcel AJ van Gerven, Increasingly complex representations of natural movies across the dorsal stream are shared between subjects. Neurolmage, 145:329-336, 2017.
[22] Tengda Han, Weidi Xie, and Andrew Zisserman. Video representation learning by dense predictive coding. In Proceedings of the IEEE/CVE International Conference on Computer Vision Workshops, pages 0-0, 2019.
[23] Julie A Harris, Stefan Mihalas, Karla E Hirokawa, Jennifer D Whitesell, Hannah Choi, Amy Bernard, Phillip Bohn, Shiella Caldejon, Linzy Casal, Andrew Cho, al. Hierarchical organi- zation of cortical and thalamic connectivity. Nature, (75(778)):195-222
2019.
[24] James Haxby, Cheryl L Grady, Barry Horwitz, Leslie G Ungerleider, Mortimer Mishkin, Richard E Carson, Peter Herscovitch, Mark B Schapiro, and Stanley I Rapoport. Dissociation of object and spatial visual processing pathways in human extrastriate cortex. Proceedings of the National Academy of Sciences, 88(5):1621-1625 1991.
[25] Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[26] Irina Higgins, Le Chang, Victoria Langston, Demis Hassabis, Christopher Summerfield, Doris Tsao, and Matthew Botvinick. Unsupervised deep learning identifies semantic disentangiement in single inferotemporal neurons. arXiv preprint arXiv:2006. /4304, 2020.
[27] Andrew D Huberman and Cristopher M Niell. What can mice tell us about how vision works? Trends in neurosciences, 34(9):644443,, 2011.
[28] Thomas W James, Jody Culham, G Keith Humphrey, A David Milner, and Melvyn A Goodale. Ventral occipital lesions impair object recognition but not abject-directed grasping: an fmri study. Brain, 126( (6((1))2463 2425222755522 2003.
[29] Judson P Jones and Larry A Palmer. An evaluation of the two-dimensional gabor filter model of simple receptive fields in cat striate cortex. Journal of neurophysiology. 58(6):1233-1258 1987.
[30] Georg B Keller, Tobias Bonhoeffer. and Mark Hübener. Sensorimotor mismatch signals in primary visual cortex of the behaving mouse. Neuron, 74(5):809-815 2012.
[31] Georg B Keller and Thomas D Mrsic-Flogel. Predictive processing: canonical cortical computation. Neuron, 100(2):424-435. 2018.
[32] Seyed-Mahdi Khaligh-Razan Razavi and Nikolaus Kriegeskorte. Deep supervised, but not un- supervised, models may explain it cortical representation. PLoS computational biology. 10(11):e1003915, 2014.
[33] Tim Kietzmann, Courtney J Spoerer, Lynn KA Sörensen, Radoslaw M Cichy, Olaf Hauk, and Nikolaus Kriegeskorte. Recurrence is required to capture the representational dynamics of the human visual system. Proceedings of the National Academy of Sciences, 116(43)22184--2166,,
2019.
12

---

[34] Talia Konkle and George Alvarez. Deepnets do not need category supervision to predict visual system responses to objects. Journal of Vision, 20(1 20(11):498 498, 2020.
[35] Eileen Kowler, Jason F Rubinstein, Elio M Santos, and Jie Wang. Predictive smooth pursuit eye movements. Anmual review of vision science, 5:223-246, 2019.
[36] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images.
2009.
[37] Marie-Eve Laramée and Denis Boire. Visual cortical areas of the mouse: comparison of parcellation and network structure with primates. Frontiers in neural circuits, 8:149, 2015.
[38] William Lotter, Gabriel Kreiman, and David Cox. Deep predictive coding networks for video prediction and unsupervised learning. arXiv preprint arXiv: 000,0004,,,, 2016.
[39] James H Marshel, Marina E Garrett, Ian Nauhaus, and Edward M Callaway. Functional specialization of seven mouse visual cortical areas. Neuron, 72(6): 14000004,, 2011.
[40] AJ McDonald and F Mascagni. Cortico-cortical and cortico-amygdaloid projections of the rat occipital cortex: a phaseolus vulgaris leucoagglutinin study. Neuroscience, 71(1):37-54, 1996.
[41] Johannes Mehrer, Courtney Spoerer, Nikolaus Kriegeskorte, and Tim C Kietzmann. Individual differences among deep neural network models. Nature communications, 11(1):1-12, 2020.
[42] Mortimer Mishkin, Leslie G Ungerleider, and Kathleen A Macko. Object vision and spatial vision: two cortical pathways. Trends in neurosciences, 6:414-417, 1983.
[43] Thao Nguyen, Maithra Raghu, and Simon Kornblith. Do wide and deep networks learn the same things? uncovering how neural network representations vary with width and depth. arXiv preprint xxXi:22010.13227 2020.
[44] Cristopher M Niell and Massimo Scanziani. How cortical circuits implement cortical computa- tions: Mouse visual cortex as a model. Annual Review of Neuroscience, 44, 2021.
[45] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, arXiv preprint arXiv:1807.0374 2018
[46] Christopher C Pack and Richard T Born. Temporal dynamics of neural solution the aperture problem in visual area mt of macaque brain. Nature,
2001.
[47] Tatiana Pasternak, Kris M Horn, and John HR Maunsell. Deficits in speed discrimination following lesions of the lateral suprasylvian cortex the cat. Visual neuroscience, 3(4):365- 375, 1989.
[48] Tatiana Pasternak and William H Merigan. Motion perception following lesions of the superior temporal sulcus in the monkey. Cerebral Cortex, 4(3):247-259, 1994.
[49] Carlos R Ponce, Will Xiao, Peter F Schade, Till S Hartmann, Gabriel Kreiman, and Margaret S Livingstone. Evolving images for visual neurons using a deep generative network reveals coding principles and neuronal preferences. Cell, 177(4))999-0099 2019.
[50] Blake A Richards, Timothy P Lillierap, Philippe Beaudoin, Yoshua Bengio, Rafal Bogacz, Amelia Christensen, Claudia Clopath, Rui Ponte Costa, Archy de Berker, Surya Ganguli, et al. deep learning framework for neuroscience. Nature neuroscience, 22 22(11):1761-1770 2019.
[51] Reuben Rideaux and Andrew Welchman. But still moves: static image statisties underlie how we see motion. Journal of Neuroscience, 40012):2538-2552, 2020.
[52] Peter H Schiller and Nikos K Logothetis. The color-opponent and broad-band channels of the primate visual system. Trends in newrosciences, 13(10):392-398 1990.
[53] Jianghong Shi, Eric Shea- Brown, and Michael A Buice. Comparison against task driven artificial neural networks reveals functional organization of mouse visual cortex. arXiv preprint 17X1vv191..07986. 2019.
13

---

[54] Kevin K Sit and Michael J Goard. Distributed and retinotopically asymmetric processing of coherent motion mouse visual cortex. Nature communications, 11(1):1-14, 2020.
[55] Darragh Smyth, Ben Willmore, Gary E Baker, Ian D Thompson, and David Tolhurst. The receptive-field organization of simple cells in primary visual cortex of ferrets under natural scene stimulation. Journal of Neuroscience, (3(1))474444759 2003.
[56] Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos the wild. arXiv preprint arXiv: 2012.
[57] Katherine R Storts, Tim C Kietzmann, Alexander Walther, Johannes Mehrer, and Nikolaus Kriegeskorte. Diverse deep neural networks all predict human it well, after training and fitting. bioRxiv, 2020.
[58] Quanxin Wang, Enquan Gao, and Andreas Burkhalter. Gateways of ventral and dorsal streams in mouse visual cortex. Journal of Neuroscience, 31(5):1955-1918, 2011.
[59] Quanxin Wang, Olaf Sporns, and Andreas Burkhalter. Network analysis of corticocortical connections reveals ventral and dorsal processing streams in mouse visual cortex. Journal of Neuroscience, 32(13):4386 4399 2012.
[60] Scott Watamaniuk and Robert Sekuler. Temporal and spatial integration dynamic random- dot stimuli. Vision research, 32(12):2341-2347 1992.
[61] Laurenz Wiskott and Terrence Sejnowski. Slow feature analysis: Unsupervised learning of invariances. Neural computation, 14(4):715-770, 2002.
[62] Daniel LK Yamins, Ha Hong, Charles F Cadieu, Ethan A Solomon, Darren Seibert, and James J DiCarlo. Performance-optiiin hierarchical models predict neural responses in higher visual Proceedings of the national academy of sciences, 11 (11(23)86619886) 8624, 2014.
cortex.
[63] Chengxu Zhuang, Siming Yan, Aran Nayebi, Martin Schrimpf, Michael C Frank, James DiCarlo, and Daniel LK Yamins. Unsupervised neural network models of the ventral visual stream. Proceedings of the National Academy of Sciences, 118(3), 2021.
[64] Josef Zihl, D Von Cramon, and Norbert Mai. Selective disturbance of movement vision after bilateral brain damage. Brain, 106(2)3113-344 1983.
Checklist
The checklist follows the references. Please read the checklist guidelines carefully for information on how to answer these questions. For each question, change the default [TODO] [Yes], [No], or [N/A]. You are strongly encouraged to include justification to your answer, either by referencing appropriate section of your paper or providing a brief inline description. For example:
Did you include the license the code and datasets? [Yes] See Section ??.
Did you include the license to the code and datasets? [No] The code and the data are proprietary.
Did you include the license to the code and datasets? [N/A]
Please do not modify the questions and only the provided macros for your answers. Note that the
Checklist section does not count towards the page limit. In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See section
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
14

---

(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code. data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material or as a URL)? [Yes] See the supplementary materials
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See section section of supplementary materials (c) Did you report error bars (e.g., with respect to the random seed after running exper- iments multiple times)? [Yes] All the values reported in the figures have error bars. The error bars represent the standard deviations estimated with bootstrapping on the neurons.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See section 3.3
4. If you are using existing assets (e.g., code, data, models) or curating/releasig new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See section 3.3
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as URL? [Yes]
We included the codes and the checkpoint files for the pretrained models.
(d) Did you discuss whether and how consent was obtained from people whose data you're
using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals. applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
15


Reviews:
[{"review_id": "dPcNMlQ6ezI", "reviewer": "Reviewer_3Khp", "summary": "Motivated by the literature showing that the primate visual system has two pathways (\"ventral\" and \"dorsal\") with distinct functional specializations, the authors try to build a model of a visual system that also has representationally and functionally distinct regions. They turn to calcium imaging data from mouse visual cortex, which allows them to compare learned representations of ANN to the neural representations found in distinct cortical areas of animals shown the same images. (There aren't publicly available datasets with simultaneous recordings from ventral and dorsal areas in primates, as far as I know.) The authors find that a two-pathway 3D ResNet model trained on the contrastive predictive coding (CPC) task and a dataset of natural videos has layers that overall explain more of the mouse calcium responses in more cortical areas than several baseline models, including the same architecture trained on the same dataset but with a supervised task. In addition, the two pathways in the self-supervised model transfer better to different tasks (object recognition and motion discrimination), which is reminiscent of some of the functional specialization between primate ventral and dorsal pathways.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I like this work overall and think the authors provide enough evidence to support their general conclusion: that it's **possible** for representationally and functionally specialized layers, resembling different regions of visual cortex, to emerge in a single computational model with the right architecture, trained on the right task. I'm not sure about two of their main specific claims, though: (1) that the particular architecture/loss function they identify here is the best model of mouse visual cortex that they could have reasonably identified, and (2) that their results have much to do with the distinction between the primate ventral and dorsal pathways. I'll discuss both of these below, but want to say up front that I think they can be remediated with some more careful model comparisons/controls and a clearer elaboration of the authors' central hypotheses. I'll start with (2) since it's a \"higher level\" response:\n\n_Does the authors' work say something about \"dorsal\" and \"ventral\" visual pathways?_\n\nAs noted in the authors' introduction, ANNs optimized for object recognition (and recently in a self-supervised manner) are the best current models of multiple cortical areas in the primate ventral visual pathway. While there are a smaller number of cases where people have compared task-optimzied ANN representations to the primate dorsal pathway, the overall approach seems sound: these models predict neural activity or fMRI responses better than prior computational models, with some evidence that different tasks better account for dorsal areas than ventral areas and vice versa [A]. It would be nice, then, if there were single ANN models that could account for neural representations across both pathways in a \"physical way\" -- i.e., distinct layers of the single ANN could be convincingly mapped to distinct areas of visual cortex, extending the results of numerous studies for the ventral pathway alone (and primary visual cortex.) \n\nBut there's another problem that seems just as important to tackle, and much closer to what the authors are actually doing here: explaining the neural activity of _rodent_ visual cortex using task-optimized ANNs. To date, success has been mixed: there's some indication that object-recognition optimized deep CNNs do no better than randomly initialized models at predicting mouse visual cortex responses [B], and the top-layer features of such models do not seem to account for rat neural representations as well as early- and mid-layer features [C]. (I'm surprised that at least the first of these papers isn't discussed in the Related Work section.) Since the task-optimized ANN theory of visual representations isn't meant to be primate-specific, it predicts that **some** combination of architecture, task, and dataset should yield good predictions of rodent visual cortex (assuming task optimization methods succeed.) It seems reasonable to explore tasks that are more ethologically relevant to mice than complex invariant object recognition, which may be a primate-specific ability.\n\nAll this is to say: I'm surprised by the framing of this work as mainly addressing the first of these problems, not the second. I don't know that there's good reason to think of the various mouse visual areas as analogous to primate ventral and dorsal pathways (or whether there's evolutionary homology); the couple of references the authors give here might try to justify this categorization, but it's not clearly explained in this paper. In looking at Figure 1, I wouldn't immediately conclude that there are two distinct types of visual area in mice; one might just as easily propose that they all belong to a single hierarchy, with AM near the top and P/LM near the bottom. So, independent of the actual results (on which I'll comment below), I'm worried that the main claim here isn't well aligned with what the paper shows; comparing to the work in [A], for instance, I'm still more inclined to think that primate ventral and dorsal pathways are better explained by adaptation for **different** tasks rather than a single task with a branched (and then merging) architecture.\n\nIf the authors were to present this work more as addressing the second problem above -- actually explaining multiple regions of rodent visual cortex with a single task-optimized model -- I think it could be a clearer contribution to the literature on explaining neural data with task-optimized ANNs. If the authors are very attached to the dorsal/ventral distinction in mice, though, I think they need to justify the connection between mouse vision and (far better studied) primate vision much better than they do here.\n\n_How close is the 2-stream CPC model to the \"correct\" architecture/task for explaining mouse visual cortex?_\n\nBeyond the concern about framing (which I don't think is semantic or trivial -- it makes a big difference for ideas about the evolution of vision in mammals!) I have some questions about the actual experiments that I hope the authors could address in a revision. I think, overall, their results are pretty solid -- and pretty consistent with concurrent work on modeling mouse visual cortex [D]. Arriving at similar conclusions (multiple pathways, non-supervised object recognition tasks) despite many differences in implementation is a reassuring sign.\n\n1. In the authors' initial model comparison (Figure 2), an ImageNet trained model (\"Object Categ\") did better than the untrained model at explaining responses in several the visual areas. This is a different result and conclusion than the one in [B]. Could the authors comment on what might have led to this difference?\n\n2. The set of baselines compared to the 1- and 2-stream CPC models is pretty limited -- as the authors themselves mention in the \"Limitations\" section. I think the paper would be much stronger if there were a broader exploration of architectures trained on the same loss function and/or a broader set of loss functions tested for the same architectures, with the dataset held constant. For instance, the 2-stream model does seem better than the 1-stream model in that it has a better match to the AM representation in its higher layers, but (as per the discussion above) it's not at all clear from anatomy or prior work that 2 is the \"correct\" number of pathways here. Why not try a 3-, 4-, etc.-stream model? Or a PredNet-like model that makes predictions in pixel space rather than latent space? I appreciate that there are resource contraints here, but I don't think we can draw very strong conclusions about which architectures and/or losses are really the best match for mouse visual cortex until there's somewhat broader sampling of model space. (Especially if the authors are attached to the dorsal-ventral distinction, it would be good to see that 2 is the \"correct\" number of pathways.)\n\n3. The object categorization model, if it's a pretrained ImageNet model, was likely trained at a different image resolution than the CPC models (unless I missed this somewhere.) It's also not the closest comparison to the CPC models that the authors could make. What about a self-supervised 2-stream CNN trained on a static contrastive loss, on the same dataset and input resolution? If that model also failed to predict RSMs in the \"dorsal\" areas, it would make for a much stronger argument that dynamic prediction is a key part of the ethologically relevant task.\n\n4. It's very interesting that the two streams learn such different representations (judging by Figures 3 and 5) without any asymmetry between the two pathways' architectures other than different initial weights (though it's reminiscent of the original AlexNet.) It would be nice to get a qualitative flavor of how they differ. Do they learn very different convolutional filters at early layers or later layers? Are they really selective for shape versus motion as the authors suggest (the \"dorsal\" pathway is only a little worse at the CIFAR-10 task.) Anything to better understand how the symmetry is \"broken\" would be interesting to see here. Is it possible to bias one pathway to \"become\" the more dorsal one by changing the architecture slightly? \n\n5. The authors use CPC as a proxy task for learning representations, but also try to argue (correctly, I think) that predicting the future of a visual scene is an ethologically important task. But we don't actually see in the paper whether the models are any good at this prediction problem, or whether the 2-stream architecture is better (gets lower loss) than the 1-stream architecture. If prediction is important and this one task/loss is supposed to explain neural processes across multiple visual areas, I would expect the authors to think that the 2-stream architecture is actually better at prediction than the 1-stream architecture! Otherwise, it doesn't seem like the 2-stream + CPC task provides an explanation of the structure of mouse visual cortex in the way that the authors want it to. So I'm eager to see whether this is the case.\n\n6. Likewise, how does the 1-stream architecture do on transferring to the two tasks in Figure 5? \n\n[A] Dwivedi, Kshitij, et al. \"Unveiling functions of the visual cortex using task-specific deep neural networks.\" bioRxiv (2021): 2020-11.\n\n[B] Cadena, Santiago A., et al. \"How well do deep neural networks trained on object recognition characterize the mouse visual system?.\" (2019).\n\n[C] Vinken, Kasper, and Hans Op de Beeck. \"Using deep neural networks to evaluate object vision tasks in rats.\" PLoS computational biology 17.3 (2021): e1008714.\n\n[D] Nayebi, Aran, et al. \"Unsupervised Models of Mouse Visual Cortex.\" bioRxiv (2021).", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I think what the authors wrote is fine, though to me the \"Limitations\" section reads more like \"Here's what we would have done if we had more time\" rather than \"Here are the questions that our experiments weren't set up to address and are beyond the scope of the work.\" I think most of the things they suggest in the Limitations section are good ideas, and don't seem prohibitively expensive to try! I don't think I can reasonably expect them to try all of them in the week-long rebuttal period, but including at least some of them in a revision would make the paper much stronger.", "review_text": "", "extra_scores": {"ethical_concerns": "No, I have no concerns.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "T05Oqcl5IlR", "reviewer": "Reviewer_qYdj", "summary": "The paper uses contrastive predictive coding to train deep networks on UCF101 videos. It then uses representational similarity to compare their representations to those of mouse calcium-imaging data from the Allen Brain Observatory. The key result is that a two-stream network is a good match to mouse visual cortex data. One of the (structurally identical) streams develops a representation that is a good match to dorsal mouse areas, and the other matches ventral mouse areas. Deeper areas correspond better to areas higher in the mouse visual hierarchy. Furthermore, the ventral stream of the model better supports transfer to CIFAR-10, while the dorsal stream better supports discrimination of random-dot motion direction. ", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: \nThere have been a few papers recently that compared deep-network representations to Allen Brain Observatory data. A recent one (Nayebi, A., Kong, N. C., Zhuang, C., Gardner, J. L., Norcia, A. M., & Yamins, D. L. (2021). Unsupervised Models of Mouse Visual Cortex. bioRxiv.) also uses contrastive objectives and considers networks with multiple streams. This paper and that one agree that contrastive objectives work well for modelling mouse data, but there is surprisingly little overlap otherwise. This paper focuses mainly on the issue of dorsal/ventral stream differentiation with a single cost function, while Nayebi et al. didn’t specifically address that point (although their StreamNets may have exhibited this differentiation). \n\nQuality: \nThe technical approach is sound and the analysis is well-informed and appropriately thorough. In a couple of cases the description of the results seems more categorical than the results themselves. For example, line 223 says “unlike ResNet-1p, which fails to model the dorsal areas”. But ResNet-1p isn’t a total failure at the dorsal stream. It’s max RSM for VISal is even a little higher than that of ResNet-2p. \n\nClarity: \nThe paper is clear and well organized. The authors shared their code in case any details were missed.  \n\nSignificance: \nIt is not too surprising that a network with two streams would develop some differentiation (Krizhevsky et al.’s version of AlexNet did this). However, it is significant that an unsupervised two-stream model with no other prodding would develop representations that are convincingly dorsal and ventral. The paper also contributes substantially to the understanding of what may lead to mouse visual representations, specifically.\n\nMinor: \nThe paper was very clean. I only have a few suggestions: \n-\tRemind the reader in the caption of Figure 3 that red is dorsal. \n-\tSwitch red and blue in Figure S4, to match the convention in the main paper. \n-\tGive parameters of the Gabor control model\n-\tUse vector graphics for the figures where possible. The file size is 10x that of the other papers I’m reviewing, and it doesn't scroll smoothly. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The work has a minor societal impact. The impact and limitations were thoughtfully addressed in the last section of the paper. ", "review_text": "", "extra_scores": {"ethical_concerns": "The work does not raise ethical concerns.  ", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours"}, "extra_text": {}}, {"review_id": "Hwa8MMI-QmX", "reviewer": "Reviewer_oMSB", "summary": "The authors show that a two-stream 3d ResNet architecture trained using a contrastive predictive coding (CPC) objective develops a representation that resembles the ventral/dorsal stream split in visual cortex. Although I do not buy into all claims of the paper, I am generally very supportive of it. The authors make their main point and the paper is well written. With some adjustments of the claims it will be a great paper.\n\n\n### Update after discussion period\n\nGiven the results from the control experiments reported by the authors, I support publication of the paper and increased my score to 7. I do ask the authors to include those results in the final version of the paper.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strengths\n+ Simple but innovative idea\n+ Demonstrates a single network architecture and objective function can generate a ventral/dorsal split as in visual cortex\n+ Well written and easy to read\n\n## Weaknesses\n- Unclear if claimed superiority of CPC over supervised pre-training holds\n- Unclear how robust the emergence of two streams is\n\n\n## Detailed comments on weaknesses\n\n### 1. Comparison to supervised pre-training\n\nUsing the CPC objective for training, the authors demonstrate nicely that the two-stream architecture results in more dorsal-like representations in its second stream than the one-stream architecture. However, I am concerned about their claim that the CPC objective is necessary.\n\n- The supervised baseline might be problematic due to the low resolution of the videos. What can be recognized at 64x64 pixels? What's the performance of the classifier on the task it was trained on? If the task cannot really be solved, I'd be worried that the network simply doesn't learn useful representations in the first place, because the semantic labels are not much related to the image information present at the low resolution. Following the authors' reasoning, supervised pre-training should perform well on the ventral stream areas. However, predictive performance of the supervised model is quite weak for all brain areas. This result seems to suggest that comparison to the supervised model is not very meaningful. Wouldn't it make more sense to compare to a well-trained action classification net that uses higher-resolution movies and take care of upsampling the videos shown to the mouse accordingly?\n\n- (Related to the previous point.) Previous work [20] has demonstrated dorsal-like hierarchical representations in a 3d CNN trained on UCF101 (Tran et al. 2014). The authors cite that paper in their related work section, but do not further comment on it. Given that their claim that supervised training is not sufficient goes directly against published work, I would have expected at least some kind of discussion/explanation, and I think the burden of proof for such a claim is on the authors.\n\n- There is an existing two-stream video recognition architecture (SlowFast networks: https://arxiv.org/abs/1812.03982), which the authors don't seem to be aware of (at least it's not cited in the related work). Given that it's publicly available, trained in a supervised fashion and features two streams, I think it should be included in the comparisons in Figs. 2–4.\n\nIn summary, if the authors want to make the point that the CPC objective is necessary and supervised training does not work, they need to provide substantially more evidence for this claim. An alternative solution could be to remove this claim and focus on making the point that CPC + two-stream architecture are a sufficient condition for achieving a ventral/dorsal split, but not claim anything about necessity.\n\n\n### Robustness of the emergence of two streams\n\nAs I understand the paper, the authors trained exactly one instance of the two-stream architecture using CPC and are basing their claims on this single instance. I wonder how robust this result is, though. Do these two streams always emerge or were the authors \"lucky\"? It would be good to have at least a second, independent verification with a different seed – and, perhaps, slightly different hyperparameters – to verify that the finding is at least somewhat robust.\n\n\n### Minor comments\n\n- How was the input resolution for the ANNs determined? Depending on the resolution one picks, different layers may be optimal simply because of the receptive field sizes of neurons. It would be good to clarify this point.\n\n- Fig. 2+3: What do the error bars depict?\n\n- Fig. 2–4: Why are the gray points not identical? I thought they depict the RSM similarity of the  input (pixels) with the respective brain area? Those shouldn't depend on the model, should they? The same goes for the untrained model in Fig. 3+4(c): Shouldn't the two also be identical? After all, it's the same architecture.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "5cNzh-tCLU", "reviewer": "Reviewer_4dC4", "summary": "The authors address the challenge of training a single deep neural network, using a single loss function, which captures the specialized neural responses in both the ventral (“what”) and dorsal (“where”) pathways of mouse visual cortex. The authors demonstrate that a deep neural network comprised of two parallel convolutional hierarchies, trained with a single self-supervised predictive objective, captures neural responses in the ventral and dorsal streams better than a neural network comprised of a single convolutional hierarchy (which captures only the ventral stream), and better than either architecture trained with a supervised objective. Furthermore, by evaluating their two-pathway model on “what” and “where” downstream tasks —object categorization and motion detection— the authors show that their model has learned functionally specialized representations in each of its two pathways.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "That the architectural choice to split a neural network into two parallel pathways encourages functionally specialized representations, even when both pathways are trained on the same objective, is on its own a remarkable and intriguing observation (although the authors do not appear to be the first to make this observation; see [1] for examples where branched architectures e.g. Alexnet exhibit specialization in their branches e.g. Gabor filters in one branch vs color detectors in another). This observation could inspire future works on the theory of deep learning, help to understand how representations are learned in neural networks, and aid in model interpretability. And it would be a resounding success if a simple anatomical constraint like this could boost the performance on specific tasks or encourage representations which better fit the brain (the authors demonstrate the latter under their representational similarity metric RSM).\n\nHowever, I have one lingering question after reading the paper: has the 2-pathway architectural choice encouraged *new* functional specialization, or has it simply isolated functional specialization that already existed in the 1-pathway network? I would have liked to see a couple more analyses to investigate this possibility. (1) I would have liked to see the performance of the 1-pathway network on the two downstream tasks. Does the 1-pathway network already contain the specialization in its representations necessary to perform both downstream tasks successfully? Or does the 2-pathway constraint actually encourage new specialization that allows it to perform better on the downstream tasks? (2) I would have liked to see at least one other metric for comparing representations in the trained model to those in the brain. While the fact that the best layer in the 1-pathway network barely outperforms raw pixels at capturing the dorsal stream under the RSM is pretty convincing, naively I can imagine a scenario where the 1-pathway network contains both ventral-like and dorsal-like units, but the variance due to the ventral-like units swamps the variance due to the dorsal-like units, and hence the ventral-like features dominate the RSM as computed by Pearson correlation (whereas the 2-pathway network overcomes this by sorting the units into distinct subpopulations). Perhaps the authors could try another metric --such as regressing the target neural responses with a small number of source units, or under an L1 penalty, or CCA or something similar-- to determine whether dorsal-like features are present in the 1-pathway network (all analyses that have been used to compare models and brains in the past). (3) Finally, is there a clustering that could be performed, either on neural activity or connectivity, by which units in the 1-pathway network could be sorted into two functionally specialized subsets -- one which captures ventral-like features, and one which captures dorsal-like features?\n\nThe paper includes several other interesting insights, including recapitulating (to some extent) the anatomical hierarchy of mouse visual cortex, and indicating an apparent tradeoff between ventral-like specialization and dorsal-like specialization. Overall I think this work is a very interesting contribution, the analyses are well-performed, and the manuscript is clearly written.\n\n[1] Voss, et al., \"Branch Specialization\", Distill, 2021.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors more than adequately address the limitations and potential societal impacts of their work, and suggest a few interesting directions for future work. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This is a clear accept.  All the reviewers were clearly positive, and I largely agree with their judgements.   I think the reviews speak for themselves, and I don't have much to add here. 
Decision: Accept (Spotlight)