A mechanistic multi-area recurrent network model of decision-making
Authors: Michael Kleinman
Chandramouli Chandrasekaran
Jonathan Kao
Venue: NeurIPS 2021 Poster (2021)
Keywords: Computational Neuroscience
Recurrent Neural Network
Interpretability
Decision-Making
Multi-Area Computation

Abstract: Recurrent neural networks (RNNs) trained on neuroscience-based tasks have been widely used as models for cortical areas performing analogous tasks. However, very few tasks involve a single cortical area, and instead require the coordination of multiple brain areas. Despite the importance of multi-area computation, there is a limited understanding of the principles underlying such computation. We propose to use multi-area RNNs with neuroscience-inspired architecture constraints to derive key features of multi-area computation. In particular, we show that incorporating multiple areas and Dale's Law is critical for biasing the networks to learn biologically plausible solutions. Additionally, we leverage the full observability of the RNNs to show that output-relevant information is preferentially propagated between areas. These results suggest that cortex uses modular computation to generate minimal sufficient representations of task information. More broadly, our results suggest that constrained multi-area RNNs can produce experimentally testable hypotheses for computations that occur within and across multiple brain areas, enabling new insights into distributed computation in neural systems.

A mechanistic multi-area recurrent network model of decision-making
Michael Kleinman' Chandramouli Chandrasekaran"* Jonathan C. Kao
UUniversity of California, Los Angeles Boston University
michael. wolkkeeinnne www.eeemennlla.....mm.. cchandr1@bu. edu kaoßseas ecla.edu
Abstract
Recurrent neural networks (RNNs) trained on neuroscience- tasks have been widely used as models for cortical areas performing analogous tasks. However, very few tasks involve single cortical area, and instead require the coordination multiple brain areas. Despite the importance of multi-area computation, there is limited understanding of the principles underlying such computation. We propose use multi-area RNNs with neuroscience-inspiree architecture constraints derive key features of multi-area computation. In particular, we show that incorpo- rating multiple areas and Dale's Law is critical for biasing the networks to lear biologically plausible solutions. Additionally, we leverage the full observability of the RNNs show that output-relevant information is preferentially propagated between areas. These results suggest that cortex uses modular computation to generate minimal sufficient representations of task information. More broadly, our results suggest that constrained multi-area RNNs can produce experimentally testable hypotheses for computations that occur within and across multiple brain areas, enabling new insights into distributed computation in neural systems.
1 Introduction
Decision Decision-making, multisensory integration, attention, motor control, and timing emerge from the coordination of multiple interconnected brain areas [1H)). While neural activity particular can contain behaviorally relevant signals, such as choices or percepts, it is often unclear if these signals originate within the neural bases these behaviors necessitates an understanding of the intra- and inter-area dynamics, that is, how neural activity evolves within and between brain areas. However, hypotheses for how distinct brain-area dynamics and connectivity relate to computation. To address this gap, we use multi-area recurrent neural networks (RNNs) model, probe, and gain insight into how behaviorally relevant computations emerge within and across multiple brain areas.
inherited from upstream brain areas (6]. Understanding the
currently lack clear
Optimized feedforward and recurrent neural networks have been used for machine learning but also emerging tools to model computations associated with visual [T. IЛ, cognitive [22] timing task inputs into behavior-related outputs through recurrent dynamics. Its artificial units often exhibit heterogeneous responses and population dynamics observed in brain and motor tasks [5] (77-20) If artificial units resemble cortical neurons, RNNs are subsequently
[ navigation
14 and motor tasks [5 16] RNNs transform experimenter-desigeee
implicated in cognitive
analyzed propose hypotheses for how local computation occurs in brain area 12 18 21. In comparison to direct experimental recordings, an important advantage of RNNs is that the activity
Joint senior author
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Inputs
Multi-area RNN
Outputs
Rlght larg
Decsioo vartable
Left barg
Terg
Check
Slim
Check Targ Stim
on on
off
Figure 1: Task. RNN configuration. The RNN receives 4 inputs. Two inputs indicate the identity of the left and right targets, which can be red or green. These inputs are noiseless. The other two inputs indicate the value of the signed color coherence (proportional and negative signed color coherence (proportional to amount of green in checkerboard), We added independent Gaussian noise to these signals (see Appendix A.2 The network outputs two analog decision variables, each of which indicates evidence towards the right target (solid line) or left target
amount of red in checkerboard)
(dashed line). A decision is made the direction of whichever decision variable passes a preset threshold (0.6) first. The time at which the input passes the threshold is defined to be the reaction time.
of all artificial units and their recurrent connectivity fully observed. It is therefore possible to engineer [22] and reverse engineer [23 RNNs by analyzing their activity and recurrent connectivity, providing mechanistic insight into cortical computation
211
Traditionally, RNNs have provided insight into local computations, and there has been limited insight into multi-area computation 7 19]. To study multi-area computation, we explicitly constrained RNNs to have multiple recurrent areas, which we refer to as multi-area RNNs. We used these multi-area RNNs to study decision- mrriiinn-maknn,. cognitive process known to involve multiple areas including the prefrontal, parietal, and premotor cortex 7 22 28 Multi-area RNNs enable us to investigate several questions. Most broadly, what are the roles of within-ares dynamics and inter-area connections in mediating distributed computations? How does the dimensionality and dynamics of neural computation differ across areas? What role do inter-area feedforward and feedback connections play in propagating information and rejecting noise? How do intra-area dynamics and inter-area connections coordinate solve task?
We use multi-area RNNs to study these questions a decision-making task where premotor cortex and upstream areas are known to perform distinct computations. We trained multi-area RNNs to perform perceptual
task (Checkerboard Task) and compared their activity to monkey neuron recordings from the dorsal premotor cortex (PMd). We found that, when incorporating Dale's 's law and maatomically-infomme emerged in multi-area RNNs. Specifically, the multi-area RNN's output area (1) resembled PMd in single unit statistics and neural population activity, and (2) only retained the "output relevant" signals. Inter-area connections preferentially propagated these output relevant signals while attenuating output irrelevant signals. Our models and analyses provide a framework for studying distributed computations involving multiple areas in neural systems.
levels of feedforward inhibition into training, PMd-resembling dynamics
2
Motivation: Decision-making involves multiple brain areas
2.1 Checkerboard Task
shown in Fig. monkey was first shown left and right targets whose color (red and green) was random on each trial. The monkey was subsequently shown a central static checkerboard composed of red and green squares. The monkey was trained to discriminate the dominant color of the static checkerboard and reach to the target matching the dominant color. Since the target colors were random each trial, this task separates the reach direction decision from the color decision 2 This task enables studying how information related to the selection of the color of the target and information related to the direction of the reach is represented.
In the "Checkerboard Task"
24
26
2.2 PMd Data during Checkerboard Task
We analyzed the activity of neurons from the dorsal premotor cortex (PMd), an area associated with somatomotor decisions, in monkeys performing the Checkerboard Task (30] Neural activity

---

PMd data
PCs
dPCA and decoding C
PSTHs
Choice probability
Varience:
Varannc wwwaoooaageo
80.25
direction
Targels
Check on
0.75
Right
1.0
0.5
PC₂
000
0
PC,
Siyned coherance
0.0
Check on
200m
RNN data
Area CCA 0.38
Area 2
Area 3
CCA 0.55
CCA 0.73
Frecton dPAC variance explained
0.5
color
context
Fight green
0.4
0.3
Loft green
0.2
PC2
0.1
PC
77.%%
Left x Right
99.2%
94.2%
h
Area
Area
Area
g
1.0
0.9
Decode
0.7
0.8
Area
Area 2
Area
Direclion CP
0.5
Check
Chick on
Color CP
0.6
Figure 2: PMd-resembling dynamics emerge in neuroscience constrained RNNs. (a) PMd neural trajectories the top 3 PCs. Color reflects signed color coherence, with darker shades red (green) indicating more red (green) checkerboards. Right (left) reaches are denoted by solid (dotted) lines. (b) (Top) Variance captured by dPCA axes for the color decision, target configuration (context), and direction decision. (Bottom) Decode accuracy of the direction decision, color decision, and context in PMd sessions with U-probes and multiple neurons. (c) Representative PMd PSTHs aligned to checkerboard onset. (d) Direction and color choice probability (CP) for all recorded PMd units. (e) Neural trajectories in the top principal components for each RNN area. (f) Variance captured by dPCA axes for color, context, and direction. (g) Non-linear tSNE embedding of peri-movement activity in each area. Each dot is trial, with red or green denoting the color decision and '.' or 'x' denoting the direction decision. Unlike Areas and 2, Area only had two clusters separated based on the direction decision.
(h) Decode accuracy of direction, color, and context in all three areas. (i) Example PSTHs in each area. (j) Choice probabilities for units in all areas (pooled over 8 RNNs).
in PMd principally reflects the direction decision (left or right) and has minimal representations associated with the dominant color of the checkerboard (red or green) [26 30 31. To summarize this phenomenon, we show the principal components (PCs) of the PMd neural population activity
in Fig. 2b. These PC trajectories separate based the eventual reach direction (right reaches in
solid, left in dotted), but not the color (red and green largely overlapping). We identified principal axes via demixed PCA (dPCA (2)) that maximized variance related to the target configuration

---

(context), color decision, and direction decision (see Appendix B.4). The direction axes captured significant variance (26.7%) while the color and context axes captured minimal variance (0.7%, 0.5%. respectively), as shown in Fig. 2 It is possible, however, that there is direction-dependen color variance that is averaged away during marginalization when computing the dPCA variance (2) Given simultaneously recorded data, more appropriate measure of representation is the decode accuracy of direction, color and context. Across sessions where we analyzed multiple simultaneously recorded units from U-probes, the direction decision could be decoded from PMd activity significantly above chance (accuracy: 0.89, < 0.01, bootstrap), but the color decision and context decode accuracy were not significantly above chance in any session (overall accuracies: 0.52 and 0.52, respectively, Fig. Dp bottom).
Single neurons also had minimal color separation in individual PSTHs (c.g., Fig. 2.. To summarize this effect in single neurons, we computed the choice probabilities (CPs) reflecting how well the direction decision (direction CP) and color decision (color CP) could be decoded. PMd units generally had near chance color CP (0.5), but moderate to high direction CP, as shown in Fig. 2t. Together, these results demonstrate that PMd largely represents direction-related signals, but not the color decision or target configuration context. Since PMd activity minimally represents the color of the checkerboard or the target configuration, we reasoned that checkerboard and target inputs are transformed into direction signal upstream of PMd and that multiple brain areas are necessary for solving this task. Brain areas, including the dorsolateral prefrontal cortex (DLPFC), and the ventrolateral prefrontal cortex (VLPFC), have been implicated in related sensorimotor transformations
33 35
3 Multi-Area RNN Training Details
We trained RNNs of the form
x(( (() + Wl) + Win + brsc t
(1)
where r(t) = relu(x(t)). is a time-constant of the network, W E NNN defines how the artificial neurons are recurrently connected, bree R defines constant bias, Win Wn E g maps the RNN's ' inputs onto each artificial neuron, and is the recurrent noise. The output of the network is given by linear readout of the network rates, i.e.,
(t) W ott)
where W... out E NN maps the network rates onto the network outputs. For 3-area RNN, W ren: is defined through the following block matrix
W
W21
0
W rec
W12
12
W22
22
W
32
0
W23
W
where Wa; refer to the recurrent connections of area i, and feedforward connections, and Wi.-1 refer to the feedback connections. Feedforward and feedback connections were only allowed between adjacent areas. Task inputs were defined to project onto the first area, and outputs were read out from the final area. In the rest of the text, we primarily focus on 3-area RNN that had approximately 10% feedforward and 5% feedback connections between areas, based on projections between prefrontal and premotor cortex in a macaque atlas [36] The network also constrained to follow Dale's law, in Song et al. (37) The RNN processed the target context and checkerboard inputs to output decision variables reflecting accumulated evidence for left and right decision (Fig. D. Further details are discussed in Appendix A.2
the convention that W Wi,41 refer to
Results
Because the Checkerboard Task involves multiple brain areas, we reasoned that a single-area RNN would not resemble PMd recordings. We first trained traditional single-area RNNs to perform the Checkerboard Task. We found that these RNN representations mixed color and direction information, as summarized in Appendix Fig- g and therefore did not resemble PMd activity. This led us to study multi-area RNN models performing the Checkerboard Task, which turn out to accurately model PMd activity.

---

d
مخه
000000
Drection soccuraa raoo
Color accaracy
0.76
0.75
0.5
101
50
100
30
100
10
20 60
% to
E
% foedback
areas
(10)
A <(x10-))
Figure 3: PMd-resembling dynamics emerge in neuroscience constrained RNNs. (a) We trained 3-area RNNs without explicit excitatory (E) or inhibitory (I) neurons. Inputs projected onto Area I, and outputs were read out from Area 3. We varied the percentage of feedforward connections and computed the color and direction accuracy in Area 3. At 1% feedforward connections, color could still be significantly decoded above chance. Dots are the mean across networks and error bars are s.c.m. For significance, * is 0.05, ** is p 0.01, and $** is p < 0.001 (with appropriate correction for multiple comparisons). We incorporated Dale's law with 80% E, 20% neurons into subsequent sweeps, (b) We varied the percentage of feedforward E-to-I connections. Minimal representations with chance color decode accuracy emerged when the percentage of feedforward E to connections was 2% less (feedforward
was fixed 10%). (c-d) Color information was relatively robust to feedforward E-E connections and feedback connections. (e) At least areas were required for the RNN's last area resemble PMd dynamics. (f) 3-area RNNs with neurophysiological constraints had minimal representations that were generally robust to machine learning hyperparameters. The only exceptions were when the number of units was relatively small, or the learning rate was relatively large.
4.1 PMd-like representations emerge in optimized multi-area RNNs with neuroscience constraints
Given the anatomical and physiological evidence suggesting that multiple brain areas are implicated in the CB task, hypothesized that the last area of an optimized multi-area RNN would more closely resemble PMd, receiving transformed direction signals computed using the checkerboard coherence and target configuration from upstream areas. We trained multi-area RNNs perform the Checkerboard Task as described in Section
The 3-area RNN had qualitatively different population trajectories across areas, shown in Fig. 2 Area 1 had four distinet trajectory motifs corresponding to the four possible task outcomes (combinations of left vs right and red vs green decisions). PC₁ primarily varied with direction, while PC2 varied with both the target context and red versus green checkerboards. In contrast, Area 2 and Area population trajectories primarily separated on direction, not color, like in PMd. Area 3 trajectories most strongly resembled PMd trajectories (canonical correlations, r = 0.38, 0.55, 0.73 for Areas 1, 2, and 3; see Appendix B.7
We quantified the variance captured by dPCA principal axes for the context, color, and direction axis. We found that color axis variance decreased in later areas (Area 1: 5.6%, Area 2: 0. 0.13%, Area 3: 0.07%, Fig. 2h. In contrast. Area had the largest direction axis variance (Area 1: 30.9%. Area 18.2%, Area 3: 48.5%, Fig. 2). An important assumption of dPCA is that the neural activity can be decomposed as a sum of terms that depend solely on particular task variables (38. The color variance found by dPCA indicate that color, on its own, did not account for large fraction of the overall neural variance. However, it is possible there significant color variance within a reach direction that dPCA, lincar dimensionality reduction technique, does not capture.
As we are interested in whether the color information is contained in the representation, a more appealing measure is decode accuracy. If the color of the target can be decoded from the representation of neural activity, then color information present in the representation. We performed nonlinear dimensionality reduction via t-distributed stochastic neighbor embedding (tSNE) (39 shown in Fig. D These results suggest that Areas and 2 contain color information, but Area does not (color decisions overlap). We decoded the color decision and context (target configuration) from RNN units in each area (Fig. 2h, Area 1, 2, and color accuracy: 0.93, 0.76, 0.51, and Area 1, 2, and 3 context accuracy: 0.99. 0.87, 0.54). Area I and 2 had above chance context and color decode
5

---

accuracies (p < 0.01/9, 1-tailed t-test with Bonferroni correction), while Area 3 color and context decode accuracies were near chance, and most similar to PMd (Fig. Dh. color: p 0.05, context:
p 0.024). The direction decision could be decoded significantly above chance in all areas (Fig. 2h, p < 0.01/9) We also observed that Area unit PSTHs more closely resembled PMd neuron PSTHs (e.g., Fig. 2), and color CP progressively decreased in later areas (Fig. 2 Area 3, like PMd, had many neurons with moderate to high direction CP, but low color CP.
We tested how robust these results were to architecture and hyperparameter selection In particular, we quantified how well color could be decoded in the multi-area RNN's last area across several hyperparameter settings. We found that architecture impacted whether optimized multi-area RNNs had PMd-like minimal representations. In particular, we found that PMd representations emerged when we incorporated anatomical and neurophysiological constraints: Dale's 's law, empirical levels of feedforward inhibition, and at least 3 areas (Fig. Bh-e). When we varied machine learning hyperparameters, we found that our results were generally robust: multi-area RNNs had PMd-like representations in their last area over a wide range of hyperparameter settings (Fig. ET Together, this constellation of results shows that Area 3 of the multi-area RNN recapitulates key features of PMd activity, making this RNN a candidate model of multi-area recision-making in the Checkerboard Task.
In the next sections, we leverage the full observability of this biologcalll-plausibbe multi-area RNN to understand the mechanisms in different areas of the network and also how the network filters color information while propagating direction information.
4.2 Separation of the color and direction decision in Area
What are the key computational features of how the multi-area RNN represents color and direction information in the Checkerboard task? We first focused our analysis on Area 1, which uniquely
has substantial variance for both color and direction decisions (Fig. 22)) implying a central role in
computing the direction choice. We performed dPCA to identify demixed principal components related to the RNN inputs (coherence and context) and decisions (color and direction) 3p
We found
demixed components that separated information related to coherence, context, the color choice, and the direction choice (Fig. !b), consistent with these quantities being decodable from activity (Fig. Bh). We subsequently identified the context, color, and direction axes as the dPCA principal axes (unit norm, analogous to PCA eigenvectors), which combine the demixed components (analogous to PCA scores) to reconstruct neural activity
[2]
We projected RNN activity and input representations onto the principal axes for context, color, and direction (Fig. Fl). We found that the context and color axis both responded to context and color inputs, and overall trajectories represented both context and color information. This suggests that color and context information are mixed in Area 1. In contrast, the direction axis strongly represented the direction choice, but did not strongly represent context or color (Fig. t, right). Strikingly, context and color inputs had nearly zero projection on the direction axis (Fig. dh. right, opaque traces at O). Consistent with these observations, we found the color and context axes were highly overlapping (dot product: 0.93), indicating that context and checkerboard variance are mixed in Area activity. In contrast, the direction axes was closer to orthogonal to the context and color axes (overlap with color and context: 0.14 and 0.09, respectively).
These conclusions were upheld when we performed targeted dimensionality reduction (TDR), where we found (1) a direction axis separating left and right choices, with negligible input projections, and (2) that color and context representations were mixed (Appendix Fig. [4] Further, this structure was unique for PMd-like 3-area RNNs. In single-area RNNs, dPCA identified nearly orthogonal context, color, and direction axes, with trajectories that separated almost exclusively based on context, color,
and direction, respectively (Appendix Fig. 13)).
This Area representation has an important property: the direction choice is represented robustly on nearly orthogonal axis that has close to zero context and color input projections, (F) This is not trivial: counter-examples, single-area RNNs use direction axes that have context and color input projections (Fig. B)). while the direction axis of an unconstrained 3-area RNN (without anatomical
"These sweeps over different random initializations and different parameter settings consisted of the most significant computational cost, roughly requiring 500 CPU hours on AWS, with each model training in approxi- mately 1-2 hours.

---

a
dPCA context axis
Color axis
Direction axis
6.
activity
20:
Projections (au)
5
10-
Inputs
Check
Targ on
b
97.7% variance captured
dPCA axes
d
Target checkerboard inputs
right
Target couio
context
Taig
Targ on epoch
left red
directon axis
checiaatonndd
PG)
Targer contg
ctx
right red
dir
Figure 4: Separation of direction and color in Area (a) The context, color, and direction axis correspond the dPCA principal axes, which are not constrained to be orthogonal. Trajectories for different contexts and colors were separable on both the context and color axis. In contrast, the direction axis separated primarily on chosen direction. The RNN input representation had strong projections on the context and color axes, but not the direction axis. (b) Top PCs of Area 1 activity, which capture 97.7% of the Area variance. In the targets on epoch, the trajectories separate two regions corresponding to the two potential target configurations (Target config 1 in blue, and Target config in purple). The trajectories separate upon checkerboard color input, leading to four total trajectory motifs: right green, left red, right red, and left green. (c) Projection of the dPCA principal axes onto the PCs. (d) Projection of the context and color inputs onto the PCs. Context inputs are shown in pink, a green checkerboard input in green, and red checkerboard input in red. Green (red) checkerboards lead to an increase (decrease) in PC2 and the color axis, and differ in magnitude depending on the location of the trajectory in PC space. Trajectories are reduced in opacity to better visualize inputs.
connectivity constraints that did not resemble PMd) has context and color information, and also receives context and color inputs (Fig. l3p). We show the axes overlapped with the PCs in Fig 4 as well as the effect of the checkerboard and target inputs, which qualitatively shows that the inputs do project onto the direction axis.
4.3 Inter-area connections preferentially propagate output-relevant direction information
The differentiating aspect of multi-area computation is that the different areas are separated. A natural question ask how then does information propagate between areas? As defined in Section 3 we
denote the feedforward connections from Area to 2 as W W2,, and from Area 2 to 3 as W 23- We present results for feedforward connections from excitatory connections to excitatory units. Based on the hypothesis that the brain uses null and potent spaces selectively filter information 40)
evaluated the effective potent and null spaces of W 12 and W23- We defined the effective potent space be the right singular vectors corresponding to the largest singular values (see Appendix B.Y.
The effective null space corresponded to the singular vectors with the smallest singular values.
We quantified how the color and direction axis were aligned with these potent and null spaces (see Appendix B.9). The projections onto the potent space are shown Fig. 5p.b for Wi2 and W23, respectively. The null projection magnitudes are equal to one minus the potent projection. We found the direction axis was more aligned with the potent space and the color axis was more aligned with the null space. In fact, the direction axis (computed using the activity in Area 1) consistently
7

---

a
Area to
b
Area 2 to 3
random direction
random drecton
1.0
0.0
Null wwwjecttooooooo
0.8
0.6-
0.4
0.4
0.2
color ax8p
direston exs
0.0
0.0
Dimension potamt sparce
Dimensson porent spaiee
Potent Projections
Figure 5: (a) Projections onto the potent space between Areas and 2 for the color and direction axis, and random vector function of effective rank for the input area to the middle area. Regardless of the dimension of the potent space, the direction axis is preferentially aligned with the potent space, indicating the information along this axis propagates. while the color axis approximately randomly aligned. Shading indicates s.e.m. (b) Same as (a) but for projections between Areas 2 and 3. (c) Illustration depicting how the orientation of the axes affect the information that propagates.
most aligned with the top singular vector (governed by the parameters of the feedforward matrix; which do not affect the activity in Area 1). In contrast, the color axis was similarly aligned to a random vector. This alignment was robust to the dimension of the effective potent space, and was consistent across networks with varying feedforward connectivity percentages (10%, 20%, 30%, 50%, 100%). This suggests that learning in the multi-area recurrent network involved aligning the relevant information (in the activations) with the top singular vector (governed by the learned parameters of the feedforward matrix). These results indicate that direction information is preferentially propagated to subsequent areas, while color information is not. This phenomena is schematized in Fig. 55. To better understand the propagation and filtering of information in networks that had color information in the output area, we performed the same analyses on networks trained without Dale's 's law and area networks, and found that these networks had significantly reduced alignment of the direction axis with the top singular vectors (Appendix Fig. 17
These results also have implications how inter-area connections relay information between areas. Color activity has significant representation in Area (see Fig. 2). Therefore, the inter-area connections must not merely propagate the highest variance dimensions of preceding area [41]. Consistent with this reasoning, we found that while the top 2 PCs capture 97.7% excitatory unit variance, the top 2 readout dimensions of W only captured 40.0% of Area I's excitatory unit neural variance (Appendix Fig. [6). Hence, inter-area connections are not aligned with the most variable dimensions, but rather aligned preferentially propagate certain types of information result consistent with a recent study analyzing links between activity in V1 and V2 41
4.4
Area 3, modeling PMd dynamics, is primarily input driven and implements bistable dynamics
We showed previously that Area 3 most closely resembled PMd's dynamics (Fig. 2 Our results suggest that a direction signal has been computed before Area 3 and is selectively propagated through the RNN's 's inter-area connections. We found that the input to Area 3 (through W 22 is graded direction signal that provides a directional evidence signal for left or right reaches (Fig. (n). This activity must be transformed into eventual DV outputs, which are the accumulated evidence for left or right reach. This is illustrated in Fig. 60, where we plot W2 (r? are the unit activations of Area 2), and r?.
To analyze Area 3's dynamics, we first observed that Wont's 's coefficients were sparse, with 44 out of 80 output weights being identically zero. We found that the readout led to two separate clusters of artificial units: units with non-zero coefficients for the left DV (orange) and those with non-zero coefficients for the right DV (blue). Artificial units projected either to the left or right DV outputs, but not both, suggesting that there are two clusters mediating left and right choices.
Based on this clustering, we sorted and visualized the connections of excitatory units of Area 3, which upon first glance generally has no discernible structure (Fig. be, left panel). After sorting, found that two self-excitatory pools of units emerged in Wree. the first pool in Fig. b (right)

---

b
Figure 6: Area 3 mechanism. (a) Projection of input and overall activity onto the direction axis identified through dPCA. The conventions the same as in Fig. 4 (b) Readout weights in W out are sparse, with many zero entries, and selective weights for a left or right reach. (c) The unsorted Projectin cnto andi (8n0)
Connections
Connections
Weight War
Artiicaal neuron number
co
Conmecions from
Right units
Aneepe tea
Rigght Ouiput
1000
Corned one
Input
20
0.0
0.0
Lelt Output
Right
15gg Check
0.5
Directional exidence
Intiraary
connectivity matrix for the nonzero readout units (left panel), and the sorted connectivity matrix when the matrix was reordered based on the readout weight pools (right). (d) Average PSTHs from units for a leftward reach and (inset) rightwards reach. When one pool increases activity, the other pool decreases activity. (e) Averaged recurrent connectivity matrix. (I) Schematic of output area. (g) Psychometric curve after perturbation experiment, where 10% of inhibitory weights to the left pool (orange) and right pool (blue) were increased (doubled). Directional evidence is computed by using the signed coherence and using target configuration to identify the strength of evidence for left reach and strength of evidence for a right reach. Increasing inhibition to the left excitatory pool leads to more right choices and vice versa.
corresponding to the left DV and the second pool corresponding the right DV. In addition these two pools, we identified pool of randomly connected excitatory units and pool of inhibitory units with strong projections from and to the two pools. The full Area connectivity matrix is shown in Appendix Fig. 18 This structure is consistent with winner-take-all network, where increasing activity in one pool inhibits activity the other pool through separate inhibition pool (Fig. (d), By taking the averaged connectivity matrix, similar to 22 we confirmed that there were two excitatory pools that received similar projections from the random excitatory pool and inhibitory pool (Fig 6e).
We summarize the behavior with schematic of the area in Fig.
We subsequently applied selective perturbations W rec determine how behavioral performance was biased. We increased inhibition to either the right or the left pool by doubling the weights of 10% of the inhibitory neurons associated with each pool. We found that this biased the network towards more left or right reaches, respectively, shown in Fig. Dg. When inhibition was increased to the right excitatory pool, the network was more likely to respond left. Conversely, when inhibition was increased to the left excitatory pool, the network was more likely to respond right.
Together, these results show that the output area, modeling PMd, robustly transforms separable direction inputs to decision variable through a winner-take-all mechanism.
Discussion
Even though behavior and cognition arise from the coordinated computations of multiple brain areas, there is limited understanding of how interacting brain areas coordinate to produce cognitive behavior this study, we used multi-area RNNs to gain mechanistic insight into how the brain computes a perceptual decision in the Checkerboard Task and transmits only the direction decision to PMd. These results propose hypotheses for computations that occur upstream PMd, particularly how neural population activity representing context, color, and direction structured, and what
41 4

---

information is propagated between areas. We found that inter-area connections were preferentially aligned the direction axis, not axes of maximal variance, leading to selective propagation of direction activity and attenuation of color activity. This role for inter-area connections is consistent with null and potent spaces for filtering and propagating information between areas
4 and
communication subspaces, which are aligned with lower variance dimensions 41
Our results suggest that cortex and multi-area RNNs may share more general principle of multi- area information processing: if information becomes irrelevant for later computations, is reduced or discarded. In the Checkerboard Task, color information is necessary to compute the direction decision, but does not need to be represented after the direction decision is computed, as in PMd 4 26 3C Bi] 45]. In deep neural networks, it believed that minimal representations simplify the This idea is consistent with (1) the multi-area RNN developing minimal (little color information) but sufficient (robust direction information) representation of task inputs, and (2) Area 3, the output area, using a simple winner-take-all readout, forming two pools of neurons representing right and left decisions (Fig.
role of the output classifier 46 17
Our analysis of the multi-area RNN leads to testable hypotheses for future experiments. First, expect that neurons in cortical areas upstream of PMd should exhibit mixed selectivity for color and direction information, consistent with studies of dorsolateral prefrontal cortex (DLPFC) and ventrolateral prefrontal cortex (VLPFC) in cognitive tasks [8 50
More specifically. our model predicts the following organization of population dynamics in these areas: neural population dynamics should diverge two regions with slow dynamics based on target configuration, with largely overlapping context and color axes, but an orthogonal direction axis. Second, due to alignment of inter-area connections, direction axis activity in WFCC/L.CPFFFF should be more predictive of activity in downstream regions such as PMdr and PMd than activity in the top PCs.
10

---

Acknowledgments and Disclosure of Funding
We thank Laura Driscoll for helpful comments on the manuscript as well as Krishna V. Shenoy and William T. Newsome for helpful discussions on earlier versions of these results. We also thank Krishna V. Shenoy for kindly allowing us to use the data collected by Dr. Chandrasekaran when he was postdoc in the Shenoy Lab. MK was supported by the National Sciences and Engineering Research Council (NSERC). CC was supported by a NIH/NINDS ROO award RO0NS092972 and R01 award NS122969, the Moorman-Simon Interdisciplinary Career Development Professorship from Boston University, the Whitehall foundation, and the Young Investigator Award from the Brain and Behavior Research Foundation. JCK was supported by NSF CAREER 1943467, NIH DP2NS122037 the Hellman Foundation, and a UCLA Computational Medicine AWS grant. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research.
References
[1] S. M. Szezepanski, C. S. Konen, and Kastner, "Mechanisms of spatial attention control in frontal and parietal cortex," Journal of Neuroscience, vol. 30, 1, pp. 148-160) 160, 2010. [Online]. Available:
[2] J. Kalaska and D. Crammond, "Cerebral cortical mechanisms of reaching movements," " Science, vol. 255, no. 5051. pp. 1517-1523, 1992. [Online]. Available:
https:
[3] E. D. Remington, D. Narain, E. A. Hosseini, and M. Jazayeri, "Flexible sensorimotor com- putations through rapid reconfiguration of cortical dynamics," Neuron, vol. 98, no. 5, pp. 1005-1019.e5, Jun. 2018.
[4] T. Yamagata, Y. Nakayama, J. Tanji, and E. Hoshi, "Distinct information representation and processing for goal-directed behavior in the dorsolateral and ventrolateral prefrontal cortex and the dorsal premotor cortex," Journal of Neuroscience, vol. 32, no. 37, pp. 12 13934 12 949, 2012.
[5] J. M. Yau, G. C. DeAngelis, and D. E. Angelaki, "Dissecting neural circuits for multisensory integration and crossmodal processing," Philosophical Transactions of the Royal Society B: Biological Sciences, vol. 370, no. 1677, p. 20140203, 2015. [Online]. Available: http://myylaaceerpbbbiiiingggggp
[6] Cisek, "Making decisions through distributed consensus." Current opinion in neurobiology. vol. 22, no. 6, pp. 927-936, 2012.
[7] L. Pinto, K. Rajan, B. DePasquale, S. Y. Thiberge, D. W. Tank, and C. D. Brody, "Task- dependent changes in the large-scale dynamics and necessity of cortical regions," Neuron, vol. 104, no. 4, pp. 810-824, 2019.
[8] M. Siegel, Buschman, and E. K. Miller, "Cortical information flow during flexible sensori- motor decisions," Science, vol. 348, no. 6241, pp. 1352-1355, 2015.
[9] D. J. Freedman and J. A. Assad, "Neuronal mechanisms of visual categorization: an abstract view on decision making," Annual review of neuroscience, vol. 39, pp. 129-147, 2016.
[10] D. L. K. Yamins, H. Hong, C. F. Cadieu, E. A. Solomon, D. Seibert, and J. J. DiCarlo, "Performance-optimize hierarchical models predict neural responses in higher visual cortex," Proc. Natl. Acad. Sci. U. S. A., vol. 111, no. 23, pp. 8619 8624, Jun. 2014.
[11] D. L. K. Yamins and J. J. DiCarlo, "Using goal-driven deep learning models to understand sensory cortex," Nat. Neurosci,, vol. 19, no. 3, pp. 356-365, Mar. 2016.
[12] V. Mante, D. Sussillo, K. V. Shenoy, and W. T. Newsome, "Context-dependent computation by recurrent dynamics in prefrontal cortex," Nature, vol. 503, no. 7474, pp. 78-84, Nov. 2013.
[13] R. Laje and V. Buonomano, "Robust timing and motor patterns by taming chaos in recurrent neural networks," Nature neuroscience, vol. 16, no. 7, p. 925, 2013.
11

---

[14] A. Banino, C. Barry, B. Uria, C. Blundell, T. Lillicrap. P. Mirowski, A. Pritzel, M. J. Chadwick, T. Degris, Modayil et al., "Vector-based navigation using grid-like representations in artificial agents," Nature, vol. 557, no. 7705, pp. 429-44299444 4033, 2018.
[15] G. Hennequin, T. Vogels, and W. Gerstner, "Optimal control of transient dynamics in balanced networks supports generation complex movements," Neuron, vol. 82, no. 6, pp. 1394-1406, Jun. 2014.
[16] J. P. Stroud, M. A. Porter, G. Hennequin, and T. P. Vogels, "Motor primitives in space and time via targeted gain modulation in cortical networks," Nature neuroscience, vol. 21, no. 12, p. 1774, 2018.
[17] O. Barak, "Recurrent neural networks versatile tools of neuroscience research," Current Opinion in Neurobiology, vol. 46, pp. 1 6, 6, 2017, computational Neuroscience. [Online]. Available:
77000229
[18] D. Sussillo, M. M. Churchland, M. T. Kaufman, and K. V. Shenoy, "A neural network that finds a naturalistic solution for the production of muscle activity," Nat. Neurosci,, vol. 18, no. 7, pp. 1025-1033, 2015.
[19] H. F. Song, G. R. Yang, and X. J. Wang, "Training excitatrryinhhiiiory recurrent neural networks for cognitive tasks: a simple and flexible framework," PLOS Comput. Biol., vol. 12,
no. 2. pp. 1-30, 2016.
[20] H. Song, G. R. Yang, and X.-J. Wang, "Reward-based training of recurrent neural networks for cognitive and value-based tasks," Elife, vol. 6, Jan. 2017.
[21] W. Chaisangmongkon, S. K. Swaminathan, D. J. Freedman, and X. J. Wang, "Computing by robust transience: how the fronto-parien parietal network performs sequential, sategory-based decisions," Neuron, vol. 93, no. 6, pp. 1504-1517.e4, 2017.
[22] F. Mastrogiuseppe and S. Ostojie, "Linking connectivity, dynamics, and computations in low-rank recurrent neural networks." Newron, vol. 99, no. 3, pp. 609 - e23, 2018. [Online]. Available: http://www.eecccccccccco.oeeeeee
[23] D. Sussillo and O. Barak, "Opening the black box: low-dimensional dynamics in high- dimensional recurrent neural networks," Neural Comput., vol. 25, no. 3, pp. 626-649, 2013.
[24] C. Chandrasekaran, D. Peixoto, W. T. Newsome, and K. V. Shenoy, "Laminar differences in decision-related neural activity in dorsal premotor cortex," " Nat. Commun., vol. 8, no. 1, p. 614, Sep. 2017.
[25] M. T. Kaufman, M. M. Churchland, S. I. Ryu, and K. V. Shenoy, "Vacillation, indecision and hesitation in moment-by-moment decoding monkey motor cortex," " Elife, vol. 4, pp. 1-21, 2015.
[26] É. Coallier, T. Michelet, and J. F. Kalaska, "Dorsal premotor cortex: neural correlates of reach target decisions based on a color-location matching rule and conflicting sensory evidence," " Journal of neurophysiology, vol. 113, no. 10, pp. 3543-3573, 2015.
[27] J. Gold and M. N. Shadlen, "The neural basis of decision making," Annual review of neuro- science, vol. 30, 2007.
[28] R. Romo and V. de Lafuente, "Conversion of sensory signals into perceptual decisions." Progress in Neurobiology, vol. 103, pp. 41 75, 2013, conversion of Sensory Signals into Perceptions. Memories and Decisions. [Online]. Available:
[29] D. Freedman and J. A. Assad, "A proposed common neural mechanism for categorization and perceptual decisions," Nature neuroscience, vol. 14, no. 2, 143, 2011.
[30] C. Chandrasekaran, D. Peixoto, W. T. Newsome, and K. V. Shenoy, "Laminar differences in decision- neural activity in dorsal premotor cortex," Nature Communications, vol. no. 1, p. 614, 2017.
12

---

[31] M. Wang, C. Montanède, C. Chandrasekaran, D. Peixoto, K. V. Shenoy, and J. F. Kalaska, "Macaque dorsal premotor cortex exhibits decision-related activity only when specific stimulus-response associations are known," Nature Communications, vol. 10, no. 1, p. 1793, 2019. [Online]. Available:
[32] D. Kobak, W. Brendel, C. Constantinidis, C. E. Feierstein, A. Kepecs, 7. F. Mainen, X.-I. Qi, R. Romo, Uchida, and C. K. Machens, "Demixed principal component analysis of neural population data," Elife, vol. 5, Apr. 2016.
[33] S. Wise and E. A. Murray, "Arbitrary associations between antecedents and actions," Trends in Neurosciences, vol. 23, no. 6, pp. 271 276, 2000. [Online]. Available:
[34] E. Hoshi, "Cortico-basal ganglia networks subserving goal-directed behavior mediated by conditional visuo-goal association," Front. Neural Circuits, vol. 7, no. October, p. 158, 2013.
[35] D. Badre, J. Hoffman, J. W. Cooney, and M. D' D'esposito, "Hierarchical cognitive control deficits following damage the human frontal lobe," Nature neuroscience, vol. 12, no. 4, p. 515, 2009.
[36] N. T. Markov, M. M. Eresey-Ravasz, A. R. Ribeiro Gomes, C. Lamy, L. Magrou, J. Ve- zoli, P. Misery, Falchier, R. Quilodran, M. Gariel, J. Sallet, Gamanut, C. Huissoud, S. Clavagnier, P. Giroud, D. Sappey-Marinier, P. Barone, C. Dehay, Z. Toroczkai, K. Knoblauch, D. C. Van Essen, and H. Kennedy, "A weighted and directed interareal connectivity matrix for macaque cerebral cortex," " Cereb. Cortex, vol. 24, no. 1, pp. 17-36, Jan. 2014.
[37] H. F. Song, G. R. Yang, and X. J. Wang, "Training excitatory-inhibice recurrent neural networks for cognitive tasks: simple and flexible framework," PLoS Comput. Biol., vol. 12, no. 2, pp. 1-30, 2016.
[38] K. W. Latimer, "Nonlinear demixed component analysis for neural population data as a low-rank kernel regression problem," Neurons, Behavior, Data Analysis, & Theory, pp. - 24, 2019.
[39] L. v. d. Maaten and G. Hinton, "Visualizing data using t-sne," Journal of machine learning research, vol. 9, no. Nov, pp. 2579-2605, 2008
[40] M. T. Kaufman, M. M. Churchland, S. I. Ryu, and K. V. Shenoy, "Cortical activity in the null space: permitting preparation without movement," Nat. Neurosci., vol. 17, no. 3, pp. 440 448, Mar. 2014.
[41] D. Semedo, A. Zandvakili, C. K. Machens, B. M. Yu, and A. Kohn, "Cortical Areas Interact through Communication Subspace," Neuron, vol. 102, no. 1, pp. 249-259.e4, apr 2019. [Online]. Available: https://doi.org/ htt.:/oogggg..10//..eeerroooo wwwo:/.nerr...211.......
[42] G. R. Yang, M. R. Joglekar, H. F. Song, W. T. Newsome, and X.-J. Wang, "Task representations neural networks trained to perform many cognitive tasks," Nature neuroscience, vol. 22, no. pp. 297-306, 2019.
[43] A. Kohn, A. Jasper, J. D. Semedo, E. Gokcen, C. K. Machens, and B. M. Yu, "Principles of corticocortical communication: Proposed schemes and design considerations," Trends in Neurosciences, vol. 43, no. 9, pp. 725-737, 2020/10/12 2020.
[44] S. D. Stavisky. J. C. Kao, S. I. Ryu, and K. V. Shenoy, "Motor cortical visuomotor feedback ac- tivity is initially isolated from downstream targets in Output-Null neural state space dimensions," Neuron, vol. 95, no. 1, pp. 195-208.e9 Jul. 2017.
[45] Cisek and J. F. Kalaska, "Neural correlates of reaching decisions in dorsal premotor cortex: Specification of multiple direction choices and final selection of action," Neuron, vol. 45, no. 5, pp. 801-814, 2005.
[46] A. Achille and S. Soatto, "Emergence of invariance and disentanglement in deep representations," Journal of Machine Learning Research, vol. 19, no. 50, pp. 1-34, 2018. [Online]. Available:
13

---

[47] S. Soatto and A. Chiuso, "Visual representations: Defining properties and deep approximation." " Proceedings of the International Conference on Learning Representations (ICLR), June 2016.
[48] M. Rigotti, O. Barak, M. R. Warden, X.-J Wang, N. D. Daw, E. K. Miller, and S. Fusi, "The importance of mixed selectivity in complex cognitive tasks," Nature, vol. 497, no. 7451, pp. 585-590, May 2013.
[49] V. Mante, D. Sussillo, K. V. Shenoy, and W. T. Newsome, "Context-dependent computation by recurrent dynamics in prefrontal cortex," Nature, vol. 503, no. 7474, pp. 78 84, Nov. 2013.
[50] S. Fusi, E. K. Miller, and M. Rigotti, "Why neurons mix: high dimensionality for higher cognition," Current opinion in neurobiology, vol. 37, pp. 66-74, 2016.
[51] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT Press, Nov. 2016.
[52] R. Pascanu, T. Mikolov, and Y. Bengio, "On the difficulty of training recurrent neural networks," International Conference on Machine Learning, Feb. 2013, pp. 1310-1318.
[53] D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint 1rXiv:1412.6980, Dec. 2014.
[54] B. W. Brunton, M. M. Botvinick, and C. D. Brody, "Rats and humans can optimally accumulate evidence for pocision-mmaking. Science, vol. 340, no. 6128, pp. 95-98, 2013.
[55] R. Ratcliff, "A theory of memory retrieval." Psychological review, vol. 85, no. 2, p. 59, 1978.
[56] D. Sussillo, M. M. Churchland, M. T. Kaufman, and K. V. Shenoy, "A neural network that finds a naturalistic solution for the production of muscle activity," Nat. Neurosci, vol. 18, no. 7, pp. 1025-1033, 2015.
[57] D. Barber and F. Agakov, "The im algorithm: A variational approach information maximiza- tion," Proceedings of the 16th International Conference on Neural Information Processing Systems, ser. NIPS' 03. Cambridge, MA, USA: MIT Press, 2003, p. 201-208.
[58] B. Poole, S. Ozair, A. Van Den Oord, A. Alemi, and G. Tucker, "On variational bounds of mutual information." in Proceedings of the 36th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, K. Chaudhuri and R. Salakhutdinov, Eds., vol. 97. Long Beach, California, USA: PMLR, 09-15 Jun 2019, pp. 5171-5180, [Online]. Available:
http:/ppoolll99..ht..
[59] M. Kleinman, Achille, Idnani, and J. Kao, "Usable information and evolution of optimal representations during training," in International Conference on Learning Representations, 2.21. [Online]. Available: http:://peeeeeeveew..et//ouu/tdp
14


Reviews:
[{"review_id": "tp7lDMKJqAn", "reviewer": "Reviewer_eHSi", "summary": "The paper presents a multi-compartment recurrent neural network model. The network is trained to perform decision tasks similarly to non-human primates, and the network’s activity is compared to population activity recorded in primate cortex. Similarity between the artificial network and real data are compared with a focus on how similarity depends on biologically plausible constraints, such as Dale’s law. Analysis of the artificial networks showed how relevant information was propagated between modules to achieve the pMD-like choice encoding in higher areas.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The results build on current applications of RNNs in systems neuroscience. The results show how easy-to-incorporate biological constraints are important for comparing artificial networks to neural data, and such considerations will be important for many experimentalists. \n\nThe methods and results were generally clear. However, a couple minor edits could improve clarity. In particular, the conclusions of Figure 3 A-E were a little difficult to interpret. Labeling which quantities on the x-axis were more “brain-like” would help.\n\nAlthough the data comparison includes only 1 region, this analysis offers some predictions about low-dimensional inter-area communication for future multi-area recordings. However, the impact of this modeling study could be more clearly outlined.\n-\tThe abstract promised “testable hypotheses” from multi-area RNNs, but I did not see concrete suggestions for experimental tests of specific hypotheses explicitly laid out beyond what’s been proposed in terms of “minimal representations” in classifiers. What computationally relevant effects does the network predict from different experimental perturbations (whether with microstimulation, inactivation, targeted probes with specific stimuli, etc.)?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "gASd5drDL7s", "reviewer": "Reviewer_SBJB", "summary": "In this study, the authors trained a three-layer recurrent network and studied how the multiple networks (brain areas) work together to complete decision making. By constraining network architecture and parameters from biology, the authors demonstrated the last layer of the trained network reproduce the characteristics of neuronal responses in the dorsal premotor cortex, i.e., only encoding the direction of decision choices but filtering out sensory information, and conducted comprehensive analyses to show how decision variables emerge from network connections. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The trained network successfully reproduces key characteristics of neurons in the dorsal premotor area, and found the inter-network connections are preferentially aligned to the choice axis (direction), but not the axis of sensory variable or the axes of maximal variance. It is a solid study by using artificial neural network training to help us understand the neural computation in the brain, and the paper did systematic state space analysis of the network. From the discussions it is not very clear whether similar work has been done before to study sensorimotor transformation in the brain.\n\n### Main suggestions\nAlthough I understand that PMd responses cannot be reproduced by a single recurrent network, I hope this study could provide more mechanistic understanding of why the three-layer network is necessary to reproduce the key results, or given a two-layer network what extra mechanism is needed to reproduce the key results. It is a bit pity that the paper only briefly mentioned with less than three areas the last layer cannot reproduce the responses in PMd (line 147 and Fig. 3e). \n\nIt is better to see some discussions that how the 1st and 2nd layer in the network model can be regarded as sensory cortex in the brain.\n\n### Clarify\nThe paper is structure-wise and clearly written in neuroscience point of view. As a NeurIPS paper I suggest the authors compress the description in Results a little bit and adding more computational justifications and motivations. When I read the papers I realize there are not sufficient technical details in the main text, and I need to look for them in the supplementary information. \n\n### Minor\nI realize there is some wrong cross-reference in the paper which is probably resulted from this is a compressed version from an earlier long manuscript submitted to a neuroscience journal. I wish the author proofread the manuscript again to correct all of these typos. \n- Figure  1 caption mentions Methods section but no Methods in either the paper or Supplementary information.\n- Figure 2C: what is the difference between the top and bottom panels?\n- Line 112: a wrong cross-reference. The details should be in Appendix 5, not 5.4.\n- line 184: there is no figure 4e in this paper.\n- line 227: what is the full name of DV? It has never been defined.\n- Appendix line 527, how Z_des represents left or right choice?\n- Figure 4a: I suggest authors arrange the three plots vertically, and plot the input trajectories separately, i.e., not overlaid with activity.\n- line 545 in Appendix: why you stop training to prevent the network from achieving nearly perfect performance? Is it necessary to reproduce ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"ethical_concerns": "No", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}, {"review_id": "XgsPA4Lu6nF", "reviewer": "Reviewer_LjFK", "summary": "In this paper the authors introduce a model consisting of multiple connected RNNs which are trained to perform decision making tasks.\nIt is shown that imposing specific biological constraints (e.g. Dale's law) is important to yield biological solutions. Their results support the modular information storage often attributed to the brain, and opens the door for comparing multi-brain area experimental observations with models.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The authors built a model consisting of 3 RNNs, and tested the necessary ingredients needed to capture experimental observations. Their model proposes an explanation for why information about colour is lost in the dorsal premotor cortex recordings, which is due to colour information becoming irrelevant in later computations. Overall this detailed study brings novel, interesting ideas and makes some points that would be primarily of interest for neuroscience, but also for deep learning. \n\nMajor points:\n - A proper discussion on related work is missing. There is work in deep learning using multiple RNNs, it would be good to discuss this. Also, there is some (not much I agree) on using multiple RNNs. For example: Kang B, Druckmann S (2020) Approaches to inferring multi-regional interactions from simultaneous population recordings, Current Opinion in Neurobiology (Review)\n - Also I failed to find a discussion on the mapping between the Area 1 and 2 with specific areas in the brain.\n - What would happen if more brain areas would be added? Presumably there are more brain areas than just one (area 2) between (the visual cortex, area 1) and the premotor cortex (area 3).\n - It should be discussed that using BPTT to train RNNs is not plausible, and that using more plausible training algorithms may impact the representations learnt and therefore the results.\n\nMinor points:\n - Fig. 1 is not very clear. The caption does not refer clearly to the panels. Why use a) if there is only one panel?\n - It would be good to make it clear which training algorithm was used in the main text. I know its stated in the supplementary material, but it can be briefly stated in the main text.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No proper contrast with previous work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper builds a multi-areal model of a decision making task. Neuroscience-inspired architecture constraints on the network is shown to be critical for learning solutions that resemble cortical dynamics. All reviewers agreed on the significance and the novelty of the work.
Decision: Accept (Poster)