Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper presents the operative dimensions -- a novel, more effective tool (e.g. compared to PCA of W, X or R) to characterize the functionally relevant subspaces of RNN weights. The tool is well designed (Sec 2.2.1), easy to compute (appendix), and able to identify previously unknown properties of RNNs, e.g. the existence of low-dimensional functional subspaces (Sec 2.2.2) and the weight-space functional modules (Sec 2.2.3).
strengths_and_weaknesses: Strengths
+ (Originality) The proposed tool (operative dimensions) is novel to the best of my knowledge, as well as the two main findings regarding previously unknown properties of RNNs (as summarized above).
+ (Quality) The proposed tool is well designed (as dimensions with highest impact on local dynamics when removed), easy to compute (closed-form for column dimensions), and carefully validated with lots of experiments and analyses in the main text and appendix.
+ (Clarity) The writing is overall clear and easy to follow. The figures are well-made and very helpful.

Weaknesses
- (Significance) Although the two tasks (context-dependent integration and sine wave generation) are well studied and commonly used in computational and theoretical neuroscience papers, it’s hard to predict if the proposed tool (validated on them) and results (derived from them) really generalize to harder, more realistic tasks. Additional experiment(s) using e.g. sequential MNIST, TIMIT, or any of the possible applications described in Sec 3, could greatly enhance the significance of this work.
questions: - The column and row dimensions’ results are always both presented in the main text. Their difference is however not clearly explained. E.g., what do they capture differently? Are they complementary to each other, e.g. stronger when combined? Why should we use both? Etc.
- choice_1 and choice_2 don’t seem to be explicitly defined in the paper. Do they mean when the output is positive or negative? If so, do sensory_1’s mean and sensory_2’s mean always have opposite signs? Or, do they mean when sensory_1 or sensory_2 is picked by the RNN? If so, how is the detection done? And shouldn’t context_1 always lead to choice_1 (and context_2 to choice_2) when the RNN is fully trained?
limitations: The authors have reasonably addressed the limitations of their work.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper works towards an understanding of how weight structure influences the dynamics of recurrent neural networks (RNNs). It is shown that the principal components (PCs) of the weight matrix do not provide a good compression that preserves dynamics, instead a more dynamically-informed low-rank compression of the weights into "operative dimensions" can do so. These are defined (locally, along the attractor, similar to Lyapunov exponents) as the direction which maximizes norm of the divergence of trajectories. Compression of the W matrix using either PCs or operative dimensions is compared with a few RNN tasks used in computational neuroscience.
strengths_and_weaknesses: I found this paper to be very well-written and interesting. I didn't read the supplement due to lack of time. The way of defining operative dimensions is elegant and seems to be somewhat numerically efficient (it is tested for weight matrices with 1000s of dimensions). I liked how Fig 2 shows how these dimensions affect the attractor structure of the RNN, and I enjoyed the discussion.

The main weaknesses I found were in the tasks that were chosen. These tasks are somewhat artificial although I like their grounding in neuroscience-related tasks. For the NeurIPS audience, it would be interesting to see some whether operative dimensions offer insight into a more "AI" RNN task. Perhaps this is beyond the scope of the current paper.

I think the paper would be strengthened by exploring the connection between your ideas and more classical dynamics ideas such as Lyapunov vectors or bred vectors. You are essentially using the same idea, calculating directions of maximal growth in trajectories, except you are doing it in the weight space rather than the state space. This is interesting.

Something that could describe the low-dimensionality that is observed here is the dimensionality of the control signal u/matrix B. It could be that this low-dimensional drive forces the RNN onto an attractor of similar dimension. That could explain the patterns you observe in Fig 4. I imagine there is extensive literature that could explain this effect. 

Some detailed comments:
Throughout the text you use "RNN" to refer to "recurrent neural networks" in the plural. I think you should change this to "RNNs" for the plural, which is more common.

148: "dimension of the connectivity a" suggest changing to unambiguous language "unit vector a"

150, eqn 6: maybe worth mentioning this is an orthogonal projection

283: "This past work had however not" awkward language. Also, I believe the low-rank RNN work from Mastrogiuseppe and Ostojic did relate to attractor structure.
questions:  If the RNN dynamics are linearized, I believe that there is a direct connection to the singular vectors of the dynamical map. Maybe PC directions, in this special case, correspond to operative dimensions? This seems like an obvious connection to explore. Also, spending a bit more time in the text talking about the methods you use to optimize Eqn 8 and 9 would be helpful.

Do you see alignment of the operative dimension components with the B matrix components? The alignment (or lack thereof) to the W matrix components is explored extensively; this seems like an obvious one to also explore.

In the discussion, it would be worth discussing connections to model reduction methods, eg for PDEs and high-dim ODEs.

Preservation of attractor basin boundaries seems like something these operative dynamics clearly address, but I don't think this was discussed.
limitations: I found the authors did a good job discussing limitations and potential ethical issues.
However, I do think they could mention that their results are limited to studying very simple tasks, much simpler than the AI type tasks where RNNs are employed in practice.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper presents a framework and method to analyze the connectivity of unconstrained RNN weights. They find that while recurrent weights are high-dimensional, the network dynamics that actually solve simple neuroscience tasks (i.e. ones that map stimulus inputs to one of two choices) are low-dimensional. These low-dimensional subspaces are estimated by removing different dimensions and examining the effect on activity state space. These are referred to as operative dimensions, which contain functional modules for the task. The operative dimensions can be related back to the recurrent connectivity matrix W by the construction of a reduced-rank connectivity W^{OP}. Finally, they illustrate how this framework allows one to relate different functional modules to subspaces in the weight matrix, which in essence relates function to structure.
strengths_and_weaknesses: The paper is well-written, with sufficient detail and supporting material in the appendix. The math is straightforward and clear. The methods are certainly based on some prior work, but the general framework and findings are novel and interesting. The authors are also very thorough with additional analyses and explore nearly all the related questions that arose as I read the paper.

Their findings are significant in that they apply to unconstrained RNNs, which brings the literature closer to real-world uses of neural networks. They also could be applied to other areas in machine learning (continual learning or weight compression). And, as the authors note, the findings could also make it easier for neuroscientists to use RNNs as models of biological datasets. This paper could clearly have impact on several sub-areas.

One of the most interesting findings, in my opinion, is that the number of operative dimensions increases with with task/network complexity. It will be interesting to test this point further by independently varying stimulus complexity and output complexity. For example, how many more dimensions are required as one increases the number of nonlinearities required to produce some output? To be clear I am not suggesting that the authors conduct another analysis now -- there is already more than enough in the current paper.
questions: * Fig. 3C shows that network performance gets worse with the first few operative dimensions before it gets better. The authors hypothesize in lines 201-206 that the first few operative dimensions may be required at state space locations explored late in the trial, rather than at the beginning of the trial. Is there any evidence to support this?
* It is interesting that the activity of the RNN is low-dimensional even before training. Why is this?
* Any thoughts from the authors on how this might scale to stacked RNN architectures?

Suggestions
* The citations in the text are numbered, but the reference list is not. Please fix.
limitations: The authors thoroughly discuss the limitations to their approach, notably their choice to sample the average trajectory in activity state space. They also provide supplementary analyses to address some of the limitations.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors study how unconstrained training of recurrent neural networks results in structured synaptic connectivity. Specifically, they define a novel decomposition of the synaptic matrix that reveals this structure. In contrast to a naïve decomposition of the matrix, this novel form is relatively low-dimensional. 
The new suggestion is to define operative dimensions as those directions in weight space that lead to the maximal disturbance to neural dynamics. These are defined locally, around points of interest, and then integrated into a global measure. 

strengths_and_weaknesses: Strengths:
This is an important problem, and there are few studies tackling it. The suggested approach is, to the best of my knowledge, novel.
Weaknesses:
A main weakness is novelty with respect to reference 20 (Schuessler et al, NeurIPS 2020). That paper tackled the same problem, and showed both numerically and analytically that the changes to connectivity (delta W) are low rank. There are certainly differences between the papers, but given the similarity, this is hardly discussed.
Another relevant reference (that is not cited) is Smith, Linderman and Sussillo “Reverse engineering recurrent neural networks with Jacobian switching linear dynamical systems”, NeurIPS 2021. In that paper, roughly speaking, trained networks are approximated by several local dynamical systems. It would seem that the analysis of A.4.3 of Vanilla might be extended to general networks by using the Jacobian at each point in phase space.
The paper also has very little analytical results – specifically, the question of why this low dimensionality develops is not addressed.

questions: 1.	Besides reference 20, there are also relevant works in the feedforward domain. A couple of examples: 
a.	Saxe, Andrew M., James L. McClelland, and Surya Ganguli. "Exact solutions to the nonlinear dynamics of learning in deep linear neural networks”
b.	Timor, Vardi, Shamir, “Implicit Regularization Towards Rank Minimization in ReLU Networks”
2.	L38-39: “… it remains unclear … findings can be generalized to RNN … not subject to such constraints”. – This sentence highlights the need to better relate to existing literature.
3.	Equation 1 and Line 57. If I understand correctly, the random noise is drawn independently for every dt. Stochastic differential equations require scaling the noise with sqrt(dt). If dt is not changed, this just amounts to an “incorrect” scaling of the noise. But, in general, it is not an exact implementation of equation 1, and could lead to problems when checking sensitivity for dt for instance.
4.	It is not entirely clear to me why the operative dimensions reveal cases similar to figure 2. If a change in W causes a trajectory to cross a separatrix, the local changes might be small, but the resulting global (t -> infty) changes could be large. The discussion in section 2.2.1 explains why a small change in local dynamics can lead to dramatic performance changes. But operative dimensions search for large local changes.
5.	L228: Another relevant reference here is Smith, Linderman and Sussillo “Reverse engineering recurrent neural networks with Jacobian switching linear dynamical systems”
6.	Is there any relation between these results and the eigenvalue spectrum of W, or of its linearized versoins? If I understand correctly, the authors show that singular value decomposition is uncorrelated to these dimensions. On the other hand (following the jacobian switching linear systems reference above, and the results in A.2.3), is there a relation to the eigenvectors of the linearized dynamics around the points used?

limitations: Yes
review_title: null

Meta-review: metareview: This paper defines a new way to identify a low-dimensional subspace in recurrent weight matrices of a recurrent neural network that is important for computation. All reviewers agreed the approach is novel and insightful.
confidence: Certain
Decision: Accept