SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure 5: The joint representation of multiple tasks as a line attractor. A. Two exemplar kappa-projections of all attractors, with differing grey shades (from dark to light) representing different level curves of the  $q$ -function ([28], see supplementary 2.2). B Attractor projections into a 1D space using arc-length parameterization, stacked upon one another and aligned to the generating functions on the left. The points are colored w.r.t. the task output. C pairwise distance between each pair of task representations in every network, binned w.r.t.  $l_{2}$  distance. On the left we show the results for networks trained on six tasks, and on the right for networks that were trained only on two tasks at a time. D same but binned w.r.t. the steepness distance.
After this computation, the resultant arc lengths are normalized to fit within the  $[0,1]$  range. This normalization produces the arc length function  $s(R_{i}(v))$  which signifies the position of  $R_{i}(v)$  on the unified one-dimensional curve.
In Fig. 5B we stack the 1D projection of the curves of each task, where on the left the corresponding functions appear. One can clearly see a gradual change in the functions, paralleling a gradual change in their representation.
4.4 Task distance
The functions in Fig. 5B seem to be ordered, but it is not immediately clear what is the metric underlying this ordering. A priori, there isn't one correct way to define the distance between tasks. When working with functions, the default way is to consider the  $l_{2}$  distance between the corresponding functions:
$$
d _ {l _ {2}} \left(\phi_ {1}, \phi_ {2}\right) = \int_ {v _ {\min }} ^ {v _ {\max }} \| \phi_ {1} (v) - \phi_ {2} (v) \| _ {2} ^ {2} d v \tag {4}
$$
Observing the ordering emerging from the networks, an alternative way is measuring the similarity between the steepness of the functions:
$$
d _ {\text {s t e e p n e s s}} \left(\phi_ {1}, \phi_ {2}\right) = \int_ {v _ {\min }} ^ {v _ {\max }} \left\| \left| \phi_ {1} ^ {\prime} (v) \right| - \left| \phi_ {2} ^ {\prime} (v) \right| \right\| _ {2} ^ {2} d v \tag {5}
$$
In practice, we approximate these integrals using Reimanns sum formula.
4.5 Representational distance
Representational distance quantifies the overlap between two distinct task representations. Leveraging the arc length function  $s$ , the distance is defined as:
$$
d \left(R _ {1}, R _ {2}\right) = 1 - \frac {\left| \left\{s \left(R _ {1} (v)\right) _ {v} \cap \left\{s \left(R _ {2} (v)\right) _ {v} \right. \right| \right.}{\left| \left\{s \left(R _ {1} (v)\right) _ {v} \cup \left\{s \left(R _ {2} (v)\right) _ {v} \right. \right. \right|} \tag {6}
$$
Figure 6: Intuition for symmetries in attractor representations, using a simplified 1D depiction. A task defined by the function  $y = v^4$  (middle) and two functions obtained by horizontal and vertical reflections. B The same three functions, now depicted as a function of  $h$ . The red lines are reflections of the transformed  $v^4$ , where it is evident that only the vertical reflection is conserved. C The nonlinear transformation from  $v$  to  $h$  (which includes the dynamics of the network) is the reason that horizontal reflections of a task yield different representation.
4.6 Function steepness shapes representations
To assess which task similarity correlates more with the representation similarity, we compute all pairwise distances between tasks - both in task space, and in representation space. The left columns of Fig. 5C,D show the relationship between either function-distance (C) or derivative distance (D) and the representation distance. The derivative distance is much more predictive, although not perfect. The existence of six simultaneous tasks could explain some of the spread in these graphs, so we also analyzed a simpler scenario. We trained full-rank networks on one pair of tasks at a time (right columns of Figure Fig. 5C,D). In this case the correlation between derivative distance is much stronger.
4.7 Symmetries in attractor representations
Why is it that steepness controls task alignment within attractors? A different perspective is to consider symmetries and invariances in task- and in representation-space [9]. Consider one specific function,  $y = v^4$ , depicted in the center of Fig. 6A. One can apply two different symmetry operations on it to obtain different functions. We know from the results of Fig. 5C,D that only the vertical reflection will preserve steepness, and hence is expected to lead to a similar representation. To gain intuition on why this happens, consider the transformations from input to output (Fig. 6C). Input undergoes a nonlinear transformation to hidden state – both due to static nonlinearities and due to the dynamics leading to the corresponding fixed point. From the hidden state to the output, however, is a linear transformation. We ignore the different input and output directions, and the curvature of the attractor and encapsulate the entire transformation as some nonlinear function – for instance, the one in Fig. 6C. If we now replot the three functions in terms of their relation to the hidden state (Fig. 6B), we see why only transformations that respect steepness lead to similar representations.
We showed that recurrent neural networks trained on multiple tasks exhibit a simplicity bias in terms of the dynamical objects formed during training. Unless prevented, networks will form the minimal number of objects needed to solve the task. We demonstrated that this bias is a consequence of recurrent dynamics and the sequential formation of attractors. First, attractors are generated one at a
time. Second, once an attractor forms, the recurrent dynamics will converge to this attractor for all tasks.
Our analysis focused on an architecture with separate input and output channels for each task, and without an explicit context channel. Furthermore, the input and output statistics were identical for all tasks. Several studies of multi-task learning used different architectures. [10] showed how modular representations can emerge from multi-task learning in feed forward networks. Their single input - many output configuration results in a joint representation of all tasks, with the nature of this representation being the object of study. Specifically, they showed that random classification tasks disentangle the representation. RNNs trained on many cognitive tasks with more complex input and output structures also show shared attractors [6, 35]. The task-set analyzed there is closer to neuroscience experiments, making it easier to compare to data. On the other hand, these tasks have a correlation between their input-output structure and their cognitive demands. Here, we opted for simpler tasks, using fundamental dynamical objects, and with identical input-output structure for all tasks. This allowed a systematic study of how dynamical objects arise during training and relate to one another.
Our choice for simple and systematic tasks also comes with several limitations. The tasks are not as close to biology as those in previous works [2, 35]. On the mahine learning side, the tasks and architectures are very far from state of the art. While we did examine GRU architectures and lazy vs rich initializations, the focus here was on the systematicity and to remain close to elementary dynamical objects.
The underlying mechanism for the simplicity bias is the sequential emergence of attractors, and the reduction of loss once enough attractors have emerged. This is reminiscent of several results showing the emergence of various network properties at different rates. Connectivity in deep linear networks is affected by leading modes of the data covariance faster than other modes [25]. RNNs trained on fixed point tasks develop rank-1 connectivity faster than higher ranks [26]. Gradient descent first learns low moments, and then high cumulants of the data [18]. In modular feed forward networks, different pathways through the network race with each other, leading to a reduced effective network [24].
The simplicity bias has several intriguing applications. First, reusing an existing representation leads to faster learning of tasks. This feature was most evident in the gated scenario, in which only one task is processed at a time. [15] highlighted a similar phenomenon as a possible normative reason for the limited capacity of cognitive control. Second, our results suggest that observing a modular representation is an indicator of some constraint on the architecture. Finally, an attractor shared between tasks is not identical to an attractor that was formed in response to a single task. For instance, if a task with constant output has an oscillatory component to its neural activity, it might suggest that the same circuit is also capable of generating such an oscillation in another context. One can imagine recording the neural activity of a subject dribbling a basketball, and being able to predict whether that subject also knows how to shoot.
This work was supported in part by the Israeli Science Foundation grant 1442/21 and an HFSP research grant (RGP0017/2021).
[1] Omri Barak. Recurrent neural networks as versatile tools of neuroscience research. Current Opinion in Neurobiology, 46:1-6, October 2017. ISSN 09594388. doi: 10.1016/j.conb.2017.06.003.
[2] Omri Barak, David Sussillo, Ranulfo Romo, Misha Tsodyks, and L.F. Abbott. From fixed points to chaos: Three models of delayed discrimination. Progress in Neurobiology, 103:214-222, April 2013. ISSN 03010082. doi: 10.1016/j.pneurobio.2013.02.002.
[3] Dean V. Buonomano. Decoding Temporal Information: A Model Based on Short-Term Synaptic Plasticity. The Journal of Neuroscience, 20(3):1129-1141, February 2000. ISSN 0270-6474, 1529-2401. doi: 10.1523/JNEUROSCI.20-03-01129.2000.
[34] Guangyu Robert Yang, Michael W Cole, and Kanaka Rajan. How to study the neural mechanisms of multiple tasks. Current Opinion in Behavioral Sciences, 29:134-143, October 2019. ISSN 23521546. doi: 10.1016/j.cobeha.2019.07.001.
[35] Guangyu Robert Yang, Madhura R. Joglekar, H. Francis Song, William T. Newsome, and Xiao-Jing Wang. Task representations in neural networks trained to perform many cognitive tasks. Nature Neuroscience, 22(2):297-306, February 2019. ISSN 1097-6256, 1546-1726. doi: 10.1038/s41593-018-0310-2.