A Recurrent Neural Circuit Mechanism of Temporal-scaling Equivariant Representation
Authors: Junfeng Zuo
Xiao Liu
Ying Nian Wu
Si Wu
Wenhao Zhang
Venue: NeurIPS 2023 poster (2023)
Keywords: temporal-scaling group
equivariant representation
disentangled representation
motor timing
continuous attractor networks
TL;DR: A biologically plausible recurrent neural circuit model is proposed that equivariantly represents temporal sequences and explicitly represent temporsal scaling operators.

Abstract: Time perception is critical in our daily life. An important feature of time perception is temporal scaling (TS): the ability to generate temporal sequences (e.g., motor actions) at different speeds. However, it is largely unknown about the math principle underlying temporal scaling in recurrent circuits in the brain. To shed insight, the present study investigates the temporal scaling from the Lie group point of view. 
We propose a canonical nonlinear recurrent circuit dynamics, modeled as a continuous attractor network, whose neuronal population responses embed a temporal sequence that is TS equivariant. Furthermore, we found the TS group operators can be explicitly represented by a control input fed into the recurrent circuit, where the input gain determines the temporal scaling factor (group parameter), and the spatial offset between the control input and network state emerges the generator.  The neuronal responses in the recurrent circuit are also consistent with experimental findings. We illustrated that the recurrent circuit can drive a feedforward circuit to generate complex temporal sequences with different time scales, even in the case of negative time scaling (''time reversal'').  Our work for the first time analytically links the abstract temporal scaling group and concrete neural circuit dynamics.

A Recurrent Neural Circuit Mechanism of Temporal-scaling Equivariant Representation
Junfeng Zuo zuoj unf eng edu. cn
Xiao Liu xiaol iu230pku edu. cn
Ying Nian 2
Si Wu si siwu@pku. edu. cn
Wen-Hao Zhang
you@stat. ucla.cllcooooooo edu
wenhao zhang@utsoutthwsstem.. edu
Peking-Tsinghuu Center for Life Sciences, Academy for Advanced Interdisciplinary Studies,
School of Psychological and Cognitive Sciences,
Beijing Key Laboratory of Behavior and Mental Health,
IDG/MeGovern Institute for Brain Research,
Center of Quantitative Biology, Peking University. DDpartment Statistics, University California, Los Angeles.
Lyda Hill Department of Bioinformatics, UT Southwestern Medical Center.
OO'Donelll Brain Institute, UT Southwestern Medical Center.
Abstract
Time perception is fundamental in our daily life. An important feature of time perception is temporal scaling (TS): the ability generate temporal sequences (e.g., movements) with different speeds. However, it is largely unknown about the mathematical principle underlying TS in the brain. The present theoretical study investigates temporal scaling from the Lie group point of view. We propose a canonical nonlinear recurrent circuit dynamies, modeled as continuous attractor network, whose neuronal population responses embed a temporal sequence that is TS equivariant. We find the TS group operators can be explicitly represented by a time-invariant control input to the network, whereby the input gain determines the TS factor (group parameter), and the spatial offset between the control input and the network state the continuous attractor manifold gives rise to the generator the Lie group. The recurrent circuit's 's neuronal responses are consistent with experimental data. The recurrent circuit can drive feedforward circuit to generate complex sequences with different temporal scales, even in the case negative temporal scaling ("time reversal")). Our work for the first time analytically links the abstract temporal scaling group and concrete neural circuit dynamics.
Introduction
We are living in dynamic world and the brain can flexibly process sensory and motor events occurring different time scales [1-]] An example of such temporal flcxibility is self-initiated movements, e.g.. singing song normal faster speed. The capability of temporally flexible movements implies the brain flexibly control the underlying neural circuit dynamics. Indeed, experimental data converges empirical principle of temporal scaling (TS) the behavioral and neuronal levels (.5). Specifically, when generating movements with longer intervals, was found the neuronal population activities evolve along the same (low-dimensional) manifold of the neuronal population
"Correspondng author.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

responses but at slower speed 22 Although this observation was reproduced in previous modeling studies that trained recurrent circuit models achieve temporal scaling (c.g., 2 6 7 9 [5). from clear about the mathematical principle governing temporal scaling in brain's 's recurrent neural circuits.
is far
Temporal scaling is also fundamental in machine learning (ML) and robotic research. Most of engineering approaches use time warping achieve temporal scaling where sinusoidal oscillatory inputs are needed (e.g., [1-19).. However, no experiments support such oscillation- bbased signals are used to represent time information ranging from milliseconds to seconds in the brain B 3 Some other ML studies directly modulated time constants of artificial neurons in recurrent neural networks to realize TS (e.g. (20).. whereas real neurons' time constant is thought be fixed due to biophysical properties. Earlier studies also introduced neurophysiologica properties to achieve temporal scaling, e.g., using synaptic shunting to adapt the effective integrating time [2]] and utilizing time-covariant time field sizes 222. However, they ((11 (22) focused on the scale invariance during the temporal scaling inference rather than generating temporal sequences with different speeds. Combined, current ML models adopt different mechanisms to realize TS than neural circuits in the brain.
To provide algebraic and dynamical understanding of temporal scaling in neural circuits, we rigorously study this question from temporal scaling (Lie) group. We contend that there are two aspects of realizing TS group transformations in recurrent neural eircuits: One is explicitly representing a TS controller corresponding to TS group operators; and the other is representing TS equivariant temporal sequence in spatiotemporal neuronal responses, referred to as TS equivariant representation. It is unknown how recurrent neural circuits represent abstract TS group operators, nor the equivariant representation of temporal sequences. And no previous MI. study investigated the equivariance of a Lie group acting on the temporal domain, nor an explicit representation of group operators.
The present study analytically derives a canonical current eircuit with continuous manifolds of attrac- tors, so-called continuous attractor networks (CANs), to equivariantly represent generic sequences with different temporal scales. And the TS operators are realized by time-invariant control input applied to the CAN: The TS factor is represented as the gain (magnitude) of the control input, and the TS (group) generator emerges from the spatial offset between the control input and the network state along the continuous attractor manifolds. Moreover, applying a negative (inhibitory) gain of control input enables the recurrent circuit to generate a fime-reversed sequence. The proposed circuit model reproduces a wide range of experimental findings of temporal scaling. We also demonstrate the recurrent circuit can drive a feedforward circuit to generate arbitrary, complex temporal sequences with different time scales, e.g., hand-written sequences. It will shed light on understanding the flexible timing in the brain, and provides new building blocks to achieve timing tasks in engineering.
The contributions of our work are as follows: It for the first time analytically links the TS group equivariant representation to biologically plausible recurrent neural circuit, and analytically identifies that TS group operators are represented by time-invariant control inputs applied to the circuit. The proposed circuit with fixed connectivity flexibly generates TS equivariant sequences by just modulating the gain of control inputs, which is novel mechanism never used in ML studies (Discussion, comparison with other works). also proposes novel circuit mechanism generating time-flipped sequences ("time reversal"") with receiving control inputs of negative gain.
2 The temporal scaling group
We formulate the temporal scaling transformation by using the Lie group. Consider a TS group whose group operator 5(a:) E & will map time scale / (the scale of the time axis) into a new scale t,
( S(a) -t lstart + al,
(1)
where the symbol denotes the action of group operators, and a is the temporal scaling factor. fstart denotes the start timing, where the lstart = to = O when > O, and otherwise sttrtt = fox regards as the end timing of the original time scale. Across this study, we use to represent a "standard" or reference physical time, and d. denotes the scaled time. Then we consider an arbitrary temporal sequence yt),, which can be high high-dimensional and may be regarded as, e.g., hand movements, vocal sequences, etc. Changing the time scale t will transform yt) into new sequence y(t).
y(t) =(a) y(t) y|S(a) t] y(at),
(2)
2

---

A
Equivariant map
B
Temporal scaling
Temporal scaling (e=:0)
Time
Sequence
y(1)
x(O)
Fooor Ongg
ya Mil
Original
y(t)
St
S(o))
Time
Time
S(a)
S(a)
reversed
Scaled seq, 300
Mfemort
seq, 3
y(==(_+od)
t=
y(t)
Time
Time
1/ox
Figure 1: (A) An equivariant map between the time and a temporal sequence. (B) Left: A TS operator S(a)) with scales the time by factor a and then scale temporal sequences accordingly. Right: A TS operator Š(a)) with a < 0 flips and scales the sequence on time axis.
To simplify notation, we suppress lstart in the last term in Eq. 2 and across the study, in that our focus is the temporal scaling of sequences rather than temporal alignment between them. Our actual calculations do align lstart appropriately. S(ar) is TS operator acting on a sequence, in contrast to S(ar) acting on the time directly. TS group operators are commutative, i.e., the effect of sequential actions of two operators can be composed into the same operator irrelevant their order, i.e., S(o) 5(3) = 5(3) S(a) = $(a33). Other properties of TS group operators can be found in Supplementary Information (SI) (Eq. ST
(0 < a 1) corresponds to speed up (slow down) the time, analogous compress (stretch) the sequence y() along the time axis (Fig. BB. left). = 0 is a particular case where the sequence remains a constant value, analogous "time freezing". a < 0 implies "time reversal", i.e., flipping the sequence over the time axis and then scaling it based on the absolute value of a (Fig. T3, right).
3 Disentangled neural representation of the sequence time and pattern
We focus how temporal sequences (() with different temporal scales can be flexibly generated by canonical neural circuits. There are two aspects of temporal sequence generation: One is representing the temporal pattern (or content) of sequences referred to "pattern" representation, c.g., representing a song A vs. song B; and another is representing the time and time scale which is referred to as "time" representation, c.g., singing song with normal speed vs. 2x speed. We reason the time and pattern of sequences can be represented separately in neural circuit [8) in that sequences can be scaled over time whatever their temporal patterns are. Therefore, we posit disentangled, two-stage circuit architecture (Fig. [A): A recurrent circuit acts as an "neural clock" whose population responses of N neurons, u4) = ي (1)) represent the time information; and a feedforward circuit converts the spatiotemporal responses u(t) into the sequence with the desired pattern, (t), e-g., a concrete song or the trajectory of hand-wring written digits. Mathematically. the disentangled circuit can be denoted as,
y(t) = Flu(t)),
where F|.] is the feedforward circuit storing the pattern information. Since F[-] is memoryless over time and maps the instantaneous response u(t) into the instantaneous complex sequence value yt), scaling the sequence y(t) over time can be realized by just scaling the recurrent 's response,
(t) S(a) y() = -(a)) Flu(O) ([Su(a)) u())
where (a) is the TS operator acting on neural responses u(t), in contrast with S(a) acting on y()). In summary, realizing TS in the disentangled circuit only needs to modulate the recurrent circuit's response u(t) without changing the feedforward circuit (FII)). In contrast, generating different patterns only needs to change (functional) feedforward circuits, while keeping the recurrent circuit's response utt) unchanged. The representation of time information in recurrent cireuits was supported by wide range of neurophysiology experiments in many brain areas, e.g., medial frontal cortex (6], hippocampus [24), striatum 25). and HVC in songbirds (26), etc. The disentangled architecture also supported by recent study on vocal motor circuits in a highly vocal rodent (27,, where the motor cortex dictates the timing of songs, while the mid-brain or brain stem areas generate music notes (patterns).
3

---

A
Disentangled
C
Recurrent circuit
E
F
circuit
Neural sequence vidi
control Input
Nou D RODO
syroqpи
Selispoe
Egenvector d/Bi
Bisoo
Enerr
Represontation
0
Elgenves.
6i4 3)
Time
D
Neuron
G
Partom rmpresentation
Fredford cnuil Fl:
Time
Mo Fee wiight
Egemmauess T6
Neuron mnee
Fiimn coto (uALl
Sequence brbbtaay)
y(t)
0 Turing digariy -d
Eigenslue index
Time
Figure 2: (A-B) A disentangled neural circuit that represents the sequence's time and pattern information in the recurrent and feedforward circuits respectively. The recurrent circuit generates neural sequence embedding the "time" manifold (B top, adapted from (33 regardless of the sequence pattern, and then the feedforward circuit maps the neural sequence into sequences with arbitrary patterns (B bottom). A control input (green circle) is supposed to modulate the temporal scale of the neural sequences generated by the recurrent circuit. (C) The recurrent circuit modeled as continuous attractor network (CAN) consisting of excitatory (E) and inhibitory (I) neurons. The E neurons are uniformly distributed in the one-dimensional 2 manifold of the generic sequence (y-axis in top figure, panel B). (D) The recurrent connections E neurons, which decays with the difference between two neurons' preferred values the generic sequence. (E) The eigenvalues and eigenvectors of the CAN, where the eigenvector with the largest eigenvalue corresponds to the movement of neuronal responses along the "time" manifold. (F) Energy landscape of the CAN without receiving the time-scale control input. (G) A generic neural sequence is generated by the recurrent circuit when receiving a time-invariant control input.
3.1 A low-dimensional "time" manifold in recurrent circuit's responses
We next investigate how the time is represented in u(t) based on experimental observations. was found that when generating movements with different time scales, recurrent circuit's 's responses u(t) evolve along the same low-dimensional manifold with different speeds 7 In contrast, the temporal responses of single neurons under different time scales may not align together: Therefore, the one-dimensional (1D) time can be represented (embedded) in ID manifold in recurrent circuit responses utt). We denote by the 1D manifold of time representation in the circuit, in order to distinguish with the reference time It is worth noting that the neural response evolution on the manifold, 2t), doesn't need to be linear with reference time t, and such linear z(t) also lacks experimental support. Mathematically, it only requires that =(t) is a one-to-one (bijective) mapping with time é (3]. Combined, the recurrent circuit's s response u(t) represents an internal "time" 2, and in the rest of the paper, we will denote the response as a function of 2t)), i.e., u[z(t)).
posit
What is the internal "time" manifold 2t) would look like? Experiments found there is a unique time for each neuron to reach its maximal response 247) Sorting neurons based on the peak response timing, the population responses u(l) resemble a bump-profile activity traveling across neurons (Fig. 2B, top). Hence, the 1D "time" manifold can be regarded as the axis of sorted neurons based on their peak timing (Fig. 3 y axis). And the evolution of neurons' bump-profile responses the manifold forms a sequence =t)) representing the internal "clapsed time" " [4] IO [1]. Eventually, the time can be explicitly read out by identifying the index of the neuron which reaches its maximal response, and the time scale (a in Eq. D reflected by the speed the neural sequence.
3.2 Temporal scaling operator acting on recurrent circuit's responses
As experiments suggested the TS transformations scale the "time" manifold rather than each individual neuron's temporal responses, we define $, in Eq. (4) as,
a[<((t)] (a) u[=(t)) u[=(S(a) t)] u(z(ott).
And such neural representation u(z(t)) is called equivariant with the temporal scaling group. To find the expression of (a), consider infinitesimal temporal scaling with close to one (a - 1).

---

Reorganizing the scaled neural response as ufa(at)] = u(z(exp(Int + ln))],, and performing first-order Taylor expansion with amount in a,
du [
dz u u[=(t)]. di 0=
u(z t + Ina)) 2 u[z(()] + (I
dlnt = [+ (Im
(6)
From the above equation, the TS generator acting on neural responses can be defined as,
dz 0 gN = dt 0= ((8,2)0,,
which characterizes the tangential direction of the TS effect neural responses. Note that Ôu cannot be simplified into 8' = td/ dt by canceling dz and Òz. This is because Iu only scales the ID "time" manifold 2() in the digeedimensional neural response u[2(t)), whereas Dd scales every neuron's responses. TS operators with arbitrary a can be derived by composing many infinitesimal scaling transformations,
lim (1+ In -ttmmgou Iu exp (In gw).
(a lim [Su(at/))"==
(8)
x-+00
Eq 89
g is also compatible with a negative a by using the generalized logarithmic function (see Eq. To find the dynamics of the scaled neural responses, taking the time derivative of the operator,
(Ina
(Ina)"
(a) = 8 exp (Ina 9.)
Σ
0.9
(1+ + g.) (0r=)),.
n!
n!
7
= exp[(Ina)(1 +1
(0) = (a,(a)(0)2),,,
where we utilize the result that 0.g." = (1+ + (0.z)) (see Eq. $13 in SI), And then the dynamics of the scaled equivariant neural sequence is,
0t)(2(t)] = (a) u[[(()] = =
a (t)) u[=(t))- (10)
Scaling the neural dynamics will modulate its time derivative by a, and the right-hand side (RHS) of the scaled dynamics (Eq. 10p depends on the scaled =(1)) rather than the original =(1). Although the TS effect the dynamics can be also derived via the chain rule (e.g., (20),, deriving such effect
from Lie group give us more insight on the algebraic structure of TS and its circuit representation.
3.3 A concrete neural representation of internal "time"
We next define a concrete neural representation of internal "time" manifold based experimental observations. We consider parametric representation where each neuron is selective for the internal value, and denote 2y as the preferred value of the j-th neuron which implies the neuron will reach its peak firing rate when =t) = z. Moreover, we consider the preferences of neurons, (x, ) )= are uniformly distributed along the z manifold to simplify the math analysis. Since the population response has bump-profile (Fig. [BB top). we model the mean firing rate (tuning) of j-th neuron as a Gaussian function over 2tt), a widely used approach neural coding studies (e.g., 30))
úg[=(() A., exp(-(xr,
(()))//4a))
(11)
where Au is the peak firing rate, and a the tuning width. Note that ū, regards as the mean neuronal response, in distinguish with the instantancous uy (t) which may deviate from the mean response. Since Lie group operators act on continuous functions (Eq. 5). to simplify the math analysis, we treat [()) continuous function corresponding to infinite number of neurons in the population (N + oo), and all neurons' preferences become a continuous variable, i.e, Tj. + x. Hence, the mean population firing rate of all neurons becomes continuous function of the =(t).
(ūg {ū,(=(00) u{=(()) = A,, expl-(e- =(1))² /4
(12)
4 A temporal scaling equivariant recurrent neural circuit model
We propose biologically plausible recurrent circuit dynamics with fixed connectivity to flexibly generate TS equivariant neural sequences v[[(()). and study how the circuit can be controlled to scale the neural sequence. Specifically, the requirements specified by the TS group and neural
representations (Eqs. 5 l2) will be used as objectives of the proposed functional recurrent circuit.
5

---

4.1 The recurrent circuit dynamics
First, we consider a special case of freezing the internal "time" by applying S(O), and hence 2(t) = 2(fo) as the initial value. This can be regarded as, e.g., the arm position is fixed over time. Then the mean, stationary neuronal responses the recurrent circuit given the =(1)) need to match the Gaussian profile as indicated by Eq. (12). Moreover, since the initial value =(4o) is arbitrary (e.g., the arm location can stable at any location any time), stationary neural responses are required to be stable at any value of z. This implies that the recurrent circuit dynamics should have a family of stationary states (attractors) along the z manifold. These kinds of networks are called continuous attractor networks (CANs), which have been widely used to explain the neural coding of continuous features (e.g.. 31 ). We firstly consider an autonomous CAN dynamics as follows 34/
T0,ul,, 1) -u(x,t) + bp] f J(2,2)) ,t)d''
(13a)
[u(z, t)} r(z, t) = G[u(x, 1 kp S(u [u(x', t) dz'
(13b)
where u(x, 1) is the synaptic input received by the excitatory neuron preferring x (Fig. c blue triangle), and T is the time constant. G(-) is nonlinear activation function modeled as divisive normalization, a canonical operation in the cortex [38. where k is the global inhibition strength coming from an inhibitory neuron pool (Fig. 2 red disk). [|3 denotes rectifying the negative part followed by a square function. is the neuronal density on the x manifold. (x,xx) denotes the synaptic weight from the pre-synaptic neuron preferring x' to the post-synaptic neuron preferring æ, and is modeled as a Gaussian function over the difference x -x'l (Fig. PP).
Jxxx') = J(z x') = Jo expl-(x (1)-(xx=))///// >/2²/="")
(14)
The Jo and in Eq. (14) denote the peak recurrent weight and the connection width respectively. It can be checked that the autonomous recurrent circuit can self-consistently generate non-zero, desired Gaussian profile population responses ū(z) specified in Eq. (12.
a(r 11r-2, 2) A,, expl- 2)) /4)) ū(=),
(15)
as long as the peak recurrent weight Jo is larger than critical value Je (can be analytically solved, Eq. [S30 for details). Note that the in the above equation is a free parameter, meaning the recurrent circuit model has a continuous family of attractors over the "time" = manifold (Fig. 2 Here we use the notation ū(r z) to denote the responses generated by the concrete recurrent circuit dynamics (Eqs. 13a 13b) distinguishing with a[=( (Eq. 12 directly specified in earlier derivations.
To generate a moving neural sequence (dynamic 2tt); Fig. 2 top), one possibility is applying a time-invarinnt control input (Ix) to the recurrent circuit (Fig. AA green disk), i.e., inserting [xx into the RHS of Eq. (Ia)) (see Discussion for other mechanisms of generating moving neural sequences),
(xx) = [<x]z>)) = lo (xp - )xx)² /40]].
(16)
In is the peak controlling imput intensity, and 2,0 is the final position of the neural sequence (the final location at y-axis of Fig. DB. top). Then if the neural response is initiated at u(ar, In) ūfr 2(1))), it will move along z manifold towards Z and generates neural sequence. Fig. [2G shows a moving neural sequence generated by the recurrent circuit model when receiving such a control input.
4.2 Identify temporal scaling operators in the recurrent circuit dynamics
We pursue a theoretical understanding of how the recurrent circuit dynamics represents temporal scaling operators. Since an TS operator S., (a) is an exponential map based on the TS factor a and the TS generator 9., (Eq. 8). we specifically identify how the recurrent circuit represents and ğu.
The TS generator (Eq. 10 requires the neural dynamics only preserves perturbations along the manifold while filters out perturbations along other directions. This is because the time derivative of the neural response is only proportional to the partial derivative over the ID = manifold, i.e., setting = in Eq. (1)) yields 0.u, [[)) (0,z) ja, =([=()). To test whether this requirement can be satisfied by the proposed recurrent circuit, we performed perturbative analysis of the circuit dynamics by considering the instantaneous network state perturbed from the attactor state, i.c.,

---

A
Neuronal population reeponse
Singlo neuron responses
Dimensionality reduction
D
Forward O
Reverse (u=00)
San.
Time intenaa cn
1.8
FIuE ganti
Fire moneiii
0.25
Inout gain
Moua scale
Temperal-scalled repponses
rotato
20ma
0.80
100mms
inpul gain
Time
(ms)
Moan
Figure 3: Temporal scaling of the neural sequence in the proposed recurrent circuit by gain modulation on the time- invariant control input. (A) Left: neural sequence generated with different control input gains. Right: The actual temporal scale factor with the control input gain. (B) An example excitatory neuron's s temporal responses (top) and responses scaled over time (bottom) in forward (left) and reverse (right) conditions. Color: input gain as suggested by dots in (A). (C) The dimensionality reduction of the neuronal responses at different control input gains. (D) The time interval (absolute value) decreases with control input gain. (H) Weber's s law in the recurrent circuit where the standard deviation of the time interval is proportional to the mean interval.
u(z,t) = ū(x - =) + Su(x, t) (ignoring the time dependence on for brevity). Substituting this expression into the network dynamics (Eq yields the perturbation dynamics,
T0,ōu( () - Ōu((x, t) + J K (zz, x ==)=))(((', ,t)da',
(17)
where K(r,, <z)) = p J(x - 0")Or("" - =)/ ///(''' =)dd". Treating K(x, a' [2) as an operator acting 8u(x, its eigenvalues Xn can be calculated (descending order, Fig. 2E) [37 39].
A 1, x =1 -I- J2/Jg. A, = 2 (n 2 3).
(18)
and the (unnormalized) eigenfunction (x]|=) with the largest eigenvalue A = 1 is calculated as,
(x122) 0,ū(x= 2) -(x 2) expl-(x 2)² 2)²//n²]:
(19)
which is the partial derivative of neural response over the 2 manifold. The above analysis proves the CAN does satisfy the requirement of the TS generator approximately (Eq. [8), i.e., only preserving the perturbation along the z manifold (A₃ = 1) and removing other perturbations since their eigenvalues smaller than 1. This is unique property of the CAN which comes from the translation-invariant
recurrent weights along the z manifold (Fq. 14).
We present simple derivations to identify the TS generator in the recurrent circuit with control input (rigorous derivations, SI. Sec. 3. The control input will deviate the network state u(x, t) from the attractor state ū(z =). Hence, our theory considers the weak limit of control input (small lo), and substitute the attractor states (Eq. [5) into the circuit dynamics (Eq. 13a).
T0,ū(z 2) ~ ū(x 2) + pJ f(x =)] + [<x]200) Re [(x)
(20)
where the decaying term ūū( 22 and the recurrent input term PJ ū(x 2) = pf [xx x') F')
z)dx' will cancel each other due to the definition of attractor states (Eq. 15 Then we treat the control input (x)lzz)) as a "perturbation" of the attractor state, and decompose it using eigenfunctions,
-ū,ū(x (x - =) R 2.0) Σ., (x) v ai fi a,0.ū(v 2),
(21)
where we ignore perturbations along directions parallel to eigenfunctions fn (x|2) (n 2 2) because they will eventually vanish (Eq. T8h. The coefficient a can be obtained by computing the inner product between the left and right-hand hand side of the above dynamics with fi (x|s), e.g. 0n = (TŌ,ū(z- 2), fi (x)=)) = + 0,ū(x =).f (>]f((//)dd
a1 -0, lo (200 2) expl- /)
(22)
Combining the above two equations, the proposed recurrent circuit with the control input approxi-
mately achieves the TS equivariance: its attractor state ū(æ 2) (Eq. 15] travels along the "time" manifold in a way consistent with the scaled dynamics derived from TS group (Eq. I0, a = 1),
0, - 2) ~ (0,2)0_a(x= 2).
7

---

66 666666660
6
Time scale
Neural response a(ati]]
control Inpuct
4C
Time Reprosentation
0.5 .0.25 0.25 0.5
Controlingut gain e (decermine temporal scale)
Time (aA))
Forward urder loo00
Reverse order (axc0)
Actull Actull tcale
9
Patteen rmpresentaiin
Foodfwdd crouut,
Target
Time
uequerice
Dialt written
an(e)
M(()
0.2
0.6 Time (ecs)
0.4
0.8
0.2
Timer (wu)
0.4
0.6
Inputgana
Figure 4: Generating a sequence pattern by feedforward circuit mapping. (A-B) A proof of concept example where the feedforward circuit transforms the neural sequence in the recurrent circuit into the x and y coordinates of a hand-written sequence of the digit '6'. (C) The hand-writter sequence at different control input gains in the recurrent circuit. Color: elapsed time (D) The temporal trajectories of the hand-written sequence at forward (left) and reverser (right) order. (E) The actual temporal scale factor of hand written sequence is proportional to the control input gain.
In practice, the actual responses u(x, t) will deviate from the attractor state ū(x - 2) due to the control input and noises. The recurrent dynamics will quickly remove distortions perpendicular to the z manifold, corresponding to pull u(2; 1) back to the continuous = manifold, and then the attractor state ū(r =) will travel along the manifold which internally represents "elapsed time".
The emergence of the TS generator. A non-vanishing TS generator gu t(0,2)0, x ta,d, (Eq. 8) needs non-zero a,, which can be satisfied with non-zero offset, &oo - 2, between the final state Zoc and the initial state 2g. Therefore, the spatial offiset of control input and the initial network state on the z manifold emerges the TS generator in the recurrent circuit dynamics.
The representation of the TS factor. Scaling neural sequences over time corresponds to multiplying the internal "time" dynamics 0,z xx 01 by the scaling factor (Eq. [T).. Since ai is proportional to the control input strength To (Eq. 22 scaling the neural sequence can be realized by the gain modulation on the control input by the scaling factor ot i.e., [(xr [0)]= + a [ ) (Eq. 16). And
then the whole TS equivariant recurrent circuit dynamics becomes (* denotes the convolution over x).
Tdu(x 8) (1x, l) + pJ r(z, () 1o-o [xxppp((a
(23)
Therefore, the TS factor a determines the control input gain in the recurrent circuit. Generating sequences with different temporal scales can be realized by simple gain modulation, which is widely observed operation in neural circuits (e.g., 440 4I). Interestingly, by applying negative input gain (a < 0) the circuit dynamics can implement "time reversal". i.e., flipped sequence over time.
Simulation experiments
We simulated the proposed circuit model to test temporal scaling via manipulating the control input's gain (SI. Sec. 4.1 network simulation detail). Changing the input gain a varies neural sequences'
the input gain as predicted by our theory (Fig. 3A. right). It is noteworthy that negative gain enables the circuit to render "time reversed" neural sequence, corresponding to flipping the original sequence on the time axis (comparing heatmaps in Fig. 3 left). At the single neuron level, temporal scaling also changes the temporal speed of single neurons' responses (Fig. 3B. The single neuron's temporal responses under gains with the same polarity, but not different polarities, can be scaled and overlap well over time. Moreover, the dimensionality reduction analysis shows neural sequences under both positive and negative control input gain all evolve along the manifold and overlap perfectly (Fig. BC). These results are consistent with the experimental finding that single neurons responses might not overlap well under different time scales, but the population responses at different time scales all evolve along the same diweeesnnional manifold and overlap perfectly 277
time scales (Fig. B left), and the actual scaling factor is proportional
A characteristic of temporal scaling at the behavioral level is Weber's law, meaning the standard deviation of the time duration sequence linearly increases with the mean duration. To reproduce this phenomenon in the model, we include independent multiplicative noise in the recurrent circuit
8

---

mimicking the Poisson spike variability (inserting VrFlu(z, ())+ t) to the RHS of Eq. (23) with F the Fano factor). The mean time duration (Eq. [S34] analytical solution) decreases with the input gain implying the internal "time" representation becomes faster (Fig. P)) Meanwhile, the standard deviation of the time duration is proportional to the mean duration, consistent with Weher's law.
The recurrent circuit's population responses u/ [z{{)] can be mapped to an arbitrary, complex sequence (t) through a feedforward circuit (Eq. 3 As a proof of concept, we use a feedforward circuit (modeled as a three-layer perceptron) to transform u(²(t)) into a 2D sequence (æ: and y coordinates) of hand-written digits, e.g., digit "6" (Fig. 4 B, see details in SI. Sec. 4.3). The feedforward circuit was trained via back proppagattn by only using the neural response and the hand-writted sequence (Fig. 4B) at only one temporal scale. After training, we test whether the whole disentangled circuit can generalize the hand written "6" sequence at other time scales by manipulating the control input's gain (a in Eq. 23). Indeed, the feedforward circuit trained at one temporal scale successfully generates hand-written sequences over other time scales (Fig. (C-D). including both positive (forward sequence) and negative (reversed sequence) temporal scales. The disentangled circuit can also generate hand- written sequences of other digits once we retrain the feedforward circuit (see SI. Figures).
6 Conclusion and Discussion
The present study investigates the neural circuit mechanism of temporal scaling group equivariant representation. We propose a disentangled neural circuit that represents the pattern (content) and the time information of temporal sequences separately. The timing information is equivariantly represented by neural sequences generated in a recurrent circuit modeled as a CAN, and the sequence pattern (content) is represented in feedforward circuit that maps the generic neural sequence to arbitrary temporal sequences. Moreover, TS operators are explicitly represented as a time- inverinvari control input applied the CAN: the TS factor determines the control input gain, and the TS generator emerges by the spatial offset between the control input and network state along the continuous attractor manifold representing "time". Eventually, modulating the control input gain enables the recurrent circuit with fixed connectivity to generate sequences with different temporal scales, including "time reversal" when using a negative gain. Our study for the first time formulates the temporal scaling problem from the Lie group, and links the abstract group to a concrete, biologically plausible neural circuit. It gives us insight into the math principle underlying TS representation in neural circuits, and the proposed model may inspire new building block for equivariant representation in ML tasks.
Comparison to other works. How temporal scaling is achieved in neural circuits has been an active topic (e.g.. 2
42 A large body of earlier neural circuit modeling studies investigated this question via training a recurrent network, e.g., a reservoir network, to represent different time scales (e.g.. the underlying math principle still lacking. contrast, the present paper analytically derives a recurrent circuit implementation of TS equivariant representation, and the explicit representation of US group operators. Moreover, our hand-crafted circuit model generalizes over temporal scales well, which outperforms recent neural circuit models where the trained network cannot generalize (extrapolate) well to temporal scales beyond the range in the training set [T 27 In addition, previous studies considered generating a moving neural sequence by adding an anti- aati-symmetric recurrent weight component in the recurrent network (e.g.. [45 ), whose magnitude increases with temporal speed (or TS factor) [37 47). Nonetheless, the TS in the brain happens in short time scales (milliseconds to a second), which is too short to modulate recurrent weights. contrast, the control input gain in the present study can be quickly modulated in short time scales. Last, previous studies proposed TS covariant networks, inspired by the fact that the width of the temporal field of hippocampal time cells covaries (increases) with the preferred elapsed time (c.g.. (22). In comparison, the TS covariant networks aim to achieve TS invariant representation that is beneficial for recognizing sequences with different temporal scales, while our network model is flexibly generating sequences with different temporal scales. When inverting the information flow and introducing a feedback loop between the control input and the recurrent network, model also has the potential recognize (infer) the input sequences and their temporal scaling factors.
6
9 13
14
6 9 14). Although the trained networks achieve the TS representation,
Equivariant representation is an active topic in ML research and lots of equivariant neural networks have been proposed (e.g., (44533)) Nonetheless, to our best knowledge, previous MI. studies exclusively investigated the equivariance under group transformations in spatial domain rather than

---

temporal domain. Moreover, there have not been many ML studies investigating recurrent network mechanisms of equivariance while most of them studied feedforward equivariant networks. Timing or TS representation in MI. models typically uses mechanism called time warping, which relies on a pacemaker (central clock) producing sine waves with different frequencies to represent times. However, the pacemaker mechanism doesn't t exist in the nervous system representing time scales ranging from the order of milliseconds to seconds [3] There are recent ML studies investigated temporal scaling recurrent networks [20], where they directly modulated the time constant of neural dynamics (i.e., T in Eq. Taa.. However, the time constant of real neurons hardly changes due to biophysical properties. And our way of achieving TS by gain modulation of the amplitude of the time-invariam control input is novel.
Limitations of the model. In order to gain theoretical understanding of the TS equivariant representation, we propose hand-crafted recurrent circuit model rather than training a neural network model as in previous studies [9 14 48 Moreover, we consider simple model whose recurrent connections do not have random components (Eq. 14). interesting to study how a TS equivariant recurrent network can be trained in unsupervised way. In addition, previously trained network models used reservoir networks containing strong random connections with disordered (chaotic) dynamics [9 4S). Although chaos provides a rich dynamical repertoire to facilitate learning, it impairs the robustness of time representation [9]. Since this study does not consider learning, ignoring the random connections will significantly help identify how structured recurrent connections achieve TS equivariance and represent operators. Statistically, a characteristic of chaotic spiking networks is the Poisson spiking variability [54] [55], whose statistical effect is equivalent to the injected multiplicative noise in our model. Furthermore, as a proof of concept, the feedforward circuit used in our model is simple and needs to be retrained to generate new sequence pattern, e.g.. other digits' written sequence, which is unlikely in the brain. One way to improve this is by using compositional model (e.g.. [56 57 which can be flexibly modulated to change its represented pattern.
Extensions of the model. The proposed neural circuit model has the potential to explain other brain functions. For example, our model may be used to explain the memory formation in the hippocampus, where it is believed that hippocampal theta oscillation compresses (speeds up) sensory input sequences by about 10 times and is beneficial for memory formation [58]. It is possible that the theta oscillation shares similar computational mechanism with the proposed temporal scaling equivariant recurrent network. At last, experiments found "time" cells in the hippocampus whose temporal firing width (duration) is proportional to their peak timing, i.e., "time" cell fires later will fire longer, and time cells' preferred peak timing is log-scaled [59]. Such relation doesn't 't exist in the proposed model, and neurons in the current model can be "deployed" in the logarithmic space of manifold to reproduce this effect. All of these will form our future research.
Acknowledgments
Y. W. is supported by NSF DMS--M15512 S. W is supported by National Science and Technology Innovation 2030 Major Program (No. 2021ZD0200204, Si Wu).
10

---

References
[1] Mare W Howard. Temporal and spatial context in the mind and brain. Current opinion in behavioral sciences, 17:14-19, 2017.
[2] Evan D Remington, Seth W Egger, Devika Narain, Jing Wang, and Mehrdad Jazayeri. A dynam- ical systems perspective on flexible motor timing. Trends in cognitive sciences, 22(10):938-99) -52, 2018.
[3] Albert Tsao, Aryana Yousefzadeh, Warren H Meck, May-Britt Moser, and Edvard Moser. The neural bases for timing of durations. Nature Reviews Neuroscience, 23(11):646-66) 2022.
[4] Shanglin Zhou and Dean V Buonomano. Neural population clocks: Encoding time in dynamic patterns of neural activity. Behavioral Neuroscience, 2022.
[5] Joseph J Paton and Dean V Buonomano. The neural basis of timing: distributed mechanisms for diverse functions. Neuron, 9844):687-705, 2018.
[6] Jing Wang, Devika Narain, Eghbal A Hosseini, and Mehrdad Jazayeri. Flexible timing by temporal scaling of cortical responses. Nature neuroscience, 21:102- 110, 2018.
[7] Nicolas Meirhaeghe, Hansem Sohn, and Mehrdad Jazayeri. A precise and adaptive neural mechanism for predictive temporal processing in the frontal cortex. Neuron, 109(18):2995330)
2021.
[8] Shreya Saxena, Abigail A Russo, John Cunningham, and Mark M Churchland. Motor cortex activity across movement speeds is predicted by network-level strategies for generating muscle activity. eLife, 1 1::66620,, 2022.
[9] Rodrigo Laje and Dean V Buonomano. Robust timing and motor patterns by taming chaos in recurrent neural networks. Nature neuroscience, (177)925-9333) 2013.
[10] Nicholas F Hardy, Vishwa Goudar, Juan L Romero-Sosa, and Dean V Buonomano. A model of temporal scaling correctly predicts that motor timing improves with speed. Nature communica- lions, (1):4733, 2018.
[11] Shanglin Zhou, Sotiris C Masmanidis, and Dean V Buonomano. Encoding time in neu- ral dynamic regimes with distinct computational tradeoffs. PLOS Computational Biology, 2022.
[12] Vishwa Goudar and Dean V Buonomano. Encoding sensory and motor patterns as time-invariant trajectories in recurrent neural networks. eLife, 7::31134, March 2018.
[13] Zedong Bi and Changsong Zhou. Understanding the computation of time using neural network models. Proceedings of the National Academy of Sciences, 11530-9:0000000040 2020.
[14] Manuel Beiran, Nicolas Meirhaeghe, Hansem Sohn, Mehrdad Jazayeri, and Srdjan Ostojic. Parametric control of flexible timing through low-dimensional neural manifolds. Neuron, 2023.
[15] Niru Maheswaranathan, Alex Williams, Matthew Golub, Surya Ganguli, and David Sussillo. Universality and individuality in neural dynamics across large populations of recurrent networks. Advances neural information processing systems, 32, 2019.
[16] Guo Guo-Zheng Sun, Hsing-Hen Chen, and Yee-Chun Lee. Time warping invariant neural networks. Advances in neural information processing systems, 5, 1992.
[17] Aleksandar Vakanski, Iraj Mantegh, Andrew Irish, and Farrokh Sanai--hharie Trajectory learning for robot programming by demonstration using hidden markov model and dynamic time warping. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 1234)::039-110 1052, 2012.
[18) Nazita Taghavi, Jacob Berdichevsky, Namrata Balakrishnan, Karla C Welch, Sumit Kumar Das, and Dan O Popa. Online dynamic time warping algorithm for human-robot imitation. In 202 IFFE International Conference on Robotics and Automation (ICRA), pages 3843-3849. IEEE, 2.21.
11

---

[19] Suhas Lohit, Qiao Wang, and Pavan Turaga. Temporal transformer networks: Joint learning of invariant and discriminative time warping. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12426-12435, 2019.
[20] Corentin Tallec and Yann Ollivier. Can recurrent neural networks warp time? arXiv preprini 1EXXv:8804.111888 2018.
[21] Robert Gütig and Haim Sompolinsky. Time-warp-a invariant neuronal processing. PLoS biology, 7(7):e 1000141, 2009.
[22] Brandon G Jacques, Zoran Tiganj, Aakash Sarkar, Mare Howard, and Per Sederberg. A deep convolutional neural network that is invariant to time rescaling. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning. volume 162 of Proceedings of Machine Learning Research, pages 9729-9778. PMLR, 17-23 Jul 2022.
[23] James Heys and Daniel A. Dombeck, Evidence for a subcircuit in medial entorhinal cortex representing clapsed time during immobility. Nature Neuroscience, 21(11)11544-1582 2018.
[24] Howard Eichenbaum. Time cells in the hippocampus: new dimension for mapping memories. Nature Reviews Neuroxcience, 15(11):732-744, 2014.
[25] Gustavo BM Mello, Sofia Soares, and Joseph J Paton. A scalable population code for time in the striatum. Current Biology, 25(9):1 2015.
[26] Galen F Lynch, Tatsuo S Okubo, Alexander Hanuschkin, Richard HR Hahnloser, and Michale S Fee. Rhythmic continuous-time coding in the songbird analog of vocal motor cortex. Neuron, 90(4):877-892, 2016.
[27] Arkarup Banerjee, Feng Chen, Shaul Druckmann, and Michael A Long. Neural dynamics in the rodent motor cortex enables flexible control of vocal timing. bioRxiv, pages 2023-01, 2023.
[28] Peter Dayan and Laurence F Abbott. Theoretical neuroscience, volume 806. Cambridge, MA: MIT Press, 2001.
[29] Alexandre Pouget, Peter Dayan, and Richard S Zemel. Inference and computation with population codes. Annual Review of Neuroscience, 26 (0(11)331-410 2003.
[30] Wei Ma, Jeffrey M Beck, Peter Latham, and Alexandre Pouget. Bayesian inference with probabilistic population codes. Nature Nearoscience, 911):1432-1438, 2006.
[31] Shun-ichi Amari. Dynamics of pattern formation in lateral-inhibition type neural fields. Biolog- ical cybernetics, (7)))777-87 1977.
[32] Si Wu, KY Michael Wong, CC Alan Fung, Yuanyuan Mi, and Wenhao Zhang. Continuous attractor neural networks: candidate of a canonical model for neural information representation. F1000Research, 5, 2016.
[33] James Knierim and Kechen Zhang. Attractor dynamics of spatially correlated neural activity in the limbic system. Annual review of neuroscience, 35:267-285, 2012.
[34] Sophie Deneve, Peter Latham, and Alexandre Pouget. Reading population codes: neural implementation of ideal observers. Nature Neuroscience, 2(8):740-745, 1999.
[35] Si Wu, Kosuke Hamaguchi, and Shun-ichi Amari. Dynamies and computation of continuous attractors. Neural Computation, 20(4):994-1025, 2008.
[36] Wen-Hao Zhang, Tai Sing Lee, Brent Doiron, and Si Wu. Distributed sampling- basedd bayesian inference in coupled neural circuits. bioRxiv, 2020.
[37] Wenhao Zhang, Ying Nian Wu, and Si Wu. Translation-equivarian representation in recurrent networks with a continuous manifold of attractors. Advances in Neural Information Processing Systems, 35:15770-157888, 2022.
12

---

[38] Matteo Carandini and David Heeger. Normalization canonical neural computation. Nature Reviews Neuroscience, 13(1):51 62, 2012.
[39] C. C Alan Fung, K. Y. Michael Wong, and Si Wu. A moving bump in a continuous manifold: A comprehensive study of the tracking dynamics of continuous attractor neural networks. Neural Computation, 22(3):752-799 2010.
[40] Emilio Salinas and Terrence J Sejnowski. Book review: gain modulation in the central nervous system: where behavior, neurophysiology. and computation meet. The Neuroscientist, 7(5):430- 440, 2001.
[41] Frances Chance, Larry F Abbott, and Alex D Reyes. Gain modulation from background synaptic input. Neuron, 35(4):773-782, 2002.
[42] Yue Liu, Zoran Tiganj, Michael E Hasselmo, and Mare W Howard. A neural microcircuit model for scalable iave-invariant representation of time. Hippocampus, 29(3):260-25) 2-440 2019.
[43] Shanglin Zhou, Sotiris Masmanidis, and Dean V Buonomano. Neural sequences as an optimal dynamical regime for the readout of time. Neuron, 108(4):651-658 2020.. 2020.
[44] Seth W Egger. Nhat M Le, and Mehrdad Jazayeri. A neural eircuit model for human sensorimo- tor timing, Nature communicationss 1111:39933, 2020.
[45] Kechen Zhang. Representation of spatial orientation by the intrinsic dynamics of the head- direction cell ensemble: a theory. The Journal of Neuroscience, 16(6):2112-2126 1996.
[46] NICHOLAS F Hardy and Dean V Buonomano. Encoding time in feedforward trajectories of a recurrent neural network model. Neural computation, 30(2):378-33) 2018.
[47] John B Issa and Kechen Zhang. Universal conditions for exact path integration neural systems. Proceedings of the National Academy of Sciences, 109(17):6716-6720 2012.
[48] Kanaka Rajan, Christopher D Harvey, and David W Tank. Recurrent network models of sequence generation and memory. Neuron, 10(1):128-142, 2016.
[49] Taco Cohen and Max Welling. Learning the irreducible representations of commutative lic groups. In International Conference on Machine Learning, pages 1755-1763. PMLR, 2014.
[50] Taco Cohen and Max Welling. Group equivariant convolutional networks. In International conference on machine learning, pages 2990-2999. PMLR, 2016.
[51] Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems, 29, 2016.
[52] Taco S Cohen, Mario Geiger, Jonas Köhler, and Max Welling. Spherical cnns. arXiv preprint 10X 2018.
[53] Risi Kondor and Shubhendu Trivedi. On the generalization of equivariance and convolution in neural networks to the action of compact groups. In International Conference Machine Learning, pages 2747-2755. PMLR, 2018.
[54] C van Vreeswijk and Haim Sompolinsky. Chaotic balanced state in a model of cortical circuits. Neural computation, 1321-113,,,,,,, 1998.
[55] Alfonso Renart, Jaime De Rocha, Peter Bartho, Liad Hollender, Néstor Parga, Alex Reyes, and Kenneth D Harris. The asynchronous state in cortical circuits. science, 327(5965):587-590 2010.
[56] Stuart Geman, Daniel F Potter, and Zhiyi Chi. Composition systems. Quarterly of Applied Mathematics, 60(4):707-736, 2002.
[57] Tai Sing Lee. The visual system's internal model of the world. Proceedings of the IEFE, 13399)11359-1377 2015.
13

---

[58] William E Skaggs. Relations between the theta rhythm and activity patterns of hippocampal neurons. The University of Arizona, 1995.
[59] Rui Cao, John H Bladon, Stephen Charczynski, Michael Hasselmo, and Marc Howard. Internally generated time in the rodent hippocampus is logarithmically compressed. Elife, 11:e75353, 2022.
14


Reviews:
[{"review_id": "evmJugin22", "reviewer": "Reviewer_tVWd", "summary": "The authors present a framework to analyze and realize temporal scaling in networks. Studies in neuroscience have thought about how the brain achieves outputs at different speeds. Here, the authors present a theoretical analysis of temporal scaling group operators, and then posit a disentangled neural circuit that is a recurrent neural network (RNN) feeding to a feedforward network to produce the actual output. The RNN takes in a time scale control input and generates the temporal scaling, while the feedforward circuit produces the pattern for the task. The authors consider an RNN with fixed weights. Finally, simulation results are shown to produce handwritten digits.", "questions": "Please see weaknesses above.", "limitations": "The authors have adequately addressed limitations of their work.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Originality: While the authors use the previously developed continuous attractor network (CAN), the analyses presented are original. A recurrent circuit that takes in a temporal scaling input and outputs signals at different frequencies has been proposed before in Maheswaranathan, Williams, et al., 2019, which is not cited. \n\nQuality: The paper considers several neuroscience studies and attempts to distill the way neurons encode time from a theoretical perspective, as well as proposes a concrete realization of such a recurrent circuit. Although the concrete nature of the representation may have its advantages to be able to analyze the network, please see weaknesses.\n\nClarity: The presentation of the paper is very clear. \n\nSignificance: The authors formulate a temporal scaling group, but it is unclear whether this leads to additional insight into networks that are able to perform temporal scaling. The recurrent network that the authors posit is a very specific one, where the neurons are a specific function of the scaling, and thus this limits the use of this network (since it cannot be learnt / trained).", "weaknesses": "The use of a fixed neural representation of the internal 'time' manifold may be a limitation of the study.\n\nThe authors claim that \"when generating movements with different time scales, a recurrent circuit's responses u(t) evolve along the same low-dimensional manifold with different speeds\", and \"overlap perfectly\". However, in most studies, the responses are in fact separated in state space. Also see Saxena, Russo, et al., 2022. In fact, even in the proposed paper, it seems that the neurons' responses are separated in PC space (Fig 3C). Please clarify.\n\nThe simulation experiments are extremely limited - the authors only consider simple x and y coordinates of hand-written digits, and only show their results for the digit '6'. However, they claim that the responses can be mapped to an arbitrary, complex sequence. Please expand.\n\nIt is unclear what we learned from this paper. The theoretical analysis formulates a temporal scaling group, but the takeaways are limited. The circuit proposed by the authors can implement temporal scaling, but so can other circuits (since the authors consider a hand-crafted network). It would be helpful if the authors can extend their results to arbitrary or trained networks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the reviewer’s comments, especially the appreciation of our theoretical derivations in identifying how temporal scaling group operators are represented in the recurrent network dynamics.\n\n> “Originality: While the authors … which is not cited.”\n\nThank you for directing us to this paper. We are happy to cite it in our revised manuscript. It aptly supports and provides motivation for our network model that receives a temporal scaling (TS) control input. Notably, we've also observed that the dimensionality reduction analysis in our network aligns closely with the phenomena presented in Maheswaranathan, 2019.\n\nWhile the continuous attractor network stands as a canonical model in neuroscience, its application to TS remains largely unexplored, as acknowledged by the Reviewer ZBA5. Numerous previous studies have delved into training a network to achieve TS (highlighted on line 26 of our manuscript). However, there remains a large gap in the theoretical analysis of the dynamics of such recurrent networks. One of the standout contributions of our study is the theoretical and analytical insight into how the abstract TS group operators manifest in recurrent circuit dynamics (as discussed in lines 231 – 243), which has not been documented before.\n\nWe believe that these theoretical insights into the abstract group representation within concrete network dynamics will greatly enhance our comprehension of the algebraic structure of network dynamics. Further, it sets the foundation for the development, learning, and analysis of network representations for even more intricate group structures in future research.\n\n> “The use of a fixed neural representation… a limitation of the study.”\n\nUnderstanding the general principle of \"time\" representation in neural circuits remains challenging. It's not yet fully clear whether neural representations of \"time\" are fixed or adaptive based on context. Our proposed neural circuit model is rooted in a recurring observation: the presence of a one-dimensional \"time\" manifold in the neural population response (Refs. [2-7]). This observation guided our construction of a neural population and its dynamics around this one-dimensional \"time\" manifold.\nOne notable advantage of having a fixed \"time\" representation is the ability to disentangle \"time\" from sequence patterns (Fig. 2A), which streamlines the joint representation of sequence speeds and patterns.\nWhile the contextually flexible neural representation of \"time\" is enticing, modeling such a neural circuit lacks robust experimental backing. Tackling this challenge will be an exciting endeavor in the future, but it is likely beyond the scope of our current research.\n\n> “The authors claim … separated in PC space (Fig 3C).”\n\nThank you for mentioning Saxena, et al., 2022. We found our network model exhibits **consistent** results with this paper and we are happy to cite it in our revised manuscript to support our model. \nThe “perfect overlap” in the original paper means the trajectories will overlap perfectly only in the **“time” manifold**, as shown in the bottom panel in Fig. 3C. This result is consistent with the result reported in Fig. 5C in Ref. 6, and is similar with the Fig. 3d in Saxena, et al., 2022 (our “time” manifold corresponds to the gray circles in the Fig. 3d). Indeed, in the proposed recurrent circuit model, the trajectories with different speeds are **separated** along other dimensions. Fig. 3C, top panel shows the trajectories are separated along the direction of the input gain $\\alpha$. We will revise the text to make it clear.\n\n> “The simulation experiments are extremely limited…”\n\nIn the disentangled neural circuit model, the feedforward circuit maps the sequential activities of the RNN to a specific temporal sequence.  We used handwritten digits to demonstrate this ability, which was widely used in neuroscience studies (Ref. 10-11).  Due to the page limit, we only presented the sequence of digit ‘6’ in the main text.  We do have the results of generating the sequence of other digits from 0 to 10, which were presented in the supplementary Fig.S3. Nonetheless, to use the proposed disentangled circuit model to generate new sequence patterns, e.g., other digits, the feedforward circuit needs to be retrained (see the discussion in the original manuscript, lines 328-331). We admit that the feedforward circuit might be over-simplified, since our main focus is on the TS in the recurrent circuit. Looking ahead, we are keen on enhancing the feedforward circuit's capabilities, allowing it greater flexibility in producing diverse sequence patterns, as suggested in lines 330-332 of the manuscript.\n\n> “It is unclear what we learned from this paper ...”\n\nPlease see *global reply* for the contributions of our work.\nA novel takeaway is the **analytical** results of the TS group representation in the nonlinear recurrent neural circuit dynamics (lines 231-243). The TS recurrent circuit models have been developed and learned (ref. [2, 6-13], and Maheswaranathan, et al., 2019), but no previous result can yield an analytical result identifying the representation of the abstract group operators in the circuit dynamics. Moreover,  our work proposes a continuous attractor network to implement the TS group for the first time, which is also confirmed by the Reviewer ZBA5. In contrast, most previous studies trained chaotic RNNs to implement TS transformations.\n\nThe inclusion of a hand-crafted network model in the present study is to obtain analytical results, and otherwise, it is unlikely to get. We are interested in using the theoretical insight obtained from the present study to design a new paradigm to train RNNs to implement TS."}, {"review_id": "yUhnmSzXci", "reviewer": "Reviewer_ZBA5", "summary": "Using Lie group algebra, this paper derives the general conditions for a network that evolves through a series of states at a continuous set of possible rates.  This would be useful for temporal rescaling, as is frequently observed in motor generation (imagine playing a melody at a different rate than it was learned).  These considerations are applied to develop a continuous attractor network that is time scale equivariant.  Simulations implement a continuous attractor network that drives a set of output neurons to draw a spatial pattern at different rates.", "questions": "What could be the benefit of a time scale-covariant network?  Lindeberg and colleagues (e.g., Lindeberg & Fagestrom, 1996) have studied the conditions for scale-covariant temporal receptive fields for many years.  What is the relationship between this work and Jacques et al., (2022; ICML)?\n\nWould this scale-equivariant network be useful for memory?\n\nIt's not clear how useful this approach is generally.  How does the number of patterns that can be generated using a single CAN scale?  What's the architecture one would need to retrieve a large number of such motor patterns?  \n\n", "limitations": "N/A", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "The rigor of this approach is admirable.  \n\nThe inclusion of a continuous attractor network is a nice addition. I'm not aware of a model addressing sequential activation that makes use of this mechanism.\n\nThis approach is vastly superior to conventional RNNs, which are ill-suited as a model of neural sequence coding or generation.", "weaknesses": "The dependence of this approach on a common zero to the scale ($t_{start}$) is not given sufficient weight.  Reversal of time in this approach (negative alpha, as in fig 1b) requires exchange of the initial condition as well.  \n\nGenerally speaking, the claims to novelty are overstated.  It is not correct to say that using gain to change the rate of a temporal representation has ``never [been] used in ML studies.'' The entire field of dynamic time warping (e.g., Sakoe and Chiba, 1978) is clearly ML (although not a neural network).  Gutig & Sompolinsky (2009, PLoS Bio) studied pattern classification in a network that exactly modulates the rate of time progression via gain.  More recently there was a workshop paper at NeurIPS last year addressing gain as a control signal in deep networks:\nhttps://memari-workshop.github.io/papers/paper_42.pdf\n\nThe Weber law is introduced via a fairly ad hoc mechanism.  Weber law emerges naturally from requiring scale-covariance, rather than scale equivariance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We deeply appreciate your positive review and insightful comments.\n\n> “The dependence of this approach on a common zero to the scale ($t_{start}$) ...\"\n\nOur analysis implicitly assumes that the sequences with different speeds start from the same $t_{start}$, and time reversal requires exchanging start timing (see details of $t_{start}$ in Eq. S11). The focus of the present study is the temporal scaling (TS) of sequences rather than temporal alignment between sequences, and the start timing is not critical in temporal scaling. Therefore, we suppress $t_{start}$ in equations to simplify notations. We will explain this more clearly in the revision.\n\n> “Generally speaking, the claims to novelty are overstated...”\n\nPlease see *global reply* for our statement about gain-modulation. \n\nThanks for highlighting those previous studies. We're keen to offer comparisons to underscore our contributions in the revision. Our study stands out for its **theoretical** and **analytical** insight into the representation of TS groups within recurrent network dynamics, which was recognized by Reviewer aiKz. While leveraging gain-modulation to manage TS has precedent (Ref. [2, 6-13], and Maheswaranathan 2019 mentioned by Reviewer tVWd), our analytical approach in identifying the TS group representation is novel. On a closer examination of the papers referenced by the reviewer, we find:\n- Sakoe and Chiba (1978) introduced a warping function to synchronize sequences with varied temporal scales. But as the reviewer rightly pointed out, it doesn't employ neural networks. It is hard to conclude that the gain modulation in the present study was also used in that paper. \n- Gutig 2009 focused on a single neuron mechanism of TS, where the input firing rate modulated the effective time constant of conductance-based integration-and-fire neurons. Even though there's a resemblance between their modulating input firing rate and our gain-modulation, the distinction is that we focused on the recurrent network mechanism of TS, unlike the single neuron mechanism.\n- The NeurIPS Workshop paper interprets the output of the inverse Laplacian transform as neural responses in the network's hidden layer (as illustrated in Eq. 1 and Fig. 1a). They then couple a factor, $\\alpha(t)$, with the decaying term, $-sF(s,t)$, which modifies the effective time constant. This is fundamentally different from our gain modulation approach.\n\nImportantly, our research represents a generative model for sequence generation, whereas the mentioned papers focused on the inference of discerning time-warped sequences.\n\n> “The Weber law is introduced via a fairly ad hoc mechanism...”\n\nWe agree that Weber’s law, referring to the std of time interval proportional to the mean, satisfies the scale-covariance. The present study does not derive Weber’s law from a normative way because it is not the main focus of the current study. Instead, we only use Weber’s law as a way to demonstrate the biological plausibility of the proposed circuit model, in that Weber’s law has been widely observed in TS experiments in neuroscience research. At the neural circuit level, we found that Weber’s law can be only realized by including the Poisson variability referred to as neuronal spiking variability  (see lines 258-264 in the manuscript), while it will deviate if the injected noise is additive (not presented in the manuscript).\n\n> “What could be the benefit of a time scale-covariant network?...”\n\nWe are happy to discuss and compare our study with these papers in the revised manuscript. The time scale-covariant networks mentioned by the reviewer focused on the temporal scale-invariant representation of sequences (e.g., Fig.3C,H in  Jacques 2022), which is beneficial for **recognizing** sequences with different time scales. In contrast, our model aims to investigate the recurrent network mechanism of flexibly **generating** sequences with different temporal scales. For the proposed network model, when inverting the information flow and introducing a feedback loop between the context input and the recurrent network, the proposed network has the potential to recognize (infer) the temporal scaling factors of input sequences.\n\n> “Would this scale-equivariant network be useful for memory?”\n\nThank you for motivating us to think of another potential function of our proposed neural circuit model, and memory is an important function of the brain. This is an open question and here we provide our thoughts that are yet to be verified by future experiments. The short answer is yes. The hippocampus is essential in forming new memories. A characteristic of hippocampal responses is the theta oscillation, where it was hypothesized theta sequences compress (speed up) the sensory input sequence by about 10 times (Skaggs, J. Neurosci., 1995). It was believed that compressing the temporal sequence by theta oscillation is beneficial for memory formation, and it is possible that the theta oscillation shares a similar computational mechanism with the temporal scale equivariant network. We will discuss this issue in our revised manuscript.\n\n> “It's not clear how useful this approach is generally...”\n\nIn the disentangled circuit, the CAN is a \"clock\" generating a _stereotypical_ neural sequence, and the feedforward circuit represents the pattern. Therefore the number of patterns that can be generated by the CAN is only one, and the number of patterns that can be generated by the whole disentangled network depends on the complexity of the feedforward circuit. \n\nSince the present study focuses on TS in a recurrent network, the feedforward circuit is a toy model where a trained feedforward circuit only generates one pattern, and it needs to be **retrained** to generate another (see line 275). By increasing the complexity of feedforward circuits, e.g., introducing a \"gate\" to modulate the **functional** feedforward connectivity, the network can generate many sequence patterns (see discussions in lines 330-332).\n"}, {"review_id": "C8bmyIiPVk", "reviewer": "Reviewer_aiKz", "summary": "This paper introduces a continuous attractor neural network that has a simple way of generating a specific sequence of outputs at a speed tunable by changing the gain of a constant (but neuron-dependent) input. The authors show how this can be used in conjunction with a multi-layer perceptron to generate arbitrary sequence at speeds that can be tuned by controlling the gain. A negative gain can be used to reverse the sequences. The authors analyze their model from the perspective of the Lie group of temporal scaling.", "questions": "Once in the text (below eq. (21)) and then again in the Appendix, the claim is made that the eigenvectors of the operator $K(x,x'\\vert z)$ from eq. (17) are orthogonal. This does not seem obvious (or even possible?), since the operator is, as far as I can tell, not symmetric. Am I missing something?\n\nOther comments:\n* in Figure 1B, I suggest putting the original on top\n* rotating panel A of Figure 1 by 90º could also help so that it aligns with panel B\n* in line 102: reference [23] is not about birds, but rodents\n* the notation $[u(x,t)]_+^2$ is used – I'm assuming the '+' stands for rectification, but this seems superfluous since the quantity is squared anyway; if the rectification is indeed not necessary, I suggest taking the '+' sign out to avoid confusion\n* caption of Figure 4: \"The temporal trajectories of the hand-written sequence at forward *(top)* and reverse *(bottom)* order\" -- should be *(left)* and *(right)*\n* on lines 210–211: it seems like the authors are referring to an approximation where only the component along the top eigenvector is kept; if so, this should be made clear", "limitations": "The authors have adequately discussed limitations of their work.", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "*Originality:*\nThe paper introduces a novel way of controlling the speed at which a network generates a temporal sequence by separating it into a component that is equivariant with time scaling and one that is independent of time, and simply transforms the output from the first component.\n\n*Quality:*\nThe paper uses detailed mathematical derivations as well as simulations to support its claims.\n\n*Clarity:*\nThe presentation is clear, with detailed proofs and derivations in the Appendix. The code for the simulations is also included in the Supplementary Material (which should be standard but sadly is not true for all submissions).\n\n*Significance:*\nThe work sheds more light on how brains can generate sequences at tunable speeds, which is important in the study of behaviors from song to speech.", "weaknesses": "I find that the machinery of Lie group theory is not really needed for this work and potentially makes it harder to understand. As I understand it, the point is that a continuous attractor network with a normalizing nonlinearity can be used to generate sequences with tunable speeds by simply controlling the gain of a specially tailored input. The need for Lie group theory was not very clear, beyond perhaps the simple point of using the infinitesimal version of a scaling transformation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your positive comments about the strengths, especially the analytical derivations, of our work. We will revise the manuscript accordingly based on your suggestions. Below please finds our detailed reply for your comments.\n\n> \"I find that the machinery of Lie group theory is not really needed for this work and potentially makes it harder to understand. As I understand it, the point is that a continuous attractor network with a normalizing nonlinearity can be used to generate sequences with tunable speeds by simply controlling the gain of a specially tailored input. The need for Lie group theory was not very clear, beyond perhaps the simple point of using the infinitesimal version of a scaling transformation.\"\n\nWe appreciate the reviewer's summary of our work from a network implementation perspective. Admittedly, resorting to Lie group theory might seem excessive when the objective is perceived as merely formulating a network model for temporal scaling (TS) transformations. However, the true depth and ambition of our study span a broader horizon.\n1. **Introduction of Equivariance**: Our study stands at the vanguard by rigorously introducing equivariance to delineate TS. Recently, the concept of equivariance has garnered substantial attention in the ML community (Cohen & Welling, 2016). Its applications in geometric machine learning, molecular design (Hoogeboom, 2022), fluid dynamics (Wang, 2020), and particle physics (Bogatskiy, 2020) have already been established. Consequently, leveraging Lie group theory to demystify cognitive computations promises to catalyze foundational advancements in neuroscience research.\n2. **Continuous Attractor Dynamics via Lie Group Framework**: Our research meticulously derives the continuous attractor dynamics through the prism of the Lie group framework, offering an innovative approach to TS transformations. This novelty has also been recognized and endorsed by Reviewer ZBA5.\n3. **Theoretical and Analytical Insights**: The cornerstone of our research lies in providing a robust **theoretical** and **analytical** understanding of the TS group representation within recurrent network dynamics (lines 231 – 243). It's pivotal to note that the Lie group is essential to this exploration. Through the Lie group, we distilled the TS operations on neural representations (Eqs. 7-10) and juxtaposed them with the theoretical analysis of network dynamics (Eq. 23), leading us to discern the TS group operator representation. Such a theoretical discourse on the abstract group's representation in concrete, biologically plausible network dynamics paves the way for a more profound comprehension of the **algebraic** structure of network dynamics. This perspective is instrumental when aiming to construct or understand network representations of intricate group structures in the future. To the best of our knowledge, no extant research has ventured into or achieved such profound theoretical insights into TS group representation.\n\nIn summary, while our approach might seem intricate on the surface, the depth and breadth of its insights, and the gaps it fills in the current literature affirm the significance of our methodology.\n\n> \"Once in the text (below eq. (21)) and then again in the Appendix, the claim is made that the eigenvectors of the operator $K(x,x'|z)$ from eq. (17) are orthogonal. This does not seem obvious (or even possible?), since the operator is, as far as I can tell, not symmetric. Am I missing something?\"\n\nReply: Thank you for pointing this out. The eigenfunctions are not orthogonal since the operator $K(x,x’|z)$ is not symmetric, but this does not affect our theoretical result in Eq 22. In the theoretical analysis, we used an orthonomal basis function set $v_n(x|z)$ (Eq. S22) to find the eigenfunctions $f_n(x|z)$ of $K(x,x’|z)$, and then the eigenfunctions are expressed as linear combinations of $v_n(x|z)$. Importantly, the eigenfunction of the \"time\" manifold, $f_1(x|z)$, is parallel to the basis function $v_1(x|z)$. Therfore the calculation of $a_1 = \\langle \\tau \\partial_t \\bar{u}(x-z), f_1(x|z) \\rangle$ utilizes the **orthogonal basis function set**, rather than orthogonal eigenfunctions.\n\nWe will correct the typo by removing the sentence \"since eigenfunctions $f_n(x|z)$ are orthogonal to each other\" in line 221.\n\n**Reply to comments on typos, figures and notations:**\n\n> “in Figure 1B, I suggest putting the original on top”\n> “rotating panel A of Figure 1 by 90º could also help so that it aligns with panel B”\n\nThanks for the suggestions. We will revise our manuscript accordingly.\n\n> \"in line 102: reference [23] is not about birds, but rodents”\n\nThanks for your advice.  We will revise it accordingly.\n\n> “the notation is used $[u(x,t)]_+^2$ … to avoid confusion”\n\nYes, the index $+$ denotes the rectification. It is necessary to rectify the negative $u(x,t)$ before taking the square. For example, say $u(x,t)=-1$, and $[u(x,t)]_+^2 = 0$ with rectification before square, whereas $[u(x,t)]^2 = 1$ will yield a non-zero positive output.\n\n> “caption of Figure 4: … -- should be (left) and (right)”\n\nThanks for pointing that out.  We will revise it accordingly.\n\n> “on lines 210–211: it seems like the authors are referring to an approximation where only the component along the top eigenvector is kept; if so, this should be made clear”\n\nThanks for the suggestion and we will revise the text to make it clearer. We mentioned this in line 220 when we only consider the component along the 1st eigenvector. To emphasize this approximation, we plan to revise the text in lines 109-111 as: \\\n_\"The above analysis proves the CAN does satisfy the requirement of the TS generator approximately (Eq. 8), i.e., only preserving the perturbation along the $z$ manifold $(\\lambda_1=1)$ and removing other perturbations since their eigenvalues are smaller than $1$.\"_\n"}, {"review_id": "gbONkJiwnJ", "reviewer": "Reviewer_NfgN", "summary": "Empirical data have shown the existence of neurons that represent time and the ability of neural circuits to generate temporal sequences at different speeds. The authors propose a model that takes into account these findings: a continuous attractor network that acts as a neural clock which can evolve with a speed proportional to its input, and a feedforward neural network that use this signal to generate a pattern. In particular, they propose a continuous attractor architecture that has attracting biologically plausible properties, analyze it from a Lie group perspective and show in simulation that their theoretical arguments hold in practice.", "questions": "--", "limitations": "Yes", "rating": 7, "confidence": 2, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The proposed approach makes a lot of intuitive sense (I am actually surprised that this approach has not been proposed before) and match experimental data. The analysis is comprehensive and illustrated with well designed figures that help getting the gist of it.", "weaknesses": "What does the Lie group perspective bring? To the best of my understanding, the Lie group perspective is not used in the analysis and requires introducing additional jargon that makes the paper harder to read. Can the authors clarify the interest of the TS group? \n\nHow novel is the idea that input gain modulates moving speed of the bump? From my understanding as an outsider of the field, similar ideas have been introduced to model head direction cells. The authors do not seem to include such references. Is it a misunderstanding from my side or are some references missing?\n\nI’m happy to increase my score once those questions are answered.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful to your positive review and honored to answer your questions.\n\n> “What does the Lie group perspective bring? To the best of my understanding, the Lie group perspective is not used in the analysis and requires introducing additional jargon that makes the paper harder to read. Can the authors clarify the interest of the TS group? ”\n\nThank you for highlighting this aspect of our research. While we indeed propose a recurrent network for the TS transformation, the true essence and significance of our study lie elsewhere. We delve deeper to furnish a **theoretical** and **analytical** framework, elucidating the TS group representation within recurrent network dynamics (lines 231 – 243).\nThe Lie group's role is paramount here. It's not just an ancillary component but the theoretical basis that enables us to derive the TS operations on neural representations, as illustrated by Eqs. 7-10. By comparing this with our theoretical analysis of network dynamics (as per Eq. 23), we're able to pinpoint the TS group operator representation. Such insights, which bridge abstract group representation with concrete, biologically plausible recurrent network dynamics, immensely further our grasp of the **algebraic structure** within these dynamics. Furthermore, this understanding will be pivotal as we investigate the neural circuit representations with even more intricate group structures in future research.\nWe'd like to underline that, to our knowledge, our study pioneers this unique theoretical insight into TS group representation—filling a significant gap in the existing literature.\n\n> “How novel is the idea that input gain modulates moving speed of the bump? From my understanding as an outsider of the field, similar ideas have been introduced to model head direction cells. The authors do not seem to include such references. Is it a misunderstanding from my side or are some references missing?”\n\nWhile prior neuroscience research has employed gain modulation for temporal scaling within recurrent network dynamics—as acknowledged in our references (Ref. 6, 9, 10, 13)—our study introduces two distinct and novel contributions:\n1. **Time Reversal through Negative Gain**: Our findings indicate that a negative gain can effectuate a \"time reversal.\" To the best of our knowledge, this phenomenon hasn't been delineated in preceding studies.\n1. **Analytical Representation of Temporal Scaling Factor**: We rigorously derive that the input gain symbolizes the temporal scaling factor, denoted as $\\alpha$ associated with TS operators. While this might be conceptually intuitive to some, our study offers the first mathematically rigorous representation of this relationship."}]

Author rebuttal:
We sincerely appreciate all reviewers' comments! Here we reply to some common questions raised by reviewers.

### Novelty and contribution
As some reviewers asked whether Lie group theory is necessary for this study, we emphasize the objectives and contributions of our work that include **three** parts: 
1. Our study provides **theoretical** and **analytical** understanding of temporal scaling (TS) group representation in the biologically plausible recurrent network dynamics (lines 231 – 243). Specifically, we start from the Lie group to derive the TS operations on neural representations (Eqs. 7-10), and then compare it with the theoretical analysis of network dynamics (Eq. 23) to identify the TS group operator representation. Nearly no previous study analytically link the TS group with a nonlinear recurrent neural circuit dynamics.
1. We rigorously derive the continuous attractor dynamics based on the Lie group framework, which is a novel implementation of TS transformation (acknowledged by the Reviewer ZBA5). 
1. We rigorously introduce group equivariance to define temporal scaling.  Equivariance has become a trending topic in machine learning (Cohen & Welling, 2016) and has been widely used in, e.g., geometric machine learning,  molecular design (Hoogeboom, 2022), fluid dynamics (Wang, 2020), particle physics (Bogatskiy, 2020), etc. We believe using the Lie group to understand the cognitive and neural computation in the brain would be fundamental in neuroscience research. 

Other new results include
1. A negative gain can induce “time reversal” which has not been reported to our best knowledge.
1. Disentanglement between time control and pattern generation, which possibly corresponds to the mismatch of neural and muscle activities that were found by Saxena, et al., 2022, mentioned by the Reviewer tVWd.

Combined, to the best of our knowledge, the present study is one of the earliest studies that link 1) the abstract group operators, 2) biologically plausible recurrent network dynamics, and 3) neurobiological experiments together. Most earlier studies only bridged the above aspects of 1) and 2), or 2) and 3), but not all three of them.  We believe having such theoretical results of abstract group representation in concrete network dynamics would greatly gain our understanding of the algebraic structure of network dynamics, and can help us build/learn network representation of more complicated group structures in the future.

### Gain modulation
The proposed neural circuit model receives a time-invariant control input whose gain can be modulated to change the temporal scale of the sequences in the recurrent circuit. The **gain-modulation** is a widely accepted neuroscience operation that refers to only modulating the **magnitude** of inputs, e.g., multiplying the control input $I(x|z_\infty)$ (vector) by the gain $\alpha$ (scalar) (see Eq. 23). Indeed, using gain-modulation to control temporal speed has been used before, e.g., previous studies trained recurrent networks that receive a gain-modulated temporal scaling input (Ref. [2, 6-13], and Maheswaranathan, et al., 2019 mentioned by the Reviewer tVWd). However, no previous study **theoretically** identified the control input gain represents the temporal scaling factor as reported in the present study. The novelty of our method was also acknowledged by the Reviewer aiKz in his/her statement of originality about our work.

Meta-review: The reviewers appreciated the novelty, originality and clarity of this manuscript, and I congratulate the reviewers for their careful and detailed rebuttals, which were helpful for clarifying the paper's theoretical contributions. Although several reviewers felt that the Lie group theory framework was not a necessary component to the manuscript's findings, the overall sentiment among reviewers was that the manuscript's contributions (e.g., continuous attractor implementation of temporal scaling, reproduction of biological findings) were novel and important enough to warrant acceptance.  Thus, I'm pleased to report that this paper has been accepted to NIPS.  Congratulations!  Please revise the manuscript to address all reviewer comments and questions.
Decision: Accept (poster)