# Working memory dynamics in a flip-flop oscillations network model with Milnor attractor

David Colliaux, Yoko Yamaguchi, Colin Molter, Hiroaki Wagatsuma

► To cite this version:

David Colliaux, Yoko Yamaguchi, Colin Molter, Hiroaki Wagatsuma. Working memory dynamics in a flip-flop oscillations network model with Milnor attractor. 14th International Conference on Neural Information Processing ICONIP 2007, 2007, Kitakyushu, Japan. pp.160-169, 10.1007/978-3-540-69158-7_18 . hal-00382373

HAL Id: hal-00382373

https://hal.science/hal-00382373v1

Submitted on 6 Apr 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Working Memory Dynamics in a Flip-Flop Oscillations Network Model with Milnor Attractor

David Colliaux<fnref n="1" />,<fnref n="2" />, Yoko Yamaguchi<fnref n="1" />, Colin Molter<fnref n="1" />, and Hiroaki Wagatsuma<fnref n="1" />

david.colliaux@polytechnique.org

**Abstract.** A phenomenological model is developed where complex dynamics are the correlate of spatio-temporal memories. If resting is not a classical fixed point attractor but a Milnor attractor, multiple oscillations appear in the dynamics of a coupled system. This model can be helpful for describing brain activity in terms of well classified dynamics and for implementing human-like real-time computation.

## 1 Introduction

Neuronal collective activities of the brain are widely characterized by oscillations in human and animals [1][2]. Among various frequency bands, distant synchronization in theta rhythms (4-8 Hz oscillation defined in human EEG) is recently known to relate with working memory, a short-term memory for central execution in human scalp EEG [3][4] and in neural firing in monkeys [5][6].

For long-term memory, information coding is mediated by synaptic plasticity whereas short-term memory is stored in neural activities [7]. Recent neuroscience reported various types of persistent activities of a single neuron and a population of neurons as possible mechanisms of working memory. Among those, bistable states, up- and down-states, of the membrane potential and its flip-flop transitions were measured in a number of cortical and subcortical neurons. The up-state, characterized by frequent firing, shows stability for seconds or more due to network interactions [8]. However it is little known whether flip-flop transition and distant synchronization work together or what kind of processings are enabled by the flip-flop oscillation network.

Associative memory network with flip-flop change was proposed for working memory with classical rate coding view [9], while further consideration on dynamical linking property based on firing oscillation, such as synchronization of theta rhythms referred above, is likely essential for elucidation of multiple attractor systems. Besides, Milnor extended the concept of attractors to invariant sets with Lyapunov unstability, which has been of interest in physical, chemical and biological systems. It might allow high freedom in spontaneous switching among semi-stable states [12]. In this paper, we propose a model of oscillation associative memory with flip-flop change for working memory. We found that

<footnote n="1">¹ Lab for Dynamics of Emergent Intelligence, RIKEN BSI, Wako, Saitama, Japan</footnote>

<footnote n="2">² Ecole Polytechnique (CREA), 75005 Paris, France</footnote>
the Milnor attractor condition is satisfied in the resting state of the model. We will first study how the Milnor attractor appears and will then show possible behaviors of coupled units in the Milnor attractor condition.

# 2 A Network Model

## 2.1 Structure

In order to realize up- and down-states where up-state is associated with oscillation, phenomenological models are joined. Traditionally, associative memory networks are described by state variables representing the membrane potential {$S_i$} [9]. Oscillation is assumed to appear in the up-state as an internal process within each variable $\phi_i$ for the $i^{th}$ unit. Oscillation dynamics is simply given by a phase model with a resting state and periodic motion [10,11]. $cos(\phi_i)$ stands for an oscillation current in the dynamics of the membrane potential.

## 2.2 Mathematical Formulation of the Model

The flip-flop oscillations network of N units is described by the set of state variables {$S_i, \phi_i$} $\in \mathbb{R}^N \times [0, 2\pi^{[N]} (i \in [1, N])]$. Dynamic of $S_i$ and $\phi_i$ is given by the following equations:

$$ \begin{aligned} \frac{dS_i}{dt} &= -S_i + \sum W_{ij} R(S_j) + \sigma(\cos(\phi_i) - \cos(\phi_0)) + I_{\pm} \\ \frac{d\phi_i}{dt} &= \omega + (\beta - \rho S_i)\sin(\phi_i) \end{aligned} \quad (1) $$

with $R(x) = \frac{1}{2}(\tanh(10(x - 0.5)) + 1)$, $\phi_0 = \arcsin(\frac{\omega}{\beta})$ and $\cos(\phi_0) < 0$.

R is the spike density of units and input $I_{\pm}$ will be taken as positive ($I_+$) or negative ($I_-$) pulses (50 time steps), so that we can focus on the persistent activity of units after a phasic input. $\omega$ and $\beta$ are respectively the frequency and the stabilization coefficient of the internal oscillation. $\rho$ and $\sigma$ represent mutual feedback between internal oscillation and membrane potential. $W_{ij}$ are the connection weights describing the strength of coupling between units i and j. $\phi_0$ is known to be a stable fixed point of the equation for $\phi$, and 0 to be a fixed point for the S equation.

# 3 An Isolated Unit

## 3.1 Resting State

The resting state is the stable equilibrium when $I = 0$ for a single unit. We assume $\omega < \beta$ so that $M_0 = (0, \phi_0)$ is the fixed point of the system. To study the linear stability of this fixed point, we write the stability matrix around $M_0$:

$$ DF|_{M_0} = \begin{pmatrix} -1 & -\sigma\sin(\phi_0) \\ -\rho\sin(\phi_0) & \beta\cos(\phi_0) \end{pmatrix} \quad (2) $$
The sign of the eigenvalues of $DF|_{M_0}$ and thus the stability of $M_0$ depends only on $\mu = \rho\sigma$. With our choice of $\omega = 1$ and $\beta = 1.2$, $\mu_c \approx 0.96$. If $\mu < \mu_c$, $M_0$ is a stable fixed point and there is another fixed point $M_1 = (S_1, \phi_1)$ with $\phi_1 < \phi_0$ which is unstable. If $\mu > \mu_c$, $M_0$ is unstable and $M_1$ is stable with $\phi_1 > \phi_0$. Fixed points exchange stability as the bifurcation parameter $\mu$ increases (transcritical bifurcation). The simplified system according to eigenvectors $(X_1, X_2)$ of the matrix $DF|_{M_0}$ gives a clear illustration of the bifurcation as

$$ \begin{aligned} \frac{dx_1}{dt} &= ax_1^2 + \lambda_1 x_1 \\ \frac{dx_2}{dt} &= \lambda_2 x_2 \end{aligned} \qquad (3) $$

Here $a = 0$ is equivalent to $\mu = \mu_c$ and in this condition there is a positive measure basin of attraction but some directions are unstable. The resting state $M_0$ is not a classical fixed point attractor because it does not attract all trajectories from an open neighborhood, but it is still an attractor if we consider Milnor's extended definition of attractors. Phase plane $(S, \phi)$ Fig. 1 shows that for $\mu$ close to the critical value, nullclines cross twice staying close to each other in between. That narrow channel makes the configuration indistinguishable from a Milnor attractor in computer experiments.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Top: Phase space $(S, \phi)$ with vector field and nullclines of the system. The dashed domain in B shows that $M_0$ have positive measure basin of attraction when $\mu = \mu_c$. Bottom: Fixed points with their stable and unstable directions for the equivalent simplified system. A: $\mu < \mu_c$. B: $\mu = \mu_c$. C: $\mu > \mu_c$.</figcaption></figure>

Since we showed $\mu$ is the crucial parameter for the stability of the resting state, we can now consider $\rho = 1$ and study the dynamics according to $\sigma$ with a close look near the critical regime ($\sigma = \mu_c$).
## 3.2 Constant Input Can Give Oscillations

Under constant input there are two possible dynamics: fixed point and limit cycle. If

$$ \left| \frac{\omega}{\beta - S} \right| < 1 \qquad (4) $$

there is a stable fixed point $(S_1, \phi_1)$ with $\phi_1$ solution of

$$ \begin{aligned} \omega + (\beta - \sigma(\cos(\phi_1) - \cos(\phi_0)) - I)\sin(\phi_1) &= 0 \\ S_1 &= \sigma(\cos\phi_1 - \cos\phi_0) + I \end{aligned} \qquad (5) $$

If condition 4 is not satisfied, the $\phi$ equation in 1 will give rise to oscillatory dynamics. Identifying S with its temporal average, $\frac{d\phi}{dt} = \omega + \Gamma\sin(\phi)$ with $\Gamma = \beta - S$ will be periodic with period $\int_0^{2\pi} \frac{d\phi}{\omega+(\beta-S)\sin(\phi)}$. This approximation gives an oscillation at frequency $\omega' = \sqrt{\omega^2 - (\beta - S)^2}$, which is qualitatively in good agreement with computer experiments Fig. 2.

<figure><img src="image_3.png" /><figcaption>Fig. 2. For each value of constant current I, maximum and minimum values of $S_1$ are plotted. Dominant frequency of $S_1$ obtained by FFT is compared to the theoretical value when S is identified with its temporal average: Frequency VS Frequency (theoretical).</figcaption></figure>

If we inject an oscillatory input into the system, S oscillates at the same frequency provided the input frequency is low. For higher frequencies, S cannot follow the input and shows complex oscillatory dynamics with multiple frequencies.
# 4 Two Coupled Units

For two coupled units, flip-flop of oscillations is observed under various conditions. We will analyze the case $\mu = 0$ and flip-flop properties under various strengths of connection weights, assuming symmetrical connections ($W_{12} = W_{2,1} = W$).

## 4.1 Influence of the Feedback Loop

In equation 1, $\rho$ and $\sigma$ implement a feedback loop representing mutual influence of $\phi$ and $S$ for each unit.

**The Case $\mu = 0$.** In the case $\sigma = 0$ or $\rho = 0$, $\phi$ remains constant $\phi = \phi_0$: the system is then a classical recurrent network. This model was used to provide associative memory network storing patterns in fixed point attractors [9]. For small coupling strength, the resting state is a fixed point. For strong coupling strength, two more fixed points appear, one unstable, corresponding to threshold, and one stable, providing memory storage. After a transient positive input $I_+$ above threshold, the coupled system will be in up-state. A transient negative input $I_-$ can bring it back to resting state.

For a small perturbation ($\sigma \ll 1$ and $\rho = 1$), the active state is a small up-state oscillation but associative memory properties (storage, completion) are preserved.

**Growing Oscillations.** The up-state oscillation in the membrane potential dynamics triggered by giving an $I_+$ pulse to unit 1 grows when $\sigma$ increases and saturates to an up-state fixed point for strong feedback. Interestingly, for a range of feedback strength values near $\mu_c$, $S$ returns transiently near the Milnor attractor resting state.

Projection of the trajectories of the 4-dimensional system on a 2-dimensional plane section P illustrates these complex dynamics Fig. 3. A cycle would intersect this plane in two points. For each $\sigma$ value, we consider $S_1$ for these intersection points. For a range between 0.91 and 1.05 with our choice of parameters, there are much more than two intersection points $M^*$, suggesting chaotic dynamics.

## 4.2 Influence of the Coupling Strength

The dynamics of two coupled units can be a fixed point attractor, as in the resting state ($I = 0$), or down-state or up-state oscillation (depending on the coupling strength), after a transient input. Near critical value of the feedback loop, in addition to these, more complex dynamics occur for intermediate coupling strength.
<figure><img src="image_4.png" /><figcaption>**Fig. 3.** A: Influence of the feedback loop- Bifurcation diagram according to $\sigma$ (Top). $S_1$ coordinates of the intersecting points of the trajectory with a plane section P according to $\sigma$ (Bottom). B: Influence of the coupling strength - $S_1$ maximum and minimum values and average phase difference ($\phi_1 - \phi_2$) according to W (Top). $S_1$ coordinates of the intersecting points of the trajectory with a plane section P according to W (Bottom).</figcaption></figure>

**Down-state Oscillation.** For small coupling strength, the system periodically visits the resting state for a long time and goes briefly to up-state. The frequency of this oscillation increases with coupling strength. The two units are anti-phase (when $S_i$ takes maximum value, $S_j$ takes minimum value) Fig. 4 (Bottom).

**Up-state Oscillation.** For strong coupling strength, a transient input to unit 1 leads to an up-state oscillation Fig. 4 (Top). The two units are perfectly in-phase at $W = 0.75$ and phase difference stays small for stronger coupling strength.

**Chaotic Dynamics.** For intermediate coupling strength, an intermediate cycle is observed and more complex dynamics occur for a small range ($0.58 < W < 0.78$
<figure><img src="image_5.png" /><figcaption>**Fig. 4.** $S_i$ temporal evolution, ($S_1, S_2$) phase plane and ($S_i, \phi_i$) cylinder space. Top: Up-state oscillation for strong coupling. Middle: Multiple frequency oscillation for intermediate coupling. Bottom: Down-state oscillation for weak coupling.</figcaption></figure>

with our parameters) before full synchronization characterized by $\phi_1 - \phi_2 = 0$. The trajectory can have many intersection points with P and $S^*$ in Fig. 3 shows multiple roads to chaos through period doubling.
# 5 Application to Slow Selection of a Memorized Pattern

## 5.1 A Small Network

The network is a set N of five units consisting in a subset $N_1$ of three units A, B and C and another $N_2$ of two units D and E. In the set N, units have symmetrical all-to-all weak connections ($W_N = 0.01$) and in each subset units have symmetrical all-to-all strong connections ($W_{N_i} = 0.1 * M$) with M a global parameter slowly varying in time between 1 and 10. These subsets could represent two objects stored in the weight matrix.

## 5.2 Memory Retrieval and Response Selection

We consider a transient structured input into the network. For constant M, a partial or complete stimulation of a subset $N_i$ can elicit retrieval and completion of the subset in an up-state as would do a classical auto-associative memory network.

<figure><img src="image_6.png" /><figcaption>**Fig. 5.** Slow activation of a robust synchronous up-state in $N_1$ during slow increase of M</figcaption></figure>

In the Milnor attractor condition more complex retrieval can be achieved when M is slowly increased. As an illustration, we consider transient stimulation of units A and B from $N_1$ and unit E from $N_2$ Fig. 5. $N_2$ units show anti-phase
oscillations with increasing frequency. $N_1$ units first show synchronous down-state oscillations with long stays near the Milnor attractor and gradually go toward sustained up-state oscillations. In this example, the selection of $N_1$ in up-state is very slow and synchrony between units plays an important role.

# 6 Conclusion

We demonstrated that, in cylinder space, a Milnor attractor appears at a critical condition through forward and reverse saddle-node bifurcations. Near the critical condition, the pair of saddle and node constructs a pseudo-attractor, which can serves for observation of Milnor attractor-like properties in computer experiments. Semi-stability of the Milnor attractor in this model seems to be associated with the variety of oscillations and chaotic dynamics through period doubling roads.

We demonstrated that an oscillations network provides a variety of working memory encoding in dynamical states under the presence of a Milnor attractor. Applications of oscillatory dynamics have been compared to classical autoassociative memory models. The importance of Milnor attractors was proposed in the analysis of coupled map lattices in high dimension [11] and for chaotic itinerancy in the brain [13]. The functional significance of flip-flop oscillations networks with the above dynamical complexity is of interest for further analysis of integrative brain dynamics.

# References

1. Varela, F., Lachaux, J.-P., Rodriguez, E., Martinerie, J.: The brainweb: Phase synchronization and large-scale integration. Nature Reviews Neuroscience (2001)

2. Buzsaki, G., Draguhn, A.: Neuronal oscillations in cortical networks. Science (2004)

3. Onton, J., Delorme, A., Makeig, S.: Frontal midline EEG dynamics during working memory. NeuroImage (2005)

4. Mizuhara, H., Yamaguchi, Y.: Human cortical circuits for central executive function emerge by theta phase synchronization. NeuroImage (2004)

5. Rainer, G., Lee, H., Simpson, G.V., Logothetis, N.K.: Working-memory related theta (4-7Hz) frequency oscillations observed in monkey extrastriate visual cortex. Neurocomputing (2004)

6. Tsujimoto, T., Shimazu, H., Isomura, Y., Sasaki, K.: Prefrontal theta oscillations associated with hand movements triggered by warning and imperative stimuli in the monkey. Neuroscience Letters (2003)

7. Goldman-Rakic, P.S.: Cellular basis of working memory. Neuron (1995)

8. McCormick, D.A.: Neuronal Networks: Flip-Flops in the Brain. Current Biology (2005)

9. Durstewitz, D., Seamans, J.K., Sejnowski, T.J.: Neurocomputational models of working memory. Nature Neuroscience (2000)

10. Yamaguchi, Y.: A Theory of hippocampal memory based on theta phase precession. Biological Cybernetics (2003)
11. Kaneko, K.: Dominance of Minlnor attractors in Globally Coupled Dynamical Systems with more than 7 +- 2 degrees of freedom (retitled from 'Magic Number 7 +- 2 in Globally Coupled Dynamical Systems') Physical Review Letters (2002)

12. Fujii, H., Tsuda, I.: Interneurons: their cognitive roles - A perspective from dynamical systems view. Development and Learning (2005)

13. Tsuda, I.: Towards an interpretation of dynamic neural activity in terms of chaotic dynamical systems. Behavioural and Brain Sciences (2001)