# Closed-Loop Current Stimulation Feedback Control of a Neural Mass Model Using Reservoir Computing
## 1 Introduction
Closed-loop tACS that matched both frequency and phase of slow-wave oscillations during sleep improved sleep quality compared to a sham condition; however, no open-loop control condition was performed to assess whether this improvement was due to the closed-loop design for the increase in sleep quality [24]. These results hint that closed-loop tES protocols can be effective, but more robust methodologies and controls are needed to produce consistent results. One way to study mechanisms of closed-loop tES while sidestepping some practical constraints is to investigate neural population behaviors using computational models. Such models avoid complications introduced by human variability and technology limitations to explore more directly how stimulation can affect the brain. The Jansen and Rit neural mass model (JRNMM) [25] provides one platform that lends itself to simulating neural dynamics and how electrical perturbation can shift population behavior. The JRNMM treats a column of cortex as interconnected layers of excitatory interneurons, pyramidal cells and inhibitory interneurons, each of which is modeled by a set of second-order nonlinear differential equations. Because the state variables encode the membrane potentials and currents of each given subpopulation, tES can easily be introduced by adding an independent external current to the internal membrane currents. This approach offers a controlled in silico environment to model neural dynamics under the influence of tES. Closed-loop stimulation can be implemented by observing the output of the JRNMM and providing desired external stimulation calculated using a closed-loop feedback controller. Closed-loop feedback controllers depend upon system identification to operate. In practice, this can be difficult to achieve due to limited access to internal model states. Some linearization techniques to control nonlinear systems require access to model state equations [26], which may not be possible in a neuroscientific setting where changes in connectivity or anatomy may alter the underlying equations that govern neural interactions. Model-free nonlinear control approaches avoid such a pitfall. One such method uses a time-series prediction scheme using echo-state networks (ESNs) to invert system dynamics by predicting the system output from a given input [27]. The ESN approach can be reversed to perform nonlinear control: the ESN is used to predict the system input that will yield a desired system output [28]. While the ESN model-free approach does not require rigorous linearization techniques, it does require adequate training data for system identification. To understand how closed-loop current stimulation can modulate neuronal dynamics, we adapted approaches that simulate electrical current stimulation to dynamical models of the brain; this allowed us to modulate endogenous neural oscillations [29; 30]. We used a JRNMM to simulate two interacting cortical sources driven by random noise [11; 25; 31; 32]. The objective of our study was to develop a closed-loop controller to control the nonlinear stochastic dynamics of the JRNMM. An echo-state network (ESN) reservoir computer was used to invert the dynamics of the JRNMM, by training the ESN to predict the input that gave rise to the observed dynamics of the PCs of the model. The ESN closed-loop controller was able to suppress the output of the JRNMM even without access to the internal states of the model or any knowledge of the state equations. The ESN controller was able to predict and then destructively interfere with the endogenous membrane currents. These findings demonstrate one possible mechanism for how the phase of current stimulation can impact the modulation of neural oscillations, providing a framework for developing robust, reliable perturbations of neural activity using closed-loop tES.