# Closed-Loop Current Stimulation Feedback Control of a Neural Mass Model Using Reservoir Computing
## 4 Discussion  
Interest in tES as a neuromodulatory technique is growing in medical and experimental settings, even though the mechanisms through which tES operates continue to be debated. Most published tES studies stimulate on a timescale of tens of minutes, using narrowband inputs whose stimulation frequency matches the resonant frequency of the targeted neural network [3]. However, there are other important considerations in tES experimental design beyond matching a brain region's resonant frequency. Carefully timing the phase onset of tES to align with ongoing neural oscillations has measurable perceptual effects [35]. Such results support the notion that closed-loop tES protocols guided by real-time neural signature recordings could achieve more robust and predictable outcomes [18]. This line of work is still in the early stages of development due to challenges in tES artifact removal in neural recordings to validate successful neuromodulation [21], subject variability (which can alter tES effects) [17] and challenges in designing a stable closed-loop controller [23]. To address these issues, we first developed a computational platform to study oscillatory dynamics of a neural system by instantiating a two-column JRNMM [25,31]. This model simulated two connected cortical sources through a set of nonlinear second-order differential equations with membrane potentials as the state variables. tES was simulated by injecting external current into the membrane currents of the PC layer of the model. This provided us with a closed-loop framework: simulated EEG recordings could be \"recorded\" from the PC layer of the model while also being stimulated by exogenous electrical current. Figure 12: PSD of PC column 1 averaged over 100 trials. Shaded regions denote \(\pm 1\) standard error of the mean. The results above are shown for the ESN network trained on all of the data (log fraction of the data = 1) and with a value of \(k=1\). Figure 13: (**A**) Single trial showing the endogenous membrane current (red) and the endogenous membrane current + the ESN current feedback (blue). Adding the ESN current feedback destructively interfered with the ongoing currents. (**B**) The mean squared membrane current without the feedback and with the feedback for all 100 trials. The results above are shown for the ESN network trained on all of the data (log fraction of the data = 1) and with a value of \(k=1\). Because latent variables and state equations cannot be observed in most neuroscientific settings, we adapted a model-free method to design a feedback controller for this nonlinear stochastic dynamical system. Specifically, we used an ESN reservoir computer as a time-series prediction model. We then inverted the dynamics of the model by predicting the appropriate current stimulation input to achieve a desired output of the model based on observing the previous outputs of the PC layer [27; 28]. In this work, we only explored one set of parameters for the JRNMM. A potential future development would be to test the effectiveness of the ESN across multiple different parameters to yield different neural mass dynamics and different neural mass models [36]. Upon being driven by random noise, the baseline JRNMM PCs generate an output with a broad-band spectra with a unimodal peak around 5 Hz (the resonant frequency of our model, given our parameter choices). Both columns 1 and 2 showed similar spectra; however, column 1 (whose PCs fed back to the excitatory interneurons in column 2; see Figure 2) always had greater power than column 2 (whose PCs fed back to column 1's PCs and inhibitory neurons). The ESN was trained by injecting randomized current into the model and using the subsequent PC output as the input to the ESN. The current injection was used as the output of the ESN, which caused the ESN to predict the current input that gave rise to the observed PC output. We varied the amount of training data used in order to assess how ESN performance would be affected in real-world settings where data are likely limited.