# Closed-Loop Current Stimulation Feedback Control of a Neural Mass Model Using Reservoir Computing
## 4 Discussion
We found that even with very limited training data (the lowest amount being 132 trials, which corresponded to around 80 s of stimulation data), the total output was reduced by about the same amount as when we used the full training data set (1 h of stimulation data). Given that some tES protocols are on the order of tens of minutes [5], acquisition of adequate amounts of data should be possible in practice, though the noise acquired in realistic settings may impede ESN training. The fact that the ESN dynamics could be learned with a small amount of training data may indicate that the state equations behave in a nearly linear fashion. Since the only nonlinearity in the model was from the sigmoidal function that converts membrane potentials into firing rates, it could mean that the model was operating near the linear part of the sigmoid function. The closed-loop controller was implemented after the ESN network was trained. During the training period, the ESN was fed the current PC output and the future PC output to predict the stimulation current to deliver in order to reduce future PC output. In the closed-loop feedback setting, the ESN was also fed the current PC output, but the future PC output had to be manually set based on the desired dynamics at the next time step. We found that setting the future PC output to be a scaled down version of the current PC output resulted in a closed-loop controller that stably suppressed the energy output of the PCs. In a perfect setting, the future PC output would be equated to zero to drive the dynamics to zero energy. However, this caused the model to become unstable. This was most likely due to the stochastic noise constantly input into the model, which prevented the model from being held at zero energy. We explored the effect of different values of the scaling parameter \(k\) on the amount of energy suppression in the PC layer and found that scaling parameters in the range of 0.9 to 1 resulted in the greatest amount of energy suppression, while lower values of 0.7 to 0.8 increased the energy of the PC layer. The optimal choices of scaling parameter \(k\) and training set size reduced the energy output of PC layer 1 by around 30%. Varying the amount of training data while keeping the scaling parameter fixed did not significantly impact the amount of energy reduction, but strongly impacted characteristics of the current stimulation. Specifically, the ESN controller learned more efficient, lower energy current stimulation feedback with more training data. The ESN controller suppressed PC energy by predicting and subsequently destructively interfering with endogenous membrane currents, demonstrating that the feedback controller successfully estimated internal states of the model from the observable PC output. It may have been the case that the ESN learned an internal representation of phase space of the endogenous currents, similar to the models in [37; 38]. Providing additional features of the data (e.g., instantaneous phase or wavelet coefficients) could assist the ESN in learning this phase representation. Because the controller successfully estimated the internal membrane currents, simply switching the sign of the injected current would positively reinforce ongoing dynamics. This approach could potentially be applied in both human or animal models by first identifying a target dipole source using source localization techniques, then modeling current flow using forward models [39]. Current source density methods could also be used to estimate the radial current flow at the scalp [40]. After estimating the current over the entire scalp, tES could be applied in an open-loop setting to destructively interfere with the estimated current. The closed-loop method proposed in this paper could be performed on real-time EEG data, given a successful tES removal artifact algorithm that could operate in real time.