Title: OMiSO: Adaptive optimization of state-dependent brain stimulation to shape neural population states
Venue: NeurIPS 2025
Accepted: yes
Keywords: closed-loop optimization

microstimulation

neural population activity

dimensionality reduction

latent variable models

reinforcement learning

behavior cloning

proximal policy optimization
Abstract: The coordinated activity of neural populations underlies myriad brain functions. Manipulating this activity using brain stimulation techniques has great potential for scientific and clinical applications, as they causally influence the nervous system. To improve the accuracy by which one can manipulate neural activity, it is important to (1) take into account the pre-stimulation brain state, which can influence the brain’s response to stimulation, and (2) adaptively update stimulation parameters over time to compensate for changes in the brain’s response to stimulation. In this work, we propose Online MicroStimulation Optimization (OMiSO), a brain stimulation framework that leverages brain state information to find stimulation parameters that can drive neural population activity toward specified states. OMiSO includes two key advances: i) training a stimulation-response model that leverages the pre-stimulation brain state, and inverting this model to choose the stimulation parameters, and ii) updating this inverse model online using newly-observed responses to stimulation. We tested OMiSO using intracortical microstimulation with a ``Utah'' array and found that it outperformed competing methods that do not incorporate these advances. Taken together, OMiSO provides greater accuracy in achieving specified activity states, thereby advancing neuromodulation technologies for understanding the brain and for treating brain disorders.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces OMiSO (Online MicroStimulation Optimization), a brain stimulation framework that optimizes stimulation parameters by explicitly modeling the pre-stimulation neural state to achieve precise control over population-level neural activity. This work proposes a closed-loop, state-aware optimization approach, which dynamically adapts stimulation protocols based on real-time neural recordings.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The framework introduces optimal parameter selection conditioned on pre-stimulation brain states—a well-established control-theoretic principle previously unrealized in MiSO systems. This enables context-aware stimulation tailored to baseline neural activity.\n\n2. By continuously refining parameters using newly observed stimulation responses, OMiSO overcomes the limitation of MiSO's restricted parameter-space search. The proposed reinforcement learning algorithm dynamically optimizes stimulation protocols, achieving superior neural control.\n\nWeaknesses:\n1. While the paper improves upon MiSO, it inherits fundamental limitations—notably, the low-dimensional stimulation parameter space (constrained by dual-electrode setups) inherently restricts the spatial scope of neuromodulation. The work focuses on algorithmic refinements rather than addressing these hardware-imposed boundaries, leaving the core challenge unresolved.\n\n2. The chosen metrics (e.g., accuracy in reaching target states) may not fully capture the biological efficacy of neural control, especially given the dynamic nature of brain states.\n\n3. Due to inter-session variability in neural activity, the claimed optimization benefits require validation across totally unseen sessions/timepoints to demonstrate robustness. The authors should discuss strategies to mitigate session-dependent confounds.\n\nQUESTIONS:\nSee Weaknesses.\n\nLIMITATIONS:\nSee Weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors developed a method called OMiSO, which, given the current neural population state and the desired state, outputs an optimal stimulation pattern that will move the state to the desired state. In this study, the authors focus on not the full neural state space, but the low-dimensional latent state obtained from factor analysis. Factor analysis is performed for each session, and the latent spaces are aligned, following the same approach as in MiSO. The authors then fit a stimulation-response model that predicts the latent state after stimulation, given the latent state before stimulation and the stimulation pattern. This model is then used to generate enough data to fit an RL model via behavioral cloning. The RL model’s job is to predict, given the pre-stimulation latent state and the target latent state, what the stimulus pattern should be. After these steps, which are so far done offline, the authors further refine the RL model online via PPO as new data comes in. The authors test their method on a real monkey electrophysiology dataset and find superior performance compared to baselines and a recent method, MiSO.\n\nSTRENGTHS AND WEAKNESSES:\n**Quality**: The authors’ claims are well supported by experiments. \n\n1. The authors claim that leveraging pre-stimulation neural activity for predicting optimal stimulation patterns can be helpful, and show experimental evidence that the latent states of the monkey before and after stimulation for two different stimuli are distinguishable. I agree that providing the model with the pre-stimulation activity (more data) is helpful, and wonder if, for their setup, providing it with experimental stimuli (the location of the target dots) can make it even better. What do the authors think about including “optional data” (e.g., target location in this case, but could be some other data in general) as one of the features of the model?\n\n2. The authors show evidence that their stimulation-response model works well, by experimenting with different neural network architectures. The authors report the hyperparameters in the Supplementary Material. The authors also justify their stimulation-response inverse model by comparing it to a “perfect” inversion model, which constructs a table of all possible combinations of stimulation patterns and their corresponding states post-stimulation. This perfect inversion model predicts the optimal stimulation patterns from the post-stimulation states using this table (a table including pre-stimulation combinations may be infeasible due to the latent space being continuous and large). Their inverse model (trained by BC) performs well compared to the perfect inversion model.\n\n3. Lastly, the authors show evidence that refining their predictions online via PPO helps in actual brain stimulation experiments. 14 sessions were used to make comparisons between MiSO, OMiSO, and their perfect inversion model without pre-stimulation states. The authors show that OMiSO performs better than the baseline methods in this experiment.\n\n\n**Clarity**: The work is overall clear and well-organized. There were a few places where I thought would benefit from clarification.\n\n1. Equations (1-4) are exactly the same as Equations (1-4) of the MiSO paper, and as far as I understand, there is no conceptual difference between the latent alignment procedure performed by MiSO and that performed in this work. I think the authors could instead cite MiSO for this Section, with just Eq. 4 in the paper, as this one is used throughout the later Sections.\n\n2. It wasn’t clear from the main text how the function f (the stimulation-response model) was modeled when it was first introduced in L118. Adding a sentence early on that the network architectures in Fig. 2B are used for their function f, and that the specific architectures are shown in Fig. S1, could be helpful. This is also true for \\phi_\\theta, first introduced in L128. Perhaps mentioning that this is an MLP early on could be helpful.\n\n3. It wasn’t clear in the beginning why Z is a matrix with t, whereas z is a vector without t? If I understand correctly, this is because for Z, the stimulation-response model f can use 1 to 5 pre-uStim time bins. I think this information could be added early on to make it clearer.\n\n4. The authors mention that “OMiSO performs R-fold cross validation where every session is used as a validation set once (see Section S4)”. S4 didn’t specify this, and perhaps mentioning S6 in S4 makes sense.\n\n5. It wasn’t clear from the main text how the experiments were run -- for each trial, how did they choose which method to use? For Fig. 4D, the authors show the average predicted post-stimulation state and target state, along with single-trial predictions of post-stimulation states. Is the plot in Fig. 4E made by first taking the Euclidean distance between the average predicted state and the target state for each session, and then computing the mean and standard error across the 14 sessions? For L312, should this be p=0.05 and p=0.01, and not p<0.05 and p<0.01?\n\n6. L157: The number of stimulus patterns is not in real space but in integer space.\n\n7. L195: What does it mean for the baseline b to be used “to evaluate the induced post-uStim activity”? How does this baseline b help here?\n\n8. After doing dimensionality reduction to m=5 dimensions, how does the user select l from m? How m and l were selected was not explained in S3 but in S1 (only for m). If l components are selected from m, I think the same state in this l-dimensional latent space doesn’t represent the same space in neural space or the m-dimensional latent space. Would results be different if the authors used l=m=5, or l=m=2? I think a justification for subselection would be helpful. How does increasing or decreasing l or m affect model performance (both accuracy and speed)? \n\n9. Consistent terminology could help: for example, sometimes the stimulation-response model is referred to as the statistical model, and pre-uStim is referred to as pre-stimulation.\n\n10. There is no Fig. 2C (L268).\n\n11. MiSO also performs online inference, but this method is called Online MiSO. Is the online-ness the most important factor that distinguishes between this approach and MiSO? A few sentences discussing in-depth how this approach differs from MiSO would be helpful.\n\n**Significance**: Optimal online control of the neural population state from one state to another is a key challenge in BCI, and OMiSO suggests a new approach to solve this problem. The authors apply their method to real brain stimulation experiments in monkeys and demonstrate its efficacy in comparison to a previous method, MiSO. Their demonstration of their method and baselines in actual experimental settings is a big strength of this paper. The method also consists of modular steps, and each step of the method can be replaced without affecting the other steps. For example, in the paper, factor analysis and the latent alignment method via Procrustes optimization were used in one of the steps, and this may be replaced by other fast dimensionality reduction methods and alignment methods in the future if one finds improvement over the current one. This modularity allows accessing each step of the model and what features most affect its performance.\n\n**Originality**: While there is previous work incorporating pre-stimulation neural state for choosing optimal stimulation patterns, the authors suggest that this is the first to incorporate pre-stimulation activity for manipulating high-dimensional neural population state. Because this work focuses on the prediction of the low-dimensional latent state, I think it would help to show prediction in the neural population state (i.e., predictions on firing rates of individual neurons) are close to the observed high-dimensional neural population state. Currently, the authors focus on the learned low-dimensional state, and it is not clear whether the learned low-dimensional state accurately captures the high-dimensional neural population state. As mentioned in Clarity point 8 above, it may also be the case that if the users subselect arbitrary dimensions in the m-dimensional latent space, it would be hard to recover the full high-dimensional neural population state. A discussion on how this can be addressed would be helpful. That using PPO to update/refine the predictions in real-time is better than the method used in MiSO is also an interesting finding. Is there a more direct way to incorporate pre-stimulation neural state into the MiSO framework and compare that to OMiSO?\n\nQUESTIONS:\nPlease see above for questions.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce an algorithm to identify desired stimulations in order to move brain activity to a target state, in a closed loop. \nThe key ingredients involve learning an inverse mapping that outputs the desired stimulation pattern required to reach a target state from the previous state of neural activity, and then updating this mapping online as more data is collected during the experiment. They show that taking into account the current state of the brain in this pipeline results in far better performance in terms of reaching the target state relative to existing methods such as MiSo. In particular, results from stimulations in macaque PFC recordings using Utah array show successful reaching of target state in a low-dimensional space.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The paper is well-written, key claims and results are easily to follow.\n2. It attempts to solve an impactful and translatable problem, and is extremely significant to research at the cross roads of machine learning and neuroscience. \n3. The methodology overall makes sense, the high-level pipeline is novel, and the experiments are thoroughly done on real macaques.\n4.  I also appreciate the supplement discussion re how the online updates are done without hindering the generation of stimulation patterns themselves. \n\nWeaknesses:\n1. My primary concern is that reinforcement learning terminology seems to be used very loosely here. While that doesn't overshadow the main contributions of the paper, this being a machine learning venue does require either further explaining or rewriting. For example, the inverse model training is really  a supervised problem on synthetic data generated from the forward model. Calling it behavior cloning is confusing in my opinion, because it is unclear what the markov decision process looks like here. In its current form \"state\" in the MDP sense includes past + current ($z_{pre}$) and future ($z_{post}$) information. If we think of the action as the stimulation, what should the next markovian state look like? Is there a sequential problem underlying this process at all that I might be missing? Relatedly, PPO also seems to be loosely used, where the advantage function typically is defined as $A(s,a) = Q(s,a)-V(s)$. I am completely fine with the authors saying that they update the model using a PPO-inspired loss function so that the inverse model doesn't deviate much between consecutive trials. Alternatively, I would like a concrete definition of the RL problem if the authors wants to stick to referring to these approaches as behavior cloning and PPO. \n\n2. Some of the choices made during the algorithm are unclear from the main text / methodology, making it hard to fully wrap one's head around it. Crucially, after learning the forward model on low-dimensional projection of the data, the authors further select $l$ of those dimensions to learn the inverse model. The rationale behind this is unclear to me. Is this is a computational bottleneck? Presumably, factor analysis already compresses neural data to a low-dimensional space, is this further compression necessary as it presumably results in not being able to control certain neural dimensions?\n\nQUESTIONS:\nAside from some of my questions listed in the weakness section, I have the following questions / comments:\n\n1. In the introduction, the authors don't make it clear what \"brain state\" means. Later on it is clear that the authors are referring to a low-d projection of neural activity, but through the introduction it is unclear whether this refers to a high-level cognitive state etc, and it would help the readers a lot to know this upfront.\n\n2. Forward model: In line 118, what kind of function does $f$ represent? The supplement suggests these are deep networks, which would be helpful to clarify here.\n\n3. What if a reaching a target brain state requires multiple steps of stimulations? How would this algorithm handle such a case given intermediate target states may be unknown in that setting?\n\nMinor comments / typos:\n1. Lines 36-39: this sentence needs to be broken into two for readability.\n2. Line 83: I am assuming this should be $e^i \\in \\\\{1,...n_i\\\\}$.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Online MicroStimulation Optimization (OMiSO), a closed-loop brain stimulation framework designed to modulate neural population activity more precisely by leveraging the brain’s ongoing state. Unlike prior methods, OMiSO conditions stimulation parameter selection on pre-stimulation neural activity and continuously adapts its model based on observed responses. The framework builds a stimulation-response model online and inverts it to choose stimulation parameters that drive neural activity toward specified target states. The approach was validated in a non-human primate experiment using intracortical microstimulation, where it outperformed baseline methods lacking brain state awareness or adaptive learning. While the method shows strong empirical results and is broadly applicable to different stimulation modalities, its effectiveness may vary depending on the stability of neural dynamics and consistency between training and deployment conditions. Overall, OMiSO presents a principled and flexible framework that enhances the precision of neural control, with promising implications for both neuroscience research and clinical neuromodulation.\n\nSTRENGTHS AND WEAKNESSES:\n## strengths\n1. This paper evaluates its proposed method on the real animal experiment, bridging the gap between machine learning framework and the real world application\n2. This work is well-motivated and application-oriented solving the challenge in neuromodulation\n\n## weakness\n1. Although authors try to provide a nice illustration in figure 1 and 2.1 overview, I still think it is not easily following for pure machine learning researchers.\n2. Limited novelty: Core components (Factor Analysis, behavior cloning, PPO) are not new and their integration is not clearly justified as novel.\n\nQUESTIONS:\n1. In Section 2.2, you mention that due to the large stimulation parameter space, only a small subset of possible patterns can be tested within a session. To address this, OMiSO first identifies a low-dimensional latent space of the high-dimensional neural population activity. Given this, could you clarify why you chose to model the latent space from spiking activity rather than directly leveraging low-dimensional biomarkers such as local field potentials (LFPs), which are commonly used in clinical settings? Additionally, do you have any experimental comparisons showing that OMiSO outperforms LFP-based stimulation methods?\n2. Could you elaborate on the key technical and application-level differences between OMiSO and previous methods such as MiSO? A clear comparison would help readers better understand the novelty and specific contributions of your approach.\n3. I would appreciate clarification on the connection between your monkey experiments and clinical deep brain stimulation (DBS). My understanding is that DBS is typically used in disease treatment contexts (e.g., Parkinson’s disease). How does your framework relate to that setting? Specifically, is the goal of OMiSO to generalize toward clinical applications, or is it primarily intended as a neuroscience tool for experimental control (e.g., guiding the monkey to desired neural states)?\n4. Could you comment on whether OMiSO aligns with model-based reinforcement learning paradigms?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}