Title: MiSO: Optimizing brain stimulation to create neural population activity states
Venue: NeurIPS 2024
Accepted: yes
Keywords: closed-loop optimization

microstimulation

neural population activity

dimensionality reduction

latent variable models

reinforcement learning
Abstract: Brain stimulation has the potential to create desired neural population activity states. However, it is challenging to search the large space of stimulation parameters, for example, selecting which subset of electrodes to be used for stimulation. In this scenario, creating a model that maps the configuration of stimulation parameters to the brain’s response can be beneficial. Training such an expansive model usually requires more stimulation-response samples than can be collected in a given experimental session. Furthermore, changes in the properties of the recorded activity over time can make it challenging to merge stimulation-response samples across sessions. To address these challenges, we propose MiSO (MicroStimulation Optimization), a closed-loop stimulation framework to drive neural population activity toward specified states by optimizing over a large stimulation parameter space. MiSO consists of three key components: 1) a neural activity alignment method to merge stimulation-response samples across sessions, 2) a statistical model trained on the merged samples to predict the brain's response to untested stimulation parameter configurations, and 3) an online optimization algorithm to adaptively update the stimulation parameter configuration based on the model's predictions. In this study, we implemented MiSO with a factor analysis (FA) based alignment method, a convolutional neural network (CNN), and an epsilon greedy optimization algorithm. We tested MiSO in closed-loop experiments using electrical microstimulation in the prefrontal cortex of a non-human primate. Guided by the CNN predictions, MiSO successfully searched amongst thousands of stimulation parameter configurations to drive the neural population activity toward specified states. More broadly, MiSO increases the clinical viability of neuromodulation technologies by enabling the use of many-fold larger stimulation parameter spaces.

MiSO: Optimizing brain stimulation to create neural population activity states

Abstract

Brain stimulation has the potential to create desired neural population activity states. However, it is challenging to search the large space of stimulation parameters, for example, selecting which subset of electrodes to be used for stimulation. In this scenario, creating a model that maps the configuration of stimulation parameters to the brain's response can be beneficial. Training such an expansive model usually requires more stimulation-response samples than can be collected in a given experimental session. Furthermore, changes in the properties of the recorded activity over time can make it challenging to merge stimulation-response samples across sessions. To address these challenges, we propose MiSO (MicroStimulation Optimization), a closed-loop stimulation framework to drive neural population activity toward specified states by optimizing over a large stimulation parameter space. MiSO consists of three key components: 1) a neural activity alignment method to merge stimulation-response samples across sessions, 2) a statistical model trained on the merged samples to predict the brain's response to untested stimulation parameter configurations, and 3) an online optimization algorithm to adaptively update the stimulation parameter configuration based on the model's predictions. In this study, we implemented MiSO with a factor analysis (FA) based alignment method, a convolutional neural network (CNN), and an epsilon greedy optimization algorithm. We tested MiSO in closed-loop experiments using electrical microstimulation in the prefrontal cortex of a non-human primate. Guided by the CNN predictions, MiSO successfully searched amongst thousands of stimulation parameter configurations to drive the neural population activity toward specified states. More broadly, MiSO increases the clinical viability of neuromodulation technologies by enabling the use of many-fold larger stimulation parameter spaces.

1 Introduction

Brain stimulation is an important tool for treating brain disorders [1-3] and for causally perturbing neural activity states to understand brain function [4]. Because complex brain functions are realized through the coordinated activity of populations of neurons, brain stimulation techniques to control neural population activity have the potential to manipulate complex brain function. Most brain stimulation studies to date have used electrical microstimulation to produce a motor response [5] or perceptual experience [6, 7], disrupt neural activity [8, 9], or shift cognitive state [10]. It has been less common to consider the response of a population of neurons to microstimulation [11-14]. In principle, different combinations of stimulation parameters make it possible to create diverse

neural population activity patterns [15-20]. However, it is challenging to identify specific stimulation parameters that achieve a desired activity manipulation for an entire recorded neural population.

Previous studies have developed closed-loop stimulation methods to search the stimulation parameter space [12, 21, 22]. A key challenge for closed-loop stimulation is to widely sample the multidimensional stimulation parameter space. It is necessary to collect many stimulation-response samples before the closed-loop stimulation procedure can begin to work effectively. A key innovation of our work is to merge stimulation-response samples across experimental sessions [23-25]. The ability to merge data leverages the finding that neural population activity tends to occupy a lowerdimensional space than the number of neurons being recorded, which enables the alignment of low-dimensional spaces across sessions. By merging data across multiple previous sessions, it is possible for the closed-loop stimulation procedure to work effectively starting from the very outset of the current session.

In this study, we propose MiSO (MicroStimulation Optimization), a closed-loop stimulation framework to drive neural population activity toward specified states by optimizing over a large stimulation parameter space (Section 2.1). MiSO consists of three key components: 1) a neural activity alignment method to merge stimulation-response samples across sessions (Sections 2.2 and 2.3), 2) a statistical model trained on the merged samples to predict the brain's response to untested stimulation parameter configurations (Section 2.4), and 3) a closed-loop optimization algorithm to adaptively update the model's predictions and then choose the next stimulation parameter configuration to test (Section 2.5). In this study, we implemented MiSO with a factor analysis (FA) based alignment method [23], a convolutional neural network (CNN) model [26], and an epsilon greedy optimization algorithm [27]. These methods were chosen to satisfy the fast online computation requirement during the closed-loop stimulation experiments with a predictive model trained on a limited amount of data. However, MiSO is a general framework and applicable to other design choices.

We tested MiSO using electrical microstimulation (uStim) in a non-human primate implanted with a multi-electrode array in the prefrontal cortex (PFC, area 8Ar). MiSO was used to optimize the location of the stimulated electrode(s) on each trial through closed-loop updates, while keeping other parameter values such as current amplitude and frequency fixed. We considered a large uStim parameter space, defined by all possible patterns in which two electrodes out of 96 electrodes are stimulated (4,560 patterns). MiSO's latent alignment method enabled us to merge neural activity across sessions to increase the number of stimulation-response samples (Section 3.1). MiSO's CNN model trained on these merged stimulation-response samples accurately predicted neural responses to untested uStim parameter configurations (Section 3.2). In closed-loop experiments with a non-human primate, MiSO successfully searched among 4,560 uStim parameter configurations to drive the neural population activity toward targeted states. By enabling the search of a larger stimulation parameter space, MiSO produced novel population activity patterns which were not achievable by searching over a smaller stimulation parameter space (Section 3.3).

2 Methods

2.1 MiSO overview

The goal of MiSO is to identify stimulation parameter configurations that produce specified neural population activity states (Fig. 1A). MiSO first identifies a reference low-d latent space of the high-d population activity to which the neural activity from all the other experimental sessions are aligned (Fig. 1B, Section 2.2). MiSO then collects stimulation-response samples on multiple experimental sessions, then merges the samples using latent space alignment (Section 2.3). The samples are used to fit a statistical model to predict the brain's response to all possible stimulation parameter configurations within the set of parameters defined by the user (Section 2.4). These model predictions obtained prior to the closed-loop experimental session(s) are used to initialize the optimization procedure. At the beginning of each closed-loop experimental session, MiSO identifies the latent space for the session. MiSO then aligns it to the reference latent space, which was used to generate the model predictions (Section 2.5). On each trial, MiSO stimulates using the chosen parameters and updates the predictions based on the responses measured online (Fig. 1C). MiSO iteratively runs this optimization over trials to produce the specified latent activity state.

In the following sections, we refer to a specific stimulation parameter configuration as a 'stimulation pattern' and the induced brain response as the 'stimulation response'. The stimulation patterns

A

B

C

Neuron2

Utaharray

Pre-uStim

uStim

post uStim

1. Choose parameters

Target activity

Target activity

state1

?

uStim

state 2

96

150ms

Time

Chosen pattern

Fixation

8Ar

(50ms x 24bins)

?

4. Update prediction

2.Perform uStim

Time

Latent dim 2

uStim

Neuron 1

Updated prediction

Pre-uStim

Target activitystate1

→Latent dim1

activity

Latent dim 2 ↑

Error

Original prediction

Pre-uStim

activity

Induced

Neuron 3

uStim

activity

3.Analyze induced activity

Time

Latent dim 1

Latent dim2

·Induced activity

Latent dim 1

tested in this study consist of all possible patterns in which one of 96 electrodes is stimulated (single electrode stimulation patterns) and/or two of 96 electrodes are stimulated (double electrode stimulation patterns). While the neural activity used in this study consists of spiking responses recorded with a planar grid of electrodes implanted in the brain and each stimulation pattern specifies the location of the electrodes used for stimulation within the grid, MiSO can be readily applied to other stimulation/recording protocols (e.g., holographic optogenetics [19]).

2.2 Latent space identification

To merge stimulation-response samples across multiple experimental sessions, MiSO identifies a low-d latent space of the high-d population activity in each session. For this study, we used Factor Analysis (FA), which provides computationally fast latent space identification and latent state estimation, compatible with closed-loop experiments. Since MiSO's ultimate goal is to create targeted changes in neural population activity that lead to changes in brain state and in turn behavior, MiSO identifies the intrinsic latent space only using non-stimulation trials.

For the i th session, MiSO extracts a list of n i 'usable' electrodes e i ∈ R n i , where each element of e i is an integer representing one of the electrodes. An electrode is deemed 'usable' if it satisfies criteria involving its mean firing rate, Fano factor, and coincident spiking with other electrodes (see Section S1). For each time bin indexed by k = 1 , ..., K noStim i , MiSO then takes spike counts during the fixation period (Fig. 1B) on each usable electrode x noStim i,k ∈ R n i . MiSO fits the following FA model using the EM algorithm:

where z noStim i,k ∈ R m , with m<n i , is the low-d latent activity for the k th time bin, Λ i ∈ R n i × m is the loading matrix whose columns define the low-d latent space, µ i ∈ R n i contains the mean spike counts for each electrode, and Ψ i ∈ R n i × n i is a diagonal matrix capturing the independent variance of the spike counts for each electrode.

2.3 Latent space alignment

To merge neural activity across experimental sessions, we used the latent space alignment method introduced in [23] (termed 'FA+Procrustes'). It has been shown to work well in a brain-computer

interface, which like MiSO is a closed-loop experimental paradigm. The FA+Procrustes method solves the Procrustes problem to find an orthogonal transformation matrix to maximize the alignment between two FA loading matrices. First, MiSO runs a single experimental session exclusively with non-stimulation trials and extracts a list of usable electrodes e 0 , which contains n 0 electrodes. MiSO uses the data from this session to identify a reference latent space Λ 0 ∈ R n 0 × m . For each subsequent session, MiSO aligns the latent space identified in the i th session Λ i to the reference latent space Λ 0 . Concretely, for the i th session, MiSO identifies an orthogonal transformation matrix ˆ O i ∈ R m × m that fulfills:

where e com = e 0 ∩ e i is a list of usable electrodes common to the reference session and the i th session, and ∥ · ∥ F is the Frobenius Norm. This optimization can be solved in closed-form [28]. The ˆ O i found is applied to Λ i to obtain the aligned latent space ˜ Λ i ∈ R n i × m :

The latent activity in the post-stimulation period (Fig. 1B) is estimated as the posterior mean from the FA model (equation (1)) using the loading matrix ˜ Λ i :

where z Stim i,k ∈ R m , x Stim i,k ∈ R n i , and β i = ˜ Λ ⊺ i ( ˜ Λ i ˜ Λ ⊺ i +Ψ i ) -1 . By using ˜ Λ i , the induced latent activity z Stim i,k resides in a common latent space across all sessions.

2.4 Stimulation-response sample collection and CNN model fitting

To predict the stimulation response to untested stimulation patterns, MiSO uses a statistical model trained on the merged neural activity across sessions. We used a CNN in this study to capture the spatial structure of the stimulation-response relationship across the multi-electrode array (see Sections 2.7 and 3.2 for model selection). To train the CNN, MiSO runs two phases of experimental sessions to collect stimulation-response samples. In the first phase, MiSO applies randomly-selected stimulation patterns to train an initial CNN model. In the second phase, MiSO uses the trained CNN to choose stimulation patterns to maximize the diversity of the observed stimulation responses.

2.4.1 Phase 1: Random stimulation pattern selection

In the first phase, we apply random stimulation patterns over R experimental sessions to train the initial CNN model. With a planar grid of electrodes of size h × v , the stimulation pattern tested in the k th trial during the i th session S i,k ∈ R h × v is encoded using a value of 1 for stimulated electrodes and 0 for all other electrodes. The induced latent activity state z Stim i,k ∈ R m is computed using equation (4). The entries of z Stim i,k corresponding to the user defined t target dimensions within the m dimensional aligned latent space ( t ≤ m ) are subselected and collected in a new vector ˇ z Stim i,k ∈ R t . All tested stimulation patterns S i,k and corresponding responses ˇ z Stim i,k across R sessions are appended to obtain S k ∈ R h × v and ˇ z Stim k ∈ R t , where k = 1 , ..., K Stim , and K Stim is the total number of stimulation trials across sessions.

The totality of the stimulation-response samples S k and ˇ z Stim k are used to train the CNN model, which maps the stimulation patterns S k to the induced responses ˇ z Stim k . Since the CNN predictions can be variable due to training using a finite number of stimulation-response samples, MiSO uses bagging to stabilize the CNN predictions [29]. In bagging, M CNN models are fit with bootstrapping and the top C performing models in testing are used to generate predicted simulation responses for all P possible stimulation patterns defined by the user (e.g., P = 4 , 560 for double electrode stimulation patterns, computed as '96 choose 2' electrodes) (Section S2). For p = 1 , . . . , P , the predicted response ˆ z p ∈ R t to the p th stimulation pattern S p ∈ R h × v is defined as the average prediction of the C CNN models.

2.4.2 Phase 2: Guided stimulation pattern selection

Because the stimulation responses do not uniformly occupy the t -dimensional response space, choosing random stimulation patterns in Phase 1 tends to yield responses primarily in the densest

portions of the response distribution. To diversify the stimulation responses for retraining the CNN, MiSO collects stimulation-response samples using a guided sample collection strategy over G experimental sessions in Phase 2. In each session, MiSO uses the CNN-predicted stimulation responses ˆ z p from Phase 1 to select stimulation patterns on the fringes (i.e., sparsely populated regions) of the response distribution. Concretely, for each stimulation pattern p = 1 , ..., P , MiSO computes the distance measure:

where N p is the set of K-nearest neighbors of ˆ z p . MiSO chooses the top E g stimulation patterns that maximize this criterion, corresponding to responses on the fringes of the response distribution. To complement these patterns, MiSO also selects E r stimulation patterns at random (from the P possible stimulation patterns, typically E r < E g ), corresponding to responses in the more densely populated regions of the response distribution. The selected E g + E r patterns are experimentally tested for one session. The resulting stimulation-response samples are appended to the samples collected in Phase 1 and thus far in Phase 2, and the CNN is retrained using the same bagging procedure as in Phase 1. The CNN predictions are then updated and used to choose stimulation patterns to test in the next experimental session. MiSO iterates these steps of selecting stimulation patterns, collecting new stimulation-response samples, and retraining the CNN over G experimental sessions. The CNN predictions ˆ z p for p = 1 , ..., P obtained during the final session are used to initialize the closed-loop optimization.

2.5 Closed-loop optimization

Each online closed-loop session starts with K noStim c calibration trials, where no stimulation is applied. MiSO extracts a list of usable electrodes e c , which includes n c electrodes, using these trials and finds the common electrodes e com = e c ∩ e 0 between this session and the reference session. The observed spike count vectors x noStim c,k ∈ R n c are used to fit the FA parameters Λ c ∈ R n c × m , µ c ∈ R n c , and Ψ c ∈ R n c × n c . The identified latent space Λ c is aligned to the reference latent space Λ 0 using the methods described in Section 2.3, yielding ˜ Λ c ∈ R n c × m .

The closed-loop optimization starts by the user defining a target state ˇ z targ ∈ R t and loading the predicted stimulation responses ˆ z p for p = 1 , ..., P obtained from Section 2.4. MiSO then randomly chooses a stimulation pattern S s from the P possible patterns ( s = 1 , . . . , P ) to start the closedloop optimization procedure. Subsequent stimulation patterns are chosen using the epsilon greedy algorithm, described below. We used the epsilon greedy algorithm due to the need for a fast online update to compensate for activity fluctuations during the closed-loop optimization.

On the k th trial during the closed-loop optimization, the selected stimulation pattern S s is applied and the induced response during the post-stimulation period x Stim c,k ∈ R n c is measured. MiSO estimates the latent activity z Stim c,k ∈ R m in real time as:

where β c = ˜ Λ ⊺ c ( ˜ Λ c ˜ Λ ⊺ c +Ψ c ) -1 . The entries of z Stim c,k corresponding to the target dimensions are subselected as ˇ z Stim c,k ∈ R t , and the prediction of the response to stimulation pattern S s is updated as:

where 0 < α s < 1 is the learning rate. This update is essential for ensuring that the CNN predictions (based on data from previous experimental sessions) are adjusted for the current closed-loop session. Although the CNN predictions used to initialize the closed-loop optimization can capture spatial structure in the stimulation-response relationship, this update is performed separately for each stimulation pattern and therefore does not leverage spatial structure (see Section 4). MiSO uses a time-varying clipped learning rate:

where 0 < α clip < 1 , and N ( s ) is the number of trials tested with the s th stimulation pattern within the current closed-loop session. The clipped learning rate allows MiSO to make larger updates at the beginning of the session, when the framework tries to adapt to stimulation responses in the current session, and smaller updates as the framework becomes more confident about the predictions.

After updating the prediction, MiSO chooses the stimulation pattern to test on the next trial using an epsilon greedy algorithm, which balances exploitation and exploration. With a probability of 1 -ε , MiSO chooses the stimulation pattern which minimizes the L1 distance between the induced activity and the target state

and applies stimulation pattern S s ∗ on the next trial. With a probability of ε , MiSO chooses a random stimulation pattern among the P possible patterns. MiSO then iterates until the experimental session is terminated.

2.6 Experiment details

We tested MiSO using electrical microstimulation (uStim) in a macaque monkey with a 96 electrode Utah array implanted in prefrontal cortex (PFC, area 8Ar). Experimental procedures were approved by the Institutional Animal Care and Use Committee (IACUC) of Carnegie Mellon University. In each experimental session, the monkey performed a visually-guided saccade task. Each trial began with the monkey fixating on a dot at the center of the screen (Section S3). On uStim trials, we applied uStim for 150 ms during the fixation period. For each session indexed by i , we identified latent dimensions by applying FA to spike counts taken in 50 ms bins from usable electrodes e i that passed a set of criteria (Section S1). Stimulation-response samples were collected starting 50 ms after uStim offset (Fig. 1B, pink) to avoid uStim artifacts.

The number of random stimulation sessions R and guided stimulation sessions G to collect stimulation-response samples for CNN training (Section 2.4) were chosen based on the number of trials the animal worked in each session (about 300-500 uStim trials per session). We performed R = 3 sessions with random uStim pattern sampling and G = 2 sessions with guided uStim pattern sampling. On each guided sampling session, we collected E g = 80 patterns based on CNN predictions and E r = 20 random patterns. We used the top C = 10 test performing models among M = 50 CNN models to obtain the CNN prediction. To calculate d p , we used 3 nearest neighbors of ˆ z p .

Each closed-loop experimental session started with K noStim c = 100 no-uStim trials for latent space calibration. For some sessions, this calibration period also included an additional 96 trials (one per single-electrode stimulation pattern). These observations were used to compare the performance of the 'MiSO with single elec., sample avg.' with the 'Single elec., same day observation' (Fig. 2B). The latter method uses stimulation-response samples collected in the current session to initialize the predictions (Section S4). We chose t = 2 target dimensions among the m = 4 latent dimensions. The latent dimensionality m was determined based on the cross-validated data likelihood. We set the learning rate α clip as 0.1, chosen manually by assessing how the latent activity state changed over trials, and ε as 0.05, chosen by running simulations with previously-collected stimulation-response samples.

2.7 Model comparison

To decide which prediction model to use in MiSO, we compared the prediction performance of uStim responses when holding out different percentages of uStim patterns in three models: Multi-Layer Perceptron (MLP) [30], Gaussian Process (GP) [31], and Convolutional Neural Network (CNN) [26] (Fig. 3). For all models, we only encoded the location of the stimulating electrodes as inputs in this study. Other stimulation parameters, such as stimulation amplitude and frequency, were constant across electrodes and trials (see Section S3). The models differed in their ability to capture any spatial structure present in the stimulation-response relationship, whereby stimulating with nearby electrodes could induce similar latent activity states.

Below we describe the input (i.e., stimulation pattern) encoding format for each model. For the k th trial on the i th session, the MLP model takes as input S MLP i,k ∈ { 0 , 1 } 96 , a one-hot vector representing each uStim pattern. Its entries have a value of 1 for each stimulated electrode and 0 for non-stimulated electrodes. Since the one-hot input does not designate which electrodes are close

A

B

Worse

↑←

FA dim1

2

NouStim

Random uStimMiSO with single elec.,sample avg.Target

100

No uStimbaseline

relativetoNouStimbaseline

N=5 sessions

0

80

Single elec.,

FA dim2

2

sameday

observation

60

0

Tested electrode

96

40

1

20

↑

96

Better

avg.

1

No uStim

Random

uStim

0

50

100

150

200

250

300

350

sample

Trial numberin session

MisO with single

elec.,

to or far from each other, this MLP model does not capture spatial structure among the stimulating electrodes. For the k th trial on the i th session, the GP model takes as input S GP i,k ∈ R 2 n , where n is the number of electrodes used for stimulation. The locations of each stimulating electrode are encoded by a pair of values { 0 , 1 , ..., 9 } × { 0 , 1 , ..., 9 } , with the stimulating electrodes sorted in ascending order by electrode id. This input encoding format for the GP model captures spatial structure using a single Radial Basis Function kernel. Note that this format requires all stimulation patterns to have the same number of stimulating electrodes. For the k th trial on the i th session, the CNN takes a grid-format input with the same layout as the electrode array S CNN i,k ∈ { 0 , 1 } 10 × 10 , where electrodes being used for stimulation have a value of 1 and all other electrodes have a value of 0. The CNN captures spatial structure using multiple 2D convolutional filters applied to this grid input.

3 Results

3.1 Merging stimulation-response samples across sessions

When the stimulation parameter space is large, the number of uStim patterns we can test within an experimental session is much smaller than the total number of possible uStim patterns. This requires merging neural activity across sessions to create a large enough set of stimulation-response samples to learn their relationship. To develop MiSO, we considered merging neural activity based on aligning latent spaces across sessions [23]. To assess the effectiveness of this method for uStim, we empirically measured the uStim response for all possible single-electrode uStim patterns (96 patterns). We ran 5 experimental sessions in which we stimulated with each of the 96 electrodes on the array for 3-7 trials per session. We found that raw uStim responses were inconsistent across sessions due to neural recording instabilities but were more consistent within the aligned latent space (Fig. S1). This opened up the possibility of estimating uStim responses based on merged stimulation-response samples collected in previous sessions.

To test this idea, we ran closed-loop experiments with all single-electrode uStim patterns in which we used the sample average of the merged stimulation-response samples from previous sessions to initialize the predicted stimulation responses ˆ z p in MiSO (Section 2.5). We call this method 'MiSO with single elec., sample avg.' (Section S4). Here, we could use this method instead of the CNN model to generate the predictions z p , given that for the small parameter space of single electrodes

?

Single electrode uStim

B

uStim response diff.

r = 0.274

C

uStim response

0.55

MLP

Test MSE

A

Sample avg.

SmalluStimpatterndiff.

0.50

■

GP

CNN

0

0.45

T

1

2

3

4

5

6

7

8

0%

25%

50%

75%

uStim pattern diff.

% of held-out patterns during training

Large uStim

pattern diff.

Double electrode uStim

D

uStim response diff.

E

r = 0.143

uStim response

GP

Test MSE

0.26

f...

MLP

0.24

CNN

0

4

8

12

16

55%

60%

70%

80%

uStim pattern diff.

% of held-out patterns during training

we could collect sufficient stimulation-response samples for all stimulation patterns to compute the average responses. We compared this method to two baselines, 'No uStim' and 'Random uStim' (Section S4), which allow us to assess whether MiSO induces activity closer to the target state than via natural activity fluctuations or random uStim patterns, respectively. On each trial, we randomly chose one of the three methods. 'MiSO with single elec., sample avg.' (Fig. 2A, blue lines and dots) drove neural activity closer to the target state than 'No uStim' and 'Random uStim' (Fig. 2A, gray and green lines). Across multiple sessions with a different target state on each session, MiSO achieved significantly smaller errors than the two baseline methods (Fig. 2B, N = 5 sessions, p < 0 . 05 for 'No uStim' and 'Random uStim', one-tailed t-test). These results indicate that MiSO can identify effective stimulation patterns by leveraging merged samples from previous sessions.

To further validate the utility of merging past observations, we compared MiSO's performance with a method where the predictions were initialized using stimulation-response samples (one uStim trial for each of 96 electrodes) acquired immediately before starting the closed-loop optimization. We call this method 'Single elec., same day observation'. This method and MiSO use different approaches to initialize the same closed-loop procedure (equations (6)-(9)). MiSO leverages stimulation-response samples collected across multiple previous sessions for initialization. By contrast, 'Single elec., same day observation' uses the stimulation-response samples that can be collected in the current session for initialization, with the advantage that it gets to observe responses in the current session. We found that 'MiSO with single elec., sample avg.' performed equivalently to 'Single elec., same day observation' (Fig. 2B, N = 5 sessions, p = 0 . 715 , two-tailed t-test). This result indicates that the merging of stimulation-response samples across previous sessions in MiSO enables the closed-loop stimulation procedure to work effectively from the very outset of the current session. Furthermore, this result using single-electrode stimulation patterns represents an important building block for optimizing over larger stimulation parameter spaces, where merging stimulation-response samples across sessions becomes essential.

A

B

Worse

2

No uStim

RandomuStim

Target

FA dim1

MiSO with single elec., CNN

MiSOwith doubleelec.,CNN

No uStim baseline

10

N=3 sessions

0

i se

relative to No uStim baseline

0.50

FA dim2

FA dim2

1

80

0

0.00

60

一

Tested electrodes

96

CNN withdoubleelec.

1

-0.50

CNN with single elec.

40

96

1

Better

No uStim

CNN

with

CNN

0

0

100

200

300

400

Random uStim

MiSO

eelec.,

FA dim1

Trialnumberinsession

single

doublee

3.2 Predicting uStim responses using a CNN

When a search space is small (e.g., 96 single electrodes), we can measure the responses to all uStim patterns to initialize MiSO (as we did with 'MiSO with single elec., sample avg.', Section 3.1). This method becomes untenable as we increase size of the stimulation parameter space (e.g., stimulating two electrodes out of 96 yields 4,560 uStim patterns, which would require more than 30 experimental sessions to collect 3 repeats of each uStim pattern). Thus, to expand the stimulation parameter space with MiSO, we need to reduce the required number of stimulation-response samples to learn their relationship.

We asked whether there was spatial structure in the stimulation-response relationship, which could be leveraged to predict the uStim response with a limited number of samples (Fig. 3A). For this, we ran 5 experimental sessions in which we stimulated with each of the 96 electrodes on the array for 3-7 trials per session. With single-electrode uStim, we indeed observed spatial smoothness in the uStim response across physical locations on the array (Fig. 3B, Fig. S2A). In other words, stimulating with nearby electrodes tended to induce similar responses. We then asked whether we could use this property to predict the response to untested uStim patterns. We found that the GP and CNN models more accurately predicted uStim responses than the MLP model when holding out uStim patterns, indicating that spatial smoothness is a useful property for generalization to untested uStim patterns (Fig. 3C).

To increase the size of the stimulation parameter space, we ran 9 experimental sessions in which we stimulated with two out of 96 electrodes (4,560 possible patterns). We were only able to test 45 percent of this large number of possible uStim patterns, with most of the uStim patterns tested for one trial each. As with the single-electrode uStim patterns, we also observed spatial smoothness in the double-electrode uStim patterns (Fig. 3D, Fig. S2B). By leveraging this spatial smoothness, the CNN achieved better generalization performance to untested uStim patterns than the other models (Fig. 3E). Although both the GP and CNN capture aspects of spatial smoothness, the CNN outperformed the GP for double-electrode uStim. Whereas the CNN uses multiple convolution filters to capture the spatial structure of uStim patterns, the GP uses just one spatial kernel, likely limiting its performance. Furthermore, there is no natural input encoding format for a GP when stimulating using more than one electrode on a 2D array. Overall, these results indicate that the CNN model can generalize its predictions to even untested uStim patterns, enabling MiSO to perform closed-loop optimization over a large stimulation parameter space with a limited number of stimulation-response samples.

3.3 Closed-loop testing of MiSO initialized using a CNN

By combining the latent space alignment method (Section 3.1) and the CNN prediction model (Section 3.2), we tested MiSO in closed-loop experiments in a non-human primate over a large uStim parameter space, defined by all possible double-electrode uStim patterns (4,560 patterns). To train the CNN model, we ran 2,122 double-electrode uStim trials across 5 sessions (representing ~30 percent of the possible patterns) and merged the stimulation-response samples using latent space alignment. Using the trained CNN model, we predicted the uStim response for all double-electrode uStim patterns. The key motivation of expanding the stimulation parameter space is to expand the range of uStim responses that can be produced. Based on the CNN predictions, we found that the range of activity states achievable by double-electrode stimulation patterns spanned novel regions of the target space that were not reachable using single-electrode uStim (Fig. 4A, Fig. S3).

To demonstrate the benefit of searching a larger stimulus parameter space, we ran closed-loop experiments in which we set the target states to those exclusively reachable using double-electrode uStim patterns (Fig. 4A, yellow star). We implemented 'MiSO with double elec., CNN', in which CNN predictions ˆ z p were used to initialize the closed-loop optimization (Section 2.5). We compared the performance of 'MiSO with double elec., CNN' with three baselines: 'No uStim', 'Random uStim', as well as MiSO optimizing over the 96 single-electrode uStim patterns (termed 'MiSO with single elec., CNN'). If the CNN predictions are meaningful and the closed-loop optimization is able to guide the search in the stimulus parameter space, 'MiSO with double elec., CNN' would outperform the three baselines. Indeed, 'MiSO with double elec., CNN' (Fig. 4B, pink lines and dots) successfully identified double-electrode uStim patterns that induced responses closer to the target state than the three baseline methods (gray, green, and blue lines). Across multiple sessions with different target states, 'MiSO with double elec., CNN' produced smaller errors than the three baseline methods (Fig. 4C, N = 3 sessions, p < 0 . 05 for 'No uStim' and 'Random uStim', and p = 0 . 079 for 'MiSO with single elec., CNN', one-tailed t-test). These results demonstrate the utility of MiSO, which enables searching over a larger stimulation parameter space than previously possible.

4 Discussion

In this work, we proposed MiSO which searches a large stimulation parameter space to drive neural population activity toward specified states. MiSO's closed-loop optimization procedure is guided by predictions of a CNN trained on merged neural activity across sessions. To our knowledge, this is the first study to apply latent space alignment in the field of brain stimulation to merge stimulationresponse samples across sessions. This enables MiSO to search larger stimulation parameter spaces than previously possible.

Although we demonstrated MiSO's functionality in a 2D target space ( t = 2 ), the neural activity corresponding to a desired brain state might need to be specified in a higher dimensional space ( t > 2 ). As the dimensionality of the target space grows, it may become necessary to expand the stimulation parameter space in order to achieve a greater variety of target states. Provided that a target state is achievable, the epsilon greedy algorithm would require more time to identify appropriate stimulus parameter configurations. The reason is that a stimulation pattern that successfully achieves a target in two dimensions would not necessarily produce the desired activity if a third dimension was added to the target specification. In this case, the use of a more efficient online algorithm would be beneficial to reduce the exploration of suboptimal stimulation parameter configurations.

To expand the range of achievable population activity states, one might consider expanding the stimulation parameter space. For example, in the context of electrical microstimulation, we could use stimulation patterns involving larger numbers of electrodes and/or different current amplitudes or frequencies. For the CNN model (or any other predictive model) to retain its prediction accuracy, it would likely need to capture additional structure in the stimulation-response relationship (e.g., similar stimulation amplitudes or frequencies lead to similar responses) and require more training data. Furthermore, MiSO's closed-loop updates of the predicted responses are performed separately for each stimulation pattern (equation (7)), which becomes untenable as the stimulation parameter space grows. In this case, incorporating a statistical model in the closed-loop updates to account for spatial (or other) structure could be beneficial, such that observing a response to one stimulation pattern leads to updates of the predicted responses for multiple stimulation patterns. These computations would need to be performed fast enough to be used in a closed-loop optimization procedure.

5 Acknowledgments

This work was supported by NIH CRCNS R01 MH118929, NSF NCS DRL 2124066, Simons Foundation 543065 and NC-GB-CULM-00003241-05, and Japan Student Services Organization fellowship. We are grateful to Samantha Schmitt for assistance with data collection, and Karen McCracken and Mary Ellen Smyth for animal care.

References

Helen S Mayberg, Andres M Lozano, Valerie Voon, Heather E McNeely, David Seminowicz, Clement Hamani, Jason M Schwalb, and Sidney H Kennedy. Deep brain stimulation for treatment-resistant depression. Neuron , 45(5):651-660, 2005.

Robert S Fisher and Ana Luisa Velasco. Electrical brain stimulation for epilepsy. Nature Reviews Neurology , 10(5):261-270, 2014.

Andres M Lozano, Nir Lipsman, Hagai Bergman, Peter Brown, Stephan Chabardes, Jin Woo Chang, Keith Matthews, Cameron C McIntyre, Thomas E Schlaepfer, Michael Schulder, et al. Deep brain stimulation: current challenges and future directions. Nature Reviews Neurology , 15(3):148-160, 2019.

Marlene R Cohen and William T Newsome. What electrical microstimulation has revealed about the neural basis of cognition. Current opinion in neurobiology , 14(2):169-177, 2004.

Charles J Bruce, Michael E Goldberg, M Catherine Bushnell, and Gregory B Stanton. Primate frontal eye fields. ii. physiological and anatomical correlates of electrically evoked eye movements. Journal of neurophysiology , 54(3):714-734, 1985.

Giles S Brindley and Walpole S Lewin. The sensations produced by electrical stimulation of the visual cortex. The Journal of physiology , 196(2):479-493, 1968.

John S Choi, Austin J Brockmeier, David B McNiel, Lee M Von Kraus, José C Príncipe, and Joseph T Francis. Eliciting naturalistic cortical responses with a sensory prosthesis via optimized microstimulation. Journal of neural engineering , 13(5):056007, 2016.

Mark M Churchland and Krishna V Shenoy. Delay of movement caused by disruption of cortical preparatory activity. Journal of neurophysiology , 97(1):348-359, 2007.

Nuo Li, Kayvon Daie, Karel Svoboda, and Shaul Druckmann. Robust neuronal dynamics in premotor cortex during motor planning. Nature , 532(7600):459-464, 2016.

Tirin Moore and Mazyar Fallah. Microstimulation of the frontal eye field and its effects on covert spatial attention. Journal of neurophysiology , 91(1):152-162, 2004.

Mark H Histed, Vincent Bonin, and R Clay Reid. Direct activation of sparse, distributed populations of cortical neurons by electrical microstimulation. Neuron , 63(4):508-522, 2009.

Sina Tafazoli, Camden J MacDowell, Zongda Che, Katherine C Letai, Cynthia R Steinhardt, and Timothy J Buschman. Learning to control the brain through adaptive closed-loop patterned stimulation. Journal of Neural Engineering , 17(5):056007, 2020.

Yuxiao Yang, Shaoyu Qiao, Omid G Sani, J Isaac Sedillo, Breonna Ferrentino, Bijan Pesaran, and Maryam M Shanechi. Modelling and prediction of the dynamic responses of large-scale brain networks during direct electrical stimulation. Nature biomedical engineering , 5(4):324345, 2021.

Amin Nejatbakhsh, Francesco Fumarola, Saleh Esteki, Taro Toyoizumi, Roozbeh Kiani, and Luca Mazzucato. Predicting the effect of micro-stimulation on macaque prefrontal activity based on spontaneous circuit dynamics. Physical Review Research , 5(4):043211, 2023.

Chieko M Murasugi, C Daniel Salzman, and William T Newsome. Microstimulation in visual area MT: effects of varying pulse amplitude and frequency. Journal of Neuroscience , 13(4):1719-1729, 1993.

Alexis M Kuncel and Warren M Grill. Selection of stimulus parameters for deep brain stimulation. Clinical neurophysiology , 115(11):2431-2441, 2004.

Meghan Watson, Numa Dancause, and Mohamad Sawan. Intracortical microstimulation parameters dictate the amplitude and latency of evoked responses. Brain Stimulation , 9(2):276-284, 2016.

Thierri Callier, Nathan W Brantly, Attilio Caravelli, and Sliman J Bensmaia. The frequency of cortical microstimulation shapes artificial touch. Proceedings of the National Academy of Sciences , 117(2):1191-1200, 2020.

Hillel Adesnik and Lamiae Abdeladim. Probing neural codes with two-photon holographic optogenetics. Nature neuroscience , 24(10):1356-1366, 2021.

Marcus Triplett, Marta Gajowa, Hillel Adesnik, and Liam Paninski. Bayesian target optimisation for high-precision holographic optogenetics. Advances in Neural Information Processing Systems , 36, 2024.

Samuel Laferriere, Marco Bonizzato, Sandrine L Côté, Numa Dancause, and Guillaume Lajoie. Hierarchical bayesian optimization of spatiotemporal neurostimulations for targeted motor outputs. IEEE Transactions on Neural Systems and Rehabilitation Engineering , 28(6):14521460, 2020.

Yuri B Saalmann, Sima Mofakham, Charles B Mikell, and Petar M Djuric. Microscale multicircuit brain stimulation: Achieving real-time brain state control for novel applications. Current Research in Neurobiology , 4:100071, 2023.

Alan D Degenhart, William E Bishop, Emily R Oby, Elizabeth C Tyler-Kabara, Steven M Chase, Aaron P Batista, and Byron M Yu. Stabilization of a brain-computer interface via the alignment of low-dimensional spaces of neural activity. Nature biomedical engineering , 4(7):672-685, 2020.

Brianna M Karpowicz, Yahia H Ali, Lahiru N Wimalasena, Andrew R Sedler, Mohammad Reza Keshtkaran, Kevin Bodkin, Xuan Ma, Lee E Miller, and Chethan Pandarinath. Stabilizing brain-computer interfaces through alignment of latent dynamics. BioRxiv , pages 2022-04, 2022.

Max Dabagia, Konrad P Kording, and Eva L Dyer. Aligning latent representations of neural activity. Nature Biomedical Engineering , 7(4):337-343, 2023.

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature , 521(7553):436-444, 2015.

Richard S Sutton. Reinforcement learning: An introduction. A Bradford Book , 2018.

Peter H Schönemann. A generalized solution of the orthogonal procrustes problem. Psychometrika , 31(1):1-10, 1966.

Leo Breiman. Bagging predictors. Machine learning , 24:123-140, 1996.

David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back-propagating errors. Nature , 323(6088):533-536, 1986.

Christopher KI Williams and Carl Edward Rasmussen. Gaussian processes for machine learning , volume 2. MIT press Cambridge, MA, 2006.

Jacob R Gardner, Geoff Pleiss, David Bindel, Kilian Q Weinberger, and Andrew Gordon Wilson. Gpytorch: Blackbox matrix-matrix gaussian process inference with gpu acceleration. In Advances in Neural Information Processing Systems , 2018.

Supplementary material

S1 Spiking activity preprocessing

To identify the latent dimensions using FA, we computed binned spike counts during the 1.2 s fixation period on each no-uStim trial (50 ms bins, yielding 24 bins per trial). Thus, K noStim i for the i th session was the number of no-uStim trials × 24. These no-uStim trials were also used to extract a list of usable electrodes e i for the i th session based on the following three criteria: mean firing rate >1 Hz, Fano factor <8, and <20 % coincident spiking with each of the other electrodes. The latent dimensionality ( m = 4 ) was chosen based on the cross-validated data likelihood across multiple sessions, then used for all sessions. To evaluate the uStim response, we computed binned spike counts starting 50 ms after uStim offset on each uStim trial (Fig. 1B) to avoid uStim artifacts. The latent activity was then computed using equation (4).

S2 Models architecture and fitting

The CNN architecture and hyperparameters were determined based on a grid search. The following describes the CNN architecture used in this work from input to output: 1. a convolutional layer with 32 channels, spatial kernel size 3x3, and stride size 1, followed by ReLU activation, 2. a convolutional layer with 64 channels, spatial kernel size 3x3, and stride size 1, followed by ReLU activation, 3. two linear layers with ReLU activation, and 4. a linear output layer of size equal to the target dimensionality without any activation. The MLP architecture used in this work includes: 1. an input layer of size 96, 2. a hidden layer of size 10, followed by ReLU activation, and 3. an output layer of size equal to the target dimensionality. The CNN and MLP were implemented in PyTorch and fit using the Adam optimizer with mean squared error loss and a learning rate of 0.001. The GP model was fitted using the GPyTorch library [32]. We used a Radial Basis Function whose length scale hyperparameter was chosen by maximizing the marginal data log likelihood using the Adam optimizer with a learning rate of 0.001. We trained all models on a local computing cluster using 4 NVIDIA GeForce RTX GPUs and 11GB of RAM. We used the same architecture and hyperparameters for all experiments.

To train CNN models using bagging (Section 2.4), the merged stimulation-response samples across sessions were split into training, validation, and test sets with a ratio of 80:10:10. Each CNN model was trained on the training data. The validation data was used for hyperparameter tuning. The test data were used to evaluate the ability of the models to generalize to untested uStim patterns, and to choose the top C performing models to generate the uStim response predictions for all possible uStim patterns. These predictions were used to initialize the closed-loop optimization (Section 2.5).

S3 Details of uStim experimental paradigm

In each experimental session, the monkey performed a visually-guided saccade task. In this task, the monkey first fixated on a dot at the center of the screen. Following a random (1.45-1.75s) fixation period, the center dot turned off and one of four peripheral targets (45°, 135°, 225°, 315°) appeared. The monkey saccaded to that target to receive a liquid reward. There were two types of visually-guided saccade trials: 'uStim trials', in which we applied uStim, and 'no-uStim trials', in which we did not apply uStim. The experimental system randomly chose which trial type to perform in an interleaved manner. On uStim trials, we applied uStim for 150 ms during the fixation period. The stimulation was biphasic with each square pulse in the biphasic pair being 250 ms in duration. We set the current amplitude low enough not to induce any eye movements (current amplitude: 25 uA for single-electrode uStim, 15 uA + 15 uA for double-electrode uStim). We changed the location(s) of the stimulated electrode(s) on each trial, while keeping other parameter values such as current amplitude and frequency (350 Hz) fixed. In each closed-loop session, we chose two target dimensions along which we could modulate neural activity and induce diverse latent activity with uStim. These were not necessarily the top FA dimensions that explained the greatest covariance among the neurons.

S4 Closed-loop performance assessment methods

The table below summarizes the methods used to assess closed-loop performance. The 'Prediction model' column indicates the method used to obtain uStim response predictions to initialize the closed-loop optimization. The 'Training data' column indicates the stimulation-response samples used to train the prediction model. The 'uStim patterns' column indicates the number of electrodes used for stimulation, both in the training data and during closed-loop optimization. The 'Online algorithm' column indicates the method used to choose the next uStim pattern. All methods that use the epsilon greedy algorithm also perform closed-loop updates of the predicted responses (equations (6)-(9)).

S5 Spatial structure of uStim responses

To evaluate the spatial smoothness of uStim responses across the multi-electrode array (Fig. 3), we measured the spatial similarity between uStim patterns ( d pattern ) and the similarity in the uStim response elicited by those patterns ( d response ). We computed these metrics for every pair of tested uStim patterns.

For single-electrode uStim patterns e 1 = [ x 1 , y 1 ] and e 2 = [ x 2 , y 2 ] (for x i and y i ∈ { 0 , 1 , ..., 9 } ), we quantified their spatial similarity using the L1 distance:

where x 1 and y 1 are the spatial coordinates on the multi-electrode grid for the first uStim electrode e 1 , and x 2 and y 2 are the spatial coordinates for the second uStim electrode e 2 .

For double-electrode uStim patterns, each pattern involves two electrodes. Let e 1 = [ x 1 , y 1 ] and e 2 = [ x 2 , y 2 ] represent the uStim electrodes used for the first pattern, and e 3 = [ x 3 , y 3 ] and e 4 = [ x 4 , y 4 ] represent the uStim electrodes used for the second pattern. To calculate the distance in this case, we computed:

and used min( d 1 pattern , d 2 pattern ) as the distance measure.

For each pair of uStim patterns, we quantified the difference in uStim response using the L1 distance:

where z 1 is the response to the first uStim pattern along a target dimension, and z 2 is the uStim response to the second uStim pattern along the same target dimension. To focus on local smoothness, we only analyzed the pairs in which d pattern was less than 8 in the single-electrode case, and less than 16 in the double-electrode case.

Using the d pattern and d response values, we computed the correlation between them to evaluate if spatial structure is present. A positive correlation indicates that uStim electrodes located closer to each other on the array tend to induce a more similar response.

S6 Summary of experimental sessions

The experimental sessions used in this study comprised 3 sessions without uStim (reference sessions), 10 sessions involving single-electrode uStim, 19 sessions involving double-electrode uStim, and 8 closed-loop sessions. Each of the entries in the tables below uses a subset of the sessions described above.

The table below summarizes the number of sessions used in each closed-loop experimental result:

The table below summarizes the number of sessions used in each offline analysis result:

S7 Code availability

Python code for MiSO is available on GitHub at https://github.com/yuumii-san/MiSO.git.

A

Session 1

Session2

Session3

Session4

Session5

Inconsistent

3

.Firing

Normalized distance

50

Rate

ofresponsemap

Avg.FiringRate

Firing

fromisession

2

Rate

FAdim1

0

B

FA dim2

2

Avg.latent

activity

FA

dim1

0

0

2

3

4

5

Consistent

Session

A

B

r = 0.211

r = 0.159

uStim response diff.

2

'P

uStim response

1

0

0

1

2

3

4

5

6

7

8

4

8

12

16

uStim pattern diff.

uStim pattern diff.

A

B

Single elec. Test MSE

0.35

Double elec. Test MSE

0.4

0.3

0.25

Single

elec.

Double

elec.

Both

Single

elec.

Double

elec.

Both

uStim patterns used for training

uStim patterns used for training

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We provide a summary of the current literature and gap, and explain the contributions of our work in the abstract and introduction. Our claims are supported by the experimental results.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]

Justification: We discuss the limitations of this work in the second and third paragraphs of the discussion section.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: This paper does not include theoretical results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We disclose the details of the proposed framework, experiments, data analyses, and modeling approaches needed to reproduce the main results. We have also released the code to replicate our framework at https://github.com/yuumii-san/MiSO.git.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The code and sample data needed to replicate the experiments are available at https://github.com/yuumii-san/MiSO.git.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The training and test details are provided in Section S2.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We include error bars in all figures and provide the outcomes of statistical tests for the main results.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide details for the computer resources in Section S2.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We reviewed the NeurIPS Code of Ethics and our work is in compliance. All experimental procedures involved in this work were conducted in accordance with the United States National Research Council's Guide for the Care and Use of Laboratory Animals, and were approved by the Institutional Animal Care and Use Committee of Carnegie Mellon University.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We discuss the potential societal impacts in the first paragraph of the introduction section.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The data do not pose a risk for release as it was not acquired from human subjects and does not contain images or identifiable information.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We cite all assets used in this paper.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: The methods section provides the detail necessary to reproduce the proposed framework. We also provide code and sample data at https://github.com/yuumiisan/MiSO.git.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This work did not involve human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors proposed a framework for closed-loop brain stimulation to optimize the search of parameters in order to guide the measured brain responses towards certain predefined states. The proposed work is built on two blocks with the first one in charge of conducting latent space alignment to combine data sets acquired from different sessions. The second block is aimed at performing predictions on the brain responses by using a convolutional neural network.\n\nSTRENGTHS:\n-A novel method for applying latent space alignment to merge neural activity across sessions in the field of brain stimulation.\n-In terms of reproducibility, the implementation aspects are well-detailed in the manuscript, except for the source code which was not included.\n\nWEAKNESSES:\n- The stimulation parameters have a large search space that is not defined.\n- The modality to measure brain responses is not specified, and its variability will pose related but different challenges.\n- The limitations are weakly addressed, and it's not clear in which situations the proposed method will fail. \n- The deceptive scenarios in the search space are not mentioned.\n\nQUESTIONS:\n-Could the authors justify the innovations given that 1) conducting latent space alignment and, 2) using a CNN to predict the effects of stimulation parameter combination have been already reported in the literature?\n-It is not clear why 4,500 uStim configurations. Could the authors clarify the figure?\n-The multi-session need mentioned is not supported with evidence. Why longer sessions cannot alleviate the problem?\n-Are there any parameter combinations irrelevant? If not, support why each of the possible 4,500 configurations is relevant and their effects mutually exclusive.\n\nLIMITATIONS:\n-It is not clear the situations in which the proposed method will fail. The deceptive scenarios in the search space are not mentioned. \n-There are no examples or further information about the massive number of stimulation configurations or the extent of them that might have redundant effects.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors propose MiSO (Microstimulation Optimization), a closed-loop stimulation framework designed to optimally generate microstimulation patterns to drive neural activity towards target states. The use of many electrodes for stimulation presents a challenge due to the curse of dimensionality, leading to an exponential increase in the number of searchable patterns. The authors address this issue with two novel contributions: \n\n1. They use  latent space alignment method to concatenate data across recording days, taking into account the drift in neural states and aligning the neural activity of subsequent days to a reference day.\n\n2. They test multiple neural network architectures, including Multi-Layer Perceptrons(MLP), Gaussian Processes (GP), and Convolutional Neural Networks (CNN), to predict the effect of stimulation for unseen patterns. Additional experiments show that spatially close electrodes can produce similar neural patterns, and CNNs are particularly effective at capturing this feature. \n\nThe authors demonstrate that using MiSO with double electrode stimulation patterns can effectively drive neural activity towards the target state.\n\nSTRENGTHS:\nThis is a well-written manuscript with interesting results. The authors tackle a significant challenge in the field of brain-computer interfaces (BCI): how to guide a population of neural activity towards a target neural state using large number of stimulation patterns. The use of a subspace alignment method is particularly noteworthy, as it enables the utilization of data across different days, which is crucial given the experimental and clinical limitations. Although the implementation of neural networks to predict the effect of stimulation in a closed-loop system has been proposed before (Rao, Current Opinion in Neurobiology 2019), and a similar methods have been used in mice (Ref 19) and  non-human primates (NHPs,Ref 20) to my knowledge, this is the first work that applies subspace alignment and CNNs in a closed loop system in NHPs.\n\nWEAKNESSES:\n1. It is unclear why the authors do not fully integrate the CNN into the closed-loop system. From my understanding, the CNN generates predictions expected to produce unique responses to stimulation for unseen patterns. These patterns (Zp) are then used to find the optimal stimulation pattern. Another approach could be to solve the inverse of the CNN to identify the best input pattern that generates the target output pattern(e.g. Bashivan et al, Science, 2019), thus fully integrating the CNN into the closed loop. In the current setup, the CNN is only trained and tested once at the beginning, with the epsilon-greedy algorithm taking over—a method already explored in Ref 19. Can authors discuss why they have chosen this framework?  \n\n2. As mentioned in the discussion, the system is only tested with up to two electrodes at a time, significantly limiting the number of possible patterns to around 4,000. This problem size does not necessarily require a CNN to solve; previous methods, such as in reference 19, have used a larger number of electrodes without CNNs. The authors should discuss how they plan to scale this approach to a higher number of electrodes and the experimental and theoretical challenges involved.\n\nQUESTIONS:\n1. The current paper reports only the L1 error as a metric for convergence. However, the speed of convergence is an important factor in evaluating an algorithm's success. Specifically, from Figure 4B, it is unclear when MiSO with double electrodes achieves convergence close to the target. Can authors show how fast each of the algorithms converge to target?\n\n2. The authors should include an example of a neural target pattern in Figure 4. Similar to Fig S1, a visual representation of target activity is important. \n\n3. Did the authors attempt to show visual stimuli to the monkey and recreate the response pattern using MiSO? If such data exists, it would be beneficial to include it.\n\n4. What are the theoretical limits of control in this setting? Specifically, what is the maximum effective dimensionality of control that can be achieved using 96 channels of stimulation? As shown in Figure 3, the spatial distance affects the pattern differences.\n\n5. What is the limitation of this system to massive state fluctuations of the brain? Can their linear Subspace alignment method capture that or they require a refit? Moreover, when do they decide to refit the CNN or add the current trials to it?\n\nLIMITATIONS:\nIn its current form, it is unclear how this system can advance the state of the art in BCI systems. The authors should discuss the challenges they face when increasing the number of channels. Additionally, given the small current of 25 µA used here, how do the authors think they can induce behavioral changes, especially since FEF saccades typically require currents around 50 µA? Lastly, if they increase the number of stimulation electrodes, what are the potential implications for tissue damage?", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents MiSO (MicroStimulation Optimization), a closed-loop framework designed to drive neural population activity towards specified states by optimizing stimulation parameters over a large parameter space. MiSO addresses the challenge of the large search space by: latent space alignment and a convolutional neural network (CNN) to predict brain responses to unseen stimulation parameters.\n\nSTRENGTHS:\n- Leveraging latent space alignment to address the challenge of a large subspace is an interesting approach. \n- Experimental evaluation on a primate subject enhances credibility to application of the approach in the real world.\n- The paper is structured well.\n\nWEAKNESSES:\n- The work is preliminary with limited subjects, baselines, model comparisons, and latent space alignment methods\n- The applicability is tested on very less data and only on one primate subject, over 5 sessions, hence it is hard to gauge the effectiveness.\n- Baselines are very less and simple.\n- Latent space alignment method chosen is a classical Procrustes method, choice behind this not explained\n- The applicability of this method looks to be in a very specialized field. The broader impact or directions, even in other stimulation protocols, like functional electrical stimulation (FES) in which this method could be applied is not explained. The paper claims to have a social impact, however, the effort in identifying the social impact outside the experimented area is not visible.\n\nQUESTIONS:\n- L 98: How does MISO identify a reference latent subspace?\n\n- How sensitive is this method to the choice of latent space alignment?\n\n- Other latent space alignment methods such as canonical correlation analyses (CCA), manifold alignment, etc. exist which may be more relevant to integrate data from different sources(e.g., electrodes). Why do the authors use the particular method?\n\n- Model comparisons were done only among MLP, Gaussian Process and Convnets, why not others, e.g., classical ML methods as well as other sequence modeling ones like LSTMN, GRU? \n\n- Is there any time-delay embedding on the inputs?\n\n- What is the computational complexity of the overall method? How does it scale with the number of electrodes? \n\n- How robust are the CNN predictions to variability in neural responses or to potential noise in the data?\n\n- How does the performance of MiSO degrade with increasing complexity of the parameter space or neural dynamics?\n\n- What are the potential impacts of recording instabilities on the latent space alignment and subsequent model predictions?\n\n- Providing a broader outlook and connecting its concepts to other similar areas and discussing its impact would be desirable. For, e.g., in the context of FES, the goal would be to map stimulation parameters to muscle responses rather than neural population activity. \n\n- Please mention the limitations of the work more clearly\n\nLIMITATIONS:\nLimitations are not stated properly", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-26"}